Source: https://velofy.co/curl_reap/troubleshooting/

[curl\_reap](https://velofy.co/curl_reap/) / Guides

# Troubleshooting and limits

Fix common curl\_reap problems: wrong version, decoding and certificate errors, blocks, doctor mismatches, known issues on main and limits by design.

This page collects the errors and surprises users run into, with what causes each one and what to do. Every message and limit here comes from the source on `main`.

## First principles

Most problems fall into one of three groups, and each group has a quick check:

*   **The request took a different path than you expected.** Another TLS backend, another protocol, another version of the library. Check `r.infos["tls_backend"]`, `r.http_version` and `reap.__version__`.
*   **The server answered, but not with the page.** A block, a challenge or a rate limit is a normal response with a status and a body. Check `r.status` and `r.block_reason`.
*   **You reached a deliberate limit.** Body caps, timeouts, the crawl budget and the queue bound are there to keep a long job from running away. The error message names the limit.

## Quick checks

```
import logging
import curl_reap as reap

logging.basicConfig(level=logging.INFO)     # debug=True logs to "curl_reap.transport"

r = reap.get("https://example.com", debug=True)
print(reap.__version__)                     # "2.0.0" for these docs
print(r.status, r.http_version, r.infos["tls_backend"], r.block_reason)
```

```
reap doctor chrome        # compare your fingerprint with the capture
```

`debug=True` logs every request and response header line at INFO level. Transport Layer Security (TLS) details are in `r.infos`; see [What Response.infos holds](https://velofy.co/curl_reap/transport-and-profiles/#what-responseinfos-holds).

## Installation

*   **`reap.__version__` is not `2.0.0`.** A plain `pip install curl_reap` reads the Python Package Index (PyPI), which serves 0.2.2, an older release built on curl\_cffi. Install from the `main` branch as shown in [Installation](https://velofy.co/curl_reap/installation/).
*   **The git install fails.** pip needs `git` on your `PATH` to install from a git URL.
*   **`ImportError: JS rendering needs Playwright`.** Install the `js` extra, then run `playwright install chromium`.
*   **`reap serve` or `reap worker` exits with status 2.** A required variable (`REAP_API_KEY`, `REAP_ALLOWED_ORIGINS`) is missing, the `server` extra is not installed, an origin is invalid, or Redis could not be set up. The message on standard error says which. See [Private job service](https://velofy.co/curl_reap/job-service/).

## Request errors

*   **`ContentDecodingError` on a `zstd` body.** 41 of the 79 targets, including `chrome` (Chrome 123 and newer) and every Firefox target, advertise `zstd` in `Accept-Encoding`. Decoding a buffered `zstd` body needs the `zstd` extra on Python 3.13 and older. Streamed bodies (`stream=True`) and downloads cannot decode `zstd` at all. For those, send `headers={"Accept-Encoding": "gzip, deflate, br"}`.
*   **`ValueError: tls='native' cannot send a client certificate`.** The native engine cannot present a client certificate. Use `tls="auto"` (the default), which sends `cert=` requests through OpenSSL. See [Client certificates](https://velofy.co/curl_reap/transport-and-profiles/#client-certificates).
*   **`NotImplementedError: HTTP/3 (QUIC) is not implemented`.** `http_version="v3"` and `"v3only"` are accepted names, but HTTP/3 does not exist in curl\_reap.
*   **`TypeError: unexpected request keyword argument`.** The keyword is not a request option. The accepted keywords are listed in [Per-request options](https://velofy.co/curl_reap/sessions/#per-request-options).
*   **`ValueError: profile and impersonate disagree; pass one`.** A request passed both with names that resolve to different targets. Pass one of them.
*   **`UnknownProfileError`.** The target name does not exist. `reap doctor --list` prints all 79 names.
*   **`TransportError: ... exceeds max_body_bytes=67108864`.** A buffered body is over the 64 mebibyte (MiB) cap. Raise `max_body_bytes` for HTTP/1.1, or use `download()` or `stream=True`. Over HTTP/2 the buffered cap is fixed. This error is a `TransportError`, so the session retries it like a network error; pass `retries=0` when a large body is expected.
*   **`ConnectTimeout` or `ReadTimeout`.** A number `timeout` bounds connect, send and receive together; a `(connect, read)` tuple bounds the connect and each read. The clock restarts on every redirect hop and retry. See [Timeouts](https://velofy.co/curl_reap/sessions/#timeouts).
*   **`TooManyRedirects` on a short chain.** With `allow_redirects="safe"`, a redirect to a private, loopback or link-local address is refused with this error. The redirect response is on `exc.response`.
*   **`URLPolicyError`.** Your `url_policy` callable returned a falsy value for the request URL or a redirect hop.
*   **`CookieConflict` from `s.cookies.get(name)`.** Several cookies share the name on different domains or paths. Pass `domain=` or `path=`.

## Blocks and challenges

*   **`r.block_reason` is `"forbidden"`, `"proxy_auth_required"` or `"rate_limited"`.** The server answered 403, 407 or 429. With a proxy pool, the session already switched proxies up to `block_rotations` times before returning this. A 429 is also retried with its `Retry-After` value.
*   **`r.block_reason` is `"challenge"`.** The body looks like a bot-wall interstitial. `render_if_empty` does not render these, and curl\_reap does not solve them. Read [Responsible use](https://velofy.co/curl_reap/responsible-use/).
*   **A crawl callback never sees some pages.** Responses with status 400 or higher and challenge-looking responses go to the request’s errback, not to `parse`. See [Crawling](https://velofy.co/curl_reap/crawling/#requests).
*   **`r.ok` is `True` for a redirect.** In 2.0, `ok` covers 200 to 399. Use `r.is_success` for 2xx only.

## Fingerprint checks

*   **`reap doctor` shows `n/a`.** `ja3` is not comparable for browsers that permute their extensions on each connection, and `headers` is not comparable when the service did not echo them. An `n/a` row does not fail the check.
*   **`reap doctor` shows `DIFFERS`.** Something changed the bytes between curl\_reap and the service. Common causes: a proxy or security product that terminates TLS and sends its own handshake, headers you set yourself, or a `ja3=` or `akamai=` override. The expected values are printed to standard error.
*   **`r.infos["tls_backend"]` is `"openssl"`.** The connection did not use the captured handshake. That happens when the server only speaks TLS 1.2, when the request sends `cert=`, with a custom `Profile` that has no template, or with `tls="openssl"`. The HTTP layer still follows the profile.

## Known issues on main

These are confirmed from the source on `main` as of 2026-10-25.

### TLS 1.2 servers and proxies

When the native handshake finds that the server only speaks TLS 1.2, the transport reconnects for the OpenSSL fallback. That reconnect does not go back through every kind of proxy:

*   **SOCKS proxies:** the fallback connection goes directly to the server, not through the Socket Secure (SOCKS) proxy, so the request leaves from your own address. Only the first, native attempt goes through the proxy.
*   **HTTP proxies:** a direct connection to the server is opened and closed before the request is tunnelled through the proxy.
*   **HTTPS proxies:** the request fails with `TransportError: TLS-1.2-only servers cannot be combined with an https proxy`.

Until this is fixed, use `tls="openssl"` on sessions that must never leave the proxy and may reach TLS 1.2 servers. Those connections use OpenSSL’s handshake from the start, through the proxy, at the cost of the captured TLS fingerprint.

### Other known issues

*   **A custom `Profile` and HTTP/2.** A hand-built profile has no HTTP/2 spec, but the transport still offers `h2`. If the server picks it, the request is written as HTTP/1.1 on that connection. Pass `http2=False` with custom profiles.
*   **Removing a profile header.** A per-request header set to `None` removes a session header but not one that comes from the profile. Use a custom profile to send fewer headers.
*   **`stream=True` over HTTP/2** buffers the body and replays it. Pass `http2=False` for a real stream.
*   **`AsyncWebSocket`** does not accept keyword arguments on `send`, `send_json` and `close`. See [WebSocket](https://velofy.co/curl_reap/websocket/#async).
*   **Cancelling an async request** stops the wait but not a socket call already in progress; the request timeout still bounds it.

## Limits by design

*   No HTTP/3, no native TLS 1.2, no cleartext HTTP/2, and no HTTP/2 multiplexing: one request per connection at a time.
*   Buffered bodies: 64 MiB by default over HTTP/1.1 (`max_body_bytes`), fixed at 64 MiB over HTTP/2. Downloads have no cap.
*   The crawl queue lives in memory. A crash loses pending requests; there is no resume.
*   Cookies are scoped by domain, path and name, but public-suffix lists are not checked.
*   WebSocket: no `permessage-deflate` compression, and a single incoming frame is capped at 64 MiB.
*   Job service: one page per job, results up to 1 MiB, at most 1000 jobs queued and running.