Skip to content
curl_reap

curl_reap / Guides

HTTP/2

Built-in HTTP/2 in curl_reap 2.0: on by default, per-target SETTINGS and HPACK shaping, decoded bodies, flow control, GOAWAY and opting out.

On this page
  1. First principles
  2. Installation and the h2 extra
  3. How negotiation works
  4. Turning it off
  5. Browser-shaped framing
  6. Response bodies
  7. Flow control
  8. Connection reuse
  9. GOAWAY and stream errors
  10. Server push
  11. Limits

HTTP/2 is version 2 of the Hypertext Transfer Protocol (HTTP). curl_reap 2.0 has its own HTTP/2 client in curl_reap._h2. It needs only the hpack package (for the Huffman code and static table of HTTP/2 header compression, HPACK), which is a required dependency, so there is nothing extra to install. HTTP/2 is offered on every https:// request by default.

import curl_reap as reap

with reap.Session() as s:
    r = s.get("https://example.com")
    print(r.status, r.http_version)    # "HTTP/2" when the server chose h2

First principles

  • HTTP/2 sends binary frames on one connection. Before the first request the client announces its settings. The values and their order are the client’s own choice, so they identify the client as clearly as its handshake does.
  • There are several valid ways to encode the same header. HPACK lets a client index a header, send it literally, or mark it never-indexed, with or without Huffman coding. Each implementation has its own habits, and those are visible too.
  • Flow control is per stream and per connection. The receiver tells the sender when it may send more with WINDOW_UPDATE frames. When to send them is another choice, so curl_reap sends them when Chrome does.
  • A server may end a connection at any moment with GOAWAY. Whether a request is safe to retry depends on whether the server processed it, and the GOAWAY frame names the last stream the server may have processed.

Installation and the h2 extra

pip install curl-reap is enough. The h2 extra from 1.1.0 still exists so that pip install "curl-reap[h2]" keeps working, but it is an empty alias: it installs nothing more. hyper-h2 is no longer used.

How negotiation works

  • The Transport Layer Security (TLS) handshake offers h2 and http/1.1 through Application-Layer Protocol Negotiation (ALPN). The server chooses. A hand-built profile without an HTTP/2 spec offers only http/1.1; see Custom profiles.
  • If the server picks h2, the request goes over HTTP/2. Otherwise it uses HTTP/1.1 on the same connection.
  • Plain http:// URLs always use HTTP/1.1.
  • It works through HTTP, HTTPS and Socket Secure (SOCKS) proxies, since the TLS handshake with the target happens inside the tunnel.
  • r.http_version is "HTTP/2" or "HTTP/1.1", and r.infos["alpn"] holds the protocol the server selected on a TLS connection.

Turning it off

s = reap.Session(http2=False)              # offer only http/1.1
s = reap.Session(http_version="v1")        # the same, spelled explicitly

http_version overrides http2 when both are given. Accepted values:

Value Effect
None (default) "v2" when http2=True (the default), "v1" when http2=False
"v1" ALPN offers only http/1.1
"v2", "v2tls" ALPN offers h2 and http/1.1
"v2_prior_knowledge" Accepted. Plain http:// still uses HTTP/1.1
"v3", "v3only" Raise NotImplementedError: HTTP/3 is not implemented

Any other value raises ValueError. http_version= is also accepted per request, but it only affects new connections: an idle pooled connection to the same host, proxy and profile is reused as it is.

On the command line, reap --http2 is still accepted and does nothing, because HTTP/2 is already negotiated by default.

Browser-shaped framing

Each impersonation target carries an HTTP/2 spec captured from the real browser, and the connection reproduces it:

  • the connection preface: one SETTINGS frame with the browser’s settings in the browser’s order, then a WINDOW_UPDATE on stream 0 with the browser’s increment, then any PRIORITY frames the browser sends;
  • request HEADERS frames with the browser’s flags and priority field;
  • the pseudo header order (m,a,s,p for Chrome, m,p,a,s for Firefox, m,s,a,p for Safari);
  • HPACK representation choices (indexed, literal with or without indexing, never indexed, Huffman or raw) that follow nghttp2’s encoder byte for byte.

Headers come from the target’s captured HTTP/2 set (Profile.h2_headers), which is not always the HTTP/1.1 set lowercased: Chrome 124 and newer and Safari 18.0 and newer add priority: u=0, i over HTTP/2, and Safari places it before accept-encoding. Your own headers replace values in place or are appended. A custom profile without an HTTP/2 set sends its HTTP/1.1 headers lowercased. Connection-specific headers (Connection, Keep-Alive, Proxy-Connection, Transfer-Encoding, Upgrade, Host) are left out, and the host goes in :authority. A te header is kept only when its value is trailers.

You can supply your own HTTP/2 fingerprint per request with an Akamai fingerprint string:

akamai = "1:65536;2:0;4:6291456;6:262144|15663105|0|m,a,s,p"
r = s.get("https://example.com", akamai=akamai)

Response bodies

  • Bodies are content-decoded exactly as over HTTP/1.1: gzip (including multi-member gzip), deflate and Brotli always, zstd when the zstd extra is installed (or on Python 3.14+). r.content and r.text hold the decoded body.
  • A buffered HTTP/2 body larger than 64 MiB raises TransportError. This cap is fixed for HTTP/2; the session’s max_body_bytes applies to HTTP/1.1.
  • Session.download() streams over HTTP/2 too: the received bytes go to a spooled temporary file (in memory up to 8 MiB, on disk beyond), then are decoded into the target file. Memory stays bounded whatever the size. See Caching and downloads.
  • Trailers are read and kept off the response headers. 1xx informational responses before the final response are skipped.
  • If the server sent a Content-Length and the stream ends with fewer bytes, IncompleteRead is raised.

Flow control

Both directions follow the flow control rules of Request for Comments (RFC) 9113 on the stream window and the connection window:

  • Request bodies are split by the server’s maximum frame size and by both send windows, and the client waits for WINDOW_UPDATE when a window is exhausted. Bodies above 16 KB work.
  • On the receive side a WINDOW_UPDATE is sent the way Chrome does it: once more than half of a window has been consumed, separately for the stream and for the connection.

Connection reuse

  • One TLS connection carries one request at a time. Concurrent requests to the same host from several threads open separate connections.
  • After a request, a healthy connection goes back to the session’s pool and is reused for the next request with the same host, port, proxy, profile, TLS settings and interface. The pool keeps up to 10 idle connections per key, each for up to 60 seconds.
  • Before reuse, an idle HTTP/2 connection is polled: frames that arrived while it sat idle are processed, so a GOAWAY the server sent in the meantime is seen and the connection is dropped instead of reused.
  • Session.close() sends GOAWAY on each pooled HTTP/2 connection before closing it.

GOAWAY and stream errors

What the server does Result
GOAWAY whose last stream id is below the request’s stream The request was never processed. The error is marked retryable.
RST_STREAM with REFUSED_STREAM Marked retryable.
Connection closed before any response headers arrived Marked retryable.
GOAWAY with an error code that covers the request’s stream, before the response finished Error, not marked retryable.
GOAWAY with NO_ERROR that covers the request’s stream The response is read to the end; the connection is not reused.
The response completed, and a GOAWAY (even with an error code) arrives in the same read The completed response is returned. The GOAWAY ends the connection, not the exchange.

Errors surface as TransportError, with the underlying curl_reap._h2.H2Error as __cause__. H2Error has error_code (the RFC 9113 code), error_name, stream_id and retryable, and the TransportError message contains (retryable) when that flag is set. The session’s retry loop retries transport failures for every method in RetryPolicy.allowed_methods (GET, HEAD, OPTIONS, PUT and DELETE by default); a connection that failed is closed and never returned to the pool. See Sessions.

Server push

Server push is refused. A PUSH_PROMISE is answered with RST_STREAM REFUSED_STREAM, and pushed data never becomes part of a response.

Limits

  • No multiplexing: each connection runs one request at a time.
  • HTTP/3 (QUIC) is not implemented. http_version="v3" or "v3only" raises NotImplementedError.
  • No cleartext HTTP/2: http:// URLs use HTTP/1.1.
  • stream=True does not stream over HTTP/2. The body is buffered (up to the 64 MiB cap) and iter_content() replays it. For large bodies use download(), or stream=True with http2=False, which streams over HTTP/1.1.
  • A buffered HTTP/2 body is capped at 64 MiB regardless of max_body_bytes.