curl_reap

A server sees how a client connects before it sees what the client asks for, and a request that does not look like a browser's can get a different answer. curl_reap is a Python library whose own transport sends a captured browser's Transport Layer Security (TLS) handshake byte for byte, and matches that browser's HTTP/2 behaviour. Self-healing selectors, structured extraction and a concurrent crawler sit on top, in one library.