WebSocket

curl_reap 2.0.0 includes an RFC 6455 WebSocket client in curl_reap/websocket.py. It is on the main branch and not released yet (see the Changelog). The client opens its connection through the same transport as HTTP requests, so a wss:// connection uses the selected profile’s TLS fingerprint and browser headers.

Connect, send and receive

import curl_reap as reap

with reap.WebSocket.connect("wss://echo.example/socket") as ws:
    ws.send("hello")              # str is sent as a text message
    print(ws.recv())              # str for text messages
    ws.send(b"\x00\x01")          # bytes is sent as a binary message
    print(ws.recv())              # bytes for binary messages

The with block calls close() on exit. Without it, call ws.close() yourself.

WebSocket.connect takes these arguments:

reap.WebSocket.connect(url, *, session=None, subprotocols=(), headers=None,
                       timeout=None, profile=None, proxy=None, verify=True)

Argument	Default	Meaning
url		A ws:// or wss:// URL. Default ports are 80 and 443.
session	None	A Session whose transport and profile to use. Session.ws_connect fills this in.
subprotocols	()	Names sent in Sec-WebSocket-Protocol. The server’s choice is in ws.subprotocol, or None.
headers	None	Extra handshake headers, merged last. A None value removes a profile header.
timeout	None	Seconds. It bounds the whole connection, not only the handshake. See Timeouts below.
profile	None	Profile or impersonate target name, such as "firefox133". Falls back to the session’s profile, then "chrome". There is no impersonate= keyword here.
proxy	None	Proxy URL for this connection.
verify	True	TLS verification, as for HTTP requests.

From a session

Session.ws_connect(url, **kwargs) calls WebSocket.connect(url, session=self, **kwargs):

import curl_reap as reap

with reap.Session(impersonate="firefox133") as s:
    ws = s.ws_connect("wss://echo.example/socket", timeout=30,
                      headers={"Origin": "https://echo.example"})
    ws.send("hi")
    print(ws.recv())
    ws.close()

The WebSocket uses the session’s connection machinery and its profile (here firefox133). It does not use the session’s default headers, cookie jar, proxy or verify setting. Pass headers, proxy and verify to ws_connect instead, including a Cookie header if the server needs one. Many servers check Origin, which is not sent unless you add it.

What goes on the wire

• Handshake. An HTTP/1.1 GET with the profile’s browser header set (for Chrome: sec-ch-ua, User-Agent, Accept, Sec-Fetch-*, Accept-Encoding, Accept-Language and so on), then Host, Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Key, Sec-WebSocket-Version: 13, the optional Sec-WebSocket-Protocol, and your headers. The client requires a 101 status and checks Sec-WebSocket-Accept.
• TLS. For wss://, the TLS handshake comes from the profile’s captured ClientHello through the native engine, with the same OpenSSL fallback as HTTP. ALPN offers only http/1.1, because the upgrade runs over HTTP/1.1.
• Frames. Every client frame is masked. No extensions are offered, so there is no permessage-deflate compression.

Messages

Method	Meaning
send(payload, chunk_size=65536)	str is sent as text (UTF-8), bytes, bytearray or memoryview as binary. Anything else raises TypeError.
send_text(text), send_bytes(data)	Same as send.
send_json(obj, **kwargs)	json.dumps(obj, **kwargs) sent as text.
recv(timeout=None)	The next data message: str for text, bytes for binary.
recv_json(timeout=None)	json.loads(recv(timeout)).

Text messages are decoded as UTF-8 with replacement characters, so invalid UTF-8 does not raise. A single incoming frame larger than 64 MiB raises WebSocketError.

send() is protected by a lock, so one thread can send while another thread receives.

Fragmentation

Outgoing messages longer than chunk_size bytes (64 KiB by default) are split into a first frame and continuation frames. Incoming fragmented messages are reassembled, and recv() returns the whole message.

ws.send("x" * 200_000)                    # four frames with the default chunk size
ws.send(b"\x00" * 200_000, chunk_size=16 * 1024)  # 13 smaller fragments

Ping, pong and keepalive

• Pings from the server are answered with a pong automatically while recv() is reading. Pongs from the server are discarded. Neither is returned by recv().
• There is no background thread, so pings are only answered while your code is inside recv(), and no pings are sent unless you send them.
• To keep a quiet connection alive, call ws.ping() yourself at an interval the server accepts. ws.pong() sends an unsolicited pong. Both take an optional str or bytes payload, cut to 125 bytes.

import curl_reap as reap
from curl_reap.exceptions import WebSocketTimeout

ws = reap.WebSocket.connect("wss://echo.example/socket")
while True:
    try:
        msg = ws.recv(timeout=20)
    except WebSocketTimeout:
        ws.ping(b"keepalive")      # nothing arrived for 20 s
        continue
    print(msg)

Closing and close codes

close(code=1000, reason="", timeout=5.0) sends a close frame, waits up to timeout seconds for the server’s close frame, and closes the socket. Calling it again does nothing. The reason is cut to 123 bytes.

When the server closes first, recv() raises WebSocketClosed and the client answers with the same code:

from curl_reap.exceptions import WebSocketClosed

try:
    while True:
        print(ws.recv())
except WebSocketClosed as exc:
    print(exc.close_code, exc.reason)   # for example 1000 ""
    print(ws.close_code, ws.close_reason, ws.closed)

On the exception the close code is close_code. exc.code is the curl-style error number that every curl_reap exception carries. A close frame without a code is reported as 1005. A connection that drops without a close frame raises WebSocketClosed with close_code 1006.

The module defines constants for the standard codes. They are not exported at the package root:

Constant	Code
CLOSE_NORMAL	1000
CLOSE_GOING_AWAY	1001
CLOSE_PROTOCOL_ERROR	1002
CLOSE_UNSUPPORTED	1003
CLOSE_NO_STATUS	1005
CLOSE_ABNORMAL	1006
CLOSE_INVALID_PAYLOAD	1007
CLOSE_POLICY	1008
CLOSE_TOO_BIG	1009
CLOSE_MISSING_EXTENSION	1010
CLOSE_INTERNAL_ERROR	1011

from curl_reap.websocket import CLOSE_GOING_AWAY

ws.close(code=CLOSE_GOING_AWAY, reason="shutting down")

Timeouts

There are two separate limits:

• timeout on connect or ws_connect is used for the TCP and TLS connect and the handshake, and it also sets a deadline for the whole connection, counted from the start of connect. Once that many seconds have passed, every recv() raises ReadTimeout("websocket deadline exceeded"), even on a healthy connection. It is also the default recv() timeout.
• recv(timeout=...) limits one wait for a frame. When it expires, recv() raises WebSocketTimeout, and the connection stays open.

For a long-lived connection, leave timeout at None on connect and pass a timeout to each recv(). With both unset, recv() waits indefinitely.

Async

AsyncSession.ws_connect(url, **kwargs) and reap.AsyncWebSocket.connect(url, **kwargs) return an AsyncWebSocket. It wraps the same client and runs each call in a thread with asyncio.to_thread, like AsyncSession.

import asyncio
import curl_reap as reap


async def main():
    async with reap.AsyncSession() as s:
        ws = await s.ws_connect("wss://echo.example/socket")
        async with ws:
            await ws.send("hello")
            print(await ws.recv(timeout=10))
            await ws.send_json({"op": "subscribe"})
            print(await ws.recv_json(timeout=10))


asyncio.run(main())

Methods: send(payload), send_json(obj), ping(payload=b""), recv(timeout=None), recv_json(timeout=None), close(), and the closed property. async with closes the connection on exit.

Limitation in 2.0.0: keyword arguments to send, send_json and close raise TypeError, so chunk_size, the close code and reason, and json.dumps options cannot be passed through the async client. There is no async pong, send_text or send_bytes.

Errors

These exceptions are not exported at the package root. Import them from curl_reap.exceptions, except UnknownProfileError, which is in curl_reap.tls. Every one is a subclass of RequestException, which is an OSError.

Exception	Raised when
InvalidSchema (also a ValueError)	The URL is not ws:// or wss://, or has no valid host.
WebSocketError	The server answers the handshake with a status other than 101, Sec-WebSocket-Accept does not match, or a frame is larger than 64 MiB.
WebSocketClosed (a WebSocketError)	The server sent a close frame, the connection dropped, or you send on a closed connection.
WebSocketTimeout (a WebSocketError and a Timeout)	recv() waited longer than its timeout.
ReadTimeout	The connection’s overall deadline from connect(timeout=...) has passed.
ConnectionError, ConnectTimeout, SSLError and other TransportError subclasses	Connecting, the proxy or the TLS handshake failed.
UnknownProfileError (a ValueError)	profile is not a known profile or target.

import curl_reap as reap
from curl_reap.exceptions import TransportError, WebSocketError

try:
    ws = reap.WebSocket.connect("wss://echo.example/socket", timeout=10)
except WebSocketError as exc:      # handshake rejected
    print("refused:", exc)
except TransportError as exc:      # could not connect
    print("connect failed:", exc)

See Sessions, retries, proxies and cookies for the session options and Transport and impersonation profiles for profiles.