RESEARCH / AREA
AI infrastructure.
The code search and web retrieval tools that agent work runs on.
First principles
An agent is only as fast and as far-reaching as the tools it calls. Two calls come up again and again: searching code and fetching pages from the web.
Code search repeats dozens of times in a session, so scanning the whole tree on every query pays the same cost again and again. An index pays it once. glep does that, with output that matches ripgrep.
A web server judges a request by how it connects before it reads what it asks for, and a request that does not look like a browser can get a different answer. curl_reap reproduces the connection of a real browser.