SearchBook a call

RESEARCH / ARCHIVE

The exit-code contract

A field dossier on Kestrel: a coding agent succeeds when its process exits zero with tests green. Harness over model, competitive frame, build plan.

Velofy ·

From the archive. This page preserves the original publication; product plans and capabilities may have changed. See current open source work.

This is a field dossier: our internal working notes on Kestrel, a one-shot CLI coding agent.

Four claims

  1. K1: Value = P(first-shot correctness) × time saved. Everything else is UX polish.
  2. K2: Base models commoditize; durable value is the harness. Context assembly, safe file operations, a verification loop.
  3. K3: “One-shot” means something operational or it means nothing. Defined here: the process exits 0 with tests or build green, or it prints a failure report.
  4. K4: Setup budget: one command plus one environment variable. A GEMINI_API_KEY and nothing else. Setup that costs more loses the audience.

Harness over model

Our own working tree supplied the evidence for the harness thesis. A pack-based design rulebook (flat color, accessibility floors, a five-step verification protocol, a stack mandate) became Kestrel's first pack: domain rulebooks injected into context.

The differentiator: Kestrel's value is not another agent loop. Competitors ship chat loops; Kestrel ships rulebooks and exit codes.

Competitive frame

  • Claude Code / Codex CLI / Gemini CLI: general interactive agents, session-based, strong but chatty.
  • Aider: git-native diffs, good, but verification remains opt-in rather than contractual.
  • Kestrel's claim: the only CLI whose contract is the exit code. Batch-friendly (CI, cron, scripts) because it is non-interactive by design.
Claude Code is a pair programmer. Kestrel is a merge gate you can run from cron.

Build plan

MVP, weeks 1–2:

  1. CLI shell: parse the task, load repo context (tree, git status, relevant files via ripgrep heuristics), call Gemini with a pack-augmented system prompt.
  2. Edit executor: unified-diff apply with dry-run preview and atomic per-file rollback on failure.
  3. Verification gate: detect the package manager, run build/test commands from config or convention, and block success on red.
  4. Report: a green summary, or a failure report carrying hypothesis, diff stat and the tail of the failing command's output.

V2, in order:

  1. Packs: markdown rulebooks loaded by domain; the design pack exists, with pack precedence rules already defined.
  2. Headless mode for CI: a --check flag that fails the pipeline if the fix does not verify.
  3. Opt-in telemetry: first-shot green rate, edits per task, token burn. These numbers are the marketing; publish them like heyIAS publishes calibration.

One harness discipline, three products

Kestrel is also internal infrastructure. Numera's reconciler and filing validators, and heyIAS's evaluation pipeline, should run as kestrel-style verified jobs: deterministic inputs, coded rules, exit codes, audit logs. One harness discipline across accounting, exam prep and coding is the velofy engineering thesis: trust scales with verification, not with intelligence.

Risks

  • R1: Latency and rate limits on large repos. Mitigate with aggressive context pruning and cached file summaries.
  • R2: Overpromising “one-shot.” Define the contract narrowly: the task must include or imply a verification command; otherwise refuse politely and suggest one.
  • R3: Model drift. Pin an evaluation suite of 50 tasks with known-green outcomes and run it against every model version before promoting defaults.

This dossier extends the coding case derived in three markets, one thesis, and implements the confirmation step of the verified agent loop.