terbium / Guides
Confidence and escalation
How terbium scores every reconstructed table, what the threshold does, and the message it prints when a page needs an AI model.
Confidence is the core feature. terbium does not present a shaky parse as solid, and that honesty is what lets it decide when an artificial intelligence (AI) call is worth the cost.
The score is a measure of shape, not of truth. terbium cannot know whether a value is correct, but it can see whether a table has headers, a title, filled cells and rows that line up with its columns. A table that lacks those is more likely to be misread, so each missing piece lowers the score.
How a table is scored
Every reconstructed or native table starts at 0.98 and loses points for specific problems:
| Signal | Penalty |
|---|---|
| No column headers detected | -0.14 |
| No title found above the table | -0.10 |
| Sparse matrix (cells missing) | up to -0.35, scaled by how empty |
| Rows that do not line up with the columns | -0.05 per row, at most -0.20 |
The reason strings are no finish/column headers detected, no product title found above the table, sparse matrix: <filled>/<total> cells filled and <n> row(s) do not line up with the columns. A row lines up when it is either empty or completely filled. Headers count as missing when every header is a placeholder such as col1.
A single-column list is exempt from the header, sparse and row-alignment penalties, because “one finish, no header” is normal, not a failure. The missing-title penalty still applies.
The final score is clamped to the range 0.05 to 1.0 and attached to the table and to every record it produces, along with reasons, the human-readable list of what went wrong:
r = doc.records[0]
r.confidence # 0.88
r.reasons # ['no product title found above the table']
A comma-separated values (CSV) file has no title, so its records score 0.88. Excel (XLSX) sheets and PowerPoint (PPTX) tables take the sheet name or slide title as their title.
Some records get fixed scores instead:
- lookbook label records score 0.82;
- a table rebuilt by the AI lane is set to 0.92, with the reason
arranged by AI (<tier>); - transaction summary records score 0.8, and page-layout transaction records 0.7 (totals) or 0.65 (line items);
- a transaction summary added by AI scores 0.85, and AI line items 0.8;
- a product enriched by
enrich_productstakes the model’s own confidence, or at least 0.9 when the model gives none; - resume header records score 0.8, and section entries 0.7.
At the default threshold of 0.72, page-layout transaction records and resume section entries always count as ambiguous. That reflects how they were found, from loose page text, not a failure, and it does not trigger an escalation message.
The threshold
parse(threshold=0.72) sets the line between confident and ambiguous. It affects labelling (doc.confident_records, doc.ambiguous_records, doc.stats) and escalation. Every record keeps its raw score, so you can apply your own cut:
solid = [r for r in doc.records if r.confidence >= 0.9]
Escalation
When one or more tables fall below the threshold:
- With an AI key, only those tables are sent to the routed model, rebuilt, and their records marked
origin="ai". See AI lane and routing. - Without a key, terbium attaches a message to
doc.escalationand, ifannounce=True(the default), prints it to stderr.
With a key, tables that are still below the threshold after the AI step (because a call failed or returned no usable JavaScript Object Notation (JSON)) keep the message in doc.escalation, but it is not printed.
The message names the count, the pages, up to three reasons and a recommended tier:
terbium: 712/725 records parsed confidently.
3 table(s) on page(s) 15, 26, 30 are ambiguous (no product title found above the table;
sparse matrix: 5/9 cells filled; 2 row(s) do not line up with the columns).
-> set ANTHROPIC_API_KEY (Claude, preferred), OPENAI_API_KEY, MOONSHOT_API_KEY (Kimi),
XAI_API_KEY (Grok), or GEMINI_API_KEY or pass ai=terbium.AI(...) recommended tier: Sonnet
The message is wrapped here for width; terbium prints the reasons and the key hint each on one line.
The recommended tier is Opus if any hard table scores below 0.4, Sonnet if any has a difficulty of 0.3 or more, and Haiku otherwise. Difficulty is explained under routing on the AI lane page.
Image-only pages
A Portable Document Format (PDF) page with images and no text layer cannot be scored as a table at all. When no AI is configured, parse adds one more line for those pages (wrapped here for width):
terbium: 2 page(s) are image-only with no text layer (e.g. 1, 2). Read them with the vision lane:
pass ai=terbium.AI(...) with a key.
Catalog builds
build_catalog has its own message, based on blank fields rather than table scores. See The catalog table.
It never fails silently, and it never bills you for the easy pages.