terbium / Guides
Schemas
The built-in terbium schemas (product, generic, furniture, transaction, resume), how the default is chosen, and how to write your own.
On this page
A schema is the one place that knows what a row of output should look like. Everything upstream is domain-agnostic: swap the schema and keep the engine.
How the schema is chosen
Pass schema= to terbium.parse to choose one. When you leave it as None (the default), terbium picks it from the document type:
catalog,lookbook,table:producttransaction:transactionresume:resumedeck,unknown:generic
With doc_type="auto" (the default) the type comes from auto-classification. On the command line, --schema applies to the records view.
The schema shapes records; it does not decide which page-level readers run. The transaction and resume readers that work from labels and headings run when the document type is transaction or resume, so pass doc_type= as well when you force one of those schemas on a document the classifier labelled differently.
Built-in schemas
product
Universal and category-agnostic. It maps each column header to a canonical product field by meaning, using synonym lists (FIELD_SYNONYMS in schema/product.py). The longest matching synonym wins, so “pack size” maps to quantity rather than dimensions.
Canonical fields: sku, name, price, dimensions, color, material, category, quantity, weight, brand.
The synonym lists are broad, so check the mapping on a new supplier’s file. For example, Description and Item map to name, Finish and Shade to color, Code and Ref to sku, Rate and Amount to price, and Type and Range to category.
- A price cell is also split into
price_amountandcurrency(from$,£,€,₹,Rs, or a code such asUSD). - A column with no canonical match keeps its own header as the field name, so a bag’s “capacity” or a lamp’s “wattage” survives as an attribute.
- The table title becomes
collection. - If no column is named like a stock keeping unit (SKU) code, the first compact code-like cell containing a digit is used, skipping price, quantity and weight columns.
- Normalized keys are added alongside the raw values:
color_family,material_familyanddimensions_mm. See Normalization and feeds.
doc = terbium.parse("stock.xlsx") # classified as "table", so schema "product"
doc.records[0].fields
# {"sku": "35155", "name": "Sequence table", "quantity": "12", "collection": "Stock"}
generic
Works for any table without domain knowledge.
- Grid tables (Excel, comma-separated values (CSV), PowerPoint, and tables rebuilt from a Portable Document Format (PDF) page): one record per body row; columns become fields named after their headers. A compact, digit-bearing code of 3 to 16 characters in the row becomes the
sku. The table title is added astitle, and any shared table attributes are copied in. - Matrix and list tables (from the furniture detector): one record per filled cell, with
row,columnandtitlefields and the cell value as thesku.
doc = terbium.parse("stock.xlsx", schema="generic")
doc.records[0].fields
# {"Article": "35155", "Product": "Sequence table", "Qty": "12", "title": "Stock"}
furniture
One record per article number, tuned for a catalogue size by finish matrix. Choosing it also switches PDF reconstruction to the specialized cross-tab detector.
doc = terbium.parse("catalogue.pdf", schema="furniture")
doc.records[0].fields
# {"product": "Barrow pouf", "size": "40 × 40 × 40 cm", "finish": "off white",
# "dimensions_cm": "40 x 40 x 40 cm", "dimensions_in": '15.5" x 15.5" x 15.5"'}
When the table carries them, composition and height are added too. The repository’s examples/furniture_catalogue.py runs this schema on a catalogue PDF you supply.
transaction and resume
transaction reads invoices, bills, receipts, purchase orders and quotes into summary, line_item and total records. resume reads a curriculum vitae (CV) into a header record and one record per line under each section; rows from a table become records with section set to table. Both are described in Invoices, receipts and resumes.
Writing your own
Subclass terbium.schema.Schema, give it a name, implement build_records(tables) and register it:
import terbium
from terbium.model.record import Record
from terbium.schema import Schema, register_schema
@register_schema
class PricelistSchema(Schema):
name = "pricelist"
def build_records(self, tables):
records = []
for t in tables:
for rh, ch, val in t.iter_cells():
records.append(Record(
sku=val, fields={"item": t.title, "variant": ch},
source_page=t.source_page, confidence=t.confidence,
reasons=list(t.reasons),
))
return records
doc = terbium.parse("prices.pdf", schema="pricelist")
build_records receives ExtractedTable objects. Each has title, row_headers, col_headers, cells (cells[row][col]), source_page (0-based), kind (matrix, grid or list), attributes, confidence, reasons and origin. iter_cells() yields (row_header, col_header, value) for every non-empty cell.
Lookbook label grids do not pass through the schema. Whatever schema you choose, each label becomes a record with a name field and, when the page has a collection title, a collection field. These records have a fixed confidence of 0.82.
You can also pass a Schema instance directly: terbium.parse(path, schema=PricelistSchema()). An unknown schema name raises ValueError listing the registered names.