Skip to content
terbium

terbium / Guides

The catalog table

Turn a vendor catalogue into one row per product with name, SKU, materials, dimensions and photo, then write it to CSV.

On this page
  1. Why the photo is the anchor
  2. From the terminal (no AI)
  3. How it works
  4. Dense and sparse photo pages
  5. OCR for image-only pages
  6. When the layout hides things: the AI pass
  7. It tells you when there is more to read
  8. Extra options
  9. Measured results

terbium’s headline job: point it at a vendor catalogue and get back a table of products, one row each, with the name, the stock keeping unit (SKU) code, the materials or ingredients, the dimensions and the path to the product’s image. Then write it to a comma-separated values (CSV) file.

import terbium

rows = terbium.build_catalog("vendor_catalogue.pdf", images_dir="images/")
terbium.to_catalog_csv(rows, "catalogue.csv")

rows is a list of dicts:

{"sku": "RG-1001", "name": "Anatolia Kilim", "materials": "wool",
 "dimensions": None, "image": "Anatolia_Kilim.jpeg", "page": 12}

catalogue.csv has the columns you would hand to a buyer or a product information management (PIM) system:

SKU,Name,Materials/Ingredients,Dimensions,Image,Page
,Kyoto Bedside Table,,,Kyoto_Bedside_Table.jpeg,2
RG-1001,Anatolia Kilim,wool,,Anatolia_Kilim.jpeg,12

to_catalog_csv(rows, path=None) always returns the CSV text and also writes it when you pass a path.

Why the photo is the anchor

Catalogues disagree about almost everything: some print codes and some do not, some put the material in a label and some in a sentence, some use tables and some use free layout. The one thing every product in a catalogue has is a photo, and the text about a product sits near its photo. So terbium builds one row per photo and reads the text around it, instead of hunting for a table that may not exist. When there are no photos to anchor on, it falls back to the product table.

From the terminal (no AI)

terbium catalogue.pdf                    # print the table; photos go to catalogue_images/
terbium catalogue.pdf --csv out.csv      # also write the CSV
terbium catalogue.pdf --csv -            # print the CSV to stdout instead of the table
terbium catalogue.pdf --limit 0          # show every row, not just the first 20
terbium catalogue.pdf --html out.html    # a self-contained HTML table
terbium prices.xlsx --no-images          # a price list: table only, no photo extraction
terbium catalogue.pdf --records          # raw parsed records instead of the table

terbium <file> classifies the file first. When it detects a catalogue, lookbook, table, deck or unknown document, it prints the catalog table with the columns SKU, Name, Materials/Ingredients, Dimensions, Image and Page. For a Portable Document Format (PDF) or PowerPoint (PPTX) file it extracts the photos to <file>_images/ unless you pass --images DIR or --no-images. It uses no artificial intelligence (AI) unless you add --ai. Example header line:

terbium 2.0.0  ·  catalogue.pdf  ·  124 products  ·  images -> catalogue_images/

--no-images also turns off the photo-anchored rows, so a photo lookbook with no product table can come back empty. Use it for price lists, not for photo catalogues.

Invoices and resumes detected by the classifier go to the records view instead. See Invoices, receipts and resumes.

How it works

build_catalog picks one of two paths.

Photo-anchored rows. For a PDF or PPTX with images_dir set, every product photo anchors a row. terbium:

  1. extracts the photo into images_dir, skipping logos, icons and banners (see Pulling out images);
  2. names the row from the caption label next to the photo, else the page’s largest short heading, else a plausible name line from the caption zone;
  3. mines the nearby text for a SKU: a SKU: label first, then a real product code that is not a page number, an ordinal, a dimension or a rate price such as 2.25/SQFT;
  4. mines the same text for materials or ingredients: a line starting with Material, Materials, Ingredients, Composition, Made of, Made from, Crafted from or Fabric, else a composition line such as 78% wool, else known material words (for example “solid mango wood” gives mango, wood);
  5. finds dimensions: the first L x W or L x W x H measurement, such as 140X35X75 CM.

Captions are scored by proximity: below the photo first, then to its right, then above it, within 200 points. Each text line is split at wide gaps first, so in a two-column grid one product’s caption does not bleed into its neighbour’s.

If the caption zone holds no code, the SKU search falls back to the whole page. A SKU: label anywhere in the zone also beats an unlabelled code. So a product that prints no code can receive a neighbour’s SKU. Check for repeated SKUs before import; Worked examples shows how.

For PPTX decks, each picture placed directly on a slide becomes a row, named from the slide title or slide text. Pictures inside a group shape or a picture placeholder are skipped, and so are pictures whose shorter side is under 180 pixels or whose aspect ratio is over 4. On a slide with no text at all, optical character recognition (OCR) reads the picture; that text is trusted for a code or dimensions but not for a free-text name.

Table rows. For a price-list style catalogue (Excel, CSV, a PDF with no photos, or when images_dir is not set), rows come from the product table through the product schema: name, sku, materials and dimensions from the mapped columns, with image left blank.

images_dir matters: without it, build_catalog does not extract photos and falls back to table rows even for an image-heavy PDF.

Dense and sparse photo pages

Label extraction runs per page. It does not require several images on a page, so sparse pages (one photo) and dense grids (eight to twelve or more products per page) are both handled, and a document can mix the two.

For thumbnail sheets, where every image is swatch-sized, the first photo pass may find nothing. In that case terbium retries once with swatch-sized images accepted (shorter side down to 64 pixels).

OCR for image-only pages

build_catalog(..., ocr="auto") runs a local Tesseract pass on PDF pages that have images and fewer than 3 words, to read a code, name or dimensions baked into the pixels. It needs no key and does nothing when Tesseract is not installed. ocr=False turns it off. ocr=True does not make Tesseract available; without it, the message below wrongly says OCR already ran, so leave the default.

When the layout hides things: the AI pass

Clean catalogues, such as a labelled photo grid, come out complete with no key. Visual brochures bury the name in a range title and the material in a paragraph. Pass an AI to read each product photo plus the nearby text and fill what is missing:

rows = terbium.build_catalog(
    "brochure.pdf", images_dir="images/", ai=terbium.AI()   # reads provider keys from env
)

Pass a terbium.AI object. ai=True is accepted, but the photo-anchored path does not resolve it, so no fill runs.

The AI pass only runs on rows that are missing a name, SKU or materials. It only fills blanks and never overwrites a value the deterministic pass found. The model is told not to invent a SKU or a material that the image or text does not support. It sees the product photo and up to 1,200 characters of nearby text. It uses the Sonnet tier unless you pin one with terbium.AI(force_tier=...). A failed call leaves that row as it was, with no error.

It tells you when there is more to read

When no AI is in play, build_catalog prints a message to stderr in two situations only:

  • some pages have a photo but no readable text, so the data lives only in the pixels;
  • a field such as SKU or materials is present on some rows but missing on most (more than 15% and fewer than 60% of rows have it).

A field that the catalogue never prints at all, such as SKUs in a photo lookbook, is not treated as a failure. This is the output for a two-page PDF that holds only photos, run without Tesseract:

terbium: 2 products - 0 named, 0 with a SKU, 0 with materials.
2 page(s) have no readable text (1, 2) - the data is only in the photos.
-> Tesseract is not installed; install it to read baked-in text locally (no key; used by default, or force with ocr=True), or pass ai=terbium.AI(...) for the vision lane.

Up to eight page numbers are listed, followed by , ... when there are more. If a local OCR pass already ran and text is still missing, the last line points to the vision lane instead. Silence the message with build_catalog(..., announce=False).

The message is built by terbium.catalog.catalog_escalation, but calling it yourself on the rows build_catalog returns gives a wrong answer: the rows no longer carry the nearby text it checks, so every photo page looks unreadable. Rely on the printed message.

Extra options

For PDF files, build_catalog passes any extra keyword arguments to export_images, so you can tune photo filtering, for example build_catalog(path, images_dir="img/", only_photos=False). The Evaluation script uses exactly that. The PPTX path ignores extra keyword arguments.

Measured results

The repository’s eval/category_bench.py recovers every name, SKU, material and image on its synthetic six-category lookbooks, and the old docs report results on six real vendor catalogues. Both are on the Evaluation page.