Skip to content
terbium

terbium / Guides

Normalization and feeds

How terbium normalizes colours, materials and dimensions, enriches product records with AI, and exports Shopify CSV and PIM JSON feeds.

On this page
  1. Deterministic normalization
  2. AI product enrichment
  3. Feed exporters
  4. Shopify product CSV
  5. PIM JSON
  6. Catalog rows versus records

Product records from the product schema are ready for a feed: terbium adds normalized keys without a model, can optionally enrich records with an artificial intelligence (AI) model, and writes Shopify and product information management (PIM) files.

The reason for normalizing is that suppliers describe the same thing in different words: “Off White”, “ivory” and “cream” are one colour family to a shop filter, and “40 x 40 cm” and “400 x 400 mm” are one size. Small fixed vocabularies fold colours and materials into families, and a unit conversion does the same for sizes, all without a model. The raw value is always kept, so nothing is lost.

Deterministic normalization

The product schema calls terbium.normalize.enrich_fields on every record. It is additive: raw values are kept and normalized keys are added next to them.

  • color_family: from color, folded into one of white, black, grey, brown, red, orange, yellow, green, blue, purple, pink, multi.
  • material_family: from material, folded into one of wood, metal, leather, textile, glass, stone, rattan, plastic.
  • dimensions_mm: from dimensions, a list of values converted to millimetres. Only metric values with a unit (mm, cm or m) are converted: 40 x 40 x 40 cm gives [400.0, 400.0, 400.0], while 12 x 10 in or a bare 40x40 adds no key. When a cell holds several measurements, only the first is converted.

The longest matching term wins, so “off white” maps to white through the “off white” entry and “stainless steel” maps to metal. The vocabularies live in COLOR_FAMILIES and MATERIAL_FAMILIES in src/terbium/normalize.py and are deliberately small.

from terbium.normalize import color_family, material_family, dimensions_mm

color_family("Off White")            # "white"
material_family("solid mango wood")   # "wood"
dimensions_mm("40 x 40 x 40 cm")
# {"values_mm": [400.0, 400.0, 400.0], "unit": "cm", "raw": "40 x 40 x 40 cm"}

Each helper returns None when nothing matches.

The same material vocabulary drives materials detection in the catalog table.

AI product enrichment

terbium.enrich_products sends product records to a routed model and returns a new list. The model infers the category and category-appropriate attributes, both explicit (in the text) and implicit (visible in a photo, such as colour, pattern, material or style), and normalizes price and dimensions. It is told not to invent facts.

doc = terbium.parse("pricelist.xlsx", schema="product")
enriched = terbium.enrich_products(doc.records, terbium.AI())

Signature: enrich_products(records, ai, image_for=None, only_below=1.01, tier="sonnet"). ai must be a terbium.AI object; True raises AttributeError.

  • image_for: a function that takes a record and returns PNG bytes of its photo, or None. When given, a vision request is made.
  • only_below: enrich only records with confidence below this value. The default enriches every record.
  • tier: the model tier; AI(force_tier=...) overrides it.

An enriched record gets origin="ai", the reason enriched by AI, and the model’s own confidence (or at least 0.9 if the model gives none). Returned sku, name, category, dimensions, color and material replace existing values; a returned price becomes price_amount and currency; other attributes are added only when the key is not already present. Records are returned unchanged when no provider is available or a call fails.

Feed exporters

Both exporters take Record objects, return the text, and also write it when you pass a path.

Shopify product CSV

terbium.to_shopify_csv(doc.records, "shopify.csv")

The file is comma-separated values (CSV) in Shopify’s product import layout. Columns: Handle, Title, Body (HTML), Vendor, Type, Tags, Published, Option1 Name, Option1 Value, Variant SKU, Variant Price, Variant Inventory Qty, Image Src.

  • Handle is a slug of the name, or of the stock keeping unit (SKU) code when there is no name.
  • Title is the name, or the SKU when there is no name. Variant SKU is the record’s SKU.
  • Body (HTML) joins material, dimensions and colour, for example Material: wool | Color: off white.
  • Vendor comes from brand and Type from category.
  • Tags are the colour family, material family and category.
  • Published is always TRUE.
  • The colour becomes the first option when present; otherwise the option is Title with the value Default Title.
  • Variant Price comes from price_amount and Variant Inventory Qty from quantity.
  • Image Src comes from an image or image_file field, copied as it is. Shopify expects a web address (URL) there, so put hosted image URLs in that field before you export.

PIM JSON

terbium.to_pim_json(doc.records, "products.json")

One JavaScript Object Notation (JSON) object per product: sku, confidence (rounded to three places) and every field.

Catalog rows versus records

build_catalog returns plain dicts, not Record objects, so write them with terbium.to_catalog_csv. The feed exporters take the Record objects from terbium.parse. To send catalog rows to Shopify, wrap each one in a Record; Worked examples shows how.