Cardinality Break Points: Context Length or Head Ceiling in Single-Pass Decision Encoders

The question

At what option cardinality does each trunk’s accuracy break under the single-pass head, and does the break point track native context length or a head-design ceiling shared by all four? The four trunks are the shortlist fixed by the stage-zero harness: ModernBERT-large, mmBERT-base, Ettin-encoder-400M, and DeBERTa-v3-large. The parent harness scores those trunks on six axes at pinned cardinalities. This paper asks the curve question the harness implies but does not run: as the option count grows on fixed items with fixed label strings, where does each trunk’s accuracy bend downward, and do the four bends land in different places or the same place?

Two hypotheses compete, and they point to different fixes. Under the context-length hypothesis, each trunk breaks when its native sequence budget can no longer hold the state plus all option strings, so the 512-class trunk breaks first and the 8K-class trunks break later. Under the head-ceiling hypothesis, all four break at roughly the same cardinality because the binding constraint is the configured split between option budget and state budget inside the head, not the backbone maximum. The option cardinality survey documents the head-budget mechanism and the hard vendor cap as separate facts. This paper designs the per-trunk sweep that separates the two soft-break explanations from each other.

This paper runs no new experiment and claims no new measurement. Velofy has not trained or shipped a decision model, and nothing below is a result from a Velofy checkpoint. It collects the verified priors that constrain where each trunk should break, defines the break point operationally so the sweep is pre-registered rather than argued after the fact, and states which outcome pattern confirms each hypothesis. Every number below is attributed to the source that reports it.

Background

A single-pass decision head places the state and every option in one sequence, scores each option marker in one forward pass, and normalizes with a softmax. The Laya teardown reads Laya code as building instructions plus masked option markers plus state, refining with a small 2-layer TransformerEncoder, then returning one logit per option. The GLiClass paper abstract describes the same one-pass idea as an alternative to cross-encoders, which it says must process text-label pairs sequentially, and the GLiClass README reports classification about 10 times faster in a single forward pass. The shared property is that option text and state text compete for one sequence budget in one encoder call.

That budget has two layers. The outer layer is the backbone native length. The ModernBERT paper abstract reports training on 2 trillion tokens with a native 8192 sequence length, and the ModernBERT-large config sets max position embeddings to 8192 with local attention over 128 tokens and global attention every 3 layers. The DeBERTa-v3-large config sets max position embeddings to 512 with 24 layers, 1024 hidden size, and 16 heads, and the DeBERTa README uses max seq len 512 in its General Language Understanding Evaluation (GLUE) recipe. The inner layer is the configured truncation. Laya’s README reports its English checkpoint defaults to 512 context with head max len 192 and about 320 tokens for state, while multilingual and typed-decisions checkpoints default to 1024 context with head max len 256 and about 768 for state. A long native length helps only if the head configuration lets option text use it. Whichever layer binds, the arithmetic is plain division: each option gets the option budget, less fixed overhead, divided by the number of options. Doubling the options halves the tokens each label can use, so some cardinality must exist where labels stop fitting. The open question is only which budget sets it.

The parent papers fix the context this paper builds on. The stage-zero harness pins one dataset revision, one label table, one training budget, and byte-identical questions per backbone, then ranks trunks lexicographically with latency and calibration vetoes. It scores flip rate at 5, 20, and 77 options and Banking77 accuracy on the full 77-label task. The option cardinality survey documents four soft-degradation mechanisms (token starvation, position effects, calibration drift, training mismatch) plus a separate hard cap, and the encoder backbones survey recommends ModernBERT-large for English with Ettin-encoder-400M as the ablation, mmBERT-base for multilingual, and DeBERTa-v3-large for zero-shot natural language inference (NLI) work. None reports a per-trunk accuracy curve against cardinality. That is the gap this paper specifies how to fill.

The break point needs an operational definition before any sweep runs. This paper defines the accuracy break point as the smallest tested cardinality at which accuracy falls by more than a pre-registered margin from the small-set baseline on the same items, with a paired bootstrap interval excluding zero. The baseline is the smallest tested cardinality, the margin is fixed before the run (5 accuracy points is the value proposed here, not a measured quantity), and only the distractor count varies. Two companion breaks use the same rule: the calibration break, where top-label expected calibration error (ECE) rises past its margin, and the stability break, where the flip rate rises past its margin. A trunk can break on calibration or stability while accuracy holds, so the sweep reports all three curves.

What the evidence says

No fetched source runs the per-trunk sweep, so there are no measured break points. What exists are priors that constrain where each trunk should break and what the sweep must control.

Native length priors have moved since the parent harness, which listed two trunks as UNVERIFIED. Config and prose fetches run for this paper on 2026-09-23 now pin them down. The mmBERT-base config sets max position embeddings to 8192 with 22 layers, 768 hidden size, and 12 heads, the mmBERT paper HTML reports context extension to 8192 sequence length, and Laya’s README notes the mmBERT-base encoder supports up to 8192 with rotary position embeddings (RoPE). The Ettin 400M encoder card reports context length up to 8K tokens with mid-training extension to 8K over 250B tokens (after 1.7T tokens of pretraining and a 100B decay phase), though its config sets max position embeddings to the odd value 7999. The practical reading is three trunks in the 8K class and one in the 512 class. Whether 7999 is a real trained length or a config typo is UNVERIFIED, but no fetched source places Ettin in the 512 class.

Discriminative priors show competence everywhere and calibration nowhere. The DeBERTaV3 abstract reports 91.37 percent on GLUE for large (1.37 over DeBERTa, 1.91 over ELECTRA) plus 79.8 percent zero-shot cross-lingual NLI (XNLI) for mDeBERTa base. The Ettin card reports GLUE of 88.9 against 88.4 at base and 90.8 against 90.4 at large, both over ModernBERT, plus retrieval of 45.7 against 43.9 at base and 48.4 against 47.0 at large. The Hugging Face blog reports ModernBERT base at 149M and large at 395M parameters. None of the four backbone sources reports Brier, ECE, flip rate, or high-cardinality behavior. Pretraining scale has converged (the ModernBERT abstract reports 2T tokens, mmBERT 3T over 1800 languages, Ettin up to 2T), but scale predicts discrimination, not the cardinality at which a head breaks.

Head-budget priors are the strongest evidence for a shared ceiling. Laya’s README states the arithmetic: options share head max len of 192 tokens on English and 256 on multilingual, so 77 options get about 3 to 4 tokens per label, computed as 256 minus 16 divided by 77. Applying the README’s own arithmetic downward gives about 12 tokens per label at 20 options and 48 at 5 on the 256 budget, and about 8 at 20 on the 192 budget. Laya’s BENCHMARKS.md reports Banking77 of 0.425 for both base checkpoints and 0.492 for typed-decisions, against a Jev figure of 0.870 cited from third-party work, and notes that both bases scoring exactly 0.425 looks like a budget ceiling rather than a capability gap. It advises keeping choice under about 20 options. Two different trunks (421M and 322M parameters per Laya’s README) breaking to the identical score points to the head, not the trunk.

Third-party Jev numbers give comparison points but no clean curve. The decision-model-benchmark README reports Jev at 76.3 percent on banking 77-way (behind gpt-oss-120b at 81.3 percent), median (p50) latency of 264 to 276 ms flat from 2 to 255 options, 100 percent at 254 and 255 options with 400 refusals at 256, 384, and 512 while every large language model (LLM) handles 512, ECE of 0.246 against an LLM range of 0.039 to 0.122, and 13 percent flips against 37 percent for the worst LLM. The jev-benchmarks README reports 300 held-out examples with 100 per condition: AG News at 4 labels with Jev 0.910 against GLiNER 0.700, Banking77/BTZSC at 72 labels with 0.870 against 0.610, DAIR Emotion at 6 labels unresolved at 0.480 against 0.440 with Brier 0.846 against 0.668 and zero probability on truth for 16 percent of examples, and latency of about 44 ms for local GLiNER against 236 to 256 ms for hosted Jev on small tasks (246 ms against 296 ms at 72 labels). TypeSafe’s post states Jev supports cardinality up to 255 with a two-stage system for higher cardinalities: independent scoring, then an explicit choice. None of these varies only the option count on fixed items.

Position literature predicts breaks unrelated to token fit. The Lost in the Middle abstract reports uneven long-context use on multi-document question answering and key-value retrieval, best at the beginning or end and worse in the middle. The multiple-choice selectors abstract reports selection bias across 20 LLMs on 3 benchmarks from token bias toward option identifiers, with a permutation-based debias. Laya’s BENCHMARKS.md reports flips of 0.150 and 0.230 on 20-option MASSIVE intent and recommends more aggressive shuffling. A trunk whose accuracy holds to 77 options but whose flips climb past 20 has still broken for automation.

Decoupled encoding shows what a non-breaking curve looks like. The Sentence-BERT abstract reports cutting 10000-sentence search from about 50 million computations and 65 hours to about 5 seconds, and the Dense Passage Retrieval abstract reports 9 to 19 percent absolute top-20 gains over BM25. Both keep per-option cost out of state context. Laya’s README documents the same escape as predict shortlist, citing a reporter figure of 54.3 to 60.8 percent with a top-20 shortlist that it marks as not remeasured. On calibration, Guo et al. call single-parameter temperature scaling surprisingly effective, and Laya’s BENCHMARKS.md confirms as-shipped ECE of 0.466 and 0.314 falling to 0.081 and 0.106 after per-bucket refits.

Analysis

The hypotheses make different falsifiable predictions. Under the context hypothesis, breaks order by native length: DeBERTa-v3-large breaks first while the three 8K trunks break later and near each other. Relaxing truncation from a pinned 512 to 2048 or higher moves the 8K breaks rightward while leaving DeBERTa pinned or moving it less, and per-example logs show DeBERTa dropping tokens where the others still fit. Under the head-ceiling hypothesis, all four break at roughly the same cardinality under identical head budgets (the Laya guidance of about 20 is the standing candidate), relaxing truncation moves all four together or none, and the identical 0.425 on two trunks is the first exhibit. A mixed outcome also informs: accuracy tracking context while flips stay shared would show ranking capacity and position stability obey different mechanisms.

The discriminating run is a two-condition sweep. Condition A pins every trunk to 512 total tokens with shared head budgets, reproducing the tightest native budget for all four. Condition B relaxes truncation to 2048 (optionally 8192 for 8K trunks, since Laya’s README lists 2048, 4096, and 8192 as supported raises), holding head implementation and training budget fixed. Cardinality sweeps 5, 10, 20, 40, and 77 options on the same items, where 77 is the intent count Casanueva et al. report for Banking77 (13083 examples over 77 intents) and 20 is the Laya guidance value. Each point reports accuracy, ECE with the parent 15-bin convention, and flip rate over the parent 5-seed permutation set, with paired bootstrap intervals over shared items following the jev-benchmarks paired target-stratified practice. Truncation is logged per example (option tokens and state tokens kept or dropped), because without that log a context story and a head story cannot be distinguished after the fact.

Two controls separate confounded mechanisms. First, a label-length control varies wording at fixed cardinality: short Banking77 labels against paraphrased longer labels on the same items. If the break tracks total option tokens, short labels break later than long labels at the same count, implicating starvation. If both break at the same count, the break implicates softmax or attention scaling over many markers rather than truncation. Second, a shuffling diagnostic repeats the sweep with option-order shuffling disabled, as the parent harness specifies. If that multiplies every trunk’s flips by a similar factor, stability is an augmentation axis. DeBERTaV3’s disentangled positions are sometimes hypothesized to reduce position bias, but no fetched source measures that on decisions, so it stays UNVERIFIED until this diagnostic runs.

Four outcome patterns cover the plausible results. If DeBERTa breaks earliest and relaxation helps only the 8K trunks, the context hypothesis wins: the fix is trunk selection plus longer truncation. If all four break together in both truncation conditions, the head ceiling wins: the follow-up is decoupled option encoding or shortlist-then-rerank as the Maestro Flash proposal sketches, not a different trunk. If all four break together at 512 but separate at 2048, both mechanisms are real with the head binding first, so head redesign precedes trunk comparison. If no trunk breaks by 77 under relaxed truncation, the single-pass head suffices at Banking77 scale on every trunk and the question moves to larger taxonomies with hierarchical labels, which the GLiClass README supports via dot notation but which no fetched source measures for decision encoders.

Three confounds need advance naming. Vocabularies differ by a factor of five (50368 for ModernBERT-large and Ettin against 128100 for DeBERTa-v3-large and 256000 for mmBERT-base, all per fetched configs), so per-option token floors must be computed in each trunk’s own tokens. Objectives differ (replaced token detection for DeBERTaV3 per its abstract against the masked language modeling (MLM) family for the ModernBERT line), so break comparisons must use each trunk’s own small-set baseline rather than absolute scores. Calibration grouping dominates trunk effects: Laya’s BENCHMARKS.md shows refits moving ECE from 0.466 and 0.314 to 0.081 and 0.106, an effect an order of magnitude larger than the 0.025 post-fit trunk gap, so every cardinality point needs its own held-out temperature fit.

Implications for small decision models like Jev, Laya, and Velofy’s proposed Maestro Flash

For Jev, the sweep changes nothing internally and everything comparatively. Jev’s architecture stays undisclosed, as the Jev survey documents, so no trunk result transfers to it. But the sweep produces the curves Jev’s third-party measurements lack: accuracy, ECE, and flips against option count on fixed items. The decision-model-benchmark README already plots accuracy and latency against option count, reporting flat p50 from 2 to 255 with sharp refusal past 255, while TypeSafe states support up to 255 with a two-stage path above. An open-trunk sweep to 77 gives the missing comparison at shared cardinalities on shared label strings. Comparisons must stay at or below 77 options, where every system answers, and must never mix the 72-label BTZSC condition with the 77-label Banking77 condition.

For Laya, the sweep tests whether its weakest numbers are trunk or head problems. BENCHMARKS.md reports 0.425 on both trunks with the explicit budget-ceiling interpretation plus 20-option flips of 0.150 and 0.230, and the README gives the 3 to 4 tokens per label arithmetic with two escapes (larger head max len and max len, or top-k shortlist). If both Laya trunks break at the same cardinality with logs showing label collapse, the under-20 guidance is confirmed as head physics and the right work is head-level: larger defaults with measured state cost, shortlist recall reported separately, and stronger shuffling. If the ModernBERT trunk breaks later under identical heads, Laya gains a trunk upgrade with no architecture change.

For Velofy’s proposed Maestro Flash, which has no weights and no results, the sweep gates any head redesign. The proposal nominates separately encoded, cacheable option representations plus per question type, option count, and language temperatures. That redesign is expensive and should only be built if the sweep shows the single-pass head binding: a shared break, or 8K breaks that relaxation cannot move. If DeBERTa breaks early while 8K trunks hold to 77 under relaxed truncation, the cheaper fix (longer truncation on the selected trunk) wins and decoupled encoding becomes a later optimization. Run the two-condition sweep first, then commit to head architecture. No training has been done for Maestro Flash, and this paper predicts no winner.

Limitations

This is a measurement design, not a measurement. No trunk was fine-tuned, timed, or scored for this paper, no Velofy checkpoint exists, and every break expectation is a prior. The proposed pre-registrations (a 5-point break margin, cardinalities at 5, 10, 20, 40, and 77, the parent 5-seed and 15-bin conventions) are defensible but not derived, and different values could place breaks differently. The specification is published so disagreement becomes a rerun.

The evidence base has the vendor-source weakness the parents note. Ettin’s deltas come from its own card, DeBERTaV3’s 91.37 percent from its own abstract, mmBERT’s breadth from its own abstract, Laya’s floors from its own BENCHMARKS.md, and Jev’s figures from two young third-party benchmarks with different protocols. Hugging Face application programming interface (API) download counts queried 2026-09-23 (625057 for ModernBERT-large, 896959 for DeBERTa-v3-large, 400800 for mmBERT-base, 9660 for Ettin-encoder-400M) measure adoption, not quality. Banking77 denominators disagree across sources (72 hypotheses in pinned BTZSC per jev-benchmarks against 77 intents per Casanueva et al.), so cross-source accuracy comparisons stay approximate until one label table is pinned.

Three quantities stay UNVERIFIED or artifact-prone. The Ettin 7999 config value against its up-to-8K card prose is unresolved, though both sit outside the 512 class. Effective usable length under fine-tuning may sit below native length everywhere, since RoPE extrapolation and DeBERTa relative-position generalization are unmeasured in fetched decision sources. The per-option token floor for distinguishable labels is reported only for Laya’s budgets (about 3 to 4 tokens per label at 77 options), not as a general law across vocabularies spanning 50368 to 256000. The sweep resolves all three by logging truncation per example in each trunk’s own tokens. The design is English-only and single-domain by intent, matching the parent sequencing that selects the trunk before adding languages.

Open questions

1. What are the measured per-trunk accuracy, ECE, and flip-rate break points on a fixed Banking77 cardinality sweep under pinned 512-token truncation with paired bootstrap intervals?
2. Does relaxing truncation to 2048 tokens and above move the 8K-class trunk breaks rightward while leaving DeBERTa-v3-large pinned, or do all four trunks move together under a shared head ceiling?
3. Does a label-length control with short against paraphrased long wordings separate token-starvation breaks from marker-count breaks, and at what per-option token floor does accuracy recover per trunk?

References

1. ModernBERT paper, arXiv:2412.13663
2. mmBERT paper, arXiv:2509.06888
3. Ettin paired encoders and decoders paper, arXiv:2507.11412
4. DeBERTaV3 paper, arXiv:2111.09543
5. Efficient Intent Detection with Dual Sentence Encoders (Banking77 dataset), arXiv:2003.04807
6. GLiClass paper, arXiv:2508.07662
7. On Calibration of Modern Neural Networks, arXiv:1706.04599
8. Lost in the Middle, arXiv:2307.03172
9. Large Language Models Are Not Robust Multiple Choice Selectors, arXiv:2309.03882
10. Sentence-BERT, arXiv:1908.10084
11. Dense Passage Retrieval, arXiv:2004.04906
12. Laya GitHub repository
13. Laya benchmarks
14. ModernBERT-large config
15. DeBERTa-v3-large config
16. mmBERT-base config
17. Ettin 400M encoder model card
18. GLiClass README
19. jev-benchmarks
20. decision-model-benchmark
21. TypeSafe, Introducing System One Models and Jev

← All papers