Skip to content

Keep per-format routing and stream traits in the chat-format registry - #1460

Merged
leehack merged 3 commits into
mainfrom
refactor/362-chat-format-traits
Oct 27, 2026
Merged

leehack merged 3 commits into
mainfrom
refactor/362-chat-format-traits

Conversation

@leehack

@leehack leehack commented Oct 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Per-format routing and streaming behavior now lives on each ChatFormatRegistration as a ChatFormatTraits value (lib/src/core/template/chat_format_registry.dart). traits is a required argument, so a format cannot be registered without them.
  • Moved out of chat_completion_stream_parser.dart: the 25-arm _toolCallOpeningFor switch, _keepsThoughtTagsAsContent, _parsesCallsInsideThoughts and _mayEmbedToolEnvelopeAfterContent.
  • Moved out of chat_template_engine.dart: _schemaDisabledFormats, _requiredKeepsLazyFormats, _toolCallGenericFormats and the Gemma branch after it (now one routesThroughGeneric trait), _toolChoiceNoneContentOnlyFormats, and the canonical JSON Schema supported set together with the incomplete-tool-call switch in _parse. Those two always listed the same six formats; they are now one leavesIncompleteToolCall check, and qualifiesJsonSchemaTools is true exactly when that check is set.
  • Behavior is unchanged. The golden test/fixtures/chat_format_traits.json was committed first (d0592cc86). At that commit its test reads main's private tables through dart:mirrors and passes against unchanged lib/. The refactor commit switches the reader to traits, and the golden is byte-identical at head. The golden also pins public-API probes that don't depend on how the tables are stored: parse outcomes for 10 incomplete-tool-call outputs × 45 formats (with tool parsing on and off), and render routing for every detection trigger under 8 tool/schema/tool_choice combinations.
  • New guards: a source scan fails if a ChatFormat set or collection literal, a format switch arm, or a statement naming three or more formats comes back in the stream parser or engine. Two near-miss detector tests close the mutants that survived the #1457 audit (the leading space in ' functools[' and Kimi K3 without <|end_of_msg|>).

Refs #362 (slice 2 of the 0.17.0 Backend SPI cleanup, #922).

Production-readiness scope

Completeness checklist

PR type guidance

  • Feature PR: N/A, refactor.
  • Bugfix PR: N/A, refactor.
  • Docs-only PR: N/A.

Test Plan

  • dart run tool/prepare_workspace.dart (clean tree afterwards)
  • dart format --output=none --set-exit-if-changed .
  • dart analyze
  • dart run tool/testing/check_platform_boundaries.dart
  • dart test -p vm -j 1 --exclude-tags local-only (macOS arm64, isolated TMPDIR): 8,547 passed, 5 skipped (an earlier run alongside the parity llama.cpp build failed one native_symbol_integration_test.dart case, which passed alone 26/26 and in this clean rerun)
  • dart test -p chrome --exclude-tags local-only test/unit/core/template test/unit/core/engine/chat_completion_stream_parser_test.dart: 1,049 passed
  • ./tool/testing/run_template_parity_suites.sh (llama.cpp b10549 corpus): detection 74, diagnostics 88, unit 1,254, upstream parser suites 10, compiled grammar 52, all pass (the grammar step hit a Dart SDK native-assets otool crash while a concurrent full VM run shared .dart_tool, and passed when rerun alone)

Mutation checks: each mutant was applied alone at head and then reverted. 21/22 were killed. Script and log: llamadart-evidence/s2-chat-format-traits/{mutate.py,mutants.log}.

Mutant Result
Drop or add a trait on one format (mistralNemo tool_choice=none, granite schema-disabled, apertus required-lazy, gemma generic routing, gemma4 envelope, gemma thought tags, kimiK2 calls in thoughts, deepseekR1 schema-disabled) killed (golden)
cohere2Moe wired to Hermes opening; laguna loses its check; Hunyuan drops plural marker; generic regex singular; Hermes check inverted; every format qualifies JSON Schema tools killed (golden)
Engine: tool_choice=none ignores hasTools; incomplete-call check ignores parseToolCalls; generic routing ignores tools killed (golden)
Stream: opening ignores the PEG parser killed (stream parser tests)
Firefunction ' functools[' without space; Kimi K3 without <|end_of_msg|> killed (new near-miss tests)
A Set<ChatFormat> added back to the engine killed (table scan)
Unknown stream format uses generic's traits survived, equivalent: generic's stream traits are all off, and the golden pins that

Matrix Evidence

Matrix row Scope covered Platform / model / backend Result Evidence / notes
structured-output-adversarial Render routing (auto/required/none, response schema, canonical schema tools), final-parse incomplete-call rejection, and stream gating traits for every format macOS arm64, VM and Chrome; llama.cpp b10549 templates Pass Golden from main plus run_template_parity_suites.sh; render, parse and stream code paths read the same values as before
high-risk-exact-head-independent-qa Exact head vs current base 184453043 / 0a33025e9 PASS Independent auditor CLEAR: 0 P1/P2; golden regenerated from main byte-identical; 10,080-case stream diff main vs head identical; 16/21 own mutants + 21/22 author mutants killed (survivors P3/equivalent). Evidence llamadart-evidence/pr-1460/evidence-1460-1844530.json; high_risk_readiness.dart exits 2.
release-metadata-verification N/A — N/A Not a release change
litert-lm-json-schema N/A — N/A LiteRT-LM schema provider untouched

Review Notes

  • Independent review status: CLEAR at 184453043 (independent auditor session). P3s: guard scan misses ChatFormat-keyed map literals, two-format helpers and case arms; pre-existing gap on generic thinkingForcedOpen in the incomplete-call check.
  • CI status / head SHA: 184453043: 32 pass, 5 skipped, 0 failing (run 40509270176).

High-risk regression review

  • Classification: high-risk (structuredOutput: lib/src/core/template/; backendRuntime: lib/src/core/engine/)
  • Implementation task: this PR (author)
  • Independent blocking QA task: independent auditor session, verdict CLEAR
  • Exact head / current base: 184453043 / 0a33025e9
  • Production-branch deletion, bypass, or miswire proof: author mutation results above. Golden probes go through production ChatTemplateEngine.render/parse; trait values are read from the registrations the engine and stream parser use.
  • Affected-family real-model/artifact evidence: N/A. Routing and gating depend only on the format, template text and model output text. Evidence is the repository templates, the llama.cpp b10549 corpus and the golden.
  • Explicit unavailable-family or other N/A evidence: No model weights used; no native or Web bridge path touched.
  • Known PR-caused P1 regressions: 0
  • Unresolved review threads: 0

Characterization golden captured from main's per-format tables before
they move into the format registry (#362).
The stream parser's tool-call opening switch and thought/envelope tables,
and the engine's schema, lazy-grammar, generic, tool_choice=none and
canonical-JSON-Schema tables move into a ChatFormatTraits value on each
ChatFormatRegistration. The pinned golden is unchanged. Refs #362.
Fails if a per-format table reappears in the stream parser or engine, and
pins the Firefunction v2 leading space and the Kimi K3 end-of-message
marker that surviving mutants dropped. Refs #362.
@leehack
leehack marked this pull request as ready for review October 27, 2026 13:46
@leehack
leehack merged commit 8dede44 into main Oct 27, 2026
37 of 38 checks passed
@leehack
leehack deleted the refactor/362-chat-format-traits branch October 27, 2026 13:46
@github-actions

Copy link
Copy Markdown
Contributor

Chat app preview removed for leehack/llamadart-chat-pr-1460.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant