Repository navigation
Conversation
The Nemotron loader and both forced-aligner loaders refused any installed Transformers version other than the one evaluated, and both asset manifests named that version, so a requirements-only bump disabled Nemotron (live labels fell back to clustering, final speakers to community-1) and Canary's word timings until a hand-made follow-up changed the constants, the manifests, the fixture records and the evaluation record together. - Drop the equality checks from `nemotron_model._build_runtime` and from `forced_aligner_native`. Class, configuration, geometry, weight-key, tensor-shape and loading-diagnostic checks stay. - Drop `transformers_version` from both asset manifests and their validators. Verification identity is now the repository, checkpoint revision and file sizes and digests. Removing the field changes the manifest digest once: existing installs re-hash their unchanged files through the existing repair paths, with no download. - Nemotron's runtime fingerprint records the installed Transformers version instead of the constant, so a saved live cache is still not resumed across versions; `bind_capture` already falls back to clustering on a mismatch. - Remove `scripts/nemotron_compatibility.py`, its test and its call from `check_held_pins.py`; it failed CI unless the pin matched the golden and evaluation records. - The opt-in CUDA golden compares outputs against the same checkpoint revision instead of requiring the baseline's runtime fingerprint. Decision record Context: Transformers 5.20.2 -> 5.21.0 gave identical outputs for both models, yet the checks turn each Dependabot bump into a silent fallback or a manual PR (Valtora#868). Agreed on Valtora#892: drop the checks, keep `transformers==` in worker.txt so each bump arrives as one Dependabot PR. Alternatives: keep the checks and re-prepare automatically on a version change (keeps the refusal, still couples library metadata to immutable files); a version range in worker.txt (lets builds install a release no PR tested); vendoring the private feature-extraction call (more code to own for a break not yet seen). Trade-offs: a release that changes behaviour without breaking a structural check now loads instead of falling back. The inference regression and the structural checks are what catch it, and the pin still moves only through a reviewed PR.
…ers-checks Brings in Valtora#894 and Valtora#893. The one conflict is the final speaker pass in docs/ARCHITECTURE.md: Valtora#894's paragraph on capped recordings and its new skip list are kept, with this branch's wording that a repair follows a checkpoint change rather than a Transformers upgrade, since the manifest no longer names a Transformers version. Valtora#894's test changes merged cleanly.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft: still in our own review. Please don't merge; we'll mark it ready.
This follows what we agreed on #892. If you're happy with it, including the inference test, which now compares outputs instead of versions but stays in, we'll mark it ready for review.
Pull Request
Description
The Nemotron loader and both forced-aligner loaders refused every Transformers release except the one we evaluated, and both asset manifests named that version. A Dependabot bump therefore switched Nemotron off (live labels fell back to clustering, final speakers to community-1) and dropped Canary's word timings, unless a hand-made PR like #868 moved the constants, manifests and evidence records together.
This removes what #725 and #735 added around the version:
nemotron_model.pyandforced_aligner_native.py;transformers_versionin both asset manifests and their validators, so a bump no longer invalidates prepared files;scripts/nemotron_compatibility.py, its test and its call incheck_held_pins.py.File hashes, the checkpoint revision, and the class, geometry, weight-key and shape checks stay. Nemotron's runtime fingerprint now records the installed Transformers version, so a saved live cache still isn't resumed across versions (the existing fallback to clustering). The opt-in CUDA golden checks the checkpoint revision and compares outputs.
transformers==stays inworker.txt.Removing the field changes the manifest digest once, so existing installs re-hash their unchanged files once through the existing startup preparation, with no download; later bumps don't. The decision record is in the commit body.
Type of change
Checks run
python scripts/check.pyon Postgres, again withCUDA_VISIBLE_DEVICES="", and again without ffmpeg/ffprobe onPATH;scripts/check_file_size.py;scripts/check_held_pins.py --offline.Migration impact
Documentation impact
docs/ARCHITECTURE.md,docs/DEPLOYMENT.md,docs/DEVELOPMENT.mdand the fixture README.Security impact
Manual verification
On an RTX 4070 Ti, a prepared Nemotron copy was verified offline (hashed, nothing downloaded) and loaded through
load_nemotron_modelon CUDA, under 5.21.0 and then 5.20.2. Under 5.20.2 the receipt still verified, the asset fingerprint matched and the runtime fingerprint differed. The CUDA golden passed under both.