fix(drift): membership test, robust to backfills and API throttling#17
Merged
Conversation
check-drift and doctor compared the latest GitHub release tag against
`public_latest_version` (the concept's newest version by `created`). That
proxy breaks after a backfill: backfilled records — historical releases and
curated major rollups alike — are created *today*, so an old rollup becomes
the newest-by-`created` version and the tag never equals its label, producing
false drift (e.g. virtualcell/vcell: GitHub 8.0.0.06 vs rollup label 8.0,
though 8.0.0.06 was in fact archived).
Replace with a membership test: is a version labeled exactly the latest
release tag published in the concept? Implemented as a single tokenless
`metadata.version:"<tag>"` search (public_tag_archived), NOT an
enumerate-all-versions scan — the unauthenticated records API is capped at
30 req/min, so paging a large concept (vcell has ~490 versions) 429s mid-scan
and would silently undercount into false drift. The phrase match is exact, so
one request answers the question precisely and deterministically.
- api: add hits_total, public_tag_archived, latest_release_archived
- api: drop now-unused (and bug-embodying) public_latest_version
- cli: wire both check-drift and doctor section C to the shared helper
- tests: hits_total total-shape parsing (int vs {"value": n})
- README: describe the membership-based drift check
Verified live: vcell now IN SYNC deterministically (was flaky 407-467 tags
under throttling); a mismatched repo/concept still DRIFTs with exit 2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BZx7w9BexQc9wWU5JoHYaF
…imits - Add the two previously-undocumented commands: list-owned (#13) and bootstrap. - Reword check-drift to describe the membership semantics from this branch. - Add an API gotcha: the public records API is 30 req/min and caps size at 25, so query one tag rather than enumerating all versions tokenlessly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BZx7w9BexQc9wWU5JoHYaF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
check-driftanddoctorflagged false drift onvirtualcell/vcell(issue #16): GitHub's latest release8.0.0.06was archived on Zenodo, yet the monitor reported drift.Root cause: the check compared the latest GitHub release tag against
public_latest_version— the concept's newest version bycreated. That proxy breaks after a backfill. vcell was backfilled on 2026-07-02 with historical per-release records and curated major rollups (7.2 … 8.0); all gotcreated=today, so the8.0rollup (label8.0, built from8.0.0.01) became the newest-by-createdversion.8.0.0.06 != 8.0→ false drift, even though8.0.0.06has its own archived record.Fix
Ask a membership question instead: is a version labeled exactly the latest release tag published in the concept?
Implemented as a single tokenless search —
q=conceptrecid:<id> AND metadata.version:"<tag>"— deliberately not an enumerate-all-versions scan. The unauthenticated records API is rate-limited to 30 req/min; paging vcell's ~490 versions (25/page → 20 pages) 429s mid-scan and would silently undercount into false drift. The phrase match is exact (verified:"8.0"does not match8.0.0.06;"8.0.0"and"8"return 0), so one request answers precisely and deterministically.Shared helper
latest_release_archivedwired into bothcheck-driftanddoctorsection C, so they can't diverge. Removedpublic_latest_version(unused, and it embodied the exact buggy notion).Trade-off
Matches on the version label, which the per-release archiver sets to the source tag. A release captured only inside a relabeled curated rollup (label ≠ tag, no per-release record) wouldn't be found — acceptable: the archiver's default labels a record by its tag, and erring toward flagging drift is the intended loud-on-failure behavior.
Verification
ruff+mypy --strictclean; 26 unit tests pass (addedhits_totaltotal-shape parsing).IN SYNCdeterministically (was flaky 407–467 "tags" under throttling with the enumeration approach I first tried);vcell-solvers/pyvcellin sync; a mismatched repo/concept pair stillDRIFTs with exit 2;doctorsection C shows ✓.drift.reusable.ymlandmonitor.ymldrive thecheck-driftCLI (interface unchanged) — no workflow edits needed.Closes the class of false positive behind #16.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BZx7w9BexQc9wWU5JoHYaF