Add captured two-run qualification evidence - #71
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
evidence-initto pin candidate name, revision, and an origin-stripped source snapshot before adapter work, then prepare two independent candidate/adapter/build/cache rootsrevisionandsnapshot_idmetadata plus all-or-none CLI identity enforcement before the first eventevidence-verifyto reject changed trees, incomplete or noncanonical bundles, mismatched metadata, invalid deterministic IDs, count or coverage drift, and byte differencesevidence-manifest.jsonfor a valid pairWhy
The held-out agent-qualification study showed that compatible-engine release evidence still failed when task identity, fresh roots, canonical bundle capture, and equality gates lived only in prose. This moves those gates into the public tool so an agent or maintainer receives a deterministic pass/fail contract and a reviewable manifest.
This remains separate from #70 and does not modify the frozen experiment cohort, intervention, or outcomes.
User impact
Ordinary adapters remain wire-compatible: the new engine metadata fields are optional unless all three
--candidate-*pins are supplied. Release and benchmark claims can require two clean, byte-identical qualifications bound to one captured candidate snapshot.The workflow verifies captured source and artifact integrity. It does not sandbox compilers or prove that an adapter avoided undeclared external caches; disposable CI or a clean host remains the strongest execution boundary.
Controlled external-engine dogfood
The workflow was exercised after review against the real external
gocronx/matcherengine at pinned revisionb8d48356c8a2677e0d8a1965d754e3c4884bb947.sha256:1055904eb895d9768f9084907701e3569917126fcbe4d1fe96be3544e507b645sha256:4f5d936af8cd1f1a85bd306129db95f9820cc19d16bef5248ef4bda52935876dsha256:1b7b8b17c73488816216e3132670385470049592a774ea0229f34859a0b502e3This is operator-run implementation evidence, not independent adoption or an untouched-defect result.
Validation