test(gate-concurrency): drop the archived receipts and replay gate - #90
Merged
Conversation
The archive was a snapshot of one day's tree, so it went stale as soon as the tree moved, and replaying it only ever proved the archive was unchanged. Fold the pair's durable result -- the two caps, timings, counts, equal failure sets and exit codes -- into a table in docs/gate-concurrency.md, and state that ongoing CI runs at the current cap are the evidence from here on. Removes the tarball, its summary, the scripts/evidence/ replay test and the path-triggered workflow that re-verified them. Keeps scripts/gate-concurrency-receipt.ts and its unit test: the runner's standalone discovery walks scripts/ (run-tests-discovery.ts COLOCATED_TEST_DIRS), so that test runs in the main gate rather than only under the deleted workflow. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
tnunamak
added a commit
that referenced
this pull request
Sep 10, 2026
The policy module cites this document as the reasoning behind its cap, but the document described the cap it replaced. It said file concurrency was 2 by default, that 8 was an override and "not the effective default on any host", and it advised against changing the default at all. A reader following the citation to check the 8 was told the 8 does not exist. It also named four tests that failed only at high concurrency, called them contention artifacts rather than code defects, and used them as the reason to stay slow. That reading was wrong, and this branch is where it was disproved: the three SQLite writer-path tests were closing the database while deferred index maintenance they had started was still running, and the upload test let a detached validation task outlive it. All four were real defects and are repaired at the root here. The document now says so, and no longer lists the tests by title, since the titles were only ever a symptom list. Rewritten to state the current policy: caps of 8 for the memory profile and 2 for PostgreSQL, clamped by CPU count and selected file count, floored at 1, with a positive override left unclamped. The wall-clock pair behind the number is kept along with its own limits, including that both runs failed identically and so establish nothing about safety on their own. Structured to match the rewrite in #90, which trims the archive references from the same file: shared headings, and the sections that PR owns are left byte-identical, so the rebase touches only the lines changed here. Documentation only. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
tnunamak
added a commit
that referenced
this pull request
Sep 10, 2026
The policy module cites this document as the reasoning behind its cap, but the document described the cap it replaced. It said file concurrency was 2 by default, that 8 was an override and "not the effective default on any host", and it advised against changing the default at all. A reader following the citation to check the 8 was told the 8 does not exist. It also named four tests that failed only at high concurrency, called them contention artifacts rather than code defects, and used them as the reason to stay slow. That reading was wrong, and this branch is where it was disproved: the three SQLite writer-path tests were closing the database while deferred index maintenance they had started was still running, and the upload test let a detached validation task outlive it. All four were real defects and are repaired at the root here. The document now says so, and no longer lists the tests by title, since the titles were only ever a symptom list. Rewritten to state the current policy: caps of 8 for the memory profile and 2 for PostgreSQL, clamped by CPU count and selected file count, floored at 1, with a positive override left unclamped. The wall-clock pair behind the number is kept along with its own limits, including that both runs failed identically and so establish nothing about safety on their own. Structured to match the rewrite in #90, which trims the archive references from the same file: shared headings, and the sections that PR owns are left byte-identical, so the rebase touches only the lines changed here. Documentation only. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
reference-implementation/docs/receipts/gate-concurrency-20260903.tar.gzheld the raw receipts and transcripts from one day of test runs: two runs of the same tree, one at file-concurrency cap 2 and one at cap 8. A test underreference-implementation/scripts/evidence/re-derived each receipt's counts, failure names and digests from those archived bytes, and a path-triggered workflow (.github/workflows/gate-concurrency-evidence.yml) ran that replay whenever any of the five files changed.That archive is a snapshot of a tree that has since moved on. Replaying it cannot tell you anything about the cap as it behaves today — it only confirms the archived bytes are unchanged, which is a property of the archive rather than of the concurrency cap. Meanwhile the archive carried real cost: 445 KB compressed, about 4 MB unpacked, two near-identical 1.9 MB transcripts, plus a workflow and a 371-line test whose only job was to re-check it. The durable result of that measurement is a handful of numbers, not four megabytes of transcript.
Change
The measurement's lasting content now lives as a table in
reference-implementation/docs/gate-concurrency.md: both caps, elapsed times, selected-file and assertion counts, the pass/fail/skip split, the fact that both runs produced the same 396 failure identities, and the exit codes. The document keeps the honest reading of that result — failure-set equality for one pair on one host, not a green suite and not proof either cap is safe — and now states that ongoing CI runs at the current cap are the evidence going forward.Removed:
reference-implementation/docs/receipts/gate-concurrency-20260903.tar.gzreference-implementation/docs/receipts/gate-concurrency-20260903.summary.jsonreference-implementation/scripts/evidence/gate-concurrency-receipts.test.ts(the directory is now empty and gone).github/workflows/gate-concurrency-evidence.ymlKept:
reference-implementation/scripts/gate-concurrency-receipt.tsandreference-implementation/scripts/gate-concurrency-receipt.test.ts. These are not archive-only. The runner's standalone discovery walksscripts/as a selection root (COLOCATED_TEST_DIRSinreference-implementation/scripts/run-tests-discovery.ts), so that unit test already runs as part of the main gate — the deleted workflow was not what kept it alive. The replay test underscripts/evidence/was the opposite case: that walk is non-recursive, so nothing selected it, which is exactly why it needed a dedicated workflow.One documentation point was worth preserving on its own merits and has been rewritten to stand without the archive:
completed_filesis derived from the exit code (run-tests.tswritesfailed ? 0 : results.length), so it reads 0 for any run that exits non-zero and is not an observed per-file completion count.How to verify
Confirm the kept test still passes:
cd reference-implementation node --test --experimental-strip-types scripts/gate-concurrency-receipt.test.tsExpect 10 passing, 0 failing.
Confirm test discovery is intact — that the kept test is still selected and that nothing points at a deleted file:
Expect 1034 discovered, the kept test selected, and an empty missing list.
Confirm no reference to a removed artifact survives anywhere:
Expect no output.
This commit is scoped
test(gate-concurrency)so it cuts no package release:.releaserc.yamlreleases only onfeat/fix/perf/revert(or a breaking marker) and only for theconnector-protocol,collector-runtimeorlocal-collectorscopes, so the type and the scope each independently rule out a publish.Assisted-by: AI