Skip to content

perf(test): raise memory-profile file concurrency to eight - #50

Open
tnunamak wants to merge 7 commits into
mainfrom
port/gate-speed-287-0902
Open

perf(test): raise memory-profile file concurrency to eight#50
tnunamak wants to merge 7 commits into
mainfrom
port/gate-speed-287-0902

Conversation

@tnunamak

@tnunamak tnunamak commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Memory-default test runs use eight file workers by default. PostgreSQL test runs retain two because their backup/restore oracle shares PDPP_TEST_POSTGRES_RESTORE_URL. A positive PDPP_TEST_CONCURRENCY still overrides either profile.

This PR also fixes the test-accounting transcript terminal-event ordering defect: malformed accounting output now resolves to the final nonzero status before the transcript is closed.

Changed

  • Updated the test runner concurrency default so memory-default uses 8 workers unless PDPP_TEST_CONCURRENCY is set.
  • Preserved the PostgreSQL profile default at 2 workers because the restore oracle shares PDPP_TEST_POSTGRES_RESTORE_URL.
  • Fixed accounting transcript close order so malformed accounting output records the final nonzero status before terminal transcript closure.
  • Added receipt-pair artifacts, verifier coverage, and guidance contract tests.

Evidence

reference-implementation/docs/receipts/gate-concurrency-memory-cap-2.receipt.json and ...cap-8.receipt.json, with their checked-in transcripts, record two clean-source memory-default runs on Node 22.23.1. Both select 1,033 files and record 6,961 assertions: 6,335 passed, 396 failed, and 230 skipped. They retain the same 396 failure identities, meaning the same failing test names are observed under both caps, and exit code 1. Cap 2 took 352.198 seconds; cap 8 took 141.066 seconds.

The receipt-pair test verifies each transcript digest and binding, then compares selection, structured outcome counts, failure identities, source digest, and timing fields. The checked-in receipt generator refuses a dirty source tree and uses the memory-default profile plus the documented cap.

This PR does not make the full test suite green and does not claim PostgreSQL concurrency equivalence.

Reviewer Findings Addressed

  • P1 — missing reviewable receipt pair: Added the generated cap-2/cap-8 receipt-and-transcript pair, their verifier, and a pair-comparison test. The artifacts bind the measured clean source head eb6a890d57658c14aef706bdbe4f756b14238d21; the later artifact-only commit carries those generated files.
  • P2 — contradictory concurrency guidance: Replaced the old universal cap-2 guidance with the runner’s profile-specific policy and added a contract test that reads both the source and the checked-in guidance.

Validation

  • node --test --import tsx for the receipt unit, receipt-pair, concurrency-cap, authority exit-order, and authority concurrency tests: 14 passing.
  • Biome check over the changed scripts: passing.
  • Mutation checks: changing the PostgreSQL default from 2 to 8 makes the cap contract fail; changing a checked-in receipt cap makes the pair verifier fail.

Assisted-by: AI

…ault

Port of PDP-Connect/pdpp PR #287 (head 07737ec79) into this repository.
pdpp froze reference-implementation/ for direct edits because the
reference server now lives here, so the change lands here instead.

run-tests.ts, the reference implementation's test runner, spawns one
child process per test file and lets a fixed number of workers pull
files from a shared queue. That number defaulted to 2 regardless of
host cores, so a 24-core machine spent most of the suite's wall time
queueing. Each child is its own process with its own event loop and
listen sockets, and any database a file needs is allocated under a
collision-proof random name, so more workers add real parallelism
without changing what a test observes.

The default cap is now 8 for the memory-default profile (tests against
the in-memory store). It stays at 2 for the postgres profile (tests
against a real PostgreSQL database): that profile's backup/restore
oracle uses one shared restore database, PDPP_TEST_POSTGRES_RESTORE_URL,
that is not allocated per file, and the cap-8 measurement never covered
it. PDPP_TEST_CONCURRENCY still overrides either default when set.

Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
Signed-off-by: tnunamak <tnunamak@gmail.com>
Assisted-by: AI
Signed-off-by: tnunamak <tnunamak@gmail.com>
Assisted-by: AI
Signed-off-by: tnunamak <tnunamak@gmail.com>
Assisted-by: AI
Signed-off-by: tnunamak <tnunamak@gmail.com>
Assisted-by: AI
@tnunamak

tnunamak commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Rebase onto current main was blocked before the first signed commit: GPG opened interactive pinentry; cancellation caused signing to fail. The batch stopped under the requested GPG rule. This rebase was aborted. Head: fe27010fe27010 (unchanged). No new CI ran; existing checks show 7 success, 4 failure, 1 skipped. Needs work: unlock GPG and retry the rebase/CI verification.

Assisted-by: AI

@tnunamak

tnunamak commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

Locally rebased onto main bac5526 with no conflicts: fe27010 → 5a2a77513f377b36c9bb6a1fca4faac99649cefc. Original patches are unchanged (range-diff and stable patch-ID comparison).

PARKED: unattended GPG signing failed with “No pinentry”. Rewritten commits have Tim’s identity and DCO but remain unsigned locally under the owner’s fallback policy; nothing was pushed. The remote head and its old hosted CI results are unchanged; no fresh hosted checks ran.

Fresh cap-2/cap-8 comparison failed: both completed 1,036 files and 10,249 assertions; cap 2 had 9,880 pass / 0 fail / 369 skip, cap 8 had 9,878 pass / 2 fail / 369 skip. The two manual-upload tests pass alone, but their cap-8 failures remain unresolved. Needs work; outcome equivalence is not established.

Assisted-by: AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant