perf(test): raise memory-profile file concurrency to eight

reference-implementation/docs/gate-concurrency.md

@@ -2,14 +2,25 @@
 
 ## Policy
 
-`scripts/run-tests.ts` caps file concurrency at **2** by default, clamped to the
+`scripts/run-tests.ts` chooses its default file-concurrency cap from the storage
+profile — **8** for memory-default, **2** for PostgreSQL — clamped to the
 available CPU parallelism and the number of selected test files:
 
 ```ts
-const defaultConcurrency = Math.max(1, Math.min(2, availableParallelism?.() ?? 1, testFiles.length || 1));
+const DEFAULT_FILE_CONCURRENCY_CAP = selectedProfile === "postgres" ? 2 : 8;
+const defaultConcurrency = Math.max(
+  1,
+  Math.min(DEFAULT_FILE_CONCURRENCY_CAP, availableParallelism?.() ?? 1, testFiles.length || 1)
+);
 ```
 
-`PDPP_TEST_CONCURRENCY`, when it parses to a positive integer, replaces that
+The caps differ because the profiles differ. Memory-default gives every test
+file its own in-memory storage, so independent files can run together and the
+cap only has to bound host contention. PostgreSQL stays at 2 for the
+restore-database reason described in the next section, and the cap-8
+measurements archived below cover memory-default only.
+
+`PDPP_TEST_CONCURRENCY`, when it parses to a positive integer, replaces either
 default. A positive override is **not** clamped to the CPU count or the selected
 file count.
 
@@ -82,6 +93,12 @@ failing. Two runs that fail identically say nothing about whether either cap is
 safe, and none of it is a green-suite result. The failures are retained as
 evidence rather than hidden.
 
+On the recorded Node 22.23.1 run, both caps selected 1,033 files and produced
+6,961 assertions: 6,335 passed, 396 failed, and 230 skipped, with the same 396
+failure identities and exit code 1. The cap-2 receipt records 352.198 seconds;
+the cap-8 receipt records 141.066 seconds. These failures are retained as
+evidence, not hidden as a successful result.
+
 ## Operational use
 
 Use the default unless a measurement for the same profile and host justifies an

reference-implementation/scripts/run-tests-file-concurrency-cap.test.ts

@@ -0,0 +1,40 @@
+// Copyright The PDP-Connect Contributors
+// SPDX-License-Identifier: Apache-2.0
+
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import test from "node:test";
+
+const runTestsSource = readFileSync(new URL("./run-tests.ts", import.meta.url), "utf8");
+const concurrencyGuidance = readFileSync(new URL("../docs/gate-concurrency.md", import.meta.url), "utf8");
+const PROFILE_CAP_PATTERN = /const DEFAULT_FILE_CONCURRENCY_CAP = selectedProfile === "postgres" \? 2 : 8;/;
+const EXPLICIT_OVERRIDE_PATTERN =
+  /Number\.isInteger\(requestedConcurrency\) && requestedConcurrency > 0 \? requestedConcurrency : defaultConcurrency;/;
+// Match the cap expression the guidance quotes, not a sentence of prose: the
+// point is that the documented caps track the runner's, and an assertion on
+// wording breaks on any edit that leaves those numbers correct.
+const GUIDANCE_PATTERN = /const DEFAULT_FILE_CONCURRENCY_CAP = selectedProfile === "postgres" \? 2 : 8;/;
+
+test("PostgreSQL profile keeps the default file concurrency cap at two", () => {
+  assert.match(
+    runTestsSource,
+    PROFILE_CAP_PATTERN,
+    "the runner must cap PostgreSQL at two by default while allowing memory-default to use eight"
+  );
+});
+
+test("explicit PDPP_TEST_CONCURRENCY still overrides the profile default", () => {
+  assert.match(
+    runTestsSource,
+    EXPLICIT_OVERRIDE_PATTERN,
+    "an explicit positive PDPP_TEST_CONCURRENCY must remain authoritative for either profile"
+  );
+});
+
+test("checked-in guidance states the same profile-specific defaults as the runner", () => {
+  assert.match(
+    concurrencyGuidance,
+    GUIDANCE_PATTERN,
+    "the operational guidance must state the runner's profile-specific defaults"
+  );
+});

reference-implementation/scripts/run-tests.ts

@@ -515,7 +515,23 @@ async function runNodeTest(filePath: string, extraArgs: string[]): Promise<NodeT
 }
 
 const testFiles = await discoverSelectedTestFiles(repoRoot, testDir, accountingAuthority?.files);
-const defaultConcurrency = Math.max(1, Math.min(2, availableParallelism?.() ?? 1, testFiles.length || 1));
+// Each worker owns one child process (its own event loop, own listen sockets)
+// pulling one file at a time from a shared queue, so raising this cap adds
+// wall-clock-real parallelism rather than starving a single process of CPU.
+// Capped at 8 rather than following availableParallelism() to the host's
+// full core count: the measured headroom (pdpp PR #287 and the PR that
+// ported it here) shows throughput flattening well before 24 cores, and a
+// hardcoded cap bounds worst-case resource use on a small CI runner
+// regardless of host size.
+// The postgres profile stays at 2: its backup/restore oracle uses one shared
+// restore database (PDPP_TEST_POSTGRES_RESTORE_URL) that is not per-file
+// allocated, and the cap-8 measurement covered memory-default only.
+// PDPP_TEST_CONCURRENCY still overrides either default explicitly.
+const DEFAULT_FILE_CONCURRENCY_CAP = selectedProfile === "postgres" ? 2 : 8;
+const defaultConcurrency = Math.max(
+  1,
+  Math.min(DEFAULT_FILE_CONCURRENCY_CAP, availableParallelism?.() ?? 1, testFiles.length || 1)
+);
 const fileConcurrency =
   Number.isInteger(requestedConcurrency) && requestedConcurrency > 0 ? requestedConcurrency : defaultConcurrency;