perf(test): clone PostgreSQL test databases from a run template

[tnunamak]

Summary

Setting up a PostgreSQL database for every test file is slow, so this change builds one schema template per test run and clones each file's database from it. The risk that comes with cloning is that a template is shared mutable state living under a predictable name: if a stale template from an earlier run, or one built by a different process against different migrations, is still present under the name this run expects, every test file that clones it silently runs against the wrong schema. Tests would pass or fail for reasons that have nothing to do with the code under test, and the cause would be invisible in the output. This PR makes that impossible by checking the template's identity immediately before each clone.

Changed

reference-implementation/ is the directory holding this repository's PDPP reference implementation — the authorization and resource servers, runtime, CLI, and black-box test suite for the Personal Data Portability Protocol. It is the component every file path and CI job below refers to. Inside it, scripts/run-tests.ts is the test runner that decides how each test file gets its database. It no longer issues a raw CREATE DATABASE ... TEMPLATE itself; it calls clonePostgresTestDatabaseFromTemplate in reference-implementation/scripts/postgres-test-template.ts, which runs the identity check described below and the clone on a single connection to the postgres maintenance database — the database CREATE DATABASE must be issued from, since a database cannot be created from a connection to itself — with an advisory lock held across the whole sequence. Doing both on one locked connection is what closes the gap: a check performed on an earlier, separate connection could be invalidated by another process between the check and the clone.

The identity is stored as a row in a table named pdpp_test_template_metadata (the pdpp_ prefix marks tables this project creates), which lives in that same maintenance database rather than inside the template, keyed by template name. It is therefore not copied into the clone — it is a record about the template, held where the clone cannot alter it. The row holds the ID of the runner process that built the template, a digest of the SQL migration files the schema was built from, the schema version those migrations produce, and an identity_digest: a SHA-256 over the template's name and those other fields together. That digest is the single value the runner passes to each child test process, so throughout this PR "identity token" means that identity_digest, and the runner ID is one of the inputs hashed into it, not a separate check.

A clone succeeds only when the digest recomputed from the stored row equals the token the runner holds, so a template with the right name but the wrong build is refused — including one built by an older version of this code, whose row predates the digest column and is NULL there. A missing identity is a hard error rather than a quiet fall back to building a fresh database, because a silent fallback would hide exactly the staleness this check exists to detect.

Not every test file can use a template. Files that run schema migrations themselves, or otherwise need a genuinely empty database, are listed as "cold-required" in reference-implementation/scripts/postgres-template-eligibility.ts and bootstrap their own.

Reviewer findings addressed

This PR has been through three rounds of independent review. Findings are labelled P1 (must fix before merge) through P3 (correctness of the description, not the code).

Round 1, P1 — the runner's own clone path bypassed identity verification. The check existed, but the runner did not go through it, so the property was unproven exactly where it mattered. reference-implementation/test/postgres-test-template-identity.test.ts adds two regression tests that drive the clone helper against a live PostgreSQL server. (Round 3 found these named as though they covered the runner's own selection of which files get a clone, which they do not; they are renamed to "clone helper ..." and that seam is now covered separately — see P1-1 below.) The first changes only the runner ID in the stored row, which invalidates the digest hashed from it; the second leaves the row alone and hands the clone a wrong expected token. Each asserts the clone is refused, and each then asserts the database that would have been created does not exist — so the test proves the refusal actually prevented the clone rather than merely logging a complaint.

Round 1, P2 — the eligible and cold-required file counts in PG-PROFILE-51-REPORT.md had drifted from the code. That report is a hand-written Markdown document checked in at the repository root, recording how this PostgreSQL work was measured; "51" is this pull request's number. Rather than hand-correcting the numbers, reference-implementation/test/postgres-template-eligibility-inventory.test.ts reads the report and derives the expected values from the two arrays that are the real source of truth, POSTGRES_TEMPLATE_ELIGIBLE_FILES and POSTGRES_TEMPLATE_COLD_REQUIRED_FILES, both exported from reference-implementation/scripts/postgres-template-eligibility.ts. The report now states 122 eligible plus 26 cold-required, 148 total, and cannot drift silently again.

Round 2, P1 (blocking) — the DCO check was red. DCO is the Developer Certificate of Origin check GitHub runs on this repository; it requires every commit to carry a Signed-off-by line whose name and email both match the commit's author or committer. The head commit was authored as Tim Nunamaker <tnunamak@gmail.com> but signed off as tnunamak <tnunamak@gmail.com> — same email, different name — so it failed while the other seven commits passed. The commit has been amended so the names match. Its tree is byte-identical to the previously reviewed head (git diff 07aa9ca2a HEAD is empty), so this is a message-only change and none of the evidence below is affected. DCO now reports pass.

Worth recording for anyone who hits this again: the commit-msg hook that runs before each commit on this machine compares the sign-off email only, so a name-only mismatch passes locally and fails in CI. That hook is developer-machine configuration held outside the repository (via git's core.hooksPath), so it is not something this PR can change.

Round 2, P3 — this description's previous claim about Biome was wrong. Biome is the linter and formatter this repository configures in reference-implementation/biome.jsonc. The corrected measurement is in the last section.

Validation

Run on Node 22.23.1 against PostgreSQL 16.15 in a Docker container bound to 127.0.0.1 only, in a throwaway database dropped afterwards.

The identity suite reports 9 passed, 1 skipped, 0 failed, in 51.3 seconds. All nine real cases ran. The skipped tenth is a placeholder that exists to report "no PostgreSQL configured" when the suite cannot run at all, so it is inert here precisely because PostgreSQL was available. The eligibility inventory suite reports 3 passed.

Both suites were then checked for teeth by mutation, because a test that passes is not yet evidence that it would catch the failure it names:

• Deleting the assertPostgresTestTemplateUsable call — the function that performs the identity check — from clonePostgresTestDatabaseFromTemplate, leaving only the bare CREATE DATABASE ... TEMPLATE, makes exactly the two new clone-helper regression tests fail: the one that alters the stored runner ID, and the one that hands the clone a wrong identity token. The other seven non-skipped cases still pass. Failing that precise pair, and only that pair, is what shows the check is load-bearing at the clone boundary rather than incidentally satisfied by something else.
• Editing the eligible-file count in PG-PROFILE-51-REPORT.md from 122 to 123 makes exactly the inventory test that reads the report fail.

Both mutations were reverted and the working tree verified clean.

Round 3: external review findings addressed

Not yet on this branch. The four repairs described in this section are committed locally but could not be pushed: this repository's pre-push hook requires every commit to carry a valid OpenPGP signature, and the signing agent's passphrase cache expired part-way through the session. The branch currently ends at 13df5c25b (the round-2 DCO fix). This section describes work that is written, run, and mutation-checked, but that a reviewer cannot yet see in the diff — treat it as a statement of intent until the commits appear.

A second, external reviewer examined the same head and raised four findings that the earlier round had not caught. All four are repaired here, each with a test that fails before the change and passes after, plus a mutation check confirming the test would catch a regression.

P1-1 — the runner's selection seam was never executed by a test. Two tests named "runner per-file clone ..." called the clone helper directly. They prove the helper is fail-closed, but they never ran the line that decides whether a file is cloned at all: useTemplate = postgresTestTemplateName !== null && isPostgresTemplateEligibleFilePath(filePath). The second half is the fail-closed default — a file not on the eligibility allowlist must get a from-scratch bootstrap even when a template exists — and deleting it failed nothing. Those two tests are renamed to "clone helper ...", and reference-implementation/test/run-tests-postgres-template-selection.test.ts now runs the real runner as a subprocess against live PostgreSQL on a cold-required migration test.

Asserting only on that file's outcome would not have worked, and this is why the defect survived review: the run's template carries a correct schema, so a cold-required file wrongly cloned from it still passes. The runner therefore now reports which path it took for each file (PDPP_TEST_DB_PROVISION, either cold-bootstrap or template-clone) and the test asserts on that. Making the decision observable is the actual repair; the assertion is just what it enables. Mutation: the exact change the reviewer named, useTemplate = postgresTestTemplateName !== null, now fails with the runner reported "template-clone".

P1-2 — the template's identity was adopted rather than decided. Three things combined. The run nonce was 32 bits, so same-name collision on a shared cluster is a birthday problem at roughly 77k runs. ensurePostgresTestTemplate returned early when a usable template already sat under the derived name, without checking that this run built it. And it returned only the name, so the runner learned the expected identity by reading the candidate back — sourcing the expectation from the thing under verification, which is circular.

The nonce is now 128 bits (a new value, kept separate from the 8-hex-character run ID that per-file database names embed and whose grammar pins that width). A pre-existing database under a fresh nonce's name is an error rather than something to reuse. The identity is computed and committed before publication and returned by the builder; the read-back that remains only confirms the published row reproduces it. Mutations: reverting the nonce to 4 bytes fails only the nonce assertion; restoring the adopt-if-usable early return fails only the adoption assertion.

P2 — verification and the clone ran on different connections. clonePostgresTestDatabaseFromTemplate opened an admin connection, then called a verifier that opened a second, independent one. The check therefore described the template as seen by a connection already closed before CREATE DATABASE ... TEMPLATE ran — a time-of-check/time-of-use window. The repair holds one connection and one advisory lock across verification, OID capture, and the clone, and re-reads the OID immediately before copying. Separately, the template's OID is now bound into the identity digest, so a drop-and-recreate that preserves the name is refused even when it happens before the call rather than during it. Mutations: removing the OID from the digest fails only the swap test; removing the advisory lock fails only the lock test.

P2 — the "schema identity" claim was broader than the check. The digest covers columns and index definitions in public and nothing else. Constraints, triggers, row-level security policies, functions, sequences, types, extensions, ownership, privileges, collation, and server version are all outside it. Rather than widen the query — a larger change than this PR should carry — the claim is narrowed: the comments now state the exclusions explicitly and say to widen the query before making the stronger claim.

All five PostgreSQL suites pass together at this head: identity 9 passed with 1 expected skip, inventory 3, provenance 3, clone atomicity 2, runner selection 1.

What this does not claim

This does not claim a new full PostgreSQL profile result, and does not make the repository's unrelated red checks green. reference-implementation gate, test reference implementation, and typecheck reference implementation are three CI jobs that lint, test, and typecheck the reference implementation. All three fail on main itself, before this branch's changes, because of an unrelated in-progress migration that left the reference implementation with type errors and failing tests — tracked as issue #55, "unblock the Move-B CI seam" — and they fail here for that same reason. I confirmed they are red on main rather than assuming it.

Corrected Biome measurement. The previous claim here — no errors and three pre-existing warnings — was wrong. Measured with this repository's own pinned toolchain (Ultracite 7.10.7, a preset wrapper that invokes Biome 2.5.11 with a shared rule set), every file this PR touches reports exactly one error, and that error is a whole-file formatting diff rather than anything about the code. It is not specific to this PR: the configuration asks for tab indentation while the reference implementation is space-indented throughout, so a sample of 60 untouched test files reports 60 errors, one each. The only diagnostics attributable to this PR are three warnings about biome-ignore comments that no longer suppress anything, in files it adds. For each of the two files it modifies, running Biome against the merge-base version and the head version gives identical counts — reference-implementation/test/helpers/postgres-temp-database.ts reports 1 error and 4 warnings both times, and reference-implementation/scripts/run-tests.ts goes from 2 errors to 1 — so the PR adds no new diagnostics to either. Neither of the two specific lint errors named in the round-2 review reproduces at this head with the pinned toolchain; a different Biome version is the likeliest explanation, which I have not confirmed. No CI job runs Biome, so none of this gates the merge.

Assisted-by: AI