epic(bench): standardize BenchExec and Divan measurement and retire custom benchmark harnesses

[DecisionNerd]

Objective

Standardize performance measurement on BenchExec for command/process-tree workloads and Divan for in-process benchmarks. Retire duplicated benchmark machinery with bounded semantic-parity evidence, preserving correctness tests and historical results. This small epic is executable and closable independently of #900; #952 remains its parent.

Maintainer decisions (2026-09-19)

• BenchExec owns process-tree timing/resources; Divan owns in-process benchmark timing/sampling. Choose by execution boundary, not an arbitrary duration cutoff. CodSpeed may continue to execute/report Divan benchmarks.
• Every performance gate consumes framework-produced measurements for the timing/resource quantities those frameworks own. Product semantic counters such as logical bytes, reader calls and publication counts remain direct deterministic evidence, distinctly named; they are not physical resource measurements.
• Shared internal phase timing may remain diagnostic-only, with explicit scopes, units and overlap semantics. It cannot supply performance threshold inputs.
• Strongly guard against new independent benchmark clocks, sampling loops and statistical harnesses. Preserve deadlines/cancellation, shared diagnostics and deterministic correctness assertions; do not ban Instant/Duration globally.
• Decouple epic(bench): standardize BenchExec and Divan measurement and retire custom benchmark harnesses #959 from test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900. Bounded migration/parity tests close this epic. The completed-ladder comparison previously required here belongs to parent epic(bench): externalize scale certification and add GDC benchmark suites #952 using test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900's single canonical run. It is not an execution or closure prerequisite for this epic or its children.
• Decouple epic(bench): standardize BenchExec and Divan measurement and retire custom benchmark harnesses #959 from live GDC suite runs. GDC measurement-boundary enforcement uses inventory, fail-closed policy tests, and checked-in tiny fixtures. Executing full GDC live lanes is not required to close this epic.

Scope and ownership

1. Policy, inventory, testing docs and CI enforcement (test(bench): define and enforce the BenchExec and Divan measurement policy #1484): a native child owns this repository-wide scope, additional to the original ladder-retirement issue. Inventory producers and consumers, measurement boundaries and dispositions: benchmark authority, diagnostic attribution, deadline/control logic or unused code.
2. Traversal/MERGE migration to Divan (refactor(bench): migrate traversal and MERGE scaling measurements to Divan #1485): a native child owns these in-process workloads, also outside the original scope. Preserve mutable fixture state, warmed traversal intent, timed setup/teardown, graph/hop parameters and deterministic assertions.
3. Lifecycle/GDC orchestration and bounded parity: retained here. Inspect certification/GDC runners and BenchExec adapters/consumers. Remove unused timers and duplicate benchmark authority; preserve or consolidate scoped shared diagnostics. Retain load/query/validation semantics, failures, cleanup and evidence on bounded tiny/shadow fixtures and policy tests. Do not split an in-memory GDC workload across processes merely to get separate timers. Establish supported framework measurement boundaries before replacing diagnostic-derived thresholds. Reconcile cgroup memory versus process RSS names where required by the migration; preserve schema compatibility. No live GDC suite execution and no test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 ladder run are closure prerequisites.
4. Existing owners remain canonical: perf(bench): make the ingest gates two-sided so improvements ratchet automatically (#1387 D1/D5) #1476 owns ingest gate/ratchet implementation and migration of its custom wall/CPU inputs; test(tck): the performance baseline is from a CI runner, so the perf gate warns on every local run #1467 owns TCK comparisons and measurement migration. Reuse their work instead of duplicating it. This epic verifies only their policy-conformance outcomes, not unrelated optimization or outlier repairs.

Keep the inventory finite and concrete: traversal/MERGE, Divan ingest including its alternate gate mode, certification/lifecycle, GDC and TCK, plus independently discovered performance-gate consumers in the same audit. Additional issues require a verified blocker or independently reviewable XL concern, not one issue per timer.

Execution order

The policy/inventory child #1484 is ready now, subject to the repository WIP limit. The Divan migration #1485 is natively blocked by #1484 and follows that policy contract. Bounded lifecycle/GDC diagnosis and fixture preparation can proceed immediately; perform its implementation against the agreed policy. Merge focused changes promptly and refresh the next candidate against current main.

Native dependencies own execution state. #956/#957/#1112 are delivered foundations, not work to repeat. #1112 already separated structural retirement, prefix parity and full completion; adapt its consumer contract to this maintainer decision without claiming an incomplete ladder is complete.

Acceptance criteria

• One measurement policy is linked from testing strategy, testing recipes, benchmarking docs and the harness README; commands and implementation agree.
• Named sites have verified dispositions. Every performance timing/resource gate uses framework-produced evidence with explicit scope/provenance; shared diagnostics cannot substitute for it. Deterministic product-counter gates remain intact.
• Fast CI rejects new independent benchmark measurement machinery and untracked/stale exceptions, with meaningful positive/negative fixtures. Ordinary control timers and approved shared diagnostics pass. No blanket directory exemptions.
• Divan workloads preserve fixture and measurement semantics and execute successfully; deterministic correctness/read-count tests remain in required product CI.
• Lifecycle migration preserves load/query/validation boundaries, result correctness, failure classes, termination/cleanup and necessary observations on bounded tiny/shadow fixtures and BenchExec consumer tests. Missing or misattributed framework evidence fails closed in those bounded tests. No test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 ladder run is required.
• GDC measurement boundaries are enforced via inventory, gdc_measurement_policy, and fail-closed unit tests on checked-in evidence shapes. Live GDC suite execution is not required for closure.
• Preserve exact legacy outcomes as checked-in fixtures and compare phases, correctness, failure classes, timing scope/provenance and teardown on bounded evidence. Every difference is explained; no unexplained semantic gap remains in the migrated behavior covered by those fixtures.
• Removed orchestration has equivalent bounded external coverage. Historical schemas/evidence remain readable. New measurement boundaries or incompatible baselines are explicitly identified, never reported as speedups.
• Required negative cases include timeout/OOM, nonzero exit/signal, malformed/missing evidence, incorrect results and failed/nonempty teardown where applicable. No scale run, test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 ladder run, or live GDC run is necessary to exercise these contracts.
• Native children and the relevant perf(bench): make the ingest gates two-sided so improvements ratchet automatically (#1387 D1/D5) #1476/test(tck): the performance baseline is from a CI runner, so the perf gate warns on every local run #1467 policy-conformance changes are merged and verified with green changed-surface checks. Do not duplicate their unrelated acceptance criteria.
• Readiness/completion predicates and docs (parity_gate, scale-parity-index.md, harness README) allow epic(bench): standardize BenchExec and Divan measurement and retire custom benchmark harnesses #959 closure on bounded outcomes only. An absent or incomplete test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 ladder and absent live GDC runs cannot prevent closure. This never marks test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900/epic(bench): externalize scale certification and add GDC benchmark suites #952 scale certification complete.
• No additional S18–S26 ladder is run for this issue; optional read-only ingest of completed test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 bundles is epic(bench): externalize scale certification and add GDC benchmark suites #952 track only.

Completion scenarios

1. Given a new custom benchmark sampler or diagnostic-based performance gate, when policy and evidence tests run, then they fail with an actionable location; deadlines and approved shared diagnostics remain usable.
2. Given equivalent fresh/warmed fixtures, when Divan workloads execute, then the same work and deterministic assertions hold and timing comes from Divan.
3. Given legacy and new bounded lifecycle fixtures, when normalized evidence is compared, then behavior agrees or every difference is explained, and failure/cleanup cases remain fail-closed. Live GDC suite execution is not required.
4. Given all bounded migration outcomes are merged and green but test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 is incomplete and no GDC live lane has run, when epic(bench): standardize BenchExec and Divan measurement and retire custom benchmark harnesses #959 readiness is evaluated, then this epic can close while test(scale): complete the S18-S26 lifecycle ladder on OVHC-AGENCY #900 and epic(bench): externalize scale certification and add GDC benchmark suites #952 continue to require their own real scale evidence.

Relationships and non-goals

Parent #952 retains the original completed-ladder comparison obligation using the single #900 OVHC-AGENCY run, including all semantic dimensions above. Neither that obligation nor #900 blocks this epic. Keep delivered #956/#957/#1112 relationships for provenance; Fly #958 is optional historical tooling.

No new framework, product-only-for-benchmarks API, numerical optimization target, weakened durability/correctness, new release cascade or scale execution. OVHC-AGENCY remains the scale acceptance host. Closing this epic proves standardized measurement and bounded migration parity, not billion-edge throughput or complete scale certification.

[DecisionNerd]

Reopening: PR #1056 merged ladder bundle ingest tooling and parity-gate fixes only. Legacy orchestration retirement remains blocked until real #900 ladder bundles are ingested and parity gate reports ready_for_retirement: true.