The replay graded persistence as recorded, so 37 OS updates filed while the apple tier was dead still read as 37 HIGH swaps

[opencdlee-dotcom]

Step S0b of PRECISION-PLAN-2026-09-23.md: backtest replay --reobserve now re-derives persistence and agent-surface findings with the current code. Before this change it replayed them exactly as recorded.

What it does

A change finding records a diff. To re-observe one, the replay rebuilds the pair that was diffed and hands it to the sensor. It does not use a second grader.

• persistence
  • New side: the sensor's own snapshot_persistence() of the item as it is now. The replay only accepts it if it still prints what the record says the item changed to.
  • Old side: what the record says the item changed from.
  • Finding the old values: the replay renders every shape _persistence_change_detail can print, with tokens in the old-side slots. The shape that matches the recorded line gives the old values.
  • Grading: check_persistence(old, rec) grades the pair. That covers the OS update, publisher-stable / relocated, payload custody and _demote.
  • Old fields no line carries: the old signer, the old program bytes on a path change, and the old payload path after an argv edit. The replay takes them from the live baseline while it still holds the recorded old side. Otherwise it tries every value the field could have had, and reports two different answers as not re-derivable.
• New persistence items: check_persistence gives a NEW item no custody rung, and the replay does the same. The step asked for a custody path here. That would have been a grader the sensor does not have, so the harness would show a fix the product lacks. That belongs in S5 or S7.
• agent-surface
  • diff_agent_surface re-grades exec targets, materialized targets and instruction-file directives against the bytes named in the fingerprint. It only does this while those are still the bytes on disk.
  • Custody is asked through _custody: the intent ledger first, then git. Both are read-only.
  • A new exec entry is graded on the config file's bytes. No record carries those, so it is counted as not re-derivable.
• Report and self-check
  • The stats line gets a "not re-derivable: N" breakdown by reason.
  • The dropped column covers both categories.
  • A persistence table shows recorded -> current.
  • The self-check asserts that every finding in a re-derivable category was re-observed, gone, or counted as not re-derivable.

Numbers (live store, 30 days, backtest replay --days 30 --reobserve)

	before (integration tip 0d7d3b6)	after (this PR)
persistence interrupt	54	9
agent-surface interrupt	16	9
total interrupt	129	77
noise re-opened	126 of 213	75 of 213
superseded re-opened	38/38	1/38
cases left OPEN	131	79
assay recall	9/21 interrupt (1 not run, 11 predicate)	9/21 interrupt (1 not run, 11 predicate)

Persistence change table:

HIGH/- -> LOW/-: 271              New item, recorded `unsigned`, now developer-id (xbar)
LOW/- -> dropped: 132             aegis's own BTM helper, which check_persistence now skips
HIGH/- -> MEDIUM/local-commit: 67
HIGH/- -> LOW/os-vendor: 37       the 2026-09-15 OS-update swaps

Not re-derivable, replayed as recorded (2392):

• the item on disk no longer shows the recorded change: 1771
• the config's content is not recorded (new agent exec entries): 313
• other persistence sensors (crontab: 174, .pth hook: 134)

What still interrupts, and why the replay leaves it

• New persistence items for the aikit jobs (#287, 288, 295, 300, 302, 307). The program is ad-hoc uv, and check_persistence grades NEW items with no custody rung. That is a sensor gap (S5/S7), not a harness gap.
• authman-healthcheck (#286, 365) and improver.weekly (#319). The pair re-derives, and current custody still gives the payload no rung. For authman, the script lives in a directory that is not in any repo.
• com.charlie.aegis.plist (#376, 377). The item has changed since, so it is not re-derivable.
• New exec entries (#297, 298, 299, 313). Not re-derivable: the config's bytes are not recorded.
• Exec targets changed (#316, 327, 453, 521). These re-derive, and current custody still leaves them HIGH: the target scripts are neither attested nor committed.
• First-sight directives (#335, 336). The files are gone from disk, so these are replayed as recorded.

Tests

• tests/test_backtest_replay_persistence.py (10 tests). It covers:
  • (a) a recorded HIGH signed-other program-bytes swap on /bin/bash re-observed as LOW os-vendor. This case is macOS-only, registered in _MAC_ONLY_CLASSES.
  • (b) a payload-bytes change whose payload now has git provenance (stubbed _git_provenance), which is demoted. It is checked both with and without the baseline, plus a payload that has moved on since the record.
  • (c) records missing fields, counted as not re-derivable and replayed as recorded, including an unrecorded old signer that decides publisher-stable.
  • agent-surface: an exec target re-graded by provenance, a first-sight directive re-graded from its markers, and a new exec entry named as not re-derivable.
• python3 -m pytest tests/test_backtest_replay.py tests/test_backtest_replay_persistence.py -q: 21 passed.
• Simulated bodies for the two replay files: win 19 passed / 2 skipped, linux 19 passed / 2 skipped, mac 21 passed.
• test_cross_platform.py under SIM_BODY=win has 3 failures, and the merge base has the same 3.
• Full suite at this commit's tree: 2164 passed, 6 skipped, 30 xfailed, 41 subtests passed in 994.39s.

🤖 Generated with Claude Code