Repository navigation
The replay that judges every precision fix now runs daily, prints in report and status, and raises its own regression finding - #62
Open
opencdlee-dotcom wants to merge 2 commits into
Conversation
…mbered to, so report and status now carry its number and a regression raises its own finding `backtest replay` is the number the precision plan is judged on, and at 3-6 minutes on the live store nothing ran it unattended. This caches it. - `_precision_measure` runs `_backtest_replay(30, reobserve=True)` and reduces its returned dict (no text scraping) to noise_reopened/noise_total, interrupts, open_cases, assay routable-interrupting/total and predicate-passing/total, plus teaching evaporation. Rule 17: the replay's own problems ride along, and the assay statuses must partition the lanes. - `_teaching_evaporation`: signal incidents opened in the window whose `_incident_identity` (exact correlation key where it cannot generalize) matches one with an earlier benign-positive/false-positive dismissals row. Machine closes at birth (auto-tolerated, learning-period, allowlisted) are teaching that held and do not count. Identity only: `_finding_classes` is not on this base (it lands with S6). - `precision.json` in STATE_DIR, resolved at call time. The scan's tail refreshes it at most once per 24 h (`_integrity_scan_due`'s rule; a failed attempt is stamped so it is retried tomorrow, not every tick). Every failure is caught and logged to the run log; the scan's result and heartbeat are untouched. `PRECISION_SNAPSHOT_EVERY_SECS <= 0` turns the scan path off, and conftest pins it to 0 so ordinary scan tests never run a replay. - `check_precision` (gather_all, next to the assay sensor) reads the snapshot and its stored baseline: any drop in routable-interrupting or predicate-passing is HIGH "A detector positive control stopped firing"; a rise of 3 or more in noise_reopened or teaching evaporation is MEDIUM "Precision regressed" (digest). Both are `self-protection` findings built with `finding()`. A first snapshot has no baseline and raises nothing. - `report` (every rendering) and `status` print one line: `Precision (30d replay, <age>): a/b judged-noise would re-alert · n interrupts · assay x/y routable + p/q predicate · teaching evaporation e`. - `aegis.py precision [--refresh]`: prints the cached snapshot, its age and its baseline; --refresh measures now. Tests: tests/test_precision_snapshot.py (throttle, failing replay vs scan, the regression rules, line format, evaporation on a synthetic store); "precision" added to the sensor roster in test_sensor_invariants.py. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s as re-alerting and nothing said so; the snapshot now counts the ledger rows it cannot verify and runs outside the scan lock The first dry run of `precision --refresh` measured a copy of ~/.aegis that left hmac.key behind. _hmac_key() minted a fresh key in its place, every custody and intent row failed its MAC, and every rung built on them was gone. The same copy, run twice back to back, 2026-09-23: no key on disk 143/214 noise re-opened, 165 interrupts, 167 open cases the real key 127/214 noise re-opened, 130 interrupts, 132 open cases The line printed both with the same confidence. - `_ledger_rows_rejected` counts the in-window custody.jsonl and intent.jsonl rows that do not verify under the key on disk, asked the way the lookups ask (_custody_carried, _intent_attested: same byte bound, same age cutoff, same MAC). It runs BEFORE the replay, because the replay's custody lookups mint a key when there is none. It never mints one itself: with no key, every in-window row counts as rejected. - The snapshot records `hmac_key` and `ledger_rows_rejected`, and the line leads with `inputs unverified: N ledger rows rejected` (plus "(no HMAC key on disk)") ahead of the numbers. The no-key copy now reads "3793 ledger rows rejected (no HMAC key on disk)". With the real key it reads "2 ledger rows rejected": two custody rows from 2026-09-19T07:45:48 that do not verify under the live key either. - The daily replay no longer holds the one-writer scan lock. cmd_scan runs `_precision_tail` after `_scan_lock` is released, and the tail takes its own `.precision.lock` (`_scan_lock` gained a `name` argument), which no scan contends on. It asks "due" again under that lock, so two scans that finish together do not both replay. Tests: a scan started from inside the replay runs to its own log line and does not replay again; a held snapshot lock skips; no key or a foreign key rejects the rows and leads the line; rows past retention are not counted. The new tests fail on dc49d69 (6 failed) and pass here. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Step S8 (Phase 4) of the precision plan: the operator sees the number the precision fixes are judged on, and a regression raises a finding without anyone asking.
What changed
_precision_measureruns_backtest_replay(30, reobserve=True)and reduces the dict it already returns (no text scraping) tonoise_reopened/noise_total,interrupts(findings the replay routes to interrupt),open_cases,assay_routable_interrupting/_total,assay_predicate_passing/_total, andteaching_evaporation. Rule 17: the replay's ownproblemstravel with the snapshot, and the assay statuses have to partition the lanes. It writesprecision.jsonunder STATE_DIR, with the path resolved when it is called. It only reads the store._incident_identity(or the exact correlation key, when the identity can't generalize) matches an incident that got abenign-positive/false-positivedismissals row earlier. Incidents the machine closed in the same pass that opened them (auto-tolerated,learning-period,allowlisted) are teaching that held, so they don't count. It matches on identity only because_finding_classesisn't on this base; it lands with S6.cmd_scanafter the scan lock is released, under its own.precision.lock(_scan_locknow takes aname), which no scan contends on. A scan started during the replay runs normally. It runs at most once per 24 h, using the same cadence rule as_integrity_scan_due, and checks again whether it is due once it holds the lock. A failed attempt gets a timestamp, so it's retried the next day and not on every tick. Any exception costs the snapshot and one run-log line, never the scan. SettingPRECISION_SNAPSHOT_EVERY_SECS <= 0turns off the scan path; conftest pins it to 0 so ordinary scan tests never run a replay._ledger_rows_rejectedcounts the in-window custody/intent ledger rows that don't verify under the HMAC key on disk. It asks the way the lookups ask, and never mints a key. The count goes intoprecision.json(hmac_key,ledger_rows_rejected), and the line starts withinputs unverified: N ledger rows rejected. Why: a copy of~/.aegiswithouthmac.keyread 143/214 noise re-opened, 165 interrupts and 167 open cases; the same copy with the real key read 127/214, 130 and 132. The earlier line printed both with the same confidence.check_precisionis registered ingather_allnext tocheck_assay. It compares the snapshot with its stored baseline, which is the previous snapshot. If the number of routable lanes that interrupt, or predicate lanes that pass, drops at all, it raises a HIGHself-protectionfinding, "A detector positive control stopped firing", which interrupts. Ifnoise_reopenedor teaching evaporation rises by 3 or more, it raises a MEDIUM "Precision regressed", which goes to the digest. Both are built withfinding(). A first snapshot raises nothing.report(every rendering) andstatusprintPrecision (30d replay, <age>): a/b judged-noise would re-alert · n interrupts · assay x/y routable + p/q predicate · teaching evaporation e.aegis.py precision [--refresh]. It prints the cached line, the open-case count, the baseline and any regression against it.--refreshmeasures now.Not in this PR
reportandstatus._finding_classesfrom S6.self:precision:*isn't a state-fingerprint prefix, so the cleared-state closer doesn't resolve it.cmd_watchprocess (52–129 s live). For that long the watch loop can't arm kqueue or start an event scan. A change in that window is caught by the next full scan, and the 600 s floor is counted from when the last full scan started, so the replay doesn't push it back. Closing the gap means running the replay in a child process. That is platform-shaped: ininstall scanmode, launchd and systemd kill children when the job exits. Left for review.package-managed) that don't verify under the live key. On this Mac the line will readinputs unverified: 2 ledger rows rejecteduntil they age out (180 d). This PR does not touch the live ledger.🤖 Generated with Claude Code