Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
f2dcc71
feat(pr-workflow): add pr-validate and falsifying-test skills
MajorLift Jul 30, 2026
76bc15d
Restore the MetaMask-planning link in the step-waiver item
MajorLift Jul 30, 2026
1e81dd3
Move `falsifying-test` to the `testing` domain
MajorLift Jul 30, 2026
f1c9023
Add `D6` substitution A/B lane and sync `pr-validate` references
MajorLift Jul 30, 2026
3f87dd6
Move the AEP run procedure behind a reference
MajorLift Jul 30, 2026
64c97a1
Restore the full `pr-validate` description
MajorLift Jul 31, 2026
1aa3bb8
Actually restore the `pr-validate` description
MajorLift Jul 31, 2026
25680db
Rename `pr-validate` to `evidence`
MajorLift Jul 31, 2026
d66ffe8
Repair unresolvable references in `evidence` and `falsifying-test`
MajorLift Jul 31, 2026
e42a77f
Follow the engine renames in the catalog and `falsifying-test`
MajorLift Jul 31, 2026
8105056
Add a lane index to the evidence catalog, and fix lane placement
MajorLift Jul 31, 2026
947c0d9
Add build-duration lanes `D7` and `G6`
MajorLift Jul 31, 2026
2fc64a3
Inline the publishing non-negotiables, which a real run ignored entirely
MajorLift Aug 1, 2026
2c981de
Add `falsify-probe.sh` — the runner that makes a lane reproducible
MajorLift Aug 1, 2026
75d89ee
Add `capture.sh` so the C9 and D3 analyses stop needing an operator t…
MajorLift Aug 1, 2026
1b34a27
Add `selector-recompute.sh` — lane C4 gets a runner
MajorLift Aug 1, 2026
572650e
Add `tsc-substitution.sh` — a runner for the tsc-blindspots lane
MajorLift Aug 1, 2026
ec04e2c
Diff the error sets rather than requiring a clean baseline
MajorLift Aug 1, 2026
6fa3c1f
Add `attest-gate.sh` — eight mechanical checks before anything is pub…
MajorLift Aug 1, 2026
e8acad3
Standardise the provenance marker across every runner
MajorLift Aug 1, 2026
50bf8f3
Add `render-count.sh` — the component half of lane C4
MajorLift Aug 1, 2026
d89bbcc
Guard the runners against the failures that masquerade as findings
MajorLift Aug 1, 2026
b351799
Give the orchestrator a runner registry, with limits and synthesis rules
MajorLift Aug 1, 2026
d000eb8
State the bar: float concerns, do not close them
MajorLift Aug 1, 2026
07002d6
Report what each measurement runner did not cover
MajorLift Aug 1, 2026
fe96614
Require a validation run to float something for review
MajorLift Aug 1, 2026
7eab362
Stop the runners from cutting the part a reader needed
MajorLift Aug 1, 2026
1c13e87
Reference artifacts by name in the publishable block
MajorLift Aug 1, 2026
b8224d3
Stop the gate failing runs whose environment is not the repo's
MajorLift Aug 1, 2026
5b8b710
Let the caller say where its tool puts the finding
MajorLift Aug 1, 2026
7a84732
Write for the reviewer, not for a user of this skill
MajorLift Aug 1, 2026
08c615a
Move the output template into the skill, where corrections can land
MajorLift Aug 1, 2026
34bd6d9
Require a finding, not a printout
MajorLift Aug 1, 2026
0174725
Check where the disclaimer is, not just that it is there
MajorLift Aug 1, 2026
40dee2e
Stop a script's summary of itself passing as a capture
MajorLift Aug 1, 2026
a56ad92
Ask for a medium, not for better text
MajorLift Aug 1, 2026
da9f593
Move the measurement to CI, where the run URL is the capture
MajorLift Aug 2, 2026
16e03a3
Pin the runner source, and say so when it is missing
MajorLift Aug 2, 2026
17f345c
Require the target repo, make install opt-out, drop the consumer-inst…
MajorLift Aug 2, 2026
da3c1ed
Make every runner say whether a reader can verify it
MajorLift Aug 2, 2026
5998799
Cite what exists; capture what you ran
MajorLift Aug 2, 2026
88f97a3
Fold the runner-workflow fixes back from where they were found
MajorLift Aug 2, 2026
9f3ffa1
Name the commit in the command line, so an A/B pair is not one line t…
MajorLift Aug 2, 2026
b276586
Stop a broken substitution reading as a divergence
MajorLift Aug 2, 2026
8a8e185
Require a run to measure the PR's range, and to say where its reach ends
MajorLift Aug 2, 2026
e0c83c2
Require an instrument to publish the effect it had, not the instructi…
MajorLift Aug 3, 2026
8ece574
Point `falsifying-test` at the harness, and separate it from its sibling
MajorLift Aug 3, 2026
76ec7f9
Rename `falsifying-test` to `red-on-base`
MajorLift Aug 3, 2026
90c10fb
Move `red-on-base` out; this PR is the instrument
MajorLift Aug 3, 2026
e82571d
Check where the run is going, not only what it says
MajorLift Aug 3, 2026
68c0587
Bring the runner fixes back from the branch CI was actually running
MajorLift Aug 3, 2026
506df35
Check that a figure in the prose traces to an exhibit
MajorLift Aug 4, 2026
8241ac1
Stop check 12 passing when it could not run
MajorLift Aug 4, 2026
a2506b8
Point references at the renamed `lavamoat-policy` skill
MajorLift Aug 4, 2026
f81508b
Give the diligence format a gate with `attest-gate.sh --diligence`
MajorLift Aug 4, 2026
0573b35
Remove private-repo and personal references from a public skill
MajorLift Aug 4, 2026
e5a33db
Take the artifact bucket and test fork out of the published text
MajorLift Aug 4, 2026
4511f65
Restore what the privacy scrub broke: a working jq filter and bucket …
MajorLift Aug 4, 2026
87b1fc5
Enforce the evidence rules where the model cannot route around them
MajorLift Aug 4, 2026
fe2eded
Add `attest` — the publish gate, ported from a personal command
MajorLift Aug 4, 2026
e636742
Add `gate-controls.sh`, and scope the gate to evidence artifacts
MajorLift Aug 4, 2026
f4bebca
Check that the gate is wired, not only that it works
MajorLift Aug 4, 2026
fa22023
Fail closed when the body cannot be read
MajorLift Aug 5, 2026
edf65a3
Ship `hooks/`, and assert that the bundle contract holds
MajorLift Aug 6, 2026
d6d0c82
Run the bundle contract check in CI
MajorLift Aug 6, 2026
bb6e744
fix(evidence-gate): require artifacts to be credible, not merely present
MajorLift Aug 6, 2026
f401caa
fix(evidence-gate): chain the publish to the gate's verdict for real
MajorLift Aug 6, 2026
966170f
fix(evidence): make the determinism check, the exit codes and the fro…
MajorLift Aug 6, 2026
f8c9e51
fix(evidence-gate): a verdict inside a code fence is a quotation, not…
MajorLift Aug 6, 2026
cf462ba
feat(evidence): measure which non-negotiables the gate actually enforces
MajorLift Aug 6, 2026
4a5b6c3
fix(evidence-gate): a CI citation is not a CI restatement
MajorLift Aug 6, 2026
f75acc5
fix(evidence-gate): a verdict token in inline code is a mention, not …
MajorLift Aug 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/lint-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,14 @@ jobs:
is-high-risk-environment: false
node-version: ${{ matrix.node-version }}
- run: yarn lint
- name: Bundle contract
shell: bash
run: |
# BUNDLE_DIRS in tools/skill-schema.mjs decides what the linter accepts beside a
# skill.md. The bundle loop in tools/install decides what reaches a consuming
# repo. A directory declared in one and missing from the other installs nothing
# and reports nothing: the skill still loads and answers without it.
tools/check-bundle-contract.sh
- name: Validate RC changelog
if: ${{ startsWith(github.head_ref, 'release/') }}
run: yarn lint:changelog --rc
Expand Down
41 changes: 41 additions & 0 deletions domains/pr-workflow/skills/attest/references/dispatched-passes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Phase 1 — the three dispatched briefs

Send each to a **fresh instance** with the artifact and nothing else: not the transcript, not
your reasoning, not what you expect it to find. Context is what you are testing for. An instance
that knows what you meant will read what you meant.

Run them concurrently — they are independent, and sequencing lets the first one's findings frame
the others.

## outframe — contest the frame

> You are reading a finished set of findings you did not produce. Do not check whether the
> findings are correct. Ask what claim was chosen and what a different framing makes visible:
> what question would a reader with different priorities have asked of the same material, what
> does the chosen frame make it impossible to notice, and which of the findings only look
> significant because of how the problem was cut. Return findings the framing hid, not a
> critique of the writing.

## missing — contest the coverage

> You are auditing a completed run for what it did not do. Enumerate: a modality that was not
> run, a claim asserted but not verified, a source cited but not read, a case the method
> structurally cannot reach. For each, say what running it would cost and what it could change.
> Do not restate what the run found. Absence is the deliverable.

## press — read it as the stranger

> You are the reviewer this lands in front of, with no context and a decision to make. Read only
> the artifact. Say what you would have to take on trust, which number you could not check if you
> wanted to, what reads as a measurement but is a sentence, and anything that assumes you were
> present for work you were not. Flag register slips: hedging that reads as concealment,
> confidence that outruns the evidence, and any place the author's process shows through.

## Reading the returns

A finding from any pass that invalidates the claim is `BLOCKED`. A finding that qualifies it is
`ATTESTED WITH` — and the caveat goes **into the published artifact**, not just into the verdict,
or the reader never sees it.

Disagreement between passes is signal, not noise: `press` clearing something `outframe` flagged
usually means the artifact reads well and is framed wrong, which is the more dangerous state.
35 changes: 35 additions & 0 deletions domains/pr-workflow/skills/attest/references/phase-0-checks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Phase 0 — what each check catches

Generated from the checks in `mms-evidence/scripts/attest-gate.sh`; that script is the
authority. Each entry exists because a run shipped without it.

| # | check | run mode | diligence mode |
|---|---|---|---|
| 1 | marker pair | ✓ | ✓ |
| 2 | canonical header | ✓ | ✓ |
| 3 | verdict line | ✓ | ✓ |
| 4 | citations pinned | ✓ | ✓ |
| 5 | captured artifact | ✓ | ✓ |
| 6 | no prescriptions | ✓ | ✓ |
| 7 | no process narration | ✓ | ✓ |
| 8 | verdict is earned | ✓ | ✓ |
| 9 | verdict matches artifact | ✓ | ✓ |
| 10 | floats something for review | ✓ | ✓ |
| 11 | disclaimer present and early | ✓ | ✓ |
| 12 | destination is open | ✓ | ✓ |
| 13 | figures trace to an exhibit | ✓ | ✓ |

Checks 1–4 differ by mode: in `--diligence` they test that contract's own marker pair, its
header, and that citations are pinned to a tag or SHA rather than a branch head, and the
verdict-line check reports SKIP because a diligence artifact renders none. Checks 8 and 9 SKIP
for the same reason. Everything from 5 down is shared, because those defects are shared.

**Check 5 is the one that matters, and it asks for a medium.** Every earlier version tested a
property of the plaintext — does it carry a marker, does the command contain a placeholder — and
each caught one defect and missed the next, because every property of plaintext is forgeable by
whatever emits the plaintext. Four runs shipped that way. A `/blob/` permalink is a citation and
does not satisfy it: it witnesses a line in a file, never a run.

**Check 12 tests the destination**, which no property of the text reveals. Across one register of
published runs, 22 of 27 comments went to pull requests that had already merged — median 22 days
after the merge, gate-clean every time.
104 changes: 104 additions & 0 deletions domains/pr-workflow/skills/attest/skill.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
---
name: attest
description: >-
The gate an evidence artifact passes before it is published to a pull request, issue or shared tracker. Two halves that do not substitute for each other — a mechanical pass that greps for the properties a reader needs (marker pair, pinned environment, a captured artifact rather than typed prose, a destination that is still open) and a dispatched pass sent to fresh instances that contest the framing, the coverage, and how it reads to a stranger. The author is the wrong checker: they remember running the check, and the memory supplies the provenance the text lacks. Verdicts are attested, attested with named caveats, blocked, or not a run — the last being common and legitimate, because a run that could not execute has produced nothing to publish. Triggers on mms-attest, or before posting any evidence, validation or diligence output to a public surface.
maturity: experimental
---

# /mms-attest

The gate an evidence artifact passes before it leaves your hands. Use before posting any
`/mms-evidence` or diligence output to a pull request, issue, or shared tracker.

## The author is the wrong reader, and the wrong checker

A validation run claims something was measured. Its characteristic failure is not a wrong number
— it is **prose that reads like a measurement**. An operator who ran the check cannot see this,
because they remember running it; the memory supplies the provenance the text lacks, before the
eye registers that it was missing.

This is not hypothetical. A run in this workflow shipped a results section whose commands, exit
codes and "reached 100%" were typed by hand, while the real logs sat unpublished on disk. The
author had the skill installed that forbids exactly that.

So the gate has two halves, and neither substitutes for the other.

**The mechanical half is not advisory.** Marker presence, a pinned environment, whether any
fenced block is a tool's output rather than the author's transcription, whether the destination
is still open — all greppable. Anything checkable is checked before a model is asked for
judgement, because a model asked "is this good evidence?" answers from inside the frame that
produced it.

**The dispatched half is positional.** Contesting the frame, the coverage, and the reading cannot
be self-run, for the same reason an author cannot proofread their own sentence for a word their
eye supplies.

## Phase 0 — mechanical, no model

```
scripts/attest-gate.sh <artifact.md> --target <owner/repo#N>
scripts/attest-gate.sh <artifact.md> --target <owner/repo#N> --diligence
```

Thirteen checks; every one a hard fail. `--diligence` swaps the four Validation-Run envelope
checks for a no-verdict contract's own and shares everything downstream. See
[references/phase-0-checks.md](references/phase-0-checks.md) for what each check exists to catch
and the run that caused it to be written.

**Run it as the same command that publishes, or it is a log line.** The gate and the write must
be one chain — `gate && publish`. Running both and reading the verdict afterwards is how a
blocked artifact reaches a public PR. The `hooks/pr-evidence-gate.py` PreToolUse hook enforces
this independently of your discipline, and fails closed; phase 0 is what you run to iterate
before it does.

## Phase 1 — dispatched, three lenses

| pass | reads for | returns |
|---|---|---|
| **outframe** | the frame — what claim was chosen, and what a different framing makes visible | findings the framing hid |
| **missing** | coverage — modality not run, claim unverified, source unread | the gap list |
| **press** | the text as it ships, as the stranger who has to act on it | leak and register findings |

Dispatch to fresh instances is the mechanism, not an optimisation: a self-run frame check is
composed inside the frame it is meant to test. Briefs in
[references/dispatched-passes.md](references/dispatched-passes.md).

Skipping a pass is allowed. Silently skipping it is not — name it as skipped in the verdict.

## Phase 2 — shape

Front-load the verdict, cut anything that does not change what the reader does, keep every
artifact and move only its placement. Shape only, after content is settled — a shape pass that
reaches content is how a capability table gets dissolved into paragraphs and the comment's
payload disappears.

## Verdict

```
ATTESTED phase 0 clean, no blocking finding from phase 1
ATTESTED WITH publishable, with named caveats carried INTO the artifact
BLOCKED phase 0 failure, or a phase 1 finding that invalidates the claim
NOT A RUN nothing was measured; there is no artifact to publish
```

`NOT A RUN` is legitimate and common. A run that could not execute its check produced no
evidence, and publishing the attempt with a disclaimer is worse than publishing nothing — the
disclaimer reads as hedging and the figure is kept anyway.

## Anti-patterns

| Bad | Good |
|---|---|
| Running phase 1 to decide phase 0 | Mechanical checks first; cheap and unarguable |
| Self-running the dispatched passes | Dispatch, or skip and say it was skipped |
| Attesting your own run | The gate is positional; an author attesting themselves attests nothing |
| Treating phase 0 items as advisory | Every one is a hard fail |
| `ATTESTED WITH` as a soft pass | The caveat goes *into the published artifact*, not just the verdict |
| Softening a check to fit the case in hand | If the new version could be satisfied by better prose alone, it is no longer the check |

## Related

- `mms-evidence` — produces the artifact this gates
- `mms-instrument-check` — prove the instrument fires before its output counts
- `mms-unmeasured-join` — audit the inference between the facts
- `mms-scope-of-search` — what a negative result is a fact about
Loading
Loading