Skip to content

A child's settle receipt records iterations:1 with 4.93M cacheRead, so post-stream-break work is unaccounted and misread as fabrication #1309

Description

@drewstone

What is wrong

A child's settle receipt records iterations: 1 alongside token totals that one model call cannot produce. The supervisor sees the child frozen for over an hour, then receives a receipt for work it never observed.

Measured on zkp-frontier-prover-a-20260918b, child :s10, Runtime 0.238.0, Tangle sandbox placement, kimi-k3:

{"iterations":1,
 "tokens":{"input":5029062,"output":55691,"freshInput":6931,
           "cacheRead":4926720,"cacheWrite":0,"cacheBreakdownKnown":false},
 "usd":0,"usdKnown":false,"ms":4303277}

iterations: 1 against cacheRead: 4,926,720. A single call cannot read 4.93M cached tokens; that is on the order of 25 to 150 calls at any plausible context size. The child ran for 4,303,277 ms (71.7 min) and its observable stream stopped early in that window.

Why it matters, and why the current reading is wrong

This run's own records interpret a frozen freshInput plus a long final message as proof that the child fabricated its report. That interpretation does not hold, and the receipt is the reason:

  • freshInput is frozen at the last streamed step in every child by construction, healthy and broken alike, so it carries no information about post-break activity.
  • The detection rule built on it scores a separately-confirmed fabricator at 0.49, inside the healthy band.
  • The number that does discriminate is cacheRead: in healthy children the recorded value matches the streamed sum; here 4.88M of cached reads never appeared in the stream.

So the evidence is consistent with a child that kept making model calls after its progress stream died, invisible to the supervisor, and whose work was then attributed to one iteration. Whether that work was real or not is a second question. The defect is that the receipt cannot tell anyone which, and the iteration count is provably wrong either way.

Downstream consequences:

  • Budget accounting. A pool that debits per iteration under-debits by the same factor. Related to budget: a settled turn with tokensKnown:false debits zero from the conserved pool #1252, which covers tokensKnown:false debiting zero; this is the case where tokens ARE known and the iteration count is not.
  • Stall detection. The supervisor's idle view and the child's real activity diverge with nothing reconciling them at settle.
  • Forensics. Operators are drawing fabrication conclusions from receipt fields that are structurally uninformative.

What would fix it

At settle, reconcile the recorded totals against the streamed steps and record the divergence rather than discarding it: the streamed sums, the recorded sums, and an explicit unobserved remainder. An iteration count that cannot be reconciled with the token totals should be recorded as unknown rather than as 1.

Repro

Any sandbox child whose stream breaks before done. The retained case above is on this host at ~/webb/_wt/lab-provider-160/pursuits/zkp-frontier-prover-a-20260918b, spawn-journal.jsonl, child :s10. The two healthy siblings in the same run show recorded totals matching their streamed sums.

Related: #1252, #1253, agent-sdk#334.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions