Skip to content

feat(derivatives): reader-facing status client for a derived scope - #206

Merged
volod-vana merged 3 commits into
mainfrom
maciej/derivative-status-client
Sep 1, 2026
Merged

volod-vana merged 3 commits into
mainfrom
maciej/derivative-status-client

Conversation

@maciejwitowski

@maciejwitowski maciejwitowski commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Every derivative question helper the SDK ships today is the builder's. registerQuestion, getQuestion, waitForQuestion and askPersonalServer all target /v1/derivatives/questions, which needs a write session — and a write session needs a write:<scope> grant entry.

The app that reads the answer usually has no write entry at all. A consent flow gives it a bare read on the derived scope, so it can never open a session. All it can call is GET /v1/data/<derivedScope>, which returns 404 for three different situations:

  • the compute is still running
  • it failed, and the server will retry
  • it failed, and nothing more will happen

This adds the client for the route that answers that: GET /v1/derivatives/status?derivedScope= (personal-server-ts#238).

Reading the status

const status = await getDerivativeStatus({
  personalServerUrl: "https://ps.example.com",
  derivedScope: "coach.weekly",
  grantId,
  signer,
});
// { derivedScope, status, lastComputedAt, derivedVersion,
//   derivedCollectedAt, errorCode, retryAfterSeconds }

Authorization is the data read's: a live grant covering the derived scope sent as the signed grantId claim, or the owner with no grant at all. Nothing is served and nothing is charged, so a priced grant raises no 402 here.

The signature covers the bare path, /v1/derivatives/status, not the query. That is the same rule the data and lineage reads follow: per-scope authorization is enforced live on every request rather than pinned by the signature. Only the write path signs path and query, where a parameter decides what gets written.

Waiting for it

const settled = await waitForDerivativeStatus({
  personalServerUrl,
  derivedScope: "coach.weekly",
  grantId,
  signer,
  timeoutMs: 60_000,
});

Returns as soon as the scope is ready, or has failed with no retry pending. Two calls worth flagging:

  • A failed status is returned, not thrown. The reader branches on errorCode. Throwing would force a try/catch around the ordinary case of "it did not work".
  • The server's retryAfterSeconds beats pollIntervalMs. inference_unavailable is the one transient class and the server retries it at 1m/5m/30m. Nothing changes before that retry fires, so polling faster only spends requests. The remaining budget still caps each sleep.

isDerivativeStatusSettled is the same predicate, exported for callers running their own loop.

Error codes

DerivativeErrorCode is the closed vocabulary the server serves:

code meaning
inference_unavailable provider or relay failed (the server retries)
source_missing a source scope is deleted or has no local data
grant_invalid the registering builder's grant no longer covers
internal anything else, including a permanent provider 4xx

It is also added to DerivativeQuestionSchema, so the builder's own view carries the class. Nullish, so a Personal Server that predates the field still parses.

The view is lifecycle only. The question text, the source scopes, the question id, the registrar and the server's raw error string never reach the reader — a test asserts the served key set and that neither the source scope nor the question id appears anywhere in it.

Testing

npm run typecheck, npm run lint, npm run format:check, npm run test:coverage all pass. 1174 tests, 71 files, 0 failures. CI green on Node 20 and 22.

20 new tests cover the request shape (path-only signature, grant in the envelope, encoded query), each lifecycle state, the owner path with no grant, 403 for an uncovered scope and for a signer with no grant, 404 for a covered scope with no question, the disclosure assertion, transport and unparseable-body failures, and the wait loop (polls to ready, returns a terminal failure, waits through a retrying one, honours retryAfterSeconds, times out, aborts).

The mock Personal Server gained the route with the real authorization model — read grant or owner, never the write-session path — plus the precedence rule among duplicate registrations and the two new fields.

The request shape is checked against the server, not just the double: personal-server-ts signs its own status tests as uri: "/v1/derivatives/status" with grantId in the Web3Signed envelope, byte-for-byte what this client sends.

Merge order

Needs a Personal Server that ships the route. An older one answers 404 for the route itself, which surfaces as DerivativeQuestionNotFoundError — the same error as a covered scope with no question behind it. Consumers pin the Personal Server by exact version, so this is only usable after that release and a bump in each surface.

One repo note, unrelated to this diff

.husky/pre-push is broken on main. It calls install-evm-key-scan-hook.sh run, but the stub that replaced the old script (#194) accepts only install|status|uninstall and exits 2 on anything else — and status refuses the repo outright because husky sets core.hooksPath. Every local push fails until it is fixed.

This branch was pushed with --no-verify after running the hook's own checks by hand (typecheck and test:coverage, both green) and grepping the diff for key material. There is none: the test accounts come from generatePrivateKey() at runtime. CI's evm-key-scan passed independently.

@vercel

vercel Bot commented Sep 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
vana-console Ignored Ignored Sep 1, 2026 9:53am UTC
vana-rbac-auditor Ignored Ignored Sep 1, 2026 9:53am UTC
vana-vibes-demo Ignored Ignored Sep 1, 2026 9:53am UTC

Request Review

@maciejwitowski maciejwitowski added the codex-review Add to run the on-demand Codex review; removed automatically when it finishes label Sep 1, 2026
The question helpers are all the builder's: registerQuestion, waitForQuestion
and askPersonalServer each need a write session, which an app holding only a
bare read entry on the derived scope cannot open. That reader is the one the
answer is for, and until now it had nothing to poll: GET /v1/data/<scope>
answers 404 whether the compute is running, retrying, or finished failing.

getDerivativeStatus reads the Personal Server's reader-facing view
(GET /v1/derivatives/status?derivedScope=). It authenticates like a data read
— a live grant covering the derived scope, or the owner — signs the bare path
with grantId in the Web3Signed envelope (the query is authorized live, per
scope, on every request), and nothing is charged, so a priced grant raises no
402.

- waitForDerivativeStatus polls until the scope is ready or has failed with no
  retry pending, and honours the server's retryAfterSeconds over
  pollIntervalMs: nothing changes before the retry the server has scheduled,
  so polling faster only spends requests. A failed status is returned, not
  thrown, so the reader branches on errorCode.
- isDerivativeStatusSettled is the same predicate for callers that poll
  themselves: ready, or failed with no retry pending.
- DerivativeErrorCode is the closed vocabulary the server serves
  (inference_unavailable, source_missing, grant_invalid, internal), added to
  DerivativeQuestionSchema too so the builder view carries it. Nullish, so a
  Personal Server that predates the class still parses.
- The mock Personal Server answers the route with the real authorization
  (read grant, not write session), the real precedence among duplicate
  registrations, and the errorCode/retryAfterSeconds fields.

Needs a Personal Server that ships the status route; an older one answers 404
for the route itself.
The test leaned on a 5ms budget to cut the server's 60s retry short, so a
slower machine blew the deadline during the first poll and the wait timed
out (CI node 20 and 22). Fake timers instead: poll one sees the retrying
failure, the clock advances the full retry, poll two sees the answer.
…e waiting

Two findings from an external review pass on the wait loop.

The server's retryAfterSeconds is now the cadence outright rather than a floor
under pollIntervalMs. A caller asking for a 60s interval against a 5s retry sat
for 60s on an answer that had existed for 55 of them, which contradicted the
documented rule that the server decides: retryAfterSeconds is when the next
compute happens, so asking sooner sees nothing new and asking later sits on a
result.

The loop also spent one request past its own budget: the deadline was checked
after the poll, so a sleep that consumed the remaining time was followed by
another read before the timeout was raised. It now gives up as soon as the
budget cannot cover the next cadence, which is the same moment expressed
honestly and one request cheaper.

getDerivativeStatus takes a signal and passes it to fetch, so an abort during a
stalled request no longer waits for the transport to notice.
volod-vana added a commit to vana-com/personal-server-ts that referenced this pull request Sep 1, 2026
…ansient retry (#238)

A builder registers a question, the Personal Server computes it, and the
answer lands in a derived scope. Until it lands, reading that scope
returns 404.

The problem is that 404 means three different things:

- the compute is still running
- it failed, and the server will retry
- it failed, and nothing more will happen

A client that cannot tell them apart either polls forever or gives up on
an answer that was seconds away. The party hit hardest is the one the
answer is for: a consent-flow app holds only a bare read entry on the
derived scope, so it cannot open a write session and cannot use any of
the question routes.

## The route

`GET /v1/derivatives/status?derivedScope=<scope>`

```json
{
  "derivedScope": "coach.weekly",
  "status": "failed",
  "lastComputedAt": "2026-08-31T09:12:44.000Z",
  "derivedVersion": 3,
  "derivedCollectedAt": "2026-08-31T09:12:44Z",
  "errorCode": "inference_unavailable",
  "retryAfterSeconds": 300
}
```

Authorization is the data read's: a live grant covering the derived
scope, or the owner. No x402 challenge and no access receipt, because
nothing is served and nothing is charged — the same bar as the lineage
read. Auth runs before the store lookup, so a caller who cannot read the
scope cannot probe which scopes have questions behind them.

`retryAfterSeconds` is the point of the whole thing: it separates a
failure still being worked on from one that is over. While a retry is
actually running there is no scheduled time, so a short poll hint (5s)
is served rather than `null`, which would read as terminal.

## Retry

`inference_unavailable` is the one transient failure class, and it now
retries at 1m, 5m, 30m before giving up. Everything else stays failed
until a source change or an explicit recompute, exactly as before.

The chain lives in memory. A restart drops it, and #236's boot
reschedule sweeps `pending` and `stale`, not `failed`, so a question
caught mid-backoff waits for a source change or an explicit recompute
instead of resuming.

## What the reader is not told

The question text, the source scopes, the question id, the registrar and
the server's raw `error` string stay owner-only. `errorCode` is a closed
vocabulary for exactly that reason — a stored message like "source scope
oura.sleep is deleted" must never reach a grantee:

| `errorCode` | meaning |
| --- | --- |
| `inference_unavailable` | provider or relay failed (retried) |
| `source_missing` | a source scope is deleted or has no local data |
| `grant_invalid` | the registering builder's grant no longer covers |
| `internal` | anything else, including a permanent provider 4xx |

`grant_invalid` needed the same care as the raw string. Only a
builder-registered question runs the live grant re-check, so serving
that class to any reader would tell them a builder registered the
question. The owner and the registrar still get it — they are the two
who can act on it — and everyone else reads `internal`.

When several questions write the same derived scope, the most optimistic
true state answers: `ready`, then `stale`, then `pending`, then
`failed`, newest first within a class. Serving data is
registration-agnostic, so a duplicate that never wrote anything must not
report away an answer the scope has.

## Rebased onto #236

Opened before per-question recompute policy landed. The three conflicts
were all "we both added a field in the same place" — `QuestionRecompute`
beside `QuestionErrorCode`, both type exports, both columns in the store
INSERT — and both sqlite migrations still run at store init.

One interaction is worth stating out loud: `recompute: "snapshot"` is
checked in `markSourceChanged` only, so a snapshot question that fails
transiently still gets the retry chain. That is the intended reading. A
retry finishes the compute the registration itself asked for; it is not
a source change, and the alternative leaves snapshot questions stuck at
`failed` for any inference blip that happens to hit their one compute.

## Testing

`npm run lint`, `npm run build`, `npm test` → **1491 passed, 112/112
files**.

New coverage: the status route in `api.test.ts` (body contract, auth
before lookup, the disclosure assertions, status precedence,
`retryAfterSeconds`), failure classification in `compute.test.ts`, the
retry state machine in `scheduler.test.ts` (backoff, give-up, source
change supersedes, stop clears), the `error_code` round-trip and
in-place migration in `question-store.test.ts`, and three composed-app
tests through the real `api-auth` with a consent-shaped grant.

Review: two adversarial rounds before opening (12 findings, all fixed
test-first — worst was the route dropping the caller's `?grantId=`,
refusing the exact credential shape the SDK sends), plus an external
pass that caught the `grant_invalid` disclosure above.

## Follow-ups

- The retry chain is in-memory (above).
- Surfacing failed questions to the owner in the personal-server UI is
unity-surfaces work.
- The SDK client for this route is vana-com/vana-sdk#206. Without it
nothing calls this: every existing SDK question helper needs a write
session the reader cannot open.

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Volod <141863857+volod-vana@users.noreply.github.com>
@volod-vana
volod-vana merged commit 10033b1 into main Sep 1, 2026
8 checks passed
@volod-vana
volod-vana deleted the maciej/derivative-status-client branch September 1, 2026 18:07
github-actions Bot pushed a commit that referenced this pull request Sep 1, 2026
## [3.22.0](v3.21.0...v3.22.0) (2026-09-01)

### Features

* **derivatives:** reader-facing status client for a derived scope ([#206](#206)) ([10033b1](10033b1))
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 3.22.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-review Add to run the on-demand Codex review; removed automatically when it finishes released

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants