From 394269e6f30552c23aa8b64c64d16da3336ec64d Mon Sep 17 00:00:00 2001 From: David Khourshid Date: Wed, 14 Oct 2026 00:34:39 +0200 Subject: [PATCH] plans: prioritized plan to close the LangGraph gaps Seven gaps (per-state update stream, time travel, memory store, subgraph mapping, published test fake, the September sweep's library gaps, durable store recipes), each with current state, design, touches, effort, risks, and the examples and docs that exercise it. Grouped into three PR-sized batches with decisions recorded. Planning only; no library code. --- plans/README.md | 6 + plans/langgraph-gaps.md | 265 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 271 insertions(+) create mode 100644 plans/langgraph-gaps.md diff --git a/plans/README.md b/plans/README.md index 80e9296b..8e1ef6f0 100644 --- a/plans/README.md +++ b/plans/README.md @@ -3,3 +3,9 @@ [Full audit](portable-agent-audit.md), based on `4c39d3681ded940b2d824a10fa1e5266eaa17729` (`origin/next`, fetched September 7, 2026). Core findings are recommendations, not implemented fixes. Three new policy examples and their tests/docs are implemented in this branch. Start with correctness findings 01–08 before API redesign. The report distinguishes reproductions, source-only findings, and design options. + +# LangGraph gaps plan + +[Full plan](langgraph-gaps.md), based on `b884c736` (`origin/next`, fetched October 13, 2026). Decisions taken with David on October 13–14, 2026. + +Seven gaps versus LangGraph, each with current state, design, touches, effort, and risks, grouped into three PR-sized batches. Planning only; nothing is implemented. Sweep gap 6b is dropped as XState v6 semantics, 6f and a `forkThread` helper were dropped as redundant, and a bundled durable store is rejected in favor of conformance-tested recipes. diff --git a/plans/langgraph-gaps.md b/plans/langgraph-gaps.md new file mode 100644 index 00000000..73c02a45 --- /dev/null +++ b/plans/langgraph-gaps.md @@ -0,0 +1,265 @@ +# Closing the LangGraph gaps: prioritized plan + +Date: 2026-10-13. Base: `origin/next` at b884c736. Worktree: `~/.codex/worktrees/agent/langgraph-gaps-plan` (branch `davidkpiano/langgraph-gaps-plan`). Planning only; no library code. + +## Governing constraints + +- `docs/scope.md` ownership table: long-term and semantic memory is "Database or memory library"; integration point is machine `input` plus `actors`. Snapshot persistence, queues, HTTP/SSE/WebSocket are host-owned, reached through "XState snapshots, callbacks, and emitted events". A bundled "memory store" is named as the anti-pattern (scope.md:12). Non-goals list vector stores explicitly. +- `docs/roadmap.md`: "Stream adapters" and "Better interaction typing" and "Stronger request/output/event inference" are in scope. XState++ rule: forward framework capabilities, no parallel adapter layers. +- `CONTRIBUTING.md:7-16`: core only when a machine cannot express portable intent or hand it to an arbitrary host without framework glue. "Observing a run without coupling it to one vendor" and "testing control flow deterministically" are core criteria. +- Ruling 2026-09-24 (memory `langgraph-examples-port`): no published model mock, demo always needs a key, `createScriptedExecutors` / `Scripted*` were deliberately removed from `@statelyai/agent` and `@statelyai/agent/testing`. `runSeam` stayed. +- Ruling 2026-09-04 (memory `log-is-source-of-truth`): log is truth, snapshot is a verified cache. + +## Decisions (walkthrough, 2026-10-13, revised after the "do we really need this" audit) + +Taken with David item by item. These override the per-gap "proposed design" text where they differ. The audit asked, for each item, whether XState or existing core already provides it; three items fell and two shrank. + +| Item | Decision | +| -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 1 streaming | `streamAgent(machine, options & { modes })` sibling of `runAgent`, returning `AsyncIterable & { result }`. **No new trace fields.** `updates` (`changed`, `previousValue`, shallow `contextPatch`) is computed inside `streamAgent` by holding the previous snapshot; `onTrace` consumers who want it do the same five lines. The shallow patch converts 1:1 to AG-UI `StateDelta` JSON Patch ops in the tanstack bridge. | +| 2a choices with payload | **Schema-driven, MCP-shaped.** No `choices` list. `getInteraction` emits, per accepted event, a JSON Schema obtained through the Standard JSON Schema interface (`~standard.jsonSchema.input()`, implemented by zod 4, valibot, arktype; `undefined` when the schema library lacks it). No converter and no zod dependency in core (zod is a devDependency only). Choices are enum fields, titled via `oneOf: [{ const, title }]`. `message` defaults to XState's native `state.description`; `meta.interaction.label` remains only for context interpolation. Helpers: `toElicitationRequest(interaction, eventType?)` → MCP `{ message, requestedSchema }`, defined for one event (auto when exactly one is accepted; multi-event states yield one request per event and the host picks), and `eventFromElicitationResult(snapshot, result)`. MCP transport and the AG-UI `Interrupt.responseSchema` bridge live in examples; the AG-UI one is promoted to `./tanstack-ai` with the stream bridge later. | +| 2e text field | **Absorbed by 2a.** Replies are `eventFromInteraction(snapshot, { type, content })`, validated by the event schema (mirrors MCP `accept`/`content` and AG-UI `resume.payload`). `{ text }` stays one release as a deprecated alias mapping to `textEvent`'s single string field. | +| 2h unparsed decision | Executor returns `{ unparsed: { reason, raw } }`; recorded as a `DecisionAttempt` with `failure: "unparseable"` (fourth variant beside `unknown-event`, `invalid-payload`, `rejected-by-guard`), re-prompted, counts toward the budget. Throws stay transport failures. | +| 2c `ContextOf` | Ship. Five-line type fix plus a type test. | +| 2d persisted snapshot | Ship. Throw a clear `AgentError`. | +| 2f `agent.actors` | **Dropped.** User-defined actors are values the user already holds; XState's `setup()` exposes no `.actors` either; request-built actors are already on `agent.requests`. | +| 3 fork lineage | `forkEventLog` and `store.fork` mint a fresh `executionId` and record `metadata.forkedFrom: { threadId, executionId, upToIndex }`. Fixes the `callKey` collision. Pre-beta. | +| 3 helper | **Docs only.** `store.fork` + `replay(prefix)` + `runAgent({ snapshot: edited })` already do it and the snapshot path already mints a fresh id. Three-call recipe in `docs/event-log.md` ("Branch with edited state"), plus `examples/time-travel`. No `forkThread` export; revisit on demand. | +| 4 testing | Docs recipes for all three adapters plus `settings(request, info)` on all three adapters (verified necessary: text executors receive `AgentTextRequest`, which carries no invoke id; `info.requestId` is the only channel). No scripted-executor export (ruling stands). Also the `createOpenAiExecutors(machine, { client })` overload for parity. | +| 5 subgraph | **Docs only, leading with nested states.** A LangGraph subgraph sharing state keys is a nested compound state (same context, no mapping). A subgraph with its own schema is an invoked child machine (`input` + `onDone`). `docs/multi-agent.md` gets a "Subgraphs" section with that two-row table and both snippets; child machine is the escalation (own schema, own budget, reuse, parallel instances). No new example; `hierarchical-teams` stays the child-machine example. No rename. | +| 6 store | Pattern only. No interface. | +| 7 durable log store (added 2026-10-14) | **No bundled store; the conformance suite is the product.** Shipping SQLite would contradict roadmap.md (storage is framework-owned) and open a queue (Postgres, Redis, D1...). Instead: store recipes as examples that pass `assertEventLogStoreConformance` in CI, a docs page that leads with "implement four methods, run the suite", and a comparison row (LangGraph bundles `SqliteSaver`; we bundle the contract and the test for yours). | +| Order | A, then B, then C. | + +### Prior art consulted for 2a/2e + +- MCP elicitation (spec 2025-11-25): `elicitation/create { message, requestedSchema }`, flat JSON Schema, primitives plus enums; titled single-select via `oneOf: [{ const, title }]`, multi-select via `array.items.anyOf`; response `{ action: "accept" | "decline" | "cancel", content }`. https://modelcontextprotocol.io/specification/2025-11-25/client/elicitation +- AG-UI 1.0 (2026-09-30): `RunFinished.outcome: { type: "interrupt", interrupts: Interrupt[] }` with `Interrupt { id, reason, message?, responseSchema?: JsonSchema, ... }`; resume via `RunAgentInput.resume: [{ interruptId, status, payload }]`; `StateDelta` is RFC 6902; `Custom { name, value }`. https://docs.ag-ui.com/concepts/interrupts +- Adaptive Cards `Action.Submit.data` and Slack Block Kit button `value`: button-list-with-payload lineage; not adopted. +- A2UI v0.9 (Google) and json-render (Vercel): component-tree rendering protocols; out of scope for core (UI is host-owned), a host can map the elicitation schema into either. + +### Effect on batches + +- Batch A: 2a is an interaction-descriptor redesign (M) rather than an additive list; split it into its own PR (A2) if review wants it separate. A1 = 2c, 2d, 2h, `settings(request, info)`, OpenAI overload, docs recipes. 2f dropped. +- Batch B shrinks: `streamAgent` plus the two example rewrites only; no trace-envelope change, so `src/serialize-trace-event.test.ts` is untouched. +- Batch C shrinks: lineage fix in `forkEventLog`/`store.fork` + conformance, the event-log docs section, `examples/time-travel`, subgraph and memory docs. No `forkThread`, no subgraph example. + +## Verification caveat + +The app worktree used for repros has a stale install: xstate 6.0.0-alpha.57 and ai 7.0.113, while `package.json` pins alpha.64 and ai ^7.0.133. Every REPRODUCES verdict below was obtained on alpha.57; the alpha.64 type declarations in `~/.codex/worktrees/agent/run-agent-sugar/node_modules` were read by hand to confirm gap 6b. Repro files: `/private/tmp/claude-501/.../scratchpad/repro/` (`b.ts`, `b2.ts`, `cf.ts`, `run.ts`, `g.ts`). Re-run `cf.ts` and `run.ts` after `pnpm install` on the implementing branch before coding. + +## Dropped or reclassified + +| Item | Verdict | Why | +| ------------------------------------------ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 6b guarded `always` arrays | **Drop.** Not a setupAgent bug | XState v6 has no `guard:` key and no transition arrays (`types.v6.d.ts`, alpha.57 and alpha.64). Fails identically with `events`, with plain `setup()`, and with `createMachine`. `docs/machines.md:301` already documents "guards are a return value". Only leftovers: two v5-style arrays in `src/workflow-config-oracle-spec.test.ts:84,99` (test fixture, not user-facing). | +| 6d `parseAgentEvent` needs a live snapshot | **Mostly fixed; one footgun** | `parseAgentEvent(machine, event)` is accepted, documented and tested (`src/run-agent.test.ts:4766-4816`, `src/events.test.ts:46`). Remaining: a _persisted_ snapshot is rejected at type level but at runtime returns the payload **unvalidated** (`src/events.ts:194-199` reads `source.machine`, absent on persisted). Keep as a small fix. | +| 6f no `setup.actors` | **Half fixed** | `DoneActorEventOf` works (`src/setup-agent.ts:815`). User-supplied `actors` are still not re-exposed. Keep the small half. | +| 6g parallel actors share a request name | **Core fixed; adapter gap remains** | Executors receive `info.requestId` (durable invoke id with the state path, e.g. `0.(machine).left.go` vs `right.go`; `src/text-logic.ts:781-797`, `src/run-agent.ts:1190,1273`). The AI SDK/TanStack `settings(request)` hooks drop `info` (`src/ai-sdk/index.ts:120-141`), so provider-level mocks cannot key on it. Reclassified as the adapter change under gap 5. | +| 4 example rename | **Reject rename** | `hierarchical-teams` is the LangGraph tutorial port and is linked from patterns.md:14, multi-agent.md:29, langgraph-comparison.md:171. `machine-as-tool` is about embedding a machine in a host tool call via `persist()` handles, not parent/child mapping. Docs section instead. | +| 3 store interface in core | **Neither; documented pattern** | See gap 3. | + +## Ranked gaps + +### 1. Per-transition update stream (LangGraph `values` / `updates` / `messages` / `custom`) + +**Current state.** All four kinds already flow through one ordered callback, `onTrace` (`src/run-agent.ts:170-243`): + +- `values` = `machine.transition { snapshot, event }`, emitted by `commit` (`run-agent.ts:2618-2625`) after every root transition, including no-ops (`:2977-2983` flags `next === state` only for `result.ignored`). +- `messages` = `stream.chunk { request, chunk: string }`, emitted by core's `bindTextLogic` when the adapter calls `info.onChunk` (`run-agent.ts:1141-1187`; AI SDK `src/ai-sdk/index.ts:417`; TanStack `src/tanstack-ai/index.ts:478`). Text-only; no reasoning or tool-input deltas. +- `custom` = `emit { event }` for root-machine `enq.emit` (`run-agent.ts:2525-2532, 2572-2581`), plus the typed `on: { [type]: handler }` sugar (`:507`). Child-machine emits are not surfaced (`:2579`). +- `updates` has no equivalent: no previous snapshot, no context diff. +- Sugar: `onTransition`, `onChunk`, `onResult`, `on`, `inspect` (`:474-527`). No async-iterable anywhere in core. `examples/tanstack-ai-stream/index.ts:188-214` hand-rolls a push queue (`createChunkQueue`) to turn callbacks into an iterable; `examples/ai-sdk-ui-stream/index.ts:159-210` wires the same callbacks into `createUIMessageStream`. Neither example uses `emit` or `onTrace`. +- `AgentEventLogStore` has no append notification (`src/event-log-store.ts:20-58`); `onEvent` (`run-agent.ts:464`) is in-process only. + +**Proposed design** (smallest; provider-free; keeps `onTrace` as the single emission point per `run-agent.ts:1069-1080`): + +1. Enrich the existing `machine.transition` trace variant rather than add a new event bus: + ```ts + | { type: "machine.transition"; snapshot; event; + changed: boolean; // next !== prev + previousValue?: StateValue; // undefined on run.start / restore + contextPatch?: Record; // shallow top-level diff, Object.is per key; removed keys as undefined + } + ``` + `commit(snapshot, event, prev)` takes `prev` from `transition()` (`:2960`) and `undefined` from `start` (`:2956`). `JsonSerializableTraceEvent` is derived, so JSON projection follows automatically; the pinned shape test in `src/serialize-trace-event.test.ts` needs updating. Optionally pass the same `{ changed, previousValue, contextPatch }` as a third argument to `onTransition`. +2. One async-iterable entry point, a sibling of `runAgent`, in a new `src/stream-agent.ts`: + ```ts + function streamAgent( + machine, + options: AgentRunOptions & { modes?: AgentStreamMode[] }, + ): AsyncIterable> & { result: Promise> }; + type AgentStreamMode = "values" | "updates" | "messages" | "custom" | "trace"; + type AgentStreamPart = + | { type: "values"; snapshot } + | { type: "updates"; value; event; contextPatch } // skipped when !changed + | { type: "messages"; request; chunk } + | { type: "custom"; event } + | { type: "trace"; event: AgentTraceEvent }; // default when modes omitted + ``` + Internals: `runAgent(machine, { ...options, onTrace: tee })` pushing into an unbounded queue (lift `createChunkQueue` from the tanstack example) with a filter per mode. Caller-supplied `onTrace`/`onChunk`/`onTransition`/`on` are chained, not replaced. `onIdle` passes produce multiple `run.start`/`run.end` pairs with distinct `runId`s; document rather than merge. + Decision: this belongs on `runAgent`'s level, not the runtime. `createAgentRuntime` users already own the loop and have `onTrace`; adding `events()` to `AgentRuntime` would duplicate that. `runAgent` gets no new `onUpdate` callback: `onTransition` plus the new fields is that callback. +3. Wire protocols stay in adapter subpaths (scope.md:41). After (2) lands, `examples/ai-sdk-ui-stream` and `examples/tanstack-ai-stream` are rewritten on `streamAgent` (delete `createChunkQueue`). Promoting them to `agentRunToUIMessageStream` / `agentRunToAgUiStream` exports under `./ai-sdk` and `./tanstack-ai` is a follow-up, in scope under roadmap "Stream adapters", not part of this PR. + +**Touches.** `src/run-agent.ts` (trace union, `commit`, `onTransition` signature), new `src/stream-agent.ts`, `src/index.ts` export, `src/serialize-trace-event.test.ts`, new `src/stream-agent.test.ts`, `docs/observability.md` ("Live progress" section lists `streamAgent` and the new fields; trace-field list at :18), `docs/langgraph-comparison.md` construct-map row "Stream events" → `streamAgent` modes, the two stream examples. + +**Effort.** M (code S for the fields, S–M for `streamAgent`, S for example rewrites). + +**Risks.** `contextPatch` is shallow; nested mutations show the whole top-level key. Say so. A no-op transition still emits `values` (LangGraph does too) but must be filtered from `updates`. Unbounded queue if the consumer stalls; acceptable for an alpha, document. Child-machine states only reach `inspect`, not `values`; parity with LangGraph `subgraphs=True` is explicitly deferred. + +**Exercised by.** `examples/ai-sdk-ui-stream`, `examples/tanstack-ai-stream`, `examples/parallel-streams` (interleaved `messages` keyed by `request.id`), `docs/observability.md`, `docs/langgraph-comparison.md`. + +### 2. Library gaps from the September sweep (6a, 6c, 6d, 6e, 6f, 6h) + +Each verified on current `next`; 6b dropped, 6g folded into gap 5 (see table above). + +| Gap | Current state | Design | Touches | Effort | +| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ | +| 6a choices with payload | `AgentInteractionDescriptor.events` is keyed by event type (`src/interaction.ts:48`), so one choice per type; `getInteraction` flattens with `Object.entries` (`:181`), `eventFromInteraction` matches `find(type === choice.type)` (`:254`). `getInteraction` output confirmed: `events: [{ type: "PICK", label, event: { size: "S" } }]` only. | Add a list form alongside the map: `choices?: Array<{ id?: string; type: TEvent; label; event?: payload }>`; `events` map stays for the one-per-type case. `getInteraction` flattens both and assigns a stable `id` (`${type}#${i}` when absent); `eventFromInteraction(snapshot, { choice: id })` resolves by id, `{ type }` keeps working when unambiguous. Validate in `interactionMetaSchema`. | `src/interaction.ts`, `src/interaction.test.ts`, `docs/human-in-the-loop.md:52`, `examples/human-in-the-loop`, the example that motivated it (the `ACCUSE { seat }` game) | M | +| 6c `ContextOf` is never | `SchemaAt` only matches `{ schemas: { context } }` (`src/type-helpers.ts:14-18`); a `createAgentSchemas` pack has `context` at top level. `EventOf`, `MetaOf` same (`:36-55`). `examples/river-crossing/index.ts:358-365` works around with `z.infer`. | Second branch in `SchemaAt`: `T extends Record ? S : never`. Same for `EventOf`/`MetaOf`. | `src/type-helpers.ts`, `expectTypeOf` cases in `src/utils.test.ts`, river-crossing cleanup | S | +| 6d persisted snapshot silently unvalidated | `src/events.ts:194-199` reads `source.machine`; a persisted snapshot has none, so the payload passes through unchecked at runtime (type-level TS2345 only). | Throw `AgentError` ("pass the machine or a live snapshot") when the source has neither `provide` nor `machine`. Keep "pass the machine" as the documented host path. | `src/events.ts`, `src/events.test.ts`, one sentence in `docs/human-in-the-loop.md` | S | +| 6e `text` field hardcoded | `eventFromInteraction(snapshot, { text })` builds `{ type: textEvent, text }` (`src/interaction.ts:232, :252`). A textEvent schema `{ body }` throws "expected string, received undefined". | Prefer inference: if the textEvent schema has exactly one string field, use it; otherwise require `textField` on the descriptor (`interaction.ts:45-50`, validated at `:127`). Default stays `"text"` so `interaction.test.ts:62` holds. | `src/interaction.ts`, tests, `docs/human-in-the-loop.md` | S | +| 6f `setup.actors` | Request actors exposed as `agent.requests.` (`src/setup-agent.ts:815`); user `actors` not re-exposed (`:643-644`). | Add `readonly actors: TActors` to `SetupAgentResult` and the `Object.assign` at `:812-816`. Guard the name against the builtins. | `src/setup-agent.ts`, `src/utils.test.ts` (`DoneActorEventOf`) | S | +| 6h decision "did not parse" | `AgentDecisionExecutor` returns `{ event, reason?, usage? }` only (`src/decision.ts:549-558`); executor awaited outside the retry `try` (`:628-641`); `DecisionAttempt.failure` has no parse variant (`:484-486`). Both adapters throw a plain `Error` on no/unknown tool call (`ai-sdk/index.ts:450-452`, `openai/index.ts:792`), which escapes retries and lands in `onError` after one call. | Executor may return `{ unparsed: { reason: string; raw?: unknown } }` instead of `event`; `resolveDecision` records a `DecisionAttempt` with `failure: "unparseable"`, feeds the reason into the next attempt's prompt like an illegal choice, and counts toward the attempt budget so it ends in `decision-exhausted`. Adapters switch to the new channel. Thrown errors keep today's behaviour (transport failure). | `src/decision.ts`, `src/decision.test.ts`, `src/ai-sdk/index.ts`, `src/openai/index.ts`, `src/tanstack-ai/index.ts`, `docs/decisions.md` | S–M | + +**Risks.** 6a and 6h change public shapes; both additive. 6e inference can be surprising when a schema later grows a second string field; the explicit `textField` escape hatch covers it. 6d turns a silent pass into a throw, which is the point, but note it in the changeset. + +**Exercised by.** `examples/human-in-the-loop`, `examples/review-tool-calls`, `examples/river-crossing`, `examples/agentic-rag` (decision with an unparseable answer scripted at the provider), `docs/human-in-the-loop.md`, `docs/decisions.md`. + +### 3. Time travel: fork from a checkpoint, edit state, resume as a branch + +**Current state.** More exists than the gap statement assumes: + +- Prefix replay: `replay(machine, entries.slice(0, n))` is a pure fold, documented as time travel (`src/event-log.ts:915-963`, `docs/event-log.md:98`). +- Fork: `forkEventLog(entries, upToIndex)` (`event-log.ts:734`) and `store.fork({ threadId, newThreadId, upToIndex })` (`event-log-store.ts:52`, conformance `event-log-store-conformance.ts:245-338`), documented at `docs/event-log.md:139-147`. +- Edit and resume: passing only `snapshot` restores via `machine.restoreSnapshot`, writes a fresh init-with-snapshot entry and mints a new `executionId` (`run-agent.ts:2747, 2857-2860`); `agentMeta` is checked only when `events` is also passed (`:2772`), so an edited `replay(...).persistedSnapshot` is accepted. Documented only as "Resuming from a snapshot with no log also yields a self-contained log" (`docs/persistence.md:93`). +- Nothing in `examples/` forks or replays except the Cloudflare store implementation (`examples/cloudflare-agent-host/event-log-store.ts:162`). +- **Latent bug.** A fork copies the init entry and so inherits the parent's `executionId` by design (`event-log.ts:640-659`). The idempotency key is `${executionId}:${siteId}#${n}` (`run-agent.ts:2888`). The first new model call on a forked branch therefore collides with the parent's call at the same site and occurrence, so a host that caches provider results by `callKey` (hosts.md idempotency) replays the parent's answer on the branch. The snapshot-edit path avoids this only because it mints a new id. + +**Proposed design.** Mostly docs, plus one small helper that also fixes the collision. + +1. `docs/event-log.md` gets a "Branch with edited state" section showing the three-line recipe (replay prefix → spread context → `runAgent(machine, { store, threadId: newId, snapshot: edited, event })`) and the rule "snapshot only at a quiescent index" (persistence.md:91). +2. Helper in `@statelyai/agent/log` (`src/event-log.ts`, re-exported from `src/log/index.ts`): + ```ts + forkThread(store, machine, { threadId, newThreadId, upToIndex, update?: (persisted) => persisted }): Promise + ``` + Without `update`: `store.fork`, then rewrite the new thread's init entry with a fresh `executionId` and `metadata.forkedFrom: { threadId, executionId, upToIndex }` (mirrors `migratedFrom` on the version bridge, `run-agent.ts:2847-2855`). With `update`: replay the prefix, apply the edit, seed the new thread with a single `initEntry(machine, { snapshot }, { metadata })`. + Decision to make explicit in the PR: whether `forkEventLog` / `store.fork` themselves should stop inheriting `executionId`. Recommendation: yes, mint a fresh one and carry lineage in `forkedFrom`, because the inherited id is what makes `callKey` collide. This changes a documented statement (`docs/event-log.md:141`) and the conformance suite. +3. New example `examples/time-travel` (or `branch-and-edit`): run to idle, fork at an earlier index, edit a context field, resume the branch, show both threads' logs diverge; test asserts the callKeys differ. + +**Touches.** `src/event-log.ts`, `src/event-log-store.ts` (+ in-memory impl), `src/event-log-store-conformance.ts`, `src/log/index.ts`, `docs/event-log.md`, `docs/persistence.md`, `docs/langgraph-comparison.md` (row for "time travel / `update_state`"), new example + test. + +**Effort.** S for docs; M with the helper and the executionId change. + +**Risks.** Changing fork lineage semantics touches the Cloudflare host store in `examples/`. Editing a snapshot bypasses the machine's typed transitions; the docs must say the edited snapshot is unverifiable against the parent log and starts a new lineage. Keep the `update` callback on the persisted snapshot, not a context-only patch, so parallel state values and children can be edited consistently (or rejected). + +**Exercised by.** new `examples/time-travel`, `examples/cloudflare-agent-host` (store fork), `docs/event-log.md`, `docs/persistence.md`. + +### 4. Published test fake (gap 5) + +**Current state.** + +- `@statelyai/agent/testing` already exists and is published (`package.json` exports, `tsdown.config.ts`): `lintAgentMachine`, `assertAgentMachine`, `canReach`, `explorePaths`, `simulateAgent` (model-free, script keyed by invoke `src`), `matchesTrajectory`, `runSeam` (`src/testing/index.ts:9-49`). +- `runSeam` already contains a name-keyed scripted text executor (`src/seam.ts:322-405`, `SeamRef.occurrence` picks the Nth call). +- `examples/mock-model.ts` (`MockLanguageModelV3` behind the real `createAiSdkExecutors`, tagged via `settings → providerOptions.mock.id`, `:201-209`) and `examples/mock-judge.ts` are the only shared doubles; 37 example tests use them; CONTRIBUTING.md:34 says no other harness. +- Docs: `docs/evals.md:5-145` "Testing without a provider" has the AI SDK `MockLanguageModelV3` recipe and a plain-function executor recipe keyed on `request.name`; `docs/models-and-providers.md:195` duplicates the first. **No OpenAI or TanStack recipe exists** (the only OpenAI double is `stubClient` in `src/openai/index.test.ts:20`). +- Core tests hand-write ~150 inline executors; the only local helper is `src/machines/machines.test.ts:50-68`. +- Adapter `settings` hooks receive `(request)` only (`src/ai-sdk/index.ts:120-141`, `src/tanstack-ai/index.ts:595-610`, `src/openai/index.ts:379-398`), so a provider-level mock cannot key on `info.requestId`. That is the surviving half of gap 6g. + +**Options considered.** + +- A. Docs-only recipes per adapter plus a "parallel requests" note. Respects the ruling fully. S–M. +- B. `createScriptedExecutors` in `/testing`. Not a model mock, but it is the exact export removed on 2026-09-24 and near-duplicates `runSeam`'s script. Requires an explicit reversal; rejected here. +- C. Widen `settings(request, info?: AgentRequestExecutorInfo)` on all three adapters, so provider-level mocks (and `examples/mock-model.ts`, unpublished) can key on `info.requestId`. A real host feature (per-call settings by invoke site), nothing mock-shaped published. S. + +**Recommendation.** A plus C. Concretely: + +1. `docs/evals.md#testing-without-a-provider`: three subsections (AI SDK with the `providerOptions` tagging trick, OpenAI via a stubbed `client.chat.completions.create`, TanStack via a fake `AnyTextAdapter`), each a copyable snippet checked by `src/docs-integration-recipes.test.ts`; a "Parallel requests" note: route on `info.requestId` or `request.input`, never on array order; `docs/models-and-providers.md:195` becomes a link. +2. `settings(request, info)` in `src/ai-sdk/index.ts`, `src/openai/index.ts`, `src/tanstack-ai/index.ts`; `examples/mock-model.ts` gains a `byRequestId` entry form; `examples/parallel-streams` or `examples/map-reduce` test switches to it. +3. Add to `docs/langgraph-comparison.md` a row: LangGraph "FakeListChatModel" → "the provider SDK's own test model behind the real adapter; see evals.md". + +**Touches.** three adapter files + their tests, `examples/mock-model.ts`, one example test, `docs/evals.md`, `docs/models-and-providers.md`, `docs/langgraph-comparison.md`, `src/docs-integration-recipes.test.ts`. + +**Effort.** S–M. **Risks.** Snippets drift from adapter wire formats (mitigated by recipe tests). `requestId` is the actor's local id, so two separately invoked instances of the same child machine in the same state can share one; document and point at `callKey` for lineage. + +### 5. Subgraph state mapping discoverability (gap 4) + +**Current state.** Mapping is plain XState invoke `input` + `onDone`, typed by the child's `input`/`output` schemas: `examples/hierarchical-teams/index.ts:125-126` (child boundary), `:162-168` (child maps input into context, projects output), `:394-431` (parent `input: ({ context }) => ({ topic, priorNotes })`, `onDone: ({ context, output }) => ({ target, context: { research: output.research, ... } })`). `examples/agent-supervisor/index.ts:571-586` factors it into `workerInputFrom` / `recordReport`. Also `game-loop-agent:249,361`, `storm-writer:517,680`. `docs/multi-agent.md` is 29 lines; "Child machines" is one sentence (`:5-7`) and never shows the shape. `docs/langgraph-comparison.md:29` says "Subgraph → Invoked child machine" with no detail. `docs/machines.md:207-258` covers invoke `input`/`onDone` for requests only. + +**Proposed design.** Docs only; no rename (see table). + +1. `docs/multi-agent.md`: new section "Map state into and out of a child machine" with the hierarchical-teams snippet, naming the LangGraph terms ("subgraph", "state transformation", "input/output keys") so search lands here. Note that the child's context is private: the parent sees only `output`, and the child sees only `input`. +2. `docs/langgraph-comparison.md`: split the Subgraph row into "Subgraph sharing state keys" and "Subgraph with its own schema → invoke `input` + `onDone`", and link the section. +3. Optional: `examples/subgraph-state-mapping`, a minimal parent and child with disjoint context keys, linked from both docs. Only if a reviewer wants a runnable focus example; hierarchical-teams already serves. + +**Touches.** `docs/multi-agent.md`, `docs/langgraph-comparison.md`, `docs/machines.md` (one cross-link), optional example. + +**Effort.** S. **Risks.** None material. + +**Exercised by.** `examples/hierarchical-teams`, `examples/agent-supervisor`, `docs/multi-agent.md`. + +### 6. Store / long-term memory abstraction (gap 3) + +**Current state.** `examples/long-term-memory/index.ts` models the store as `string[]` in machine `input` `{ userId, memories }` (`:233-236`) and `output.memories` (`:402-420`); the host persists output and feeds the next run (`:534-536`). Recall is an `actors` entry (`searchMemories`, `:124-150, :246-249`), save is a pure `choice` state with dedupe and a `MAX_MEMORIES` cap (`:383-397`). Header says outright there is no `BaseStore` handle (`:50-53`). The only store-like interface in `src/` is `AgentEventLogStore` (`src/event-log-store.ts:20-58`): append-only, thread-keyed, optimistic concurrency, no key/value, namespace, search, update or delete. + +**Decision: neither a core interface nor a new abstraction.** `docs/scope.md:36` assigns long-term memory to a database or memory library with `input` + `actors` as the integration point; `scope.md:12` names a bundled memory store as the anti-pattern; CONTRIBUTING.md:16 lists vector stores as out. A `BaseStore` with namespaces and search is a storage product, not control flow, and would conflict with the append-only, log-is-truth design of the one store core does own. Shipping it would also set the precedent scope.md exists to prevent. + +**What ships instead** (docs only): + +1. `docs/patterns.md`: a "Long-term memory" pattern section: the store is host-owned; a recall actor and a capped save state are the machine's two touchpoints; memory handles (ids, not blobs) live in context; the host chooses KV, vector, or SQL. +2. `docs/langgraph-comparison.md`: a construct-map row "`BaseStore` / `store.put` / `store.search`" → "host store behind an `actors` entry; see patterns.md and `examples/long-term-memory`", and a sentence in the prose list after the map saying this is deliberate. +3. Fix the stale wording at `docs/scope.md:40`, which still lists snapshot persistence as `result.persist()` only, to mention the event log store. + +**Touches.** `docs/patterns.md`, `docs/langgraph-comparison.md`, `docs/scope.md`. + +**Effort.** S. **Risks.** Users coming from LangGraph may still ask for it; the comparison row is the answer. + +**Exercised by.** `examples/long-term-memory`, `docs/patterns.md`. + +### 7. Durable log store: recipes, not a package (added 2026-10-14) + +**Why it's here.** After A–C, the first obstacle a LangGraph user hits is "where do I put the log?" LangGraph ships `MemorySaver`, `SqliteSaver`, `PostgresSaver`. We ship the interface, an in-memory implementation, and a conformance suite. + +**Current state.** `AgentEventLogStore` is four methods: `append`, `read`, `length`, `fork` (`src/event-log-store.ts:20-58`). `createInMemoryEventLogStore` is the reference (`:92-169`). `assertEventLogStoreConformance(createStore, { describe, it })` is exported from `@statelyai/agent/log` (`src/log/index.ts:44-45`) and documented in two sentences (`docs/event-log.md:177-183`). `examples/cloudflare-agent-host/event-log-store.ts` is the only host-written store, and it runs the suite (`:24`). No SQLite or Postgres recipe exists anywhere. + +**Decision.** No bundled store. It would not break "the machine runs anywhere" (the machine never names a store), but it contradicts `docs/roadmap.md` ("storage, queues, retries, and interruption policy: host frameworks") and `docs/scope.md:40`, and it starts an open-ended adapter queue. Same policy as the model mock: show it, don't ship it. + +**Proposed design** (docs and examples only): + +1. `docs/event-log.md` "Stores" section is rewritten to lead with the contract and the suite: "implement four methods, run `assertEventLogStoreConformance`, you're done". Include the SQL shape (`(thread_id, index)` primary key, append in one transaction with a length check). +2. `examples/stores/` (or three sibling example dirs, per the one-flat-directory rule): `sqlite` (`better-sqlite3`, ~60 lines), `postgres` (`pg`, ~60 lines), and a pointer to `cloudflare-agent-host`'s Durable Object store. Each has a test that runs the conformance suite; SQLite in-memory runs in CI, Postgres is gated on a `DATABASE_URL` env var and skipped otherwise. +3. `docs/langgraph-comparison.md`: a construct-map row "`MemorySaver` / `SqliteSaver` / `PostgresSaver` → `createInMemoryEventLogStore`, or your store checked by `assertEventLogStoreConformance`; see the store recipes". +4. `docs/persistence.md` "Framework storage" links the recipes. + +**Touches.** `docs/event-log.md`, `docs/persistence.md`, `docs/langgraph-comparison.md`, new example dirs with tests, `examples.test.ts` registration if required. No `src/` change. + +**Effort.** S–M (the recipes are small; the Postgres CI gating is the fiddly part). + +**Risks.** Recipes drift from the conformance suite as it grows; running the suite in CI is the mitigation. A user may still ask for a package; the comparison row answers it. + +**Exercised by.** the new store examples, `examples/cloudflare-agent-host`, `docs/event-log.md`. + +## PR-sized batches + +### Batch A: sweep fixes and the testing seam (one PR, `next`) + +- Gap 2: 6c, 6d, 6e, 6f, 6h. (6a can ride along or split into its own PR if the choices API needs discussion; it is the only M item.) +- Gap 5: `settings(request, info)` on the three adapters, `examples/mock-model.ts` `byRequestId`, the three docs recipes, comparison row. +- Changeset: minor. Tests: unit per gap, `pnpm check`, `pnpm docs:check`, `pnpm vitest run examples`. + +### Batch B: streaming (one PR, `next`) + +- Gap 1: trace fields, `streamAgent`, two stream examples rewritten, observability and comparison docs. +- Changeset: minor. Follow-up issue: adapter-subpath UI-protocol bridges; child emits; richer chunk parts. + +### Batch C: persistence and composition docs (one PR, `next`) + +- Gap 3: event-log "Branch with edited state" section, `forkThread`, fresh `executionId` on fork + `forkedFrom`, `examples/time-travel`. +- Gap 4 docs section and comparison rows. +- Gap 6 docs section, comparison row, scope.md:40 wording. +- Gap 7: store recipes (sqlite, postgres) with conformance tests, event-log.md Stores rewrite, comparison row. +- Changeset: patch for the fork change (call it out as a lineage-semantics change). + +## Recommendation + +Do Batch A first. Every item in it reproduces today on `next` with a two-line fix path, none depends on an API discussion except 6a (which can be split out), and it lands `settings(request, info)` which the Batch B stream examples' tests and any provider-level mock need to script parallel requests deterministically. It also closes the testing story in the only form the September ruling allows, so the LangGraph comparison can honestly say "test at the provider, we show you how for all three adapters". Batch B is the headline parity feature and should follow immediately; it is self-contained in `run-agent.ts` plus one new file, and shipping it after A means its examples can be tested with the improved mock. Batch C is docs-heavy and carries the one semantic decision (fork lineage and `callKey`), so it benefits from being reviewed on its own rather than buried under code.