Skip to content

feat(infinity-provider-protocol,infinity-provider-bedrock,infinity-agent-core)!: provider-classified errors, provider-owned timeouts, and oversized-input recovery - #121

Open
shadaj wants to merge 1 commit into
mainfrom
sandbox-23ca95d7-b2d9-4ea4-a456-3eeaeb5b1749
Open

feat(infinity-provider-protocol,infinity-provider-bedrock,infinity-agent-core)!: provider-classified errors, provider-owned timeouts, and oversized-input recovery#121
shadaj wants to merge 1 commit into
mainfrom
sandbox-23ca95d7-b2d9-4ea4-a456-3eeaeb5b1749

Conversation

@shadaj

@shadaj shadaj commented Sep 4, 2026

Copy link
Copy Markdown
Member

infinity-provider-protocol: structured, classified errors

  • New ErrorClass enum (Transient / Throttled / ContextOverflow /
    Fatal) — providers declare how callers should react; the agent runtime
    never parses error message strings.
  • CompletionError::ProviderError now carries { message, class }
    (breaking); added CompletionError::provider(class, msg) and
    CompletionError::class() (RequestError/JsonError → Fatal,
    ResponseError → Transient).
  • Remote wire protocol forwards errors as WireError { message, class } so
    classification survives the Unix-socket boundary (Fatal default for
    older providers). Covered by error_class_survives_the_wire.

infinity-provider-bedrock: timeouts + classification live here now

  • 60s request-initiation timeout around ConverseStream (Bedrock
    occasionally black-holes requests), reported as Transient so core
    retries. Deliberately no mid-stream/inactivity timeout anywhere: once
    a stream is live it is never artificially cut off by us. Tested with a
    never-responding HTTP connector under paused time.
  • classify_bedrock_error maps service codes + message heuristics (moved
    out of core) to ErrorClass: throttling → Throttled, input-too-long /
    context-limit → ContextOverflow, backend hiccups → Transient, rest
    Fatal. Mid-stream recv errors are classified the same way.
  • New non-default live-tests feature (tests/live.rs) hitting real
    Bedrock with local credentials — overflow classification, unknown-model
    fatality, happy path — instead of tautological string-table unit tests.
    All pass (and caught a bad token estimate during development).
  • infinity-provider-rig classifies rig errors from common
    OpenAI-compatible phrasings.

infinity-agent-core: no timeouts, class-driven retries, oversized-input fix

run_completion drops the 60s initiation timeout and the 120s mid-stream
stall timer; retries are driven purely by ErrorClass (Throttled → 30s
backoff, Transient → short retry, Fatal → give up), capped at
MAX_COMPLETION_RETRIES.

Three-phase pending items (fixes: an oversized user input / tool result
used to be persisted and permanently wedge the thread):

  • HistoryManager inputs move through not-known-safe
    (unvalidated_items) → known-safe (pending_items, promoted the
    moment the model streams any output — proof the context didn't overflow;
    model-produced content goes here directly) → synced. sync() only
    persists known-safe items; a sequentiality assert makes committing model
    output over unvalidated inputs impossible.
  • Interrupts: cancelling before any model output keeps the inputs in memory
    (re-sent with the interrupting message next round) but persists nothing;
    after output, input + partial output persist as before.
  • safe_spawn_point() stays before unvalidated inputs (children inherit
    history from the store).

Overflow recovery (recover_from_overflow):

  • Exactly one unvalidated item and it's replaceable (tool result /
    subscription event): the culprit is unambiguous → replace its body with
    TOOL_RESULT_TOO_LARGE_PLACEHOLDER, retry. (The replace helper skips
    bodies already equal to the placeholder, so a second overflow falls
    through to drop mode instead of retrying forever.)
  • Anything else (user text present, multiple items, or the retry overflowed
    again): settle replaceables with the shared TOOL_CALL_INTERRUPTED_TEXT
    (a "too large" note could mislead the model into re-running the tool),
    drop the user inputs (drop_unvalidated_user_inputs, dedup IDs
    forgotten), and stop — dropping the user's words is never followed by
    a silent retry. Tool calls always stay answered in memory; nothing
    unvalidated ever persists. TODO left for blocking on a lower-threshold
    compaction when committed history itself is too close to the limit.

Tests (each behavior landed red-first)

  • event_processor: oversized user input dropped & unpersisted; oversized
    tool result / subscription event → placeholder retry; tool result + user
    input → interrupted-settle + drop + no retry; second overflow gives up
    keeping the call answered; interrupt before/after model output;
    throttled/transient retry vs fatal no-retry; sync() skips unvalidated;
    flush-with-unvalidated panics; spawn-point exclusion.
  • system: interrupt-before-output persists nothing until validated; overflow
    after interrupt drops both inputs and stops (thread recovers on next
    message); oversized tool result never wedges the thread end-to-end;
    shutdown persists validated but not unvalidated tool results; rebooted
    session settles a persisted unanswered tool call via the synthetic
    "interrupted" result.

./check.bash: fmt, clippy, and full workspace tests pass (99 core tests +
all other crates); the web-e2e step stalled on a sandboxed npm ci
(network), unrelated to these changes.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 4, 2026

Copy link
Copy Markdown

Deploying infinity with  Cloudflare Pages  Cloudflare Pages

Latest commit: 132dfef
Status: ✅  Deploy successful!
Preview URL: https://29e41bf1.infinity-dc7.pages.dev
Branch Preview URL: https://sandbox-23ca95d7-b2d9-4ea4-a.infinity-dc7.pages.dev

View logs

…ent-core)!: provider-classified errors, provider-owned timeouts, and oversized-input recovery

## `infinity-provider-protocol`: structured, classified errors

* New `ErrorClass` enum (`Transient` / `Throttled` / `ContextOverflow` /
  `Fatal`) — providers declare how callers should react; the agent runtime
  never parses error message strings.
* `CompletionError::ProviderError` now carries `{ message, class }`
  (breaking); added `CompletionError::provider(class, msg)` and
  `CompletionError::class()` (RequestError/JsonError → Fatal,
  ResponseError → Transient).
* Remote wire protocol forwards errors as `WireError { message, class }` so
  classification survives the Unix-socket boundary (`Fatal` default for
  older providers). Covered by `error_class_survives_the_wire`.

## `infinity-provider-bedrock`: timeouts + classification live here now

* 60s request-initiation timeout around `ConverseStream` (Bedrock
  occasionally black-holes requests), reported as `Transient` so core
  retries. Deliberately **no** mid-stream/inactivity timeout anywhere: once
  a stream is live it is never artificially cut off by us. Tested with a
  never-responding HTTP connector under paused time.
* `classify_bedrock_error` maps service codes + message heuristics (moved
  out of core) to `ErrorClass`: throttling → `Throttled`, input-too-long /
  context-limit → `ContextOverflow`, backend hiccups → `Transient`, rest
  `Fatal`. Mid-stream recv errors are classified the same way.
* New non-default `live-tests` feature (`tests/live.rs`) hitting real
  Bedrock with local credentials — overflow classification, unknown-model
  fatality, happy path — instead of tautological string-table unit tests.
  All pass (and caught a bad token estimate during development).
* `infinity-provider-rig` classifies rig errors from common
  OpenAI-compatible phrasings.

## `infinity-agent-core`: no timeouts, class-driven retries, oversized-input fix

`run_completion` drops the 60s initiation timeout and the 120s mid-stream
stall timer; retries are driven purely by `ErrorClass` (Throttled → 30s
backoff, Transient → short retry, Fatal → give up), capped at
`MAX_COMPLETION_RETRIES`.

**Three-phase pending items** (fixes: an oversized user input / tool result
used to be persisted and permanently wedge the thread):

* `HistoryManager` inputs move through **not-known-safe**
  (`unvalidated_items`) → **known-safe** (`pending_items`, promoted the
  moment the model streams any output — proof the context didn't overflow;
  model-produced content goes here directly) → **synced**. `sync()` only
  persists known-safe items; a sequentiality assert makes committing model
  output over unvalidated inputs impossible.
* Interrupts: cancelling before any model output keeps the inputs in memory
  (re-sent with the interrupting message next round) but persists nothing;
  after output, input + partial output persist as before.
* `safe_spawn_point()` stays before unvalidated inputs (children inherit
  history from the store).

**Overflow recovery** (`recover_from_overflow`):

* Exactly one unvalidated item and it's replaceable (tool result /
  subscription event): the culprit is unambiguous → replace its body with
  `TOOL_RESULT_TOO_LARGE_PLACEHOLDER`, retry. (The replace helper skips
  bodies already equal to the placeholder, so a second overflow falls
  through to drop mode instead of retrying forever.)
* Anything else (user text present, multiple items, or the retry overflowed
  again): settle replaceables with the shared `TOOL_CALL_INTERRUPTED_TEXT`
  (a "too large" note could mislead the model into re-running the tool),
  drop the user inputs (`drop_unvalidated_user_inputs`, dedup IDs
  forgotten), and **stop** — dropping the user's words is never followed by
  a silent retry. Tool calls always stay answered in memory; nothing
  unvalidated ever persists. TODO left for blocking on a lower-threshold
  compaction when committed history itself is too close to the limit.

## Tests (each behavior landed red-first)

* event_processor: oversized user input dropped & unpersisted; oversized
  tool result / subscription event → placeholder retry; tool result + user
  input → interrupted-settle + drop + **no retry**; second overflow gives up
  keeping the call answered; interrupt before/after model output;
  throttled/transient retry vs fatal no-retry; `sync()` skips unvalidated;
  flush-with-unvalidated panics; spawn-point exclusion.
* system: interrupt-before-output persists nothing until validated; overflow
  after interrupt drops both inputs and stops (thread recovers on next
  message); oversized tool result never wedges the thread end-to-end;
  shutdown persists validated but not unvalidated tool results; rebooted
  session settles a persisted unanswered tool call via the synthetic
  "interrupted" result.

`./check.bash`: fmt, clippy, and full workspace tests pass (99 core tests +
all other crates); the web-e2e step stalled on a sandboxed `npm ci`
(network), unrelated to these changes.

Co-authored-by: Infinity 🤖 <infinity@hydro.run>
PR: #121
@shadaj
shadaj force-pushed the sandbox-23ca95d7-b2d9-4ea4-a456-3eeaeb5b1749 branch from 8f30f19 to 132dfef Compare September 4, 2026 20:59
@shadaj
shadaj marked this pull request as ready for review September 4, 2026 23:16
@shadaj
shadaj requested a review from a team September 4, 2026 23:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant