Skip to content

fix(agent): isolate analytics from workflow correctness - #196

Merged
iamjr15 merged 1 commit into
mainfrom
fix/analytics-engine-isolation
Aug 9, 2026
Merged

fix(agent): isolate analytics from workflow correctness#196
iamjr15 merged 1 commit into
mainfrom
fix/analytics-engine-isolation

Conversation

@iamjr15

@iamjr15 iamjr15 commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • prevent Analytics Engine binding errors or quota exhaustion from failing product operations
  • replace redundant per-tool lifecycle analytics with one completion event
  • emit model-resolution analytics only for the initial selection and actual model changes

Architecture

Analytics is now explicitly outside the product correctness boundary in the shared observability package. Agent Workflows retain durable step and transcript history for lifecycle debugging while Analytics Engine records compact product outcome events.

Decisions Made

Decision Choice Alternatives Considered Reasoning
Analytics failures Swallow binding exceptions at the shared emitter boundary Catch at every call site Every emitter is telemetry-only; the invariant belongs in one package and protects every Worker.
Tool telemetry Keep tool_invoked; remove step_started and step_completed Keep all three events The Workflow and transcript already own step lifecycle truth; duplicate events consumed the 250-write invocation allowance.
Model attribution Emit on selection changes Emit every model turn Attribution changes only on initial resolution or fallback, so per-turn rows added volume without information.

Edge Cases Handled

Scenario Handling
Analytics Engine per-invocation limit is exhausted The user run continues and terminal state still persists.
Analytics binding is absent Existing no-op behavior remains.
Model fallback changes the selected model A new resolution event is emitted for the actual change.
Long tool-heavy run Redundant lifecycle writes are removed; durable Workflow history remains complete.

Verification

  • pnpm lint
  • pnpm typecheck
  • pnpm turbo build --force
  • pnpm deadcode
  • pnpm architecture:check
  • pnpm turbo skills:build
  • direct emitter simulation with an Analytics binding that throws a quota error
  • production failure reproduced as Analytics Engine write limit exceeded after a 67-tool run

Keep Analytics Engine quotas outside the run failure boundary.
Remove duplicate tool lifecycle rows and emit model resolution only
when the selected model changes.
@vercel

vercel Bot commented Aug 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
cheatcode Skipped Skipped Aug 9, 2026 11:22am

@iamjr15
iamjr15 merged commit b9e93fa into main Aug 9, 2026
6 checks passed
@iamjr15
iamjr15 deleted the fix/analytics-engine-isolation branch August 9, 2026 11:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant