Conversation
github-actions
Bot
added
the
missing-template
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
WalkthroughChangesUsage persistence and budget accounting
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 2 | ❌ 3❌ Failed checks (3 warnings)
✅ Passed checks (2 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
removed
the
missing-template
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (3)
docs/use-with-claude-code.md (1)
159-165: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy liftAvoid rescanning the entire transcript tree every three seconds.
Each loop rereads every transcript from the beginning, while
seengrows for the lifetime of the process. Long-running users will incur increasing disk/CPU work and memory usage. Track per-file offsets or use a file-watching mechanism instead.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/use-with-claude-code.md` around lines 159 - 165, Update the transcript polling loop around turns() to avoid rescanning the entire transcript tree on every iteration. Track each transcript file’s read offset, or use the existing file-watching mechanism if available, so only newly appended events are processed while preserving the current backlog exclusion and seen-event behavior.tests/integration/test_require_pricing.py (1)
99-116: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winTighten the assertion to match the documented expectation.
The docstring claims the call "fails as a provider error, never a 402," but the test only asserts
!= 402. Asserting the actual expected status (e.g.== 502) would verify the documented behavior precisely, instead of just ruling out one code.🧪 Suggested tightening
- assert resp.status_code != 402 + assert resp.status_code == 502🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/test_require_pricing.py` around lines 99 - 116, Update test_budget_exempt_key_skips_pricing_gate to assert the documented provider-error status, 502, instead of merely checking that the response is not 402; preserve the existing request setup and exemption behavior.src/gateway/api/routes/batches.py (1)
293-307: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider adding integration coverage for batch budget-exemption.
test_exclude_from_budget.pycovers chat completions but I don't see a batch-specific counterpart in this cohort exercisingcreate_batch/retrieve_batch_resultswith an exempt key end-to-end (reservation skipped, spend untouched,counts_toward_budget=Falseon the logged row). Given the exactly-once accounting logic here is fairly intricate, a dedicated test would be a good safety net.Also applies to: 343-371
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/gateway/api/routes/batches.py` around lines 293 - 307, The batch budget-exemption flow around create_batch and retrieve_batch_results lacks end-to-end integration coverage. Add a dedicated test mirroring test_exclude_from_budget.py that uses an exclude_from_budget key to create and retrieve a batch, verifies reservation is skipped, user spend remains unchanged, and the logged cost row records counts_toward_budget=False while preserving exactly-once accounting.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/use-with-claude-code.md`:
- Around line 153-157: Update ship to call urlopen with a finite timeout and
retry transient HTTP or network failures using backoff, without terminating the
watcher. Preserve unseen events across failed uploads and remove or advance them
only after a successful acknowledgment.
- Around line 104-117: Update the “Get a budget-exempt import key” section to
explicitly state that exclude_from_budget also bypasses gateway reservations,
spend updates, and budget enforcement, so it is not import-only. Instruct
readers to use a dedicated user and key exclusively for imports, restrict
distribution to the importer, and handle the credential as a sensitive secret;
do not present it as a generally shareable key.
- Around line 180-186: Update the always-on pipeline guidance near the Claude
Code telemetry instructions to explicitly require OTEL_LOGS_EXPORTER=otlp and
document the OTLP endpoint and protocol settings alongside
CLAUDE_CODE_ENABLE_TELEMETRY=1. Keep the existing Collector and external-events
mapping guidance unchanged.
- Around line 159-164: Update the outbound loop around turns() so unseen events
are split into batches within the ingestion endpoint’s bounded limit. Send each
chunk independently, and only add that chunk’s source_event_id values to seen
after its ship call succeeds; preserve already-processed events and retry
behavior for failed chunks.
In `@src/gateway/services/external_usage_service.py`:
- Around line 54-69: Update the validation definitions for source_event_id,
provider, model, and session_label in the event model to enforce a bounded
identifier-safe grammar, allowing only the intended identifier/label characters
and lengths while rejecting arbitrary prompt, response, or payload text before
persistence. Preserve the existing required/optional semantics and length
bounds, and do not include sensitive values in errors or logs.
- Around line 313-345: Refactor the event-processing flow around
_resolve_target_user and _resolve_pricing to eliminate per-event database
queries: collect distinct target user IDs and provider/model pricing keys first,
fetch active users with a single IN-based query, and load the applicable pricing
histories in bulk. Resolve user validity and timestamp-effective pricing from
those in-memory results while building rows, preserving rejection, duplicate
handling, and pricing behavior.
In `@src/gateway/static/dashboard/assets/UsagePage-CR8SlxkC.js`:
- Line 1: The cost display paths in UsagePage’s `we`, `O`, and `Y` functions
incorrectly format null costs as zero. Preserve null pricing as `—` or
“Unpriced” in summary cards, tooltips, and source/model rows, and exclude null
costs from cost totals and percentage calculations so their bars have no
misleading width; retain existing formatting for numeric costs.
In `@web/src/pages/ActivityPage.tsx`:
- Around line 363-370: Add provenance display to the activity records near the
source filter: update the activity table or its expanded RequestDetail rendering
to show each entry’s entry.source or source_label, while preserving the existing
source filtering behavior. Use the existing activity row/detail symbols and
sourceLabel helper where applicable so gateway and imported records are visually
distinguishable.
In `@web/src/pages/KeysPage.test.tsx`:
- Around line 300-324: Add a test alongside the existing budget-exemption tests
that uses mockApi with an existing key, opens its edit flow, enables the “Exempt
this key from budget” toggle, and saves the key. Inspect the captured PATCH
request to /v1/keys and assert its JSON body contains exclude_from_budget set to
true, following the existing create test’s request-matching pattern.
In `@web/src/pages/UsagePage.test.tsx`:
- Around line 38-41: Add a user-facing test around the by_source fixture that
verifies the “Cost by source” section renders the friendly “Claude Code” label,
clicks it, and asserts the resulting navigation includes source=claude_code. Use
the existing UsagePage test setup and preserve the current fixture data.
---
Nitpick comments:
In `@docs/use-with-claude-code.md`:
- Around line 159-165: Update the transcript polling loop around turns() to
avoid rescanning the entire transcript tree on every iteration. Track each
transcript file’s read offset, or use the existing file-watching mechanism if
available, so only newly appended events are processed while preserving the
current backlog exclusion and seen-event behavior.
In `@src/gateway/api/routes/batches.py`:
- Around line 293-307: The batch budget-exemption flow around create_batch and
retrieve_batch_results lacks end-to-end integration coverage. Add a dedicated
test mirroring test_exclude_from_budget.py that uses an exclude_from_budget key
to create and retrieve a batch, verifies reservation is skipped, user spend
remains unchanged, and the logged cost row records counts_toward_budget=False
while preserving exactly-once accounting.
In `@tests/integration/test_require_pricing.py`:
- Around line 99-116: Update test_budget_exempt_key_skips_pricing_gate to assert
the documented provider-error status, 502, instead of merely checking that the
response is not 402; preserve the existing request setup and exemption behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 130251bf-3751-4f8d-9abf-05ca645a4d50
⛔ Files ignored due to path filters (1)
docs/public/openapi.jsonis excluded by!docs/public/openapi.json
📒 Files selected for processing (45)
TODOS.mdalembic/versions/b8c1d2e3f4a5_add_usage_source_and_budget_exclusion.pydocs/api-reference.mddocs/external-usage.mddocs/use-with-claude-code.mdsrc/gateway/api/routes/_passthrough.pysrc/gateway/api/routes/_pipeline.pysrc/gateway/api/routes/batches.pysrc/gateway/api/routes/keys.pysrc/gateway/api/routes/usage.pysrc/gateway/models/entities.pysrc/gateway/services/budget_service.pysrc/gateway/services/external_usage_service.pysrc/gateway/static/dashboard/assets/ActivityPage-BYun6blj.jssrc/gateway/static/dashboard/assets/ActivityPage-C0gVlZW-.jssrc/gateway/static/dashboard/assets/AliasesPage-91aYO2TQ.jssrc/gateway/static/dashboard/assets/BudgetsPage-BAkOGUnL.jssrc/gateway/static/dashboard/assets/KeysPage-BhG598Pa.jssrc/gateway/static/dashboard/assets/KeysPage-vrCpn4_2.jssrc/gateway/static/dashboard/assets/ModelScopeControl-DEMY0vFB.jssrc/gateway/static/dashboard/assets/ModelsPage-uB0xWqdk.jssrc/gateway/static/dashboard/assets/OverviewPage-BcmQl9vA.jssrc/gateway/static/dashboard/assets/ProvidersPage-Jyzddv9l.jssrc/gateway/static/dashboard/assets/SettingsPage-B-7cjmL2.jssrc/gateway/static/dashboard/assets/ToolsGuardrailsPage-Dp2kIXPX.jssrc/gateway/static/dashboard/assets/UsagePage-BVzIiui8.jssrc/gateway/static/dashboard/assets/UsagePage-CR8SlxkC.jssrc/gateway/static/dashboard/assets/UsersPage-5a0iIZCd.jssrc/gateway/static/dashboard/assets/index-CSyrpBqZ.jssrc/gateway/static/dashboard/assets/index-DcR_oS0q.jssrc/gateway/static/dashboard/index.htmltests/integration/test_exclude_from_budget.pytests/integration/test_external_usage_events.pytests/integration/test_require_pricing.pytests/integration/test_usage_endpoint.pytests/unit/test_pipeline_vision_billing.pyweb/src/api/hooks.tsweb/src/api/types.tsweb/src/pages/ActivityPage.test.tsxweb/src/pages/ActivityPage.tsxweb/src/pages/KeysPage.test.tsxweb/src/pages/KeysPage.tsxweb/src/pages/OverviewPage.test.tsxweb/src/pages/UsagePage.test.tsxweb/src/pages/UsagePage.tsx
💤 Files with no reviewable changes (4)
- src/gateway/static/dashboard/assets/UsagePage-BVzIiui8.js
- src/gateway/static/dashboard/assets/KeysPage-BhG598Pa.js
- src/gateway/static/dashboard/assets/ActivityPage-C0gVlZW-.js
- src/gateway/static/dashboard/assets/index-CSyrpBqZ.js
Member
Author
@coderabbitai thanks for the review. All 10 findings are addressed in cc0f8df:
The one I did not change structurally is the per-group unpriced count in the breakdown; I kept it at the totals level to avoid threading a new field through the top-N fold, which felt heavier than the Minor rating warranted. Happy to reconsider if you disagree. |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
git show --no-ext-diff --stat --oneline cc0f8df3
printf '\n--- changed files ---\n'
git diff-tree --no-commit-id --name-only -r cc0f8df3
printf '\n--- focused diff ---\n'
git show --no-ext-diff --format= -- \
src/gateway/services/external_usage_service.py \
src/gateway/api/routes/usage.py \
web/src/pages/UsagePage.tsx \
web/src/pages/ActivityPage.tsx \
docs/external-usage.md \
docs/use-with-claude-code.md \
tests/integration/test_external_usage_events.py \
web/src/pages/KeysPage.test.tsx \
web/src/pages/UsagePage.test.tsx \
cc0f8df3
printf '\n--- service outline ---\n'
ast-grep outline src/gateway/services/external_usage_service.py --items all --view expanded
printf '\n--- relevant symbols and tests ---\n'
rg -n -C 3 \
'unpriced_requests|token-only|pattern=|source_event_id|session_label|select\(.*User|Pricing|IN \(|by_source|Source|session|urlopen|timeout|OTEL_LOGS_EXPORTER|exclude_from_budget' \
src/gateway/services/external_usage_service.py \
src/gateway/api/routes/usage.py \
web/src/pages/UsagePage.tsx \
web/src/pages/ActivityPage.tsx \
docs/external-usage.md \
docs/use-with-claude-code.md \
tests/integration/test_external_usage_events.py \
web/src/pages/KeysPage.test.tsx \
web/src/pages/UsagePage.test.tsxLength of output: 91846 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- pricing service outline ---'
ast-grep outline src/gateway/services/pricing_service.py --items all --view expanded
printf '%s\n' '--- pricing resolution implementation ---'
rg -n -A100 -B12 \
'^(async )?def (find_model_pricing|default_model_pricing|normalize_effective_at)\b' \
src/gateway/services/pricing_service.py
printf '%s\n' '--- external resolver and historical pricing test ---'
sed -n '160,230p' src/gateway/services/external_usage_service.py
sed -n '280,320p' tests/integration/test_external_usage_events.py
printf '%s\n' '--- pricing service test candidates ---'
fd -t f -i 'pricing' tests src | sort | while IFS= read -r file; do
printf '\n### %s\n' "$file"
rg -n -C 2 \
'find_model_pricing|default_model_pricing|effective_at|legacy|model_key' \
"$file" || true
doneLength of output: 58779
In particular, the bulk resolver preserves I also confirmed the validation, unpriced total/UI disclosure, Activity expanded provenance, documentation hardening, and the added PATCH/drilldown coverage. I agree with keeping unpriced counts at the totals level. It fixes the misleading “$0 means free” interpretation without adding disproportionate complexity to the folded breakdown API. 🐇 ✅ If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! |
njbrake
marked this pull request as draft
July 23, 2026 18:39
…cking Squashed rebase of fix-378 onto main. External usage ingestion (POST /v1/usage/external-events), OTLP receivers (/v1/logs, /v1/traces) for Claude Code, Codex, and GenAI-instrumented apps, a per-key exclude_from_budget flag threaded through every billable site, a source-aware read surface and dashboard, and review hardening (gzip and size bounds, master-key refusal, partialSuccess reporting, batch-envelope extra-field rejection, collision-retry fallback, exempt-batch accounting). Rebase resolutions: migration re-parented onto a9c1e3b5d7f9 (dashboard sessions) to keep a single Alembic head; docs/index.md bullets re-slotted into main's sectioned list; UsagePage keeps main's sparkline chart plus the unpriced-count hint; dashboard bundle rebuilt against main's recharts dependency; OpenAPI regenerated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 21:03 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
njbrake
marked this pull request as ready for review
July 23, 2026 21:03
The Cost by source card and the Activity Source filter were removed in earlier iterations, leaving UsageFilters.source, the hooks query param, UsageSummary.by_source, and BreakdownTable's columnLabel/formatKey props with no consumer. Delete them; the backend API keeps its source filter and by_source breakdown, and the Activity detail still shows provenance via source/source_label. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 21:06 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (1)
web/src/pages/UsagePage.tsx (1)
67-86: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winMissing "Spend by source" wiring is the root cause behind both the unused
BreakdownTableprops and the missing drill-down test.columnLabel/formatKeywere added to support a friendly per-source breakdown (mirroringsourceLabelinActivityPage.tsx), andby_sourcedata is fetched and fixtured in tests, but noBreakdownTablecall actually renders it — so there's nothing for a "Cost by source" test to click on either, despite a past-review thread marking that test as already addressed.
web/src/pages/UsagePage.tsx#L67-L86: wire up a thirdBreakdownTable(e.g. "Spend by source") usingdata?.by_source, asourceLabel-styleformatKey, and adjust the breakdowns grid at L455-472 accordingly — or confirm this was intentionally dropped and remove the now-deadcolumnLabel/formatKeyprops if so.web/src/pages/UsagePage.test.tsx#L37-L40: once the source breakdown is wired up, add the drill-down test the earlier review asked for (click "Claude Code", assert navigation includessource=claude_code).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@web/src/pages/UsagePage.tsx` around lines 67 - 86, Wire a third BreakdownTable in web/src/pages/UsagePage.tsx around lines 67-86 using data?.by_source, a sourceLabel-style formatKey, and the appropriate “Spend by source” labeling; update the breakdowns grid around lines 455-472 to include it. In web/src/pages/UsagePage.test.tsx around lines 37-40, add coverage that clicks “Claude Code” and asserts navigation includes source=claude_code.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/gateway/static/dashboard/assets/ModelsPage-DUKmygcZ.js`:
- Line 1: Update the current-price selection in gt so models with only future
effective_at entries do not select a[0] as active. Restrict the fallback to rows
whose effective_at is at or before the supplied timestamp i; leave the model
without an active rate when no such row exists, while preserving existing
sorting and selection for eligible rows.
In `@src/gateway/static/dashboard/assets/UsagePage-BiTBt0H6.js`:
- Line 1: Update the UsagePage empty-state message and the ActivityPage page
description to acknowledge that data may come from externally ingested events as
well as gateway-served requests. In
src/gateway/static/dashboard/assets/UsagePage-BiTBt0H6.js:1-1, revise the “No
usage yet” text; in
src/gateway/static/dashboard/assets/ActivityPage-gqHNxdk2.js:1-1, revise the
page description. Keep the existing behavior unchanged.
- Line 1: Complete source-aware dashboard reporting in UsagePage’s Ue component
by adding source to filter state, query parameters, controls, and clear-filter
handling, and render the API’s per-source summary alongside the existing model
and user breakdowns. In ActivityPage-gqHNxdk2.js, update the Activity filter
state and request-parameter construction to include source. Apply these changes
at UsagePage-BiTBt0H6.js:1-1 and ActivityPage-gqHNxdk2.js:1-1.
---
Nitpick comments:
In `@web/src/pages/UsagePage.tsx`:
- Around line 67-86: Wire a third BreakdownTable in web/src/pages/UsagePage.tsx
around lines 67-86 using data?.by_source, a sourceLabel-style formatKey, and the
appropriate “Spend by source” labeling; update the breakdowns grid around lines
455-472 to include it. In web/src/pages/UsagePage.test.tsx around lines 37-40,
add coverage that clicks “Claude Code” and asserts navigation includes
source=claude_code.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 976687e3-8b78-44b7-95e9-582223ccfb22
⛔ Files ignored due to path filters (2)
docs/public/openapi.jsonis excluded by!docs/public/openapi.jsonuv.lockis excluded by!**/*.lock,!**/uv.lock
📒 Files selected for processing (50)
TODOS.mdalembic/versions/b8c1d2e3f4a5_add_usage_source_and_budget_exclusion.pydocs/api-reference.mddocs/external-usage.mddocs/index.mddocs/use-with-claude-code.mddocs/use-with-codex.mdpyproject.tomlsrc/gateway/api/main.pysrc/gateway/api/routes/_passthrough.pysrc/gateway/api/routes/_pipeline.pysrc/gateway/api/routes/batches.pysrc/gateway/api/routes/keys.pysrc/gateway/api/routes/otlp.pysrc/gateway/api/routes/usage.pysrc/gateway/models/entities.pysrc/gateway/services/budget_service.pysrc/gateway/services/external_usage_service.pysrc/gateway/static/dashboard/assets/ActivityPage-gqHNxdk2.jssrc/gateway/static/dashboard/assets/AliasesPage-CI0LrDEg.jssrc/gateway/static/dashboard/assets/BudgetsPage-zaCyrkoB.jssrc/gateway/static/dashboard/assets/KeysPage-CoSXJN5o.jssrc/gateway/static/dashboard/assets/ModelScopeControl-DSH2_jCr.jssrc/gateway/static/dashboard/assets/ModelsPage-DUKmygcZ.jssrc/gateway/static/dashboard/assets/OverviewPage-D1H6fN9J.jssrc/gateway/static/dashboard/assets/ProvidersPage-_OHjx3AR.jssrc/gateway/static/dashboard/assets/SettingsPage-DvoxV4gJ.jssrc/gateway/static/dashboard/assets/ToolsGuardrailsPage-YLfgPmBP.jssrc/gateway/static/dashboard/assets/UsagePage-BiTBt0H6.jssrc/gateway/static/dashboard/assets/UsersPage-Byt0myM4.jssrc/gateway/static/dashboard/assets/index-BNKS2gvW.jssrc/gateway/static/dashboard/index.htmltests/integration/test_exclude_from_budget.pytests/integration/test_external_usage_events.pytests/integration/test_otlp.pytests/integration/test_require_pricing.pytests/integration/test_usage_endpoint.pytests/integration/test_usage_summary.pytests/unit/test_pipeline_vision_billing.pytests/unit/test_usage_timestamp_serialization.pyweb/src/api/hooks.tsweb/src/api/types.tsweb/src/pages/ActivityPage.test.tsxweb/src/pages/ActivityPage.tsxweb/src/pages/KeysPage.test.tsxweb/src/pages/KeysPage.tsxweb/src/pages/OverviewPage.test.tsxweb/src/pages/OverviewPage.tsxweb/src/pages/UsagePage.test.tsxweb/src/pages/UsagePage.tsx
🚧 Files skipped from review as they are similar to previous changes (19)
- web/src/pages/OverviewPage.test.tsx
- src/gateway/static/dashboard/index.html
- TODOS.md
- tests/integration/test_usage_endpoint.py
- tests/integration/test_require_pricing.py
- web/src/pages/KeysPage.test.tsx
- tests/integration/test_exclude_from_budget.py
- web/src/api/hooks.ts
- docs/api-reference.md
- alembic/versions/b8c1d2e3f4a5_add_usage_source_and_budget_exclusion.py
- src/gateway/api/routes/_passthrough.py
- src/gateway/models/entities.py
- web/src/pages/KeysPage.tsx
- docs/external-usage.md
- src/gateway/services/budget_service.py
- src/gateway/api/routes/batches.py
- tests/integration/test_external_usage_events.py
- src/gateway/api/routes/_pipeline.py
- src/gateway/services/external_usage_service.py
main added the postman-check CI gate after this branch was cut; the rebase brought the check in without the regenerated collection covering /v1/usage/external-events, /v1/logs, and /v1/traces. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 21:14 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
- Cap token and duration fields at the 32-bit usage_logs column width so an absurd count is a 422, not a database error that 500s the batch. - Chunk the candidate-user IN query like the event-id and pricing lookups; a full batch with per-event user_ids could exceed SQLite's bind-variable limit. - Reserve source "gateway": a REST import claiming it is refused with a 422, and an OTLP client name sanitizing to it falls back to "otel", so imported rows can never masquerade as native traffic. - Correct the migration docstring header to match the re-parented down_revision; document the global (per-source, not per-user) idempotency scope. Verified end to end against a live gateway with real clients: claude -p over http/protobuf and http/json, codex exec via its [otel] config, with per-event costs matching seeded rates and User.spend unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 21:50 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 22:16 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
… gateway The same call would land twice (source=gateway enforced, source=claude_code or codex exempt); budgets stay correct but cost analytics double-count. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
July 23, 2026 22:56 — with
GitHub Actions
Inactive
GitHub Actions
Inactive
Member
Author
|
As mentioned to Khaled over Slack: merging in order to give time for testing and stabilizition before I head of on vaca. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds an external usage ingestion API so subscription-backed usage (Claude Code first) shows up in Otari at API-equivalent cost, plus a general per-key budget-exemption flag the same usage rides on.
POST /v1/usage/external-events: content-free batches (token counts and metadata only; any prompt/completion/tool field is rejected with 422 at both the event and the batch envelope, never stored), idempotent on(source, source_event_id), priced at the effective rate for each event's timestamp (historical pricing, Anthropic cache read/write) orcost: nullwhen the model is unpriced. Authenticated with a budget-exempt API key (usage binds to the key's user and stamps its id) or the master key (may name any user).sourceis a generic provenance dimension;claude_codeis the first source.POST /v1/logsandPOST /v1/traces: OTLP receivers, so an OpenTelemetry export can point straight at Otari with no collector or sidecar. The logs signal recognizes Claude Code (api_request) and Codex (codex.sse_event/codex.api_request); the traces signal maps any app instrumented with the GenAI semantic conventions (gen_ai.*, withotari.client_nameas provenance). Both protobuf and JSON payloads are accepted, optionally gzipped, with body/event caps and 4xx responses for malformed bodies so exporters never retry-loop. Only content-free usage attributes are read; prompts, tool results, the event's owncost_usd(Otari re-prices at its own rate), and user identity are ignored. Usage binds to the import key's user; the master key is refused (it has no user to bind to), and rejected events are reported through the OTLPpartialSuccessresponse. Token conventions are normalized per emitter: Claude Code's additive cache counts are booked at the 1-hour write rate, while OpenAI-shaped emitters' cached tokens are de-included from input so cache reads are never double-charged.General
exclude_from_budgetflag on API keys: imported usage is retrospective and can never be blocked, so an import key must be budget-exempt (a budgeted key is refused with 403). The same flag governs a key's live proxied traffic, which is logged with cost but never reserved, folded intoUser.spend, or gated byrequire_pricing. Implemented by threading acounts_toward_budgetdecision throughReservationHandleand gating the spend write inreconcile_reservation, so every billable site (chat, messages, responses, passthrough, batches, streaming, vision, attachment top-up) inherits it.Read surface and dashboard: a
sourcefilter and aby_sourcebreakdown across list, count, summary, and CSV; anunpriced_requestscount in the summary so a $0 total is not read as free; an API key column on the Activity page, with source and session in the expanded detail; a Tracked cost total (with an unpriced-request count) and an API key filter on the Usage page, keeping Spend for the enforced ledger; an Exempt from budget control (under Advanced) and a Budget-exempt badge on the Keys page. One reversible migration adds the columns and a(source, source_event_id)unique constraint; existing rows backfill tosource = gateway.Docs: OpenTelemetry is the single documented import path, with dedicated setup guides for Claude Code and Codex (point the telemetry at Otari with an exempt key);
external-usage.mdcovers the endpoints, attribute mapping, pricing, and the budget-exempt invariant. Deferred to follow-up issues: per-model and per-request exemption, a subscription-versus-API break-even view, more vendor special-cases, and OTLP over gRPC.PR Type
Relevant issues
Closes #378
Note: #378 explicitly deferred a full OTLP receiver from the first version. This PR ships one anyway (replacing the issue's collector-plus-importer-script criterion) because it removes the sidecar/collector operational burden and the same mapping generalizes to Codex and any GenAI-instrumented app. Flagging the scope change for the record rather than treating it as incidental.
Checklist
tests/unit,tests/integration).make lint,make typecheck,make test).uv run python scripts/generate_openapi.py).AI Usage
AI Model/Tool used: Claude Code (Claude Opus 4.8, review hardening by Claude Fable 5)
Any additional AI details you'd like to share:
The scope, design decisions, and prioritization are @njbrake's, worked out through back-and-forth; the implementation and prose were written by Claude. The change was driven end to end against a live gateway: real
claude -psessions reporting over OpenTelemetry, unpriced turns landing withcost: nulland priced turns computing full cost including cache, with a non-exempt key correctly refused andUser.spendunchanged throughout. A six-agent review pass then hardened the OTLP receiver (gzip and size-bound handling, master-key refusal, partialSuccess reporting) and ingestion edge cases, with tests for each fix.NOTE:
When responding to reviewer questions, please respond yourself rather than copy/pasting reviewer comments into an AI and pasting back its answer. We want to discuss with you, not your AI :)
Summary
User.spend, enforcement, or pricing gating.Technical notes
(source, source_event_id).source = gateway, and imported events cannot claim thegatewaysource.