Skip to content

Claude: cost per session and per project - #5026

Open
oryband wants to merge 14 commits into
steipete:mainfrom
oryband:feat/claude-session-cost
Open

oryband wants to merge 14 commits into
steipete:mainfrom
oryband:feat/claude-session-cost

Conversation

@oryband

@oryband oryband commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #4930, for Claude only, as suggested there. The Pi version is #5025.

What it does

  1. codexbar cost --provider claude --group-by session and --group-by project now work.
  2. The Cost menu and Usage & Spend show Claude projects and conversations, like Codex.
  3. A project is the session's cwd. A conversation shows its short session id, as Codex text output does.
  4. JSON cost output now includes Claude projects[]. Like Codex, it has no per-session rows.
  5. Vertex AI is unchanged.

How it works

  • Attribution comes from the rows the scanner already reconciles. Each winning row (after the existing message and request dedup, and the sidechain and subagent rules) belongs to its sessionId. Sessions and projects are built from those same rows, so they add up to the Claude total.
  • Each session keeps its own first cwd, even when one file holds several sessions.
  • Appends start from the cached cwd map. A full parse or a forced rescan rebuilds it.
  • The cache artifact moves to v6, so history rebuilds once from the transcripts. Older report memos still decode and rebuild their breakdowns once.
  • Breakdowns are saved in the report memo, so a relaunch does not decode the full cache again.
  • When Pi rows are merged into Claude, Claude shows Pi as one "Unknown project" row and no conversations, the same as Codex today.

Cost

Benchmarks against main, debug build, 3 interleaved rounds:

  • Full rebuild, 5,000 files and 250k rows: 2.85 s on main, 3.29 s on this branch (+15%).
  • Refresh after an append, 24k rows: about +0.075 s per refresh.
  • Session and project reports skip the hourly and quota buckets, so each row is priced at most 3 times.

Tests

All fixtures are synthetic.

  • Ownership: rows go to the right session and project, and the sums match the total.
  • Two sessions in one file get separate projects.
  • Append, rewrite, a response copied across resumed transcripts, and sidechain and subagent copies.
  • Cache upgrade to v6, and a relaunch that restores breakdowns without decoding the cache.
  • An unpriced day stays unpriced in every group.
  • CLI text and JSON for both groupings.

Commands run:

swift test --filter 'CostUsageClaude|CostUsageScannerClaude|CostUsageFetcherClaude|CLICost|CostHistoryChartMenuView|PiInclusive'
make check
CODEXBAR_REBUILD_BENCHMARK=1 swift test --skip-build --filter CostUsageClaudeRebuildBenchmarkTests
CODEXBAR_REFRESH_BENCHMARK=1 swift test --skip-build --filter CostUsageClaudeRefreshBenchmarkTests

@clawsweeper

clawsweeper Bot commented Nov 8, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-14T13:36:07.230516Z c064945 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Nov 8, 2026
@clawsweeper

clawsweeper Bot commented Nov 8, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge.

What this changes

This PR adds Claude session and project cost breakdowns to CLI output, the Cost menu, and Usage & Spend.

Example: Inspect session-alpha with 110 recorded tokens under /Users/test/projects/alpha.

  • Before: Claude session grouping is skipped because only Codex supports it.
  • After: Claude session grouping lists session-alpha, and project grouping attributes its usage to alpha.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The implementation has substantial accounting coverage, but a concrete presentation regression remains and required shipped-behavior and latest-stable upgrade evidence is absent.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is necessary before merge. See Before merge.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Product

Kind: Feature · Worth it: Needs a maintainer decision · Fix scope: Partial
User problem: Users can inspect aggregate Claude costs but cannot identify each conversation’s or project’s contribution through the existing grouped views.
Reason: The implementation supplies the focused Claude contribution requested for reconsideration in #4930, while Pi and optional task aggregation remain outside its scope.

Merge readiness

⛔ Blocked before merge - 4 items remain

Keep this PR open: it delivers a distinct Claude capability invited for reconsideration, but grouped output loses an established incomplete-usage warning and the feature still needs an owner decision.

Priority: P2
Reviewed head: 914377c665799bde474836ee9f563f19f2d21b91
Owner decision: Required. See Decision needed.

Decision needed

  • Question: Should Claude session/project breakdowns ship with persisted working-directory attribution and a one-time Claude/Vertex cache rebuild?
  • Recommendation: Accept the focused Claude feature: Approve the direction, with the correctness finding and contributor proof gates resolved before merge.
  • Why: The owner invited a focused implementation for reconsideration but has not approved this implementation or its storage and performance tradeoffs.

Before merge

  • Add real behavior proof - The supplied evidence exercises scanner, fetcher, and rendering code through synthetic tests and benchmarks, but shows no shipped CLI output or app capture demonstrating Claude project/conversation attribution after the change; provide redacted terminal output and screenshots or a recording, removing API keys, IPs, and non-public endpoints. After adding proof, update the PR body, then request a new review with @clawsweeper re-review. The PR author or someone with repository write access can request it.
  • Preserve incomplete-usage warnings in Claude grouped reports (P2) - With one completed Claude response and one preliminary proxy response (stop_reason: null, positive input, zero output, and no cache counters), the scanner excludes the preliminary response and retains an incomplete-request count. Enabling Claude in these early-return branches bypasses the existing exclusion warning below, and neither grouped renderer reads that count. Both --group-by session and --group-by project therefore present the completed-response subtotal without disclosing the missing usage, contrary to the established contract in fix(cost): avoid pricing incomplete Claude proxy usage #3688. Carry the existing incomplete/unpriced notices into grouped output and add a mixed completed/preliminary regression.
  • Add data-model compatibility proof - The review found that existing stored data may not work after upgrade. Show that existing data still loads and works with this change.
  • Explain or restore the original intent of Incomplete Claude usage presentation - fix(cost): avoid pricing incomplete Claude proxy usage #3688: Exclude preliminary proxy requests while clearly marking remaining known amounts as partial subtotals across user-facing reports.

Findings

  • [P2] Preserve incomplete-usage warnings in Claude grouped reports — Sources/CodexBarCLI/CLICostCommand.swift:213-220

Provenance

Tests

  • Missing end-to-end proof: Supply shipped CLI and app results for Claude grouping, including incomplete usage, plus a latest-stable upgrade with existing state preserved; the new warning regression has no demonstrated base-fail/head-pass run.
Agent review details

How this fits together

The local cost scanner reads Claude transcripts, reconciles duplicate usage records, persists generated caches, and supplies estimated costs to the CLI and app views.

flowchart LR
  A[Claude transcripts] --> B[Parser and reconciliation]
  B --> C[Generated cache]
  C --> D[Daily and grouped reports]
  D --> E[Cost fetcher]
  E --> F[CLI output]
  E --> G[Menu and Usage Spend]
Loading

Technical review

Best possible solution:

Use the existing scanner and presentation owners for Claude breakdowns while preserving incomplete and unpriced subtotal warnings in grouped output.

Do we have a high-confidence way to reproduce the issue?

Current-main provider filtering and tests establish that Claude grouped text output is unavailable; source inspection also establishes the introduced warning bypass without executing target code.

Is this the best way to solve the issue?

Partitioning existing reconciled rows into existing breakdown models is appropriate, but reusing the Codex renderers must preserve Claude’s incomplete-usage contract.

Full review comments:

  • [P2] Preserve incomplete-usage warnings in Claude grouped reports — Sources/CodexBarCLI/CLICostCommand.swift:213-220
    With one completed Claude response and one preliminary proxy response (stop_reason: null, positive input, zero output, and no cache counters), the scanner excludes the preliminary response and retains an incomplete-request count. Enabling Claude in these early-return branches bypasses the existing exclusion warning below, and neither grouped renderer reads that count. Both --group-by session and --group-by project therefore present the completed-response subtotal without disclosing the missing usage, contrary to the established contract in fix(cost): avoid pricing incomplete Claude proxy usage #3688. Carry the existing incomplete/unpriced notices into grouped output and add a mixed completed/preliminary regression.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.94

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 7447bc77c249.

Provenance checked

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

Likely related people:

  • Peter Steinberger: Raw commit 1e49619 adds Sources/CodexBarCore/CostUsageFetcher.swift:197 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 1e49619e7330; files: Sources/CodexBarCore/CostUsageFetcher.swift)
  • Yuxin Qiao: Raw commit 839464d adds Sources/CodexBarCLI/CLICostCommand.swift:319 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 839464de743b; files: Sources/CodexBarCLI/CLICostCommand.swift)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Labels

Label changes:

  • add P2: This adds useful cost-inspection capability with limited blast radius; existing aggregate reporting remains available.
  • add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
  • add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Label justifications:

  • P2: This adds useful cost-inspection capability with limited blast radius; existing aggregate reporting remains available.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Provide redacted CLI results and app captures showing the changed Claude grouping behavior.
  • Demonstrate an upgrade from v0.84.0 with existing settings, source transcripts, and cost results preserved.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

Reviewed November 8, 2026, 9:52 AM ET / 14:52 UTC.

@oryband

oryband commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

Real behavior proof, from the CLI built from this branch (head b8b066983).

Setup:

  • 4 synthetic Claude Code transcripts in 3 projects, made by a small script. No real transcripts.
  • A throwaway home folder (HOME and CFFIXED_USER_HOME), so the real cache stays untouched.
  • Times are UTC.

1. Upgrade from 0.81.0

CodexBar 0.81.0 ran first on the same home and built its Claude cache. Then the branch build ran:

0.81.0  $ codexbar cost --provider claude
Today: $0.24 · 235K tokens
Last 30 days: $0.24 · 235K tokens

branch  $ codexbar cost --provider claude
Today: $0.24 · 235K tokens
Last 30 days: $0.24 · 235K tokens

The totals match.

2. Sessions and projects

$ codexbar cost --provider claude --group-by session
Conversations (Last 30 days):
Session f6f6...00000004: $0.02 · 24K tokens · 1 requests
claude-sonnet-4-5 · Nov 8, 11:52
Session e5e5...00000003: $0.10 · 94K tokens · 4 requests
claude-sonnet-4-5 · Nov 8, 11:22
Session d4d4...00000002: $0.05 · 47K tokens · 2 requests
claude-sonnet-4-5 · Nov 8, 10:02
Session d4d4...00000001: $0.07 · 70K tokens · 3 requests
claude-sonnet-4-5 · Nov 8, 09:12

$ codexbar cost --provider claude --group-by project
Projects (Last 30 days):
alpha: $0.12 · 118K tokens
  /Users/test/projects/alpha
beta: $0.10 · 94K tokens
  /Users/test/projects/beta
gamma: $0.02 · 24K tokens
  /Users/test/projects/gamma

The projects add up to the $0.24 total. Each session's time is its last reply.

3. Append

I appended 2 replies to session d4d4...0001:

$ codexbar cost --provider claude --group-by session --refresh
Session d4d4...00000001: $0.12 · 118K tokens · 5 requests
claude-sonnet-4-5 · Nov 8, 14:48
(other 3 sessions unchanged)

$ codexbar cost --provider claude --group-by project
alpha: $0.17 · 164K tokens
beta: $0.10 · 94K tokens
gamma: $0.02 · 24K tokens

Only that session and its project changed. The new total, $0.29, equals the sum of the projects.

The session view also prints the existing "Pi/OMP usage is omitted" warning, because the default run can merge Pi rows into Claude.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fa7e607aca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +219 to +220
if groupBy == .session, Self.costGroupingProviders.contains(provider) {
return Self.renderSessionCostText(header: header, snapshot: snapshot, provider: provider)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve incomplete-usage warnings in Claude grouped reports

When Claude logs contain both completed usage and a proxy request lacking final usage, these new grouped paths return before the incompleteRequestCount warning logic. Neither grouped renderer checks that count, while a fully parsed transcript still has historyIsFullyScanned == true, so --group-by session and --group-by project show completed-only cost/token subtotals without any Incomplete marker; the default report correctly warns for the same snapshot. Carry the incomplete-usage notice into grouped output, or mark affected groups, so users can tell these amounts exclude known requests.

Useful? React with 👍 / 👎.

steipete and others added 3 commits November 14, 2026 02:35
Merge current main without losing session and project ownership or shared exact-event buffers. Preserve incomplete-usage notices in grouped CLI output and cover attribution alongside slice sharing.
Merge current main without rewriting the contributor history. Share local cost grouping and preserve incomplete-usage notices, Pi omission rules, and exact quota-event storage.

Co-authored-by: Ory Band <oryband@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants