Repository navigation
Conversation
…hat need them OpenAI-compatible upstreams such as OpenRouter cache Claude only when the request marks what to cache, and the Claude-to-OpenAI translation drops the client's markers, so requests through such a provider were never cached. support-cache-control (per openai-compatibility provider, like support-prompt-cache-key) adds Anthropic-style cache_control to the last system message and the latest user/tool message of chat-completions requests that carry none, mirroring CPA's own Claude breakpoints. A Claude-format client's ttl "1h" is kept.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c0c88ce9d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // SupportCacheControl adds Anthropic-style cache_control breakpoints to chat-completions requests that carry | ||
| // none, for upstreams that accept them (e.g. OpenRouter, which caches Claude only when asked). The client's | ||
| // requested lifetime (ttl "1h" in a Claude-format request) is kept. | ||
| SupportCacheControl bool `yaml:"support-cache-control,omitempty" json:"support-cache-control,omitempty"` |
There was a problem hiding this comment.
In Home-enabled deployments, resolveCompatConfig rebuilds provider options from the selected auth's credential_options (openai_compat_executor.go:1154-1182), but that DTO and returned literal omit this new field and there is no support_cache_control attribute fallback. A Home credential that opts in therefore still reaches applyOpenAICompatCacheControlDoc with SupportCacheControl == false; propagate the flag through this resolver and the corresponding Home credential payload.
AGENTS.md reference: AGENTS.md:L63-L63
Useful? React with 👍 / 👎.
| part, err := ojson.SetBytes([]byte(`{"type":"text","text":""}`), "text", text) | ||
| if err != nil { | ||
| return | ||
| } | ||
| part, err = ojson.SetRawBytes(part, "cache_control", []byte(cacheControl)) |
There was a problem hiding this comment.
When a selected system/user/tool message has string content, this branch builds its replacement by calling ojson.SetBytes and ojson.SetRawBytes on raw byte slices. That bypasses the required single-*ojson.Doc mutation pipeline and serializes an intermediate request fragment; build the part with an ojson.Doc or mutate the existing request Doc directly, then serialize once at the final exit.
AGENTS.md reference: AGENTS.md:L53-L53
Useful? React with 👍 / 👎.
| executor := NewOpenAICompatExecutor("openai-compatibility", &config.Config{ | ||
| OpenAICompatibility: []config.OpenAICompatibility{{Name: "router", SupportCacheControl: true}}, | ||
| }) |
There was a problem hiding this comment.
When support-cache-control is combined with a payload filter or override targeting the generated marker, this test fixture has no payload rules and therefore never proves that the finalizer remains authoritative; moving the injection after ApplyPayloadConfigWithRequestDoc would silently restore configured-away fields while the test stayed green. Extend this regression to remove or override a marker and assert the wire body for both Execute and ExecuteStream.
AGENTS.md reference: AGENTS.md:L54-L54
Useful? React with 👍 / 👎.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Superseded by a broader OpenRouter change without a config flag: OpenAI-compatible entries pointing at OpenRouter get cache breakpoints, session affinity and model limits automatically (detected by base URL, like the existing reasoning-signature handling). New PR follows. |
Problem
OpenAI-compatible upstreams such as OpenRouter cache Claude only when the request marks what to cache (
cache_controlon content parts). CPA sends them the OpenAI chat format:So every request to Claude through such a provider re-bills the whole prompt. Measured through CPA 8.0.53: Claude via an OpenRouter entry never got a cache hit, while GPT models there (cached automatically) did.
Change
New opt-in per
openai-compatibilityprovider, next tosupport-prompt-cache-key:When it's set, chat-completions requests that carry no
cache_controlget Anthropic-style breakpoints:ttl: "1h"on any block) keeps it.It's opt-in because strict OpenAI-compatible servers may reject unknown fields in content parts. OpenRouter documents
cache_controlfor Anthropic models and ignores it elsewhere.Verification
openai_compat_cache_control_test.go:Live, real OpenRouter,
anthropic/claude-haiku-5.5, two identical Claude-format requests (~12.6k tokens):support-cache-controltruecache_creation_input_tokens: 12620cache_read_input_tokens: 12620falseinput_tokens: 12624input_tokens: 12624(no cache)go build -o test-output ./cmd/server; gofmt clean on changed files;go test ./internal/runtime/executor/ ./internal/config/ ./internal/translator/... ./sdk/...green.