Skip to content

fix(agent-core): gate prompt_cache_key behind isOfficialOpenAIBaseUrl for third-party endpoints #2761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
creatiVision wants to merge 2 commits into
base: main
Choose a base branch
from
+39 −5
Open

fix(agent-core): gate prompt_cache_key behind isOfficialOpenAIBaseUrl for third-party endpoints #2761

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e430cb3
fix(agent-core): gate prompt_cache_key behind isOfficialOpenAIBaseUrl
Aug 9, 2026
601e041
chore: add changeset for prompt_cache_key fix
Aug 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/strong-ears-kick.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@moonshot-ai/kimi-code": patch
---

Stop sending the `prompt_cache_key` parameter to third-party OpenAI-compatible providers (NVIDIA NIM, Azure Foundry, etc.) that reject it with a 400 error. Only official OpenAI endpoints and Kimi now receive this parameter.
39 changes: 34 additions & 5 deletions packages/agent-core/src/session/provider-manager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,23 @@ import {
} from '../config';
import { ErrorCodes, isKimiError, KimiError } from '../errors';

/**
* Check whether `baseUrl` points to an official OpenAI API endpoint.
* `undefined` means the client defaults to OpenAI. Only official endpoints
* should receive the `prompt_cache_key` field, because strictly-validating
* OpenAI-compatible endpoints (NVIDIA, Azure Foundry, etc.) reject unknown
* parameters with HTTP 400.
*/
function isOfficialOpenAIBaseUrl(baseUrl: string | undefined): boolean {
if (baseUrl === undefined) return true;
try {
const hostname = new URL(baseUrl).hostname;
return hostname === 'api.openai.com' || hostname.endsWith('.api.openai.com');
} catch {
return false;
}
}

export interface BearerTokenProvider {
getAccessToken(options?: { readonly force?: boolean }): Promise<string>;
}
Expand Down Expand Up @@ -327,9 +344,14 @@ function toKosongProviderConfig(
offEffort,
// Session affinity: route every request of this session through the
// same provider-side prompt cache (the OpenAI analog of Anthropic
// `metadata.user_id` above). Undefined values are stripped at
// generate time, matching the `kimi` branch below.
generationKwargs: { prompt_cache_key: promptCacheKey },
// `metadata.user_id` above). Only sent to official OpenAI API
// endpoints — strictly-validating OpenAI-compatible third-party
// endpoints (NVIDIA, Azure Foundry, etc.) reject unknown parameters.
...(promptCacheKey !== undefined && isOfficialOpenAIBaseUrl(

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add a changeset for this provider fix

This is a user-visible provider behavior change, but the commit only changes provider-manager.ts and adds no .changeset/ entry, so changeset-based release/version notes can omit the third-party endpoint fix. Add an appropriate changeset before this PR is submitted.

AGENTS.md reference: AGENTS.md:L84-L85

Useful? React with 👍 / 👎.

modelBaseUrl ?? providerValue(provider.baseUrl, provider.env, 'OPENAI_BASE_URL'),
)
? { generationKwargs: { prompt_cache_key: promptCacheKey } }
: {}),
...defaultHeadersField({
...envCustomHeaders,
...kimiUserAgentHeader(kimiRequestHeaders),
Expand All @@ -342,7 +364,8 @@ function toKosongProviderConfig(
model,
baseUrl: modelBaseUrl ?? providerValue(provider.baseUrl, provider.env, 'KIMI_BASE_URL'),
apiKey: providerApiKey(provider),
generationKwargs: { prompt_cache_key: promptCacheKey },
// Kimi's API accepts prompt_cache_key on any endpoint.
...(promptCacheKey !== undefined ? { generationKwargs: { prompt_cache_key: promptCacheKey } } : {}),
...defaultHeadersField({
...envCustomHeaders,
...kimiRequestHeaders,
Expand Down Expand Up @@ -372,7 +395,13 @@ function toKosongProviderConfig(
offEffort,
// Session affinity: same `prompt_cache_key` intent as the `openai`
// branch; the Responses API accepts it as a top-level request field.
generationKwargs: { prompt_cache_key: promptCacheKey },
// Only sent to official OpenAI API endpoints — third-party endpoints
// reject unknown parameters.
...(promptCacheKey !== undefined && isOfficialOpenAIBaseUrl(
modelBaseUrl ?? providerValue(provider.baseUrl, provider.env, 'OPENAI_BASE_URL'),
)
? { generationKwargs: { prompt_cache_key: promptCacheKey } }
: {}),
...defaultHeadersField({
...envCustomHeaders,
...kimiUserAgentHeader(kimiRequestHeaders),
Expand Down