Skip to content

fix(anthropic): fail over on upstream transport errors instead of immediate 502 - #5670

Open
genhoi wants to merge 1 commit into
Wei-Shaw:mainfrom
genhoi:fix/anthropic-transport-failover
Open

fix(anthropic): fail over on upstream transport errors instead of immediate 502#5670
genhoi wants to merge 1 commit into
Wei-Shaw:mainfrom
genhoi:fix/anthropic-transport-failover

Conversation

@genhoi

@genhoi genhoi commented Aug 15, 2026

Copy link
Copy Markdown

Problem

On the Anthropic channel, a transport-level upstream failure (connection refused, dial/DNS/TLS error — the HTTP round-trip never completed) is written to the client as an immediate 502 straight from the service layer, and a plain error is returned:

  • backend/internal/service/gateway_anthropic_passthrough.go (API-key passthrough path)
  • backend/internal/service/gateway_forward.go (regular forward path)

Because the returned error is not an *UpstreamFailoverError, the handler's failover loop never tries the other accounts in the group — even when a healthy account is available. A single dead upstream endpoint (or a dead egress proxy) hard-fails every request routed to that account.

The OpenAI channel already handles this correctly via handleOpenAIUpstreamTransportError (returns a failover error, does not write the response, temp-unschedules the account on durable faults). The Anthropic channel simply never got the same treatment.

Fix

Port the OpenAI pattern to the Anthropic channel (backend/internal/service/anthropic_upstream_transport_error.go):

  • handleAnthropicUpstreamTransportError returns *UpstreamFailoverError{502} without writing the response, so the handler fails over to a healthy account within the same client request. context.Canceled still short-circuits with a plain error (client gone: no failover, no eviction).
  • Durable faults (connection refused, no route, network unreachable, DNS failure, rejected proxy credentials — classified by the existing channel-agnostic classifyOpenAITransportError) additionally temp-unschedule the account for 10 minutes (DB-persisted, matching this channel's other auto-unschedule helpers), so subsequent requests skip the dead account too.
  • Both call sites (passthrough + regular forward) now delegate to the helper. Ops error logging (status 0, kind=request_error) and Ollama activity accounting are preserved as-is.

Client-visible behavior when every account fails stays identical: the handler's failover-exhausted path emits the same 502 body as before.

Tests

  • Flipped TestGatewayService_AnthropicAPIKeyPassthrough_ForwardDirect_UpstreamRequestError to the new contract: transport failure surfaces as *UpstreamFailoverError{502} and the service does not write the response.
  • New ..._PersistentTransportErrorTempUnschedules: connection refused → failover error + one SetTempUnschedulable call.
  • New ..._TransientTransportErrorKeepsAccountSchedulable: dial timeout → failover error, account stays schedulable.
  • Full ./internal/service (with and without -tags unit) and ./internal/handler/... suites pass.

Live verification

Verified on a real deployment (two platform=anthropic, type=apikey accounts in one group, priorities 10/90): with the primary account's base_url pointed at a dead endpoint, the same client request that previously got an instant 502 is now transparently served by the fallback account (HTTP 200), the dead account is temp-unscheduled for 10 minutes with a clear reason, and gateway.failover_switch_account shows switch_count=1. After restoring the endpoint and the cooldown clearing, traffic returns to the primary account.

🤖 Generated with Claude Code

…ediate 502

When DoWithTLS fails at the transport level (connection refused, dial/DNS/TLS
error — no HTTP status received), the Anthropic channel wrote a 502 to the
client straight from the service and returned a plain error, so the handler's
failover loop never tried the other accounts in the group. The OpenAI channel
already handles this correctly via handleOpenAIUpstreamTransportError.

Port that pattern to the Anthropic channel:
- new handleAnthropicUpstreamTransportError returns *UpstreamFailoverError(502)
  without writing the response, so the handler retries the request on another
  account; context.Canceled still short-circuits (client gone: no failover,
  no eviction);
- durable faults (connection refused, no route, DNS failure, rejected proxy
  credentials — classified by the existing channel-agnostic classifier) also
  temp-unschedule the account for 10 minutes so subsequent requests skip it;
- both the API-key passthrough path and the regular forward path are covered;
  Ops error logging and Ollama activity accounting are preserved as-is.

Client-visible behavior when every account fails stays the same: the handler's
failover-exhausted path emits the identical 502 body.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@genhoi

genhoi commented Aug 15, 2026

Copy link
Copy Markdown
Author

Thank you for your contribution! Before we can merge this PR, we need you to sign our Contributor License Agreement (CLA).

To sign, please reply with the following comment:

I have read the CLA Document and I hereby sign the CLA

You only need to sign once — it will be valid for all your future contributions to this project.

I have read the CLA Document and I hereby sign the CLA

You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

I have read the CLA Document and I hereby sign the CLA

@genhoi
genhoi marked this pull request as draft August 15, 2026 10:16
@genhoi
genhoi marked this pull request as ready for review August 15, 2026 10:17
@genhoi

genhoi commented Aug 15, 2026

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant