fix(anthropic): fail over on upstream transport errors instead of immediate 502 - #5670
Open
genhoi wants to merge 1 commit into
Open
fix(anthropic): fail over on upstream transport errors instead of immediate 502#5670genhoi wants to merge 1 commit into
genhoi wants to merge 1 commit into
Conversation
…ediate 502 When DoWithTLS fails at the transport level (connection refused, dial/DNS/TLS error — no HTTP status received), the Anthropic channel wrote a 502 to the client straight from the service and returned a plain error, so the handler's failover loop never tried the other accounts in the group. The OpenAI channel already handles this correctly via handleOpenAIUpstreamTransportError. Port that pattern to the Anthropic channel: - new handleAnthropicUpstreamTransportError returns *UpstreamFailoverError(502) without writing the response, so the handler retries the request on another account; context.Canceled still short-circuits (client gone: no failover, no eviction); - durable faults (connection refused, no route, DNS failure, rejected proxy credentials — classified by the existing channel-agnostic classifier) also temp-unschedule the account for 10 minutes so subsequent requests skip it; - both the API-key passthrough path and the regular forward path are covered; Ops error logging and Ollama activity accounting are preserved as-is. Client-visible behavior when every account fails stays the same: the handler's failover-exhausted path emits the identical 502 body. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
|
All contributors have signed the CLA. ✅ |
Author
I have read the CLA Document and I hereby sign the CLA |
genhoi
marked this pull request as draft
August 15, 2026 10:16
genhoi
marked this pull request as ready for review
August 15, 2026 10:17
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
On the Anthropic channel, a transport-level upstream failure (connection refused, dial/DNS/TLS error — the HTTP round-trip never completed) is written to the client as an immediate 502 straight from the service layer, and a plain error is returned:
backend/internal/service/gateway_anthropic_passthrough.go(API-key passthrough path)backend/internal/service/gateway_forward.go(regular forward path)Because the returned error is not an
*UpstreamFailoverError, the handler's failover loop never tries the other accounts in the group — even when a healthy account is available. A single dead upstream endpoint (or a dead egress proxy) hard-fails every request routed to that account.The OpenAI channel already handles this correctly via
handleOpenAIUpstreamTransportError(returns a failover error, does not write the response, temp-unschedules the account on durable faults). The Anthropic channel simply never got the same treatment.Fix
Port the OpenAI pattern to the Anthropic channel (
backend/internal/service/anthropic_upstream_transport_error.go):handleAnthropicUpstreamTransportErrorreturns*UpstreamFailoverError{502}without writing the response, so the handler fails over to a healthy account within the same client request.context.Canceledstill short-circuits with a plain error (client gone: no failover, no eviction).classifyOpenAITransportError) additionally temp-unschedule the account for 10 minutes (DB-persisted, matching this channel's other auto-unschedule helpers), so subsequent requests skip the dead account too.kind=request_error) and Ollama activity accounting are preserved as-is.Client-visible behavior when every account fails stays identical: the handler's failover-exhausted path emits the same 502 body as before.
Tests
TestGatewayService_AnthropicAPIKeyPassthrough_ForwardDirect_UpstreamRequestErrorto the new contract: transport failure surfaces as*UpstreamFailoverError{502}and the service does not write the response...._PersistentTransportErrorTempUnschedules: connection refused → failover error + oneSetTempUnschedulablecall...._TransientTransportErrorKeepsAccountSchedulable: dial timeout → failover error, account stays schedulable../internal/service(with and without-tags unit) and./internal/handler/...suites pass.Live verification
Verified on a real deployment (two
platform=anthropic, type=apikeyaccounts in one group, priorities 10/90): with the primary account'sbase_urlpointed at a dead endpoint, the same client request that previously got an instant 502 is now transparently served by the fallback account (HTTP 200), the dead account is temp-unscheduled for 10 minutes with a clear reason, andgateway.failover_switch_accountshowsswitch_count=1. After restoring the endpoint and the cooldown clearing, traffic returns to the primary account.🤖 Generated with Claude Code