fix: survive transient MCP transport faults on preflight and exhausted retries#26
Conversation
…d retries Two transient-fault-is-fatal failure modes in the MCP HTTP path are now absorbed: - Preflight: retry transient transport faults (ConnectError/RemoteProtocolError) with capped exponential backoff before declaring a server unreachable, mirroring what RetryTransport does for in-session calls. Persistent errors (DNS/SSL/auth) still surface on every attempt. - Exhausted retries: rather than re-raising into the MCP SDK's session-scoped task group (which crashes the group and takes the whole agent task down at teardown), synthesize an in-band JSON-RPC error response echoing the request id. The SDK demuxes it to the foreground waiter as a catchable McpError. Falls back to re-raising when there's no id to route a response to.
| _PREFLIGHT_BACKOFF_SECONDS = 1.0 | ||
|
|
||
|
|
||
| async def _run_preflight(client, url: str, server_name: str) -> None: |
There was a problem hiding this comment.
🤖 [Quality] Minor: _run_preflight(client, ...) leaves client untyped while the rest of the module annotates public-ish signatures (CLAUDE.md calls for type hints on APIs). Since httpx is imported lazily inside the function body, a string annotation works without reordering: client: "httpx.AsyncClient". Not blocking — just keeps the signature self-documenting given the docstring already commits to it being an httpx.AsyncClient.
demianbrecht
left a comment
There was a problem hiding this comment.
🤖 [Security] No findings. Reviewed the MCP HTTP fault-handling changes (mcp.py preflight retry, oauth.py synthesized exhausted-retry response) plus test additions.
Critical: 0
High: 0
Medium: 0
Low: 0
Scope note: this PR is transport-resilience only — no auth/authz, injection sink, secret, or network-egress changes. Checked: (1) the synthesized JSON-RPC error parses the client's own outgoing request body (json.loads(request.content)), not untrusted external input — no injection vector; (2) the error message embeds the httpx transport exception string, which carries the config-supplied URL/host (already operator-known) but no credentials/tokens; (3) the preflight retry is bounded (3 attempts, capped backoff) with a narrow retryable set — no DoS amplification. The documented double-execution-of-non-idempotent-tools tradeoff on transport-fault retry is a correctness call the author accepted, consistent with the project's single-user local-only design intent — not a security defect. Tests use fictional hosts, no hard-coded real credentials.
demianbrecht
left a comment
There was a problem hiding this comment.
🤖 [Quality] Focused, well-executed fix for two transient-fault-is-fatal paths in the MCP HTTP layer. CI is green across the matrix (test 3.12/3.13/3.14, e2e, lint, SAST all passing).
Strengths:
- The why is documented exceptionally well — both new functions carry comments explaining the SDK task-group teardown mechanism, and the
RetryTransportidempotency caveat (retrying a transport fault can double-execute a non-idempotenttools/call) is called out honestly rather than papered over. - Test coverage is meaningful and exercises the real branches: preflight succeeds/timeout-benign/retry-then-raise/retry-then-succeed/non-transient, plus exhaustion-with-id (synthesizes), exhaustion-without-id (re-raises), and the connect-error path. Assertions check the actual JSON-RPC
idecho, error code, and attempt counts — not framework noise. - The in-band-error synthesis correctly degrades to re-raise when there's no JSON-RPC
idto route a response to (notification / unparseable body), which is the right boundary. - Patch version bump (0.10.1 -> 0.10.2) is appropriate for a bugfix.
One non-blocking nit posted inline: _run_preflight's client parameter is untyped while the surrounding code annotates signatures.
Observations (no action required):
- Preflight backoff adds up to ~3s (1s + 2s) of startup latency on a server that flaps transiently before failing, and sessions start sequentially — acceptable for the failure mode this targets, but worth being aware of if many HTTP servers are configured.
- The error-code-consistency claim (
httpx.codes.REQUEST_TIMEOUTmatching the SDK'sBaseSessionread-timeout path) depends on the pinned MCP SDK internals; if the SDK version moves, worth a re-check, but the chosen code is harmless either way since JSON-RPC permits any integer.
Summary
Closes two transient-fault-is-fatal failure modes in the MCP HTTP path, where a momentary transport hiccup would abort an entire agent task.
Preflight retry (
mcp.py)The startup connectivity probe now retries transient transport faults (
ConnectError/RemoteProtocolError) with capped exponential backoff before declaring a server unreachable. This mirrors whatRetryTransportalready does for in-session tool calls, but on the startup path theRetryTransportisn't in play yet. Persistent errors (DNS / SSL / auth) still fail on every attempt and surface, preserving diagnostic value. A GET timeout remains a benign "reachable" signal (MCP endpoints are POST-oriented).In-band error on exhausted retries (
oauth.py)The MCP streamable-HTTP SDK runs each id-bearing
tools/callas a background task in a session-scoped anyio task group, outside its owntry/except. Re-raising a transport fault after retries are exhausted crashes the whole group, cancels the foreground request, and detonates at session teardown — taking the entire agent task down (a transient codesearch timeout should never be fatal).Instead,
RetryTransportnow synthesizes an in-band JSON-RPC error response echoing the requestid(using the SDK's ownREQUEST_TIMEOUTcode for consistency). The background task completes normally; the SDK demuxes the response to the foreground waiter, which raises a catchableMcpErrorthat the tool loop already handles. Falls back to re-raising when the request has no usable JSON-RPCid(notification / unparseable body) — there's no foreground waiter to route a response to.Testing
make test— 741 passed, 7 skippedTestPreflightRetry(succeeds first try, timeout-is-benign, retry-then-raise, retry-then-succeed, non-transient-no-retry) andRetryTransportexhaustion tests (synthesizes in-band error with id, re-raises without id, connect-error path).