Skip to content

fix(client): cap readiness probes by remaining wait budget - #5745

Open
linjiaye929 wants to merge 1 commit into
bentoml:mainfrom
linjiaye929:codex/client-readiness-budget
Open

linjiaye929 wants to merge 1 commit into
bentoml:mainfrom
linjiaye929:codex/client-readiness-budget

Conversation

@linjiaye929

@linjiaye929 linjiaye929 commented Oct 14, 2026 •

Copy link
Copy Markdown

What does this PR address?

Cap readiness-probe HTTP timeouts by the remaining server_ready_timeout budget in both public HTTP clients, while preserving a shorter configured request timeout. Reject a successful readiness response if it arrives after the monotonic deadline.

Previously the loop checked elapsed time only before GET /readyz, whose timeout remained the ordinary request timeout (30 seconds by default). With server_ready_timeout=1, a synthetic server delaying /readyz for 2 seconds could return 200 and the client would accept it and fetch the schema after the readiness budget had elapsed.

The change is local to the two readiness loops. None still falls back to the request timeout; nonpositive initialization timeouts still skip readiness waiting, while direct nonpositive waits still raise ServiceUnavailable. Ordinary endpoint timeouts and the exception message are unchanged.

Limit: HTTPX applies timeouts to individual request phases / inactivity. This caps each probe's phase timeout and rejects late success; it does not introduce a hard end-to-end deadline across all transport phases or client initialization.

Validation

Python 3.12.14, pytest 7.4.0 / pytest-asyncio 0.21.1 as declared by the project:

  • 26 deterministic public-client cases: 10 failed, 16 passed before the fix; 26 passed afterward. They use a controlled monotonic clock and real HTTPX request construction with MockTransport. MockTransport itself does not enforce timeouts; the cases inspect budgets, retries and late-response decisions.
  • New regressions plus the existing session-manager test: 27 passed, independently repeated.
  • Separate loopback-only HTTP reproduction with a 1-second readiness budget, delayed 200/503 responses and fast controls: 4 failed, 2 passed before; 6 passed after, independently repeated. No model/service deployment or external provider request.
  • Configured pre-commit run --all-files: passed; the two Buf hooks had no applicable files. The unchanged base passes the same checks.
  • New test file: strict Pyright 0 errors, 0 warnings. Production-file Pyright reports 9 errors / 5 warnings, identical to the unchanged base after excluding paths and shifted line numbers; unrelated typing issues are left unchanged.
  • Ruff 0.15.12 and git diff --check pass. Python 3.9 grammar was checked, but other Python runtimes, the full test suite and deployment/load tests were not run.

Before submitting

  • Conventional Commits title.
  • Configured all-file pre-commit checks completed.
  • Regression tests cover the changed behavior.
  • Full test/deployment matrix (limits above).

AI assistance was used for analysis, implementation and the stated automated checks. No human approval or hosted CI success is claimed.

Hosted checks

At head 374caa045d46f4def16355c875db96dc72a76015, pre-commit.ci succeeded. GitHub Actions CI, code-quality, and CodeQL are completed with action_required, each with zero jobs; there are zero check runs. Those Actions have not executed their tests/scans. This is not a claim that all hosted CI passed.

@linjiaye929
linjiaye929 marked this pull request as ready for review October 14, 2026 20:57
@linjiaye929
linjiaye929 requested a review from a team as a code owner October 14, 2026 20:57
@linjiaye929
linjiaye929 requested review from jianshen92 and removed request for a team October 14, 2026 20:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant