fix(flights): stop a dead tile server freezing the scroll - #272
Conversation
Map tiles are fetched from inside get_vegas_content(), which the Vegas
coordinator calls on the render thread. So every second spent waiting on
a tile is a second the marquee is stopped -- and nothing bounded that
wait: a 10s timeout, tried against each of two URLs, for every tile in
the grid.
Found by profiling the live rig with py-spy, which sampled the main
thread stuck 5.30s in one getaddrinfo:
coordinator.run_iteration -> stream_manager._fetch_plugin_content
-> plugin_adapter._get_native_content
-> get_vegas_content -> _render_map_image -> _get_map_background
-> _fetch_tile -> requests.get -> socket.getaddrinfo
The timeout is now 3s, generous for a tile that normally arrives in tens
of milliseconds. More importantly, one failure switches tiles to
cache-only for five minutes: a tile failing means the server or the
network is gone, and every other tile in the grid is about to discover
that at the same price. The map draws with the tiles it already has.
This does not move the fetch off the render thread, which is the real
architectural fix -- it bounds what the render thread can lose. Doing it
properly means fetching tiles during update() and compositing from cache
only, which is a larger change than this one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5
|
Warning Review limit reached
Next review available in: 51 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 16 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
|
@coderabbitai review |
|
2 participants
The freeze, finally located
Map tiles are fetched from inside
get_vegas_content(), which the Vegas coordinator calls on the render thread. Every second waiting on a tile is a second the marquee is stopped — and nothing bounded it: a 10s timeout, tried against each of two URLs, for every tile in the grid.Found by profiling the live rig with py-spy, which — unlike an in-process watchdog — can sample a thread blocked in a syscall. The main thread, stuck 5.30 seconds in one
getaddrinfo:This is the stutter chased across several PRs. It is not GIL contention, not the strip rebuild, not logo decoding, and not the leaderboard — each of which I measured and eliminated first, several after predicting they were the cause.
The fix
Worst case per render goes from (10s × 2 URLs × every tile in the grid) to one 3s wait per five minutes.
What this does NOT do
It does not move tile fetching off the render thread. That is the real architectural fix — fetch during
update(), composite from cache only — and it is a larger change. This bounds the damage rather than removing the cause; the render thread can still lose ~6s once per cooldown window.A second, separate freeze remains untouched: the main loop blocking in
_update_modules -> execute_with_timeout -> threading.join(), profiled at 9.34s, bounded only by the executor's 30s timeout. Core-side, needs its own PR.Tests
test_tile_network_bound.py: the timeout is short enough for a render thread, one tile cannot exhaust a 30s update budget, a failing tile stops the remaining 40 in the grid from trying (2 network attempts total, not 82), the block expires and the network is retried, cached tiles are still served while blocked, and an uncached one returnsNonein under 50ms rather than hanging.test_no_disk_write_on_poll.py(from #269) still passes. Collisions clean across 43 plugins; safety harness passes all 8 sizes.🤖 Generated with Claude Code
https://claude.ai/code/session_01Udr6MfaFLUPhX5Fgo67Jf5