HS-1258249: fix uncatchable realtime 1008 error and endless 1-second reconnect loop - #327
HS-1258249: fix uncatchable realtime 1008 error and endless 1-second reconnect loop#327claudear wants to merge 2 commits into
Conversation
…o the zone A server-sent realtime `error` frame with code 1008 was rethrown from inside the WebSocket stream listener. A throw there is forwarded to the zone's uncaught-error handler, so it never reached the `try/catch` in `_createSocket`, never cancelled the subscription, never closed the socket and was never delivered to `subscription.stream.listen(onError:)` — app code had no way to catch it and re-authenticate. The only thing that stopped the reconnect loop afterwards was the incidental `_websok?.closeCode == status.policyViolation` check in `_retry`, which is unreliable: `closeCode` is null when the failure surfaces through the channel's onError path, and 1006/1000 when a proxy such as a Cloudflare tunnel terminates the connection without forwarding the server's 1008 close frame. In those cases the client reconnected into the same rejection once a second, forever. - `handleError` now records the fatal state, stops the heartbeat, cancels the subscription, closes the socket and delivers the `AppwriteException` to every subscriber. - `_retry` is gated on that recorded state rather than the racy `closeCode` check, and no longer resets `_reconnect = true` on its early return. - `_retries` is reset on the application-level `connected` event instead of right after the transport connects, so the backoff escalates when the server keeps rejecting the connection. - onError/onDone no longer schedule two reconnects for one failure, and the stale stream subscription is cancelled before being replaced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Greptile SummaryThe PR makes policy-violation errors catchable, terminates rejected sockets, and prevents endless immediate reconnects.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains. Important Files Changed
Reviews (2): Last reviewed commit: "fix(realtime): don't reuse a rejected so..." | Re-trigger Greptile |
…failure Addresses two gaps in the previous commit found in review: - `_createSocket` reused the existing socket when the URL matched and `closeCode` was still null. After a policy violation that close is still in flight, so re-subscribing after re-authenticating pushed the pending subscribes into a dying connection and left `_fatalError` set, leaving the client permanently disconnected. The reuse branch now also requires that no fatal error is recorded. - `handleError`'s recoverable branch called `_retry()` directly, so an error frame followed by the stream terminating scheduled two reconnects and counted one failure twice. Retry deduplication moved from a closure local to `_createSocket` into `_scheduleRetry`, which all three paths now use. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fix Confidence: 88/100The reproducing test failed before the change with the exact stack frames from the customer report (realtime_mixin.dart 385:7 handleError ← 107:13) and passes after. Full suite of 388 existing tests still passes, analyze shows no new issues, and CI is green. I verified the socket-reuse guard is load-bearing by removing it and watching the test fail. Remaining uncertainty: the tests use a fake WebSocketChannel, so real-world platform close-code timing (IO vs HtmlWebSocketChannel) isn't exercised end-to-end, and the reconnect/backoff paths involve real-time delays that are covered only indirectly. The retry-dedup change alters retry counting on paths not directly asserted by a test. |
Problem
Reported in HS-1258249: a self-hosted user behind a Cloudflare tunnel gets this on Flutter web after the socket has been idle for a couple of hours, repeating once per second forever:
Root cause
handleErrortreated a 1008 policy violation as fatal bythrowing — but it runs inside the_websok.stream.listenonData callback. A throw there is forwarded to the zone's uncaught-error handler, so it:try/catchin_createSocket(which has already returned),subscription.stream.listen(onError:), so app code cannot catch it, prompt for re-auth, or stop the client.The fatal signal was simply discarded. The only thing that then stopped the reconnect loop was the incidental
_websok?.closeCode == status.policyViolationcheck in_retry, and that check is unreliable —closeCodeisnullwhen the failure surfaces through the channel's onError path, and1006/1000when a proxy or tunnel terminates the connection without forwarding the server's 1008 close frame. In all of those cases_retry()reconnected straight back into the same rejection.On top of that,
_retries = 0was reset immediately after the WebSocket object was created, before the application-levelconnectedhandshake. When the transport connects fine but the server rejects at the app level, the counter was wiped on every attempt, so_getTimeout()always returned 1 — matching the repeatedReconnecting in 1 seconds.in the report.Fix
handleErrorno longer throws. It records the fatal state, stops the heartbeat, cancels the stream subscription, closes the socket, and delivers theAppwriteExceptionto every subscriber's controller so applications can handle it._retryis gated on that recorded state instead of the racycloseCodecheck, and no longer resets_reconnect = trueon its early return (which made the old guard one-shot)._retriesis reset in the'connected'case, after the handshake succeeds, so the backoff actually escalates on post-connect failures.onErrorandonDoneno longer schedule two reconnects for a single failure (the browser channel emits error-then-done), and the stale_websocketSubscriptionis cancelled before being replaced.A fresh
_createSocket()— i.e. an explicitsubscribeTo/upsertPresenceafter the app re-authenticates — clears the recorded fatal state, so the client is usable again.Tests
New
test/src/realtime_mixin_test.dart:handleError→_createSocket.<fn>stack trace from the ticket and no error delivered to the subscriber. Now theAppwriteExceptionreachessubscription.stream, the socket is closed, and no reconnect is scheduled.Full suite: 388 existing tests + 2 new ones pass.
flutter analyzereports no new issues in the touched files.🤖 Generated with Claude Code