You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#157 fixed recovery after EngineIO errors, and it works. But the recovery path is gated on the error being classified as a connection error. When the WebSocket is re-established at transport level while the application session is gone, the resulting error is ServerError — which is not in that list — so the cached client is kept indefinitely and the sync stays dead until the container is restarted.
What we observed
The reverse proxy sitting in front of Uptime Kuma was restarted, causing a short outage (seconds).
Uptime Kuma was reachable again within ~2 minutes. This is not an assumption: a different AutoKuma instance was started during that window and connected successfully on its first cycle.
The already-running instance never synced again — 13 consecutive 60s cycles — until we recreated the container. It came back immediately after.
The health endpoint added in rc.2 correctly reported the problem (503 on /health, container went unhealthy). That is how we caught it. But nothing recovered on its own.
We could not capture the exact log line, as the container was replaced before we could pull its logs. What the behaviour does establish is that the error was not in the is_connection_error set, otherwise the client would have been dropped and reconnected on the following cycle.
Why (reading the code)
In Sync::run (autokuma/src/sync.rs), the client is only invalidated for connection errors:
ifletErr(err) = self.do_sync().await{warn!("Encountered error during sync: {}", err);ifSelf::is_connection_error(&err){debug!("Connection error detected, will reconnect on next cycle");self.client = None;}}
get_connection reconnects only when self.client.is_none(), so anything outside that set means the dead client is reused forever.
Not covered, in particular:
ServerError(String) — the variant behind "Server responded with an error: You are not logged in", which is exactly the symptom of a lost application session
LoginError(String)
InvalidResponse(..) / UnsupportedResponse
Suggestion
Two options, not mutually exclusive:
Treat a lost session as a connection problem — either add ServerError (possibly only when the message indicates an auth/session failure) and LoginError to is_connection_error, or re-login instead of reusing the cached client.
More robustly: drop the client after N consecutive failed sync cycles, whatever the error class. Any error that persists across several cycles is, in practice, indistinguishable from a dead connection — and this also covers variants nobody has enumerated yet. The current design fails closed only for known error types, which is what let this one through after EngineIO read error after some days of uptime #157.
Happy to test a patch — we hit this reproducibly whenever the proxy in front of Uptime Kuma restarts.
AutoKuma: 2.1.0-rc.2 (also present in 2.0.0)
Uptime Kuma: 2.2.1 (MariaDB backend)
Sync interval: 60s
Summary
#157 fixed recovery after EngineIO errors, and it works. But the recovery path is gated on the error being classified as a connection error. When the WebSocket is re-established at transport level while the application session is gone, the resulting error is
ServerError— which is not in that list — so the cached client is kept indefinitely and the sync stays dead until the container is restarted.What we observed
The reverse proxy sitting in front of Uptime Kuma was restarted, causing a short outage (seconds).
503on/health, container wentunhealthy). That is how we caught it. But nothing recovered on its own.We could not capture the exact log line, as the container was replaced before we could pull its logs. What the behaviour does establish is that the error was not in the
is_connection_errorset, otherwise the client would have been dropped and reconnected on the following cycle.Why (reading the code)
In
Sync::run(autokuma/src/sync.rs), the client is only invalidated for connection errors:And
is_connection_errormatches only:get_connectionreconnects only whenself.client.is_none(), so anything outside that set means the dead client is reused forever.Not covered, in particular:
ServerError(String)— the variant behind"Server responded with an error: You are not logged in", which is exactly the symptom of a lost application sessionLoginError(String)InvalidResponse(..)/UnsupportedResponseSuggestion
Two options, not mutually exclusive:
ServerError(possibly only when the message indicates an auth/session failure) andLoginErrortois_connection_error, or re-login instead of reusing the cached client.Happy to test a patch — we hit this reproducibly whenever the proxy in front of Uptime Kuma restarts.