fix(relay): clean up in-flight websocket requests

[stuartwdouglas]

Summary

• track in-flight WebSocket REQ handlers by subscription ID and generation
• cancel pending handlers when clients send CLOSE, replace a subscription ID, or disconnect
• serialize REQ registration with CLOSE/disconnect cleanup so a late handler cannot reattach subscription state
• make local subscription, fan-out index, and Redis topic-refcount transitions cancellation-safe
• enforce the per-connection subscription ceiling at the serialized insertion point

Root cause

REQ handlers run in detached tasks because historical reads can take time. CLOSE and disconnect cleanup run independently.

A history timeout or other early cleanup could therefore produce this ordering:

1. client sends REQ
2. handler waits on access/database work
3. client sends CLOSE (or disconnects)
4. cleanup finds no registered subscription and completes
5. the detached handler resumes and registers filters, fan-out indexes, and a Redis desired-topic refcount

On a long-lived WebSocket, that state was effectively leaked. Disconnect had the same race because its registry cleanup was one-shot and did not synchronize with REQ registration.

The fix uses cooperative cancellation plus the existing per-connection subscription mutex as the registration/cleanup barrier. Once registration begins, the registry and pub/sub transition runs to completion; teardown then removes it. If teardown wins the mutex, the REQ observes cancellation and does not register.

Testing

• cargo test -p buzz-relay connection::tests:: --lib
• cargo clippy -p buzz-relay --lib -- -D warnings
• pre-push Rust tests and desktop Tauri checks

A full cargo test -p buzz-relay --lib run completed 852 tests successfully; nine unrelated database-backed admin/media tests failed because the local Postgres pool timed out.

[wesbillman]

Carl reviewing on behalf of Wes. The lifecycle barrier is directionally sound, but the NIP-50 path still permits output from a cancelled generation, so I don't think this head fully establishes the advertised CLOSE/replacement guarantee. Please also add a deterministic lifecycle regression that stages a blocked REQ, processes CLOSE/disconnect/replacement, resumes it, and asserts the local subscription map, fan-out registry, and Redis desired-topic refcount—not just token bookkeeping. Focused cargo test -p buzz-relay connection::tests:: passes at this exact head (10 tests), but those tests do not exercise the state race.

[wesbillman]

P1 — CLOSE/replacement can still be followed by stale search output. This outer select! only observes cancellation when handle_search_req yields. After its page hydration await, handle_search_req synchronously walks the hydrated page and calls conn.send for every accepted event, then sends EOSE (req.rs:719-765), with no cancellation check or yield. On a multi-thread runtime, the recv task can process CLOSE (or a replacement REQ) and enqueue CLOSED/new-generation output while this task continues enqueueing old-generation EVENTs and EOSE. Please pass the generation token into the search helper and check it before each emission and before EOSE (and make awaited phases cancellation-aware), with a deterministic regression proving no old-generation output is emitted after CLOSE/replacement wins.

[stuartwdouglas]

Fixed in cd62a82. NIP-50 search now observes cancellation around search/hydration awaits and before every EVENT/EOSE emission. Each emission also holds the active request-generation guard through the enqueue, so CLOSE removal or same-ID replacement is serialized with old-generation output; once either operation wins, the old generation cannot enqueue another frame. Added deterministic CLOSE/replacement output coverage plus blocked-registration lifecycle regressions for CLOSE, disconnect, and replacement. Those tests poll the real cleanup/replacement path to Pending behind the connection subscription mutex, then resume registration and assert the connection-local map, fan-out registry, and PubSub topic refcount. Validation: connection tests (14), REQ tests (47), clippy -D warnings, and pre-push hooks all pass.

[wesbillman]

Carl re-reviewing on behalf of Wes at cd62a82676772e718b3c9825821780e27f6994e1.

The new lifecycle barriers and deterministic state/refcount regressions are substantial improvements, and the NIP-50 helper correctly serializes each enqueue against CLOSE/replacement by holding the DashMap generation guard through conn.send. However, the ordinary historical-delivery path still uses a separate cancellation check followed by an unfenced enqueue (req.rs:367-403), and EOSE has the same check/enqueue gap (req.rs:411-414). A CLOSE or same-ID replacement can remove/cancel the old generation and enqueue CLOSED/new-generation output after the check, then the old task can resume and enqueue a stale EVENT or EOSE afterward. That is the same lifecycle ordering defect now fixed for NIP-50.

Please route all old-generation historical EVENT/EOSE (including the DB-error EOSE at req.rs:317-323) through the generation-fenced send helper, and add a deterministic regular-REQ regression that pauses an old generation immediately before enqueue, lets CLOSE/replacement win, resumes it, and proves no stale frame follows. The current close_and_replacement_suppress_cancelled_search_output test invokes the helper only after cancellation; it proves the helper predicate but not the contested check/enqueue ordering in the regular handler.

Focused validation at this exact head passes: 14 connection tests, 47 REQ tests, and git diff --check. CI is green. Requesting changes because the core no-output-after-CLOSE/replacement guarantee remains incomplete outside NIP-50.