feat(xiaohongshu): live-comments — stream a live room's comments via an in-page MutationObserver watcher - #2472
Open
yisiliu wants to merge 3 commits into
Open
Conversation
…an in-page MutationObserver watcher Captures the comment stream of a xiaohongshu live room with full structured metadata (msg, nickname, userId, commentId, commentType, fans-group) rather than scraped text. Architecture is the MutationObserverWatcher pattern from DimensionDev/Holoflows-Kit, re-implemented clean-room (that library is AGPL-3.0, this repo is Apache-2.0): a DOM mutation on the chat area is only the TRIGGER — the data source is the structured __INITIAL_STATE__.liveStream.comments store, merged into a capped in-page buffer deduped by commentId. The watcher lives on the persistent site tab ACROSS CLI invocations: install once, then repeated `--duration 0` calls drain exactly what arrived since the previous call — a streaming loop out of one-shot commands that misses nothing when the page's own comment window slides (verified live: a drain returned comments buffered between two invocations minutes apart). The command deliberately navigates with a plain goto: the extension's same-URL fast-path is what keeps the watcher alive. Empty drains are diagnosable: the result says whether the watcher survived (quiet room) or was freshly installed (tab had navigated away). Verified live in a 170-viewer room: 20s collection returned real comments with @-mentions and fan-group fields; a later pure drain returned only the in-between arrivals. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
UI-driving write on the room's contenteditable composer: focus, insert the text via execCommand (verifying the box shows exactly the requested text before anything is clicked - garbled input is never sent), click the send control that appears on input, then poll a postcondition before reporting success: the input cleared AND the comment appeared in the structured liveStream.comments store. Failure hints name the likely cause (logged out / room ended / fans-only or slow-mode restrictions). Lives in its own module: the command shares parseRoomUrl with live-comments but has different columns, and the silent-column-drop audit is per source file. Navigates with a plain goto so a live-comments watcher on the same warm tab stays alive; as a persistent write it also gains session-lease arbitration. Mechanics recon was done without posting: typing into the composer makes a .send control appear, and select-all + delete cleans up - the send path itself is covered by unit tests and deliberately NOT live-tested without explicit approval (it posts into a real streamer's room). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… is frozen Live testing with the account owner falsified v2's data model twice: 1. `__INITIAL_STATE__.liveStream.comments` is NOT a live window — it holds the SSR-initial batch plus (sometimes) local echoes of the viewer's own sends. Crowd comments stream over WebSocket straight into the DOM and never touch the store: in a 1600-viewer room the store stayed frozen at 5 entries while the chat DOM added 15 nodes in 12 seconds. v2 therefore captured the initial batch and then nothing, while reporting the watcher "alive". 2. An observer attached to the chat container dies silently when the SPA re-renders it. v3 fixes both: the watcher observes document.body and captures every `.virtual-list-item` node exactly once (WeakSet on node identity, so identical repeated texts stay distinct events), classifying items by shape (chat / enter / like / notice). Rows become seq/kind/nickname/msg - the honest schema for what the web chat stream carries; userId and commentId were only ever available for the frozen initial batch. Drain performs a final sweep, and the watcher is versioned so an outdated in-page instance is replaced instead of kept. Verified live: 24 crowd events in 10s where v2 captured zero. live-comment-send's postcondition had the same store dependency and produced false negatives - a send it reported as failed was visible in chat through the watcher. Verification is now DOM-primary (own nickname + exact text rendered in chat, own-nickname check so someone else posting the same text never confirms), with the store echo kept only as a best-effort source for the server comment id. Verified live end-to-end: send confirmed, then drained back through the watcher. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two commands covering a live room's comment surface:
xiaohongshu live-comments <room-url> [--duration N]captures the room's event stream —kind(chat/enter/like/notice),nickname,msg. The streaming loop:--duration Ncollects for N seconds;--duration 0drains exactly what arrived since the previous call, because the in-page watcher survives on the persistent site tab across CLI invocations. A caller loops one-shot commands and misses nothing.xiaohongshu live-comment-send <room-url> <text>posts a comment by driving the web composer, refusing to click send unless the box shows exactly the requested text, and confirming success with DOM-primary evidence: the input cleared and the viewer's own nickname + exact text rendered in chat (someone else posting the same text never confirms; a store echo, when present, additionally supplies the servercomment_id).Architecture — and why the DOM is the data source
The watcher is the MutationObserverWatcher pattern from DimensionDev/Holoflows-Kit, re-implemented clean-room (AGPL-3.0 vs this repo's Apache-2.0 — the pattern is adopted, not the code), fused with this repo's install-hook idiom: an idempotent, versioned in-page IIFE observes
document.bodyand captures every.virtual-list-itemchat node exactly once via a WeakSet on node identity — so identical repeated texts stay distinct events, with no external id needed.The DOM being the source is a finding, not a shortcut:
__INITIAL_STATE__.liveStream.commentslooks richer (userId/commentId per entry) but holds only the SSR-initial batch plus occasional local echoes of the viewer's own sends — crowd comments stream over WebSocket straight into the DOM and never touch that store. Measured in a 1600-viewer room: the store stayed frozen at 5 entries while the chat DOM appended 15 nodes in 12 seconds. A store-based v1 of this PR captured the initial batch and then nothing (and gavelive-comment-senda false-negative postcondition — a send it reported as failed was visible in chat). Round-trip testing — send a comment, then require it to reappear through our own watcher — falsified that design and produced this one.Robustness details, each regression-tested: observe
document.body(the SPA tears down and recreates the chat container, silently killing an observer attached to it); a final sweep on drain so items rendered without a caught mutation are never lost; a versioned watcher state so outdated in-page instances are replaced, not kept; empty drains are diagnosable ("watcher stayed alive, room quiet" vs "freshly installed — the tab had navigated away"). The commands navigate with a plain goto on purpose: the extension's same-URL fast-path is what keeps the watcher alive between invocations.Verification
--duration 0drain returned comments buffered between invocations; full loop closed — a sent comment (主播加油) confirmed by the command and then drained back through our own watcher among 31 crowd events.Recon context (FLV endpoints, the frozen-store finding, goods shelf being App-only on web) is documented in the umbrella issue #2470.
🤖 Generated with Claude Code