pi: live view, viewer chat, and human handoff via browser_handoff - #74
Open
ekenberg wants to merge 1 commit into
Open
pi: live view, viewer chat, and human handoff via browser_handoff#74ekenberg wants to merge 1 commit into
ekenberg wants to merge 1 commit into
Conversation
Register a browser_handoff tool in the native Pi extension so a Pi agent can share the live viewer (start/status/stop) and pause for human hands (wait, backed by waitForHandoff) on its own in-process session. Start mirrors the MCP server's gating: a non-loopback bind requires the deployer opt-in BETTERWRIGHT_LIVE_VIEW=1; expose "local" never does. liveViewFromEnv and isLoopbackHost move to live-view-config.ts so both surfaces share them (re-exported from mcp-server, their public home). While a view runs, freeform viewer-dock messages are polled (BETTERWRIGHT_PI_CHAT_POLL_SECONDS, default 3 s, error backoff, unref'd timers) and delivered as follow-up user guidance that wakes an idle agent; browser-step notes mirror back into the dock. Status strips the capability token and URL; re-issuing start re-shares the same URL, and a handoff timeout resolves as a normal, resumable result.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The live-view docs list a surface for every agent host — the standalone
harness gets
live_view/handofftools, MCP clients getbrowser_handoff—but the native Pi extension had none. On a headless box (my case: a VPS
driven by Pi over Tailscale) that leaves no way to compose "the agent drives
its persistent in-process session" with "a human watches, coaches, or takes
over", because
betterwright viewattaches to the daemon sessions, not theextension's private worker.
This registers a
browser_handofftool in the Pi extension, modeled on theMCP server's handler, with one Pi-specific addition each way:
start/status/stop, plus a blockingwaitbackedby
waitForHandoff()— the viewer elevates with the reason and Done /Cancel, and the tool resolves when the human finishes. A timeout resolves
as a normal
{action: "timeout"}result (resumable), never a tool error;an abort is honored without leaking listeners.
extension polls
liveViewDrainChat()(default 3 s,BETTERWRIGHT_PI_CHAT_POLL_SECONDS, exponential backoff on transientfailures, unref'd timers) and delivers messages via Pi's
sendMessage({deliverAs: "followUp", triggerTurn: true})— so a messagetyped in the dock wakes an idle agent instead of waiting for the next tool
call. Browser-step notes mirror into the dock via
liveViewPostChat, asthe MCP server does per call.
(lan/tailscale) requires
BETTERWRIGHT_LIVE_VIEW=1;expose: "local"never does.
liveViewFromEnvandisLoopbackHostmove tolive-view-config.tsso both surfaces share them (re-exported frommcp-server, their public home — no caller changes).statusstrips the capability token and URL, exactly like MCP;re-issuing
startre-shares the same URL (useful after a lost terminalscrollback — the view survives worker restarts on the same port+token).
browser_handoffis deliberately not inBROWSER_TOOL_NAMES: apending handoff (or a user still watching) must survive step-budget
exhaustion.
Divergence from MCP worth a reviewer's eye: the
reasonparameter issurfaced during
wait(the viewer banner) rather than appended to thestartresponse text, since Pi has a real blocking handoff instead ofstatus-polling.
Testing
npm run release:checkgreen (lint, typecheck ×3, build, unit tests,hand-written types, package).
relay, token/URL stripping, idempotent re-start, chat delivered as
follow-up guidance and stopped with the view, opt-in gating incl.
expose: local/tailscale, wait outcomes done/timeout, budget-exhaustionsurvival).
chat-wakes-idle-agent, take-control handoff round-trip, tmux
detach/reattach, viewer close/reopen on the same URL, and a worker
SIGKILL revived the view on the same port+token with the viewer
auto-reconnecting.