Context
I'm embedding BetterWright from a non-Node host: koto, a Rust CLI that gives agents a bounded instruction language for desktop control. Its web instructions drive BetterWright through a small persistent node sidecar that calls the documented JS API (bw.run(snippet)) — one long-lived worker, NDJSON over stdio. The integration works well; the pruned [ref=eN] snapshot in particular is a big improvement over raw accessibility-tree dumps.
Three things would make programmatic embedding materially more robust, and none of them are specific to my use case — they apply to anyone driving BetterWright from outside Node or outside the MCP server.
1. Structured error codes in the run envelope
Today a failed snippet surfaces error as a message string only. Embedders that need to distinguish failure classes (my case: a strict exit-code contract where timeout ≠ element-not-found ≠ ambiguous-selector ≠ backend-failure) are forced to classify Playwright message strings with regexes — /strict mode violation/, /Timeout \d+ms exceeded/, and so on — which is fragile across Playwright and BetterWright upgrades.
A small enum field alongside the message would eliminate that entirely:
(Or errorCode as a sibling field, for envelope compatibility.)
2. A handoff gate in the snippet sandbox
The human-intervention gate currently exists only as the browser_handoff MCP tool. The snippet sandbox has human.click/type/scroll (humanized input) and screenshot({kind: "question"}), but no way for a snippet to block until a human completes an intervention in the live view.
Something like:
const outcome = await handoff("solve the payment step", { timeout: 300000 });
// outcome: { completed: boolean, declined?: boolean }…resolving through the same live-view flow the MCP tool uses, would let non-MCP embedders offer the same "script pauses, human takes over, script resumes" experience. Right now the only options are reimplementing it with question-screenshots and polling heuristics, or not offering it at all (I chose the latter).
3. Documented envelope and artifact contract
The run envelope (ok, result, artifacts, challenges, warnings, durationMs, …) and the artifact shape (the MEDIA:<path> convention, string vs object entries) are observable but not pinned down in docs/browser-api.md. Embedders currently normalize defensively. A short documented schema — even just a versioned "these fields are stable, everything else may change" list — would let integrations upgrade BetterWright without re-reverse-engineering the shapes.
Happy to provide more detail on any of these from the embedder side. Thanks for the project — the token-economy design is exactly right.
Context
I'm embedding BetterWright from a non-Node host: koto, a Rust CLI that gives agents a bounded instruction language for desktop control. Its
webinstructions drive BetterWright through a small persistent node sidecar that calls the documented JS API (bw.run(snippet)) — one long-lived worker, NDJSON over stdio. The integration works well; the pruned[ref=eN]snapshot in particular is a big improvement over raw accessibility-tree dumps.Three things would make programmatic embedding materially more robust, and none of them are specific to my use case — they apply to anyone driving BetterWright from outside Node or outside the MCP server.
1. Structured error codes in the run envelope
Today a failed snippet surfaces
erroras a message string only. Embedders that need to distinguish failure classes (my case: a strict exit-code contract where timeout ≠ element-not-found ≠ ambiguous-selector ≠ backend-failure) are forced to classify Playwright message strings with regexes —/strict mode violation/,/Timeout \d+ms exceeded/, and so on — which is fragile across Playwright and BetterWright upgrades.A small enum field alongside the message would eliminate that entirely:
{ "ok": false, "error": { "code": "timeout" | "not-found" | "ambiguous" | "policy-denied" | "navigation" | "internal", "message": "..." } }(Or
errorCodeas a sibling field, for envelope compatibility.)2. A handoff gate in the snippet sandbox
The human-intervention gate currently exists only as the
browser_handoffMCP tool. The snippet sandbox hashuman.click/type/scroll(humanized input) andscreenshot({kind: "question"}), but no way for a snippet to block until a human completes an intervention in the live view.Something like:
…resolving through the same live-view flow the MCP tool uses, would let non-MCP embedders offer the same "script pauses, human takes over, script resumes" experience. Right now the only options are reimplementing it with question-screenshots and polling heuristics, or not offering it at all (I chose the latter).
3. Documented envelope and artifact contract
The run envelope (
ok,result,artifacts,challenges,warnings,durationMs, …) and the artifact shape (theMEDIA:<path>convention, string vs object entries) are observable but not pinned down in docs/browser-api.md. Embedders currently normalize defensively. A short documented schema — even just a versioned "these fields are stable, everything else may change" list — would let integrations upgrade BetterWright without re-reverse-engineering the shapes.Happy to provide more detail on any of these from the embedder side. Thanks for the project — the token-economy design is exactly right.