FE-1664: Experiment Live Full Brunch Integration

.changeset/tidy-voice-dock.md

@@ -2,4 +2,5 @@
 "@hashintel/petrinaut": patch
 ---
 
-Allow host-owned Voice sessions to report a neutral connected state and omit unsupported microphone, resume, reconnect, and playback controls from the dock. Keep the viewport controls above a collapsed Voice dock instead of reserving full-height chat panel space.
+Allow host-owned Voice sessions to report a neutral connected state and omit unsupported microphone, resume, reconnect, and playback controls from the dock.
+Show session notices in persistent error notifications instead of obscuring the dock's status and controls, and keep viewport controls above a collapsed Voice dock instead of reserving full-height chat panel space.

apps/petrinaut-website/MISSION.md

@@ -1,92 +1,23 @@
-# Experiment Live Full-Duplex Migration
-
-## Status
-
-Live experiment scope; local implementation prepared for Kostandin's manual
-witness, not conversational-quality acceptance.
-[FE-1663](https://linear.app/hash/issue/FE-1663/experiment-live-full-duplex-migration)
-is related to FE-1661, which remains the migration-effort assessment.
-This website experiment uses Petrinaut's existing Voice dock, with optional
-session controls and local audio-activity indicators, not inferred turn boundaries.
-It does not recut Brunch's existing Mission 7a or promote its future-planning drafts.
-
-## Imperative
-
-Learn whether GPT-Live-1 natively makes process interviewing feel fluid:
-concise relevant follow-ups, room for hesitation and elaboration, and natural
-interruption/correction without losing meaning. Smooth audio alone is not success.
-
-## Throughline
-
-Existing website Voice entry → explicit experimental consent/Start → trusted
-website `/api/voice/live-session` → client-delegated GPT-Live-1 WebRTC → microphone
-and speaker → existing Voice dock → local End/Exit. Local disconnection is
-not a remote-closure claim.
-`main → FE-1663 → Experiment Live Full Brunch Integration` is the intended stack;
-only FE-1663 is authorized now. No relay, rephrasing, or harness code is a donor.
-
-## Proof
-
-- `openai-live-session.test.ts` checks provider/default selection, enablement,
-  trusted session payload, origin/content validation, credential privacy, and
-  no retry. Existing Realtime policy and transport tests remain regression oracles.
-- `live-conversation.test.ts` checks waiting for `session.started`, no duplicate
-  start, no transcript/delegation execution, remote audio attachment, connection
-  failures, late microphone permission, and capture/playback/transport cleanup.
-- `live-conversation-control.test.tsx` checks explicit consent/Start, provider
-  pinning, canonical isolation, host state reporting, panel closure and stale callbacks.
-- Petrinaut's assistant-panel and contents tests check optional action visibility
-  and retain regression coverage for Realtime's playback and lifecycle controls.
-- Rendered real-editor consent, simulated error, and mocked active controls
-  must be inspected without a real microphone or provider session.
-- Kostandin's [10–15 minute manual procedure](README.md#manual-test--1015-minutes)
-  is the oracle for conversational quality and actual media behavior. Provider-free
-  tests and screenshots do not establish these or end-to-end Brunch compatibility.
-
-## Constraints
-
-Unset `PETRINAUT_VOICE_PROVIDER` means `realtime`; only `realtime` and `live` are
-valid. Keep existing enablement/credential plumbing. Pin configuration per
-mounted conversation; never switch providers or resubmit input automatically.
-Keep Realtime's prompts, admission, queue, settlement and playback unchanged.
-
-Live has no access to canonical submission, Brunch/Petrinaut tools or chat
-history. Its guidance does not grant domain authority. Transcript deltas are
-not finalized utterances; delegation events are metadata; commentary (unused
-here) is paraphrasable context limited to 500 tokens, and append acknowledgements
-are not speech/playback completion. Do not invent response terminals or infer
-authoritative completion from silence. Stop ends local media, not canonical work.
-
-No paid sessions, synthetic recordings, audio evaluation harness, deployment,
-Notion writes, second issue, or integration implementation are authorized.
-The follow-up permits cleanup, a clean commit without Amp thread IDs, and a draft PR.
-Preserve other worktrees and uncommitted work.
-
-## Fog-line
-
-Native finalization and enforceable output gating remain unresolved for PR 2.
-Observe whether interruption preserves corrections, whether the user feels
-heard, whether questions advance process elicitation, whether pauses/one-word
-answers survive, and whether output is concise. Observe transport failures and
-remote closure separately. A pleasant standalone conversation cannot answer
-whether canonical admission/settlement delays retain this quality.
-
-## Stop or reorient
-
-Stop at an authority change: autonomous domain speech, local substantive
-follow-ups in the integrated workflow, hybrid authoritative transcription, or
-buffered/gated output requires an explicit recut for Lu's approval. Neither
-this prompt nor success here reopens FE-1624. Provider-free verification cannot
-be replaced with paid calls or a synthetic audio campaign.
-
-## Deferred
-
-PR 2 must preserve: finalized input → existing composer → Flue → Brunch →
-authorized Petrinaut execution → complete-turn settlement → frozen canonical
-reply → Live delivery. Brunch alone authors substantive domain answers and
-questions; Flue is canonical conversation; Petrinaut is execution authority.
-Proposal approval, full-turn settlement and playback eligibility remain distinct.
-Unknown execution outcomes must not be reapplied or spoken as success. Workpiece,
-basis and tool payloads are not assistant prose. See the existing Brunch
-[future spine](../../libs/@hashintel/brunch-agent/MISSION.next.md#voice-after-the-live-transport-cut)
-for retained context, not implementation authority.
+# Experiment Live Full Brunch Integration
+
+The child branch's sole execution authority is the
+[Brunch mission](../../libs/@hashintel/brunch-agent/MISSION.md).
+This file is a pointer, not a second mission.
+
+The accepted experiment uses native Live with best-effort speech instructions and
+separate authoritative transcription. Brunch retains canonical answers and tool
+authority; settlement gates Brunch context sent to Live, not all audible speech.
+Independent questions and unsupported claims are manual-test failures.
+
+Experiment 1's original standalone comparison remains at
+[771712c1afe2d4f3d3e5ee8fac39d17303bee7a1](https://github.com/hashintel/hash/commit/771712c1afe2d4f3d3e5ee8fac39d17303bee7a1).
+The child's restacked parent is
+[b72c2ac9f83875d34f585bc672ef62496144fe6a](https://github.com/hashintel/hash/commit/b72c2ac9f83875d34f585bc672ef62496144fe6a).
+This child does not close or modify that experiment. The child now selects the
+integrated path, with provider-free checks passing but a reported connection failure
+still unresolved. No live end-to-end or naturalness acceptance is established.
+
+Keep `PETRINAUT_VOICE_PROVIDER=realtime|live`, with unset meaning `realtime`.
+The owner authorizes committing and publishing the draft child PR against the
+parent. No paid sessions, automatic microphone access, merge or deployment are
+authorized. Kostandin performs live testing manually.

apps/petrinaut-website/api/voice/transcription-session.ts

@@ -0,0 +1,10 @@
+import { createOpenAITranscriptionSessionHandler } from "../../src/server/voice/openai-transcription-session.js";
+
+declare const process: { env: Record<string, string | undefined> };
+
+export default {
+  fetch: createOpenAITranscriptionSessionHandler({
+    environment: process.env,
+    fetch: globalThis.fetch.bind(globalThis),
+  }),
+};

apps/petrinaut-website/src/main/app/local-storage-demo/local-storage-demo-app.test.tsx

@@ -89,10 +89,20 @@ describe("local storage demo Brunch voice integration", () => {
     expect(getBrunchVoiceMode(null)).toBeUndefined();
   });
 
-  test("installs the app-owned voice control for a configured Brunch transport", () => {
+  test("installs the app-owned voice control for a configured Brunch transport", async () => {
     const config = { available: true as const, connectionTimeoutMs: 15_000 };
     const tracker = new BrunchPanelConversationTracker();
-    const voiceMode = getBrunchVoiceMode(config, tracker);
+    const snapshot = {
+      conversationId: "petrinaut-preview:net-1",
+      messages: [],
+      settlements: [],
+    };
+    const voiceMode = getBrunchVoiceMode(
+      config,
+      tracker,
+      snapshot.settlements,
+      snapshot,
+    );
     const renderControl = () =>
       voiceMode?.({
         canAcceptVoiceInput: true,
@@ -121,6 +131,20 @@ describe("local storage demo Brunch voice integration", () => {
     if (!isValidElement(control)) {
       throw new Error("Expected the configured composer control to render.");
     }
+    expect(control.props).toHaveProperty("snapshot", snapshot);
+    let finishSubmission = () => {};
+    const pending = tracker.trackSubmission(
+      new Promise<void>((resolve) => {
+        finishSubmission = resolve;
+      }),
+    );
+    const whilePending = renderControl();
+    expect(isValidElement(whilePending) && whilePending.props).toHaveProperty(
+      "snapshot",
+      undefined,
+    );
+    finishSubmission();
+    await pending;
     const failureListener = vi.fn();
     const responseCompletedListener = vi.fn();
     const responseStartedListener = vi.fn();

apps/petrinaut-website/src/main/app/local-storage-demo/local-storage-demo-app.tsx

@@ -3,7 +3,11 @@
  * @role Editable demo shell: nets in local storage, one live document handle
  */
 
-import { createFlueClient, type FlueConversationSettlement } from "@flue/sdk";
+import {
+  createFlueClient,
+  type FlueConversationSettlement,
+  type FlueConversationState,
+} from "@flue/sdk";
 import { castDraft, produce } from "immer";
 import {
   use,
@@ -148,6 +152,7 @@ export const getBrunchVoiceMode = (
   config: OpenAIVoiceConfig | null | undefined,
   tracker?: BrunchPanelConversationTracker,
   settlements?: readonly FlueConversationSettlement[],
+  snapshot?: FlueConversationState,
 ): PetrinautAiVoiceMode | undefined => {
   if (!config) return undefined;
 
@@ -175,6 +180,7 @@ export const getBrunchVoiceMode = (
       {...context}
       config={config}
       settlements={settlements}
+      snapshot={tracker?.canReplaceMessages(snapshot) ? snapshot : undefined}
       resolveInputSubmission={resolveInputSubmission}
       resolveResponseSubmission={resolveResponseSubmission}
       subscribeToResponseMessageCompleted={subscribeToResponseMessageCompleted}
@@ -730,8 +736,14 @@ export const LocalStorageDemoApp = ({
         openAIVoiceConfig,
         conversationTracker,
         flueHistory.settlements,
+        flueHistory.snapshot,
       ),
-    [conversationTracker, flueHistory.settlements, openAIVoiceConfig],
+    [
+      conversationTracker,
+      flueHistory.settlements,
+      flueHistory.snapshot,
+      openAIVoiceConfig,
+    ],
   );
   const crewReservationSession = useCrewReservationFixtureSession({
     clientPromise: flueClientPromise,