Add Pi as a conversational backend over `pi --mode rpc`

[dcellison]

Why

Pi has an established user base with subscription configurations (ChatGPT Plus/Pro through Codex, Claude Pro/Max, GitHub Copilot) plus a multi-provider API-key surface (Anthropic, OpenAI, DeepSeek, Google Gemini, Mistral, Groq, Cerebras, NVIDIA, Together, Xiaomi, OpenRouter-like custom setups). Adding Pi as a Kai backend lowers adoption friction for users already in Pi's ecosystem: they bring their existing subscription and familiar model choices to Kai instead of standing up a separate backend just to onboard. Same logic that brought Claude, Codex, Goose, and OpenCode in.

Scope (this issue)

Pi as a full first-class backend on parity with Claude / Codex / Goose / OpenCode. That means both the conversational transport AND the one-shot reasoner ship together so a user selecting Pi gets memory extraction, PR review, triage, and behavioral judge routed through Pi without needing a different backend configured for those roles. The two surfaces share the same JSONL protocol parser, so splitting them into separate issues would mostly defer config and dispatch wiring without reducing real implementation risk.

The seven things this issue ships:

1. Conversational backend over pi --mode rpc with JSONL framing on stdin and stdout, JSON command objects in, JSON response and event objects out.
2. PiOneShotReasoner over pi --mode rpc --no-session --no-tools, terminating after the final event. Shares the protocol parser with the conversational backend; the difference is launch flags, session lifecycle, and termination logic.
3. Per-user auth/config home isolation: Pi state lives under each target OS user's ~/.pi/agent/ via sudo -H -u <os_user>, mirroring the existing per-user pattern for Claude/Codex/Goose/OpenCode.
4. Provider/model config: pi added to VALID_BACKENDS and BACKEND_PROVIDERS; structural model validation accepts <provider>/<model> plus optional :<thinking> suffix; small curated tier defaults in _BACKEND_PROVIDER_TIER_MODELS; user can type Pi-native model strings without a registry edit.
5. Safe default launch flags: --no-approve always, no implicit project-local .pi/ extension trust. A Telegram-driven session does not load arbitrary TypeScript extensions even when the operator has set defaultProjectTrust in their Pi config. The opt-in extension trust posture is a separate future issue, not this one.
6. Streaming: convert Pi message_update text deltas into StreamEvent.text_so_far; treat the documented terminal event as final completion; surface tool-execution events as logs only, never mixed into assistant text.
7. Backend discovery: missing Pi binary is a deferred failure (only when the backend is selected), not an import-time crash. The wizard / installer must work without Pi installed.

Implementation shape

New module src/kai/pi.py with PiBackend(AgentBackend). Pi documents its own RPC protocol, not ACP, so this is not an AcpBackend subclass. The shape is closer to Codex: a custom JSONL protocol adapter, simpler than Codex because the event schema is more linear.

Transport:

• Spawn pi --mode rpc --no-approve with cwd=self.workspace.
• Pass --provider, --model, --thinking, --session-dir, and optionally --no-session from Kai config.
• Launch under sudo -H -u <os_user> following the existing isolation pattern.
• Read stdout as strict byte-readline() split on LF (strip optional trailing CR); do NOT use generic text readline that reinterprets Unicode line separators.
• Write JSON command objects to stdin, one per line.
• Correlate command responses by id.
• Treat agent events as a separate stream from command responses.

Streaming:

• message_update text deltas → StreamEvent.text_so_far.
• Terminal event (turn_end / agent_end per documented schema) → final completion after a prompt.
• Tool execution events (tool_start, tool_update, tool_end) → logs or optional future user-facing progress; never assistant text.
• Error handling: extension_error, failed command responses, EOF, invalid JSON, and timeout all surface as backend errors.

Session:

• Decision point in the implementation PR: let Pi own ~/.pi/agent/sessions/ for conversational continuity, or point Pi at a Kai-controlled --session-dir under the user's Kai home for tighter cleanup / export control. The research note leans toward the former.
• Use RPC get_state after startup to capture sessionId and the session file path.
• Workspace change → kill and respawn (matches existing backends).
• Restart → kill and respawn.
• Shutdown: no Claude-style prompt-saving needed; Pi sessions auto-save.

Provider and model:

• BACKEND_PROVIDERS["pi"] row populated with the providers Kai already supports where Pi names align (Anthropic, OpenAI, DeepSeek, Gemini, GitHub Copilot, plus subscription providers Pi documents).
• Tier defaults: small curated set per provider (one or two model strings each) so /model has sane defaults without trying to mirror Pi's full release-driven catalog.
• validate_model_for_backend(..., "pi", ...): structural; accept <provider>/<model>[:<thinking>] and bare <model> only when the provider is set explicitly.

Environment:

• PI_BIN env var to pin the binary path (mirrors CODEX_BIN, GOOSE_BIN, OPENCODE_BIN).
• Preserve provider API key env vars through sudo: ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY, DEEPSEEK_API_KEY, plus any other provider vars enabled in BACKEND_PROVIDERS["pi"].
• Preserve TMPDIR, PATH, and any Pi-specific config override variables the installed Pi version documents.
• Prefer per-user ~/.pi/agent/auth.json for subscription login and durable API-key auth; env-var auth is acceptable when the user has configured it that way.

Configuration flow (mirrors existing backends):

• Config dataclass field for Pi-specific knobs (whatever the wizard collects).
• Validated parsing in load_config() with SystemExit on bad input.
• Pass-through in pool.py to PiBackend initialization.
• Wizard prompt in install.py _cmd_config() with validation loop, gated on the operator choosing to enable Pi.
• .env.example documents PI_BIN and the provider key vars Pi cares about.
• New _CONFIG_ENV_VARS entries in tests/test_config.py.

One-shot reasoner (src/kai/oneshot.py addition):

• New PiOneShotReasoner alongside the existing Claude/Codex/Goose/OpenCode reasoners.
• Launch: pi --mode rpc --no-session --no-tools (sessionless, no tool use) under sudo -H -u <os_user>. The same protocol parser the conversational backend uses ingests stdout; the one-shot path drives a single prompt command and waits for the terminal event before tearing down the subprocess.
• No tools by default for memory extraction, PR review, triage, and behavioral judge; callers that need tools must opt in explicitly.
• ONESHOT_REASONER_BACKENDS includes pi so role routing in get_model_for(role, ...) can dispatch one-shot calls through Pi when the operator has set it as their reasoner backend.
• BACKEND_PROVIDERS["pi"] provider list is reused; one-shot does not need its own provider matrix.
• Memory extraction, PR review, triage, and behavioral judge call sites pick up Pi automatically through the existing _run_reasoner / get_model_for indirection; no per-caller changes beyond the dispatch table.

Trust posture

Hard rules for this issue:

• Pi launches with --no-approve always.
• No code path passes --approve from a Telegram-driven session in any form.
• defaultProjectTrust from the operator's ~/.pi/ config is ignored for Kai-driven launches; Kai sets trust per-invocation, not by inheriting Pi's default.
• The extension hook surface is Powerful + TypeScript-native. Kai does not load project-local .pi/ extensions from a chat-driven session under any circumstance covered by this issue. Opt-in extension trust is explicitly deferred to a separate future issue, not a config knob shipped here.

Pi itself does not provide a built-in filesystem / process / network / credential sandbox; it runs with the launching user's permissions. Kai's existing per-OS-user isolation remains load-bearing. Stronger isolation, if ever wanted, belongs in containerization or macOS sandboxing, not in prompt wording.

Decisions to make in the implementation PR (or in a quick design comment before it opens)

• Open-ended Pi model strings vs curated registry. Recommendation: open-ended /model entry with small curated tier defaults; Pi's model surface is release-driven and a rigid registry will go stale quickly.
• Session directory ownership: Pi-owned ~/.pi/agent/sessions/ vs Kai-controlled --session-dir.
• How aggressive to be about preserving env vars across sudo -H (broad allowlist vs minimal per-provider set).

Tests

Conversational backend:

• JSONL framing and CR/LF tolerance.
• Prompt acceptance: a single prompt command produces streaming message_update events, a terminal event, and a command response correlated by id.
• Streaming delta accumulation reaches StreamEvent.text_so_far in order.
• Final completion logic on the terminal event.
• Tool events do NOT bleed into assistant text.
• extension_error, failed command responses, EOF on stdout, invalid JSON on stdout, and timeout all surface as backend errors with the right shape.

One-shot reasoner:

• Launch flags assertion: every PiOneShotReasoner invocation passes --mode rpc --no-session --no-tools --no-approve; the test suite fails if --approve or tool flags leak in.
• Single-prompt happy path: drive one prompt command, observe streaming deltas, return the final text on the terminal event.
• Subprocess teardown: the reasoner kills the Pi process after the terminal event; no orphaned children across successive calls.
• Error surfaces: extension errors, failed command responses, EOF, invalid JSON, and timeout all surface with the right structured error from the reasoner contract.
• Routing: get_model_for(role, ...) returns the configured Pi model for each reasoner role (memory extraction, PR review, triage, behavioral judge) when pi is the operator's reasoner backend.

Config and discovery:

• VALID_BACKENDS and ONESHOT_REASONER_BACKENDS include pi.
• BACKEND_PROVIDERS["pi"] populated.
• Model validation accepts/rejects the documented patterns.
• Registry completeness across all roles.
• Per-user overrides.
• Pool dispatch routes pi to PiBackend.
• Smoke test that detects a missing Pi binary without requiring Pi to be installed for the Kai test suite to pass.

Out of scope (this issue)

• Loading .pi/ project extensions or supporting the --approve flag from a chat-driven session.
• Stronger filesystem / process / network sandbox beyond per-OS-user isolation.
• Multi-backend model-overlap policy: when a model is reachable via Pi AND via a direct backend Kai already has, the user picks; Kai does not silently route.
• Curated Pi model matrix mirroring Pi's full provider catalog.
• One-shot calls that need Pi tools enabled (read/write/edit/bash/grep/find/ls). All Kai one-shot roles ship with --no-tools; tool-enabled one-shot is a future decision per caller.

Acceptance

Conversational backend:

• pi --mode rpc --no-approve subprocess launches under sudo -H -u <os_user> with cwd=workspace.
• Streaming assistant deltas reach StreamEvent.text_so_far in real time during a conversation.
• /backend pi works in chat; session continuity holds across messages within a workspace.
• /model accepts a structurally valid Pi model string and rejects malformed ones with a clear error.

One-shot reasoner:

• pi --mode rpc --no-session --no-tools --no-approve is the launch shape; the test suite asserts no other flag combination ships in production code paths.
• Memory extraction, PR review, triage, and behavioral judge run end-to-end through Pi when the operator has set pi as the reasoner backend.
• Subprocess teardown is clean after every one-shot call: no orphans, no leaked stdin/stdout handles.
• Reasoner error contract matches the existing Claude/Codex/Goose/OpenCode reasoners.

Shared:

• Selecting pi when the binary is missing produces a deferred error message, not an import-time crash.
• Tests cover JSONL framing, prompt/response correlation, streaming text, tool-event isolation, EOF, invalid JSON, timeout, and the one-shot launch-flag assertion above.
• Wizard prompts for PI_BIN when the operator enables Pi.
• .env.example documents PI_BIN and the provider key vars Pi consumes.
• Pi launches always pass --no-approve and never --approve; the test suite asserts this.

Notes

Research note prepared 2026-06-16 enumerates the Pi protocol surface, RPC command families (prompt, get_state, get_messages, set_model, cycle_model, queue controls, compaction, retry, bash, session ops, abort), the event schema, the provider model, and the trust posture. Primary sources: pi.dev docs (/latest/rpc, /latest/sdk, /latest/sessions, /latest/providers, /latest/settings, /latest/custom-provider, /latest/extensions), the earendil-works/pi GitHub repo, and the @earendil-works/pi-coding-agent npm package. Implementation work should re-verify the protocol surface against the installed Pi version before locking schema-level assumptions in code.

[dcellison]

Triage by Kai

Triage summary: Adds Pi as a conversational Kai backend over pi --mode rpc, including transport, config, streaming, per-user isolation, model validation, installer/docs, and tests. The scope is well specified but sizable and security-sensitive due to subprocess launch, sudo environment handling, and extension trust posture.

Priority: high
Labels applied: enhancement