Repository navigation
Run every agent in an NVX microVM of its own - #975
Merged
Merged
Conversation
The connector no longer starts agents in the system's process sandbox (MXC with Seatbelt, bubblewrap or ProcessContainer), and has no unsandboxed mode. Every coding agent, Planner backend (cold or spare), agent check and image build runs in an NVX microVM, through one seam (Sandbox.spawn), whose guest is decided by one pure function (vmSpec). - A guest boots its agent's image: an EROFS layer built once per recipe in a microVM, with Node, pinned git packages, the agent at the version this release pins (or the owner's --install commands) and an account for the connector's own user. The host needs neither the agent nor Node. - An agent works in a repository of its own inside its guest: a run clones with --shared into memory, a Planner with --reference --dissociate onto a sparse disk that lasts between its starts. Shares of the person's checkout and git directory are read-only; the host runs no git in a tree an agent wrote, and a run leaves no worktree. - The only way out is the connector's allow-list proxy, through NVX's proxy-only network. The environment goes in a 0600 file, never on a command line. - The run's MCP bridge runs in the guest on the image's Node; the binary carries it. - A Planner's disk is held while a launch uses it, and goes when Chopin says its document is archived or gone, after 30 days unused, or when the machine is unpaired or revoked. - A machine that can't run NVX is refused by pair and connect before anything is paired, with what it needs and the fix. One already paired tells Chopin, which keeps it (migration 051) and shows it on the Machines page. Chopin takes connections only from connector protocol 8 saying its agents run in microVMs, and tells older connectors to update. Tests of what the connector does around an agent run it on the host through a stand-in (testing/host-sandbox.ts, testing/chopin-on-host.ts), never built into chopin. Real-microVM tests run where KVM and an NVX package are available, including a new CI job. Co-authored-by: Alex Lavaee <lavaman131@github.com>
Co-authored-by: Alex Lavaee <lavaman131@github.com>
- vmSpec refuses a folder shared writable that holds what a launch may only read (the checkout, its git directory, its tokens), and refuses checkouts at /work or /opt/chopin. - Agent state a guest writes is never written or read through a link it left there. - A dead connector's microVM is ended only while it is still the process recorded; on Windows each nvx.exe is in a kill-on-close job, stopped by ending its workload first, and a VM that outlives its stop is reported. - Planner disks are made sparse on the host, held per process against this process's own sweep, and only the server a disk's document is on may remove it; unpair and revoke take unassigned spares' disks too. - A Planner checkout's hidden .git is shown again on Windows; Windows without the Hypervisor Platform, and root, are refused with the fix. - connector.js carries the bridge its microVMs run. - An older connector is kept on the machine as one that can't run work, and the Machines table says why and the fix while a machine is away. Co-authored-by: Alex Lavaee <lavaman131@github.com>
- A disk's lock is written whole and linked into place, so no process takes one half written. - --sandbox-write can't name a folder inside chopin's state directory. - On Windows a launch whose job object can't be made never runs. - A service on a machine that can't run microVMs says so once and waits; service install checks the machine even with --skip-check. - The Machines page's "Can't run work" has browser tests, and setup agents show what the machine needs above their own steps. - Docs for operators, setup agents, what a guest can read, leftovers, and the disk locks; comments no longer describe the process sandbox. Co-authored-by: Alex Lavaee <lavaman131@github.com>
Chopin now refuses a Planner credential, a GitHub or MCP token, an AI launch and a machine's database login, as well as a connection, to a connector that would start agents outside a microVM. A connector speaking a protocol before 8 is told to update whatever it says of its agents, and a current one must say on the request that its agents run in NVX. The connector says so on every request while a `chopin pair` or `connect` that checked the machine runs. Tests that drove older protocols through these routes now speak protocol 8; tests of what an older connection did are gone, since none connects. Each route has a test that an older connector, or one that doesn't say, is refused before anything is minted. Also from review: a launch that never starts (as on Windows without its job object) removes its environment file at once, and a setup agent's dialog no longer says chopin installs it. Co-authored-by: Alex Lavaee <lavaman131@github.com>
Both create and drop databases, which wait on a checkpoint that took over four seconds on a busy CI runner, so they timed out at five and raced the test after them. Co-authored-by: Alex Lavaee <lavaman131@github.com>
A reset in Chopin now starts a machine's Planner afresh, as it does in Chopin. The reset route records when it happened (`channels.planner_reset_at`, expand-only migration 052), before it ends the Planner's link, and each reset is later than the one before. Each Planner credential names that time. The connector stamps a disk with the reset it was made after (`<slot>.reset`), and removes a disk from before the latest reset before the Planner's next launch. It also removes one at the hourly `planner/kept` sweep, which now names each disk's reset and is answered with the documents reset since. Either way, it says that commits never pushed went with the disk. Co-authored-by: Alex Lavaee <lavaman131@github.com>
The credential helper named the token file by its host path. A Windows guest has that file under /mnt/<drive>, so the helper's `cat` found nothing, and a Build's or Planner's push to GitHub got an empty password. The helper now reads `$CHOPIN_GITHUB_TOKEN_FILE`, which every caller of `gitCredentials` sets. A microVM's environment gives that variable the path its guest sees, and host git reads it as before. Co-authored-by: Alex Lavaee <lavaman131@github.com>
A Windows connector passed its own profile variables to every agent's Linux guest: APPDATA, USERPROFILE, PSModulePath, ComSpec, Path, a PowerShell SHELL and the like. Those name nothing the guest has except the host user's profile paths. The allow-list for agents on Chopin's AI accounts now names only Linux's variables. Whatever the host, a microVM is given no variable only Windows sets, from that list or from a launch's own environment, and on a Windows host it is given neither its `Path` nor its `SHELL`. Co-authored-by: Alex Lavaee <lavaman131@github.com>
`mkfs.ext4` left the inode tables for the guest's kernel to zero after the first mount. On a 32 GiB disk that is about 512 MiB of zeros, and both NTFS and Linux allocate zeros written into a sparse file, so every new Planner disk grew to about 520 MiB on the host within a minute with nothing written. A new disk is a fresh sparse file, which reads as zeros, so `mkfs.ext4` is now told that (`assume_storage_prezeroed=1`) and no lazy init runs. A KVM test boots a new disk for 40 seconds and checks that it takes under 64 MiB on the host. Before this change it took 519 MiB. Co-authored-by: Alex Lavaee <lavaman131@github.com>
A second `chopin connect` on a base dir that a running connector holds opened its microVMs before it found the lock. Opening them built its agent's image and pruned the running connector's image, which broke that connector's next launch. Then the second connect never exited, because its proxy and AI relay sockets were never closed. `connect` now takes the base dir's lock before it opens the microVMs. It lets go of both the microVMs and the lock when opening them or its agent's start check fails. The service install check, which holds no base dir, removes no images, because the running service may use one another agent's recipe doesn't name. Co-authored-by: Alex Lavaee <lavaman131@github.com>
This was referenced Nov 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every agent the connector starts now runs in an NVX microVM of its own: each coding agent of a Build, Run, Retry, investigation or prototype, each Planner backend (cold or spare), the agent checks, and the builds of agent images. This replaces the process sandbox (MXC with Seatbelt, bubblewrap and ProcessContainer) completely. There is no fallback and no unsandboxed mode, and a machine that can't run NVX is refused at pairing and connect.
The design, and what was built, is in docs/nvx-sandbox.md. The guide for people who run
chopinis updated in docs/chopin-cli.md, and the AGENTS.md sandbox rule is rewritten.What changes for people
/dev/kvm(or/dev/mshv), or Windows on x86_64 with the Windows Hypervisor Platform. On a Mac, an Arm machine, or a machine whose user can't use KVM,chopin pairandchopin connectrefuse before they pair or spend a key. The refusal says what the machine needs and, where there is one, the fix (sudo usermod -aG kvm $USER, …).chopinbuilds an image for the agent once, in a microVM, with the version this release pins (Copilot CLI 1.0.114; Atomic 0.10.2 with atomic-acp 0.0.18). Another ACP agent needs--install '<command>', and a Planner command needs--planner-install. The first connect downloads NVX (about 275 MB, checked against a pinned SHA-256) and builds the image (about 40 s).runs/keeps each run's record and its agent's state.git branchonly after agit fetch.planners/<owner>/<repo>/<document>.disk, sparse, up to 32 GiB) with its clone. The disk lasts between the Planner's starts. It goes when Chopin says the document is archived or gone, after 30 days unused, or when the machine is unpaired or revoked, never while a launch holds it. Commits on it that were never pushed go with it, and the connector says so.--require-sandboxand--no-sandboxare gone. Saved settings that had them keep working without them.--sandbox-host,--sandbox-writeand--sandbox-hidestay.--envgives it.How it works
Sandbox.spawn(launch)inapps/connector/src/sandbox.tsis the only way to start an agent.vmSpecinapps/connector/src/nvx.tsis a pure function that decides everything a guest gets: its image and user, its shares and their modes, its proxy, its environment and its command.nvxArgumentsturns that into onenvx sandbox run. The environment goes in a0600file, never on a command line.PREPARE). A run usesgit clone --sharedfrom a read-only share of the checkout's git directory, into memory. A Planner uses--reference --dissociateonto its disk. S1's measurements drove this, and the supervisor approved it: a worktree shared over virtio-fs took 1.7 to 11 s forgit status, while an in-guest clone took 32 ms.safe.directory=*. Images are rebuilt only when their recipe changes.--network-egress=deny --host-loopback=deny --network-proxy=10.0.0.1:<port>: the connector's allow-list proxy is the only way out. The AI relay stays on host loopback and is reached through the proxy.chopinbinary carries (guest-bridge.ts). An investigation'supload_image_filereads the image inside the guest's repository.nvxruns in its own process group, which is ended with its launch. Launch records under<base>/nvx/launcheshold the start IDs of the launch and its connector. The next connector kills only a process that still has its recorded start, never one that took over its number, and never a live chopin's launch. On Windows eachnvx.exegoes in a kill-on-close job object, so a connector crash ends its VMs too. A stop first ends the workload's stdin and waits; only then does it kill the job. A VM still running 10 s later is reported as wedged rather than taken for gone.realDirectory,writeOwn,readOwn). A writable share never holds anything the launch may only read, and--sandbox-writecan't name a folder inside the state directory.<disk>.held, linked into place atomically and counted per process), and a sweep takes the same hold before it removes one. A disk is removed only for the server its document is on (<slot>.server). A Planner reset in Chopin removes the document's disk: the reset is recorded (channels.planner_reset_at, migration 052, expand-only), and each Planner credential names it. The connector stamps each disk with the reset it was made after (<slot>.reset) and removes an older disk before the next launch, or at the hourlyplanner/keptsweep, saying that unpushed commits went with it.microvmsOnly). An older connector is told to update (connector-update-required) whatever it says of its agents, and a refused connection is kept on the machine as one that can't run work. A protocol 8 connector that doesn't say so getsmicrovm-required, and a machine's words about why are kept. The connector says so on every request oncechopin pairorconnecthas checked the machine.POST /api/connector/capabilityrecords why a paired machine can't run them (machines.vm_problem, migration 051, expand-only), and the Machines listing shows it while the machine is away.POST /api/connector/planner/kepttells a machine which Planner disks it may forget: each archived, gone, or reset since the disk was made.What read a run's worktree on the host, and where it went
upload_image_file)git worktree addunder/workin the guest--keep-runs)read_rebuildHEADmovedInvariants and their tests
nvx.test.ts(the agent isnvx sandbox run's workload);nvx-vm.test.ts(ending a launch ends OpenVMM)nvx.test.ts;nvx-vm.test.ts(writing into the person's.gitfails read-only)nvx.test.ts;run.test.ts;planner.test.ts;nvx-vm.test.tsnvx.test.ts;nvx-vm.test.ts(no direct connection, no DNS);sandbox-proxy.test.tsnvx.test.ts;nvx-vm.test.ts(no secret on any host command line);launch-handoff.test.tsplanner-disks.test.ts(including a sweep racing a launch in the same process, and two servers)nvx.test.ts(each shape, plus a property check)machine-store.test.ts;nvx-vm.test.ts(the guest plants the link)sandbox.test.ts(a reused pid survives, a matching orphan is killed)nvx.test.ts,main.test.ts,messages.test.ts,routes.test.ts,machine-contract.tsroutes.test.ts(connections, GitHub token, MCP token, AI launch, database login);planner-dispatch.test.ts(Planner credential); each refusal comes before anything is minted.machine.test.ts(the connector says so on each request)sandbox.test.tsplanner-disks.test.ts(sweep and launch);planner.test.ts(a launch after a reset starts on a new disk);routes.test.ts(planner/kept);planner-dispatch.test.ts(the credential names the reset);channels/routes.test.ts; the storage contract on memory and PostgreSQLValidation
On the final head:
bun run fixmade no changes.bun run ciandbun run typespassed.bun test apps/connector apps/server/src/{experiments,tasks,harness,auth,ai,machine-store,mcp,channels,storage/memory} apps/web/src/experiments packages/protocolgave 2190 tests: 0 failed, 28 skipped.TEST_DATABASE_URL=… bun test apps/server/src/storage/postgres(a throwawaypostgres:17-alpine) gave 222 passed.apps/server/src/machine-store/server.test.tsagainst the same database gave 18 passed.CHOPIN_NVX_ROOT=<nvx-0.1.0-linux-kvm> bun test apps/connector/src/nvx-vm.test.tsgave 8 passed. Those tests cover:.git;bun run e2efor setup-agents, machines, implementation-launcher, planner-spare, experiments, planner-picker, handover and build-restart, underflockwith an own port base and database. 119 passed. One machines test, "the sidebar's Machines menu sets the default machine and leads to the page", failed once under load and then passed 3 times out of 3 on its own.Windows
The three gaps found while measuring on WHP are closed, and
windows-host.test.tspassed on a Windows Server 2025 host with WHP:FSCTL_SET_SPARSEandSetEndOfFile) and formatted withnodiscard..gitis shown again.nvx.exeruns in a kill-on-close job, as described under Teardown. A machine without the Windows Hypervisor Platform is refused with the command that turns it on.On that host, the Windows binary ran
--versionandstatus. Pair, connect and a Build or Planner end to end on Windows are the next check.Not done, or not verified
install.shandinstall.ps1, is the next step.nvx sandbox runleaves its temporary folder when a launch is ended by force; sweeping it isn't built.