Repository navigation
feat: let hosts allow and build projects' sandbox images - #88
Merged
kalaspuff merged 8 commits intoOct 18, 2026
Merged
Conversation
…ositories Hosts list the images they run for projects that choose their own (allowed_images in host.toml: references, or prefixes in /*, optionally pinned to digests) and the repositories whose Dockerfile or devcontainer.json they build images from (allowed_builds). Both reach the fleet as optional fields in the hello; FleetDO schedules sessions only on runners that run or build their image, and a session that no runner takes says why. Hosts refuse sessions they don't allow, for control planes that send one anyway. A project's image_build setting names a Dockerfile (with a context) or a dev container configuration in its repository. A host that allows it reads the file on the default branch, keys the build by the files it reads, downloads only what the build needs, and builds it once, one build at a time, within build_timeout_minutes; an image over max_build_mb is removed. The next build of a project removes its earlier one, and while GitHub can't be reached sessions start from the last build. Refs #16
fctl-agent merged nothing into ~/.claude/settings.json: it overwrote it, so an image's plugins, marketplaces, hooks and permission rules were lost. It now writes its own keys over the image's settings and drops what could move Claude off the subscription or out of bypass mode (apiKeyHelper, ANTHROPIC_* and provider variables in env, permissions.disableBypassPermissionsMode). Before starting Claude it runs claude --version and fails the session with a clear message when Claude Code is missing or older than 2.1.101, the first release that asks its host for fresh OAuth tokens. The agent's hello reports the version. Refs #16
fctl projects set gets --image, --build and --build-context, --setup-script (the script, @file or - for stdin), --env and --unset-env, and --cpus and --memory (default for each runner's own), and fctl projects show says where sessions start from, their size, setup script and variables. fctl host install takes --allow-image and --allow-build for allowed_images and allowed_builds, refusing entries that match nothing; fctl host status and doctor, and fctl admin runners, say which images a host runs and which repositories it builds images from. Refs #16
Settings → Sandbox → Image picks each runner's own image, an image from a registry, or one built from the repository (a Dockerfile and the directory it builds in, or a devcontainer.json), and says which drivers use images and which runners run or build them. Admin → Runners shows each runner's image and what else it runs and builds. The docs describe projects' own images with a worked example, builds from the repository, which images a host runs, private registries, what an image needs and what fctl writes into a sandbox; ADR 0017 records the decisions. Refs #16
"A project's own image" made a different anchor on fctl.sh than on GitHub; "Your own image" makes the same. The .NET example's profile.d line fits its code block.
Two sessions that finished getting their images at once could each save builds.json without the other's project. The docs now say that a build whose directory is the repository's root runs again after any change, and how to avoid that.
- A tag in allowed_images no longer allows a reference that pins a digest to it: a pull by digest ignores the tag. - References follow Docker's grammar: uppercase makes the first part a registry, repositories are lowercase, separators and sha256 digests are checked. Docker Hub prefixes written with docker.io or index.docker.io match its images. - Hosts read dev container configurations in one pass, and no file over 1 MiB. - Only an unreachable GitHub starts sessions from the last build, and only one built for the same setting; a missing file fails the session. - Builds use the Dockerfile's text as GitHub sends it, never a file a link in the repository points to on the host, and unpack nothing through a link. - Sessions that need the same build wait for the one that builds it, and each downloads before waiting for another build, so its token can't expire in the queue. A stopped build removes its files, and hosts clear leftovers when they start. - Download errors leave out signed links' tokens; a dev container's image must be a valid reference. - fctl projects set --build x.json drops the Dockerfile's directory; the web form brings back what was typed when switching back; an image and a build together fail on the image's field, which older forms show. Refs #16
This was referenced Oct 18, 2026
kalaspuff
deleted the
fctl/39-can-you-check-if-something-is-left-to-do
branch
October 18, 2026 18:23
kalaspuff
pushed a commit
to hyperstructure/fctl-site
that referenced
this pull request
Oct 18, 2026
The privacy policy says what `fctl host` downloads and what app.fctl.sh keeps about hosts. hyperstructure/fctl#88 changes both, for hyperstructure/fctl#16: - **What hosts download.** Hosts pull the images projects choose, from those images' registries. Where their owners allow it, they also download a project's repository from GitHub to build the image the project builds from it. - **What app.fctl.sh keeps.** Hosts report the sandbox image they use, and the images and repositories they allow. app.fctl.sh keeps these with the rest of a host's details. The policy's `updated` date is already today's. Refs hyperstructure/fctl#16 Pairs with hyperstructure/fctl#88. ## How I verified it `npm run check` passes, with the same two warnings as `main`. `FCTL_DOCS_DIR=<that branch> npm run build` renders the policy and the docs from the fctl branch. Every link and anchor between the docs pages resolves, including the new sections of "Sandbox images". Session: https://app.fctl.sh/s/ses_01M57YPG1ATK9ARAXTZG43G51J Co-authored-by: fctl-sh[bot] <343971779+fctl-sh[bot]@users.noreply.github.com>
fctl-sh Bot
added a commit
that referenced
this pull request
Oct 18, 2026
Resolves the conflicts with #88. Its build tests and the image refresh's share one stand-in CLI, driver/fake.rs, which gains #88's builds, image sizes and failing builds; the two remove_image methods it and this branch added were the same, so one stays. docs/sandbox-images.md keeps #88's wording for project images, and says that a host on another image removes none of the default image's: it does remove old project builds.
kalaspuff
pushed a commit
that referenced
this pull request
Oct 18, 2026
Hosts on the default sandbox image pulled their tag once and kept it. hyperstructure/sandbox's Image workflow rebuilds the newest release's image when Claude Code has a newer release, since fctl turns off Claude Code's auto-updater. But a host that already had the tag kept its first pull until it was upgraded or someone ran `fctl host image pull`, so its sessions' Claude Code aged between upgrades anyway. And every upgrade left the previous version's image on the host, as did the `latest` a host pulls while its release's image isn't published yet. Nothing removed them. The two decisions #47 left to a person were made in this session, and are recorded in [the issue](#47 (comment)). The host tells a local build from the published image by the `org.opencontainers.image.revision` label, which only the Image workflow sets. It pulls again at start and every 6 hours. ## What changed **Refreshing** (`crates/fctl-host/src/refresh.rs`, `daemon.rs`). The host's background pull used to run once, at start, and only for a missing image. Now it runs at start and every 6 hours: - An image that isn't here is pulled as before, falling back to `latest` while the version's image can't be pulled. - On the default image, the host pulls its tag again, so new sessions start from the newest build. Sessions never wait for it: they start from the build that is there. When the tag hasn't changed, the pull costs one manifest request. - It never pulls over an image without the revision label. That covers builds from `fctl host image build` or by hand: the sandbox's `Dockerfile` sets no labels, and its base, `debian:bookworm-slim`, has none. An image whose description can't be read counts as a local build, so the host leaves it alone, as it did before. - A host whose `host.toml` sets another image pulls it only when it isn't there, as before. **Removing** (`refresh::clean_up`). At start, after each pull and after sweeps that remove sandboxes, the host removes these, without force: - builds that a newer one replaced; - other versions' images (`X.Y.Z` tags); - `latest`, once the version's image is here. Until then, sessions start from it. Images without the revision label stay, and so do tags hosts don't pull, like `0.5`, which projects may use. A host on another image removes nothing. **How a replaced build stays findable.** A pull that moves a tag leaves the old build without a name. Docker keeps it as `<none>`, and with the containerd image store nothing finds that again by name. Apple's `container` keeps its blobs and snapshot orphaned until an `image delete` runs its cleanup. So before a pull (or `fctl host image build`) replaces a published build, the host gives that build a second name, `<tag>-replaced-<the start of its ID>`, and removes the name right after: - If the build is unchanged, removing the name only untags it. - If it was replaced and nothing uses it, the build goes. - With Docker, if it was replaced and a sandbox still uses it, Docker refuses. The name stays until a cleanup after the sweep removes that sandbox (`keep_stopped_hours`). This needs no state file, and a host that stops halfway leaves a name the cleanup finds. A name whose build is still the one sessions start from is left alone, so a cleanup that runs during a pull can't strand the build. **Apple.** `container image delete` doesn't check whether a container uses the image. It's still safe: each container gets a clone of the image's filesystem when it is created (`Bundle.cloneContainerRootFs` in apple/container), and starting it again uses only that clone. So on Apple the host removes these right away, and stopped sandboxes still start. **Driver** (`driver/container.rs`, `driver/mod.rs`): - `image`: ID and labels from `image inspect`, in Docker's, Podman's and both of Apple's shapes (its JSON changed at 1.0). - `tag_image`. - `remove_image`: `image rm` or `image delete`, never forced. - `images`: `image ls --format …` or `image list --quiet`. **CLI** (`crates/fctl/src/cmd/host.rs`): - `fctl host image pull` also removes what sessions no longer start from, and says what it did: ``` ✓ Pulled ghcr.io/hyperstructure/sandbox:0.5.0, a newer build (Claude Code 2.1.310) Removed ghcr.io/hyperstructure/sandbox:0.4.0 The build it replaced stays until no sandbox uses it; the host removes it then. ``` `--json` adds `newer` (whether the pull brought a newer build) and `removed` (the names it removed) to `image` and `instead_of`. - `fctl host doctor` shows the image's Claude Code version, or that the host doesn't pull over a local build: ``` ✓ sandbox image ghcr.io/hyperstructure/sandbox:0.5.0 (Claude Code 2.1.310) ✓ sandbox image ghcr.io/hyperstructure/sandbox:0.5.0 (built here: the host doesn't pull over it) ``` **Docs:** - `docs/sandbox-images.md` says when the host pulls again, how it tells a local build, and what it removes and when, with Docker and with Apple. - `docs/architecture.md` and `docs/cli.md` say the same briefly. - AGENTS.md lists the revision label among what the host relies on in hyperstructure/sandbox. hyperstructure/sandbox#3 does the same there. **Tests.** The driver tests' stand-in for the `docker` and `container` CLIs moved to `driver/fake.rs`, shared by the driver tests, these tests and #88's build tests. It gained image IDs, labels, tags, listing, and Docker's refusal to remove an image's last name while a container uses it. It keeps #88's builds, image sizes and failing builds. **With #88.** `main` gained #88 (projects' own images and builds) while this was open, and is merged in. Both added the same `remove_image` to the driver, so one stays. Project images and builds aren't the default image, so the host doesn't pull them again here or remove them in this cleanup: #88 keeps each project's newest build itself. ## Compatibility This changes only the host: no protocol or control-plane change, and it reaches hosts with the next release. `fctl host image pull --json` gains two fields. - At its first start after the upgrade, a host removes the previous version's image, unless a stopped sandbox uses it (Docker). - Builds that earlier pulls left as `<none>` stay: nothing tells them from other unnamed images. - Two hosts sharing one Docker daemon on different versions would each remove the other's image when no sandbox uses it, and pull it again later. That costs extra downloads; nothing breaks. ## Verification - **Unit tests:** 14 new; fctl-host has 57 with #88's. - `removable`: what goes and what stays, including `latest` before and after the version's image arrives, local builds, `X.Y` tags, and a replaced name whose build is still current. - Version tags, replaced names, and inspect output from Docker, Podman, Apple 0.x and Apple 1.x. - Flows against the stand-in: - a newer build replaces one a stopped sandbox uses, which goes once the sandbox is gone; - an unchanged build is only tagged and untagged; - the host never pulls over a local build, and `fctl host image pull` does; - a build over the published image (and a failed one); - `latest` going once the version's image is published; - other versions going after an upgrade, unless in use; - Apple removing a replaced build that a sandbox was made from; - a host on another image doing nothing. - **The published image** (read from ghcr.io anonymously): the config of `ghcr.io/hyperstructure/sandbox:0.5.0` has `org.opencontainers.image.revision` (`12685ad…`) and `sh.fctl.claude-code.version` (`2.1.302`). `debian:bookworm-slim`, the base of local builds, has no labels. - **With real Podman** (rootless, `FCTL_DOCKER_BIN=podman`): I imported images of a few KB under the default image's names. The host's `0.5.0` was a local build, and a stopped `fctl-ses-…` container used `0.3.0`. `fctl host run`: - logged that it doesn't pull over the local build; - removed `0.4.0` (only untagged, since `0.4` names the same build), `latest`, and a `0.5.0-replaced-…` build; - kept `0.3.0` because Podman refused ("image is in use by a container"), and kept the local `0.2.0` and the `0.4` tag. `fctl host doctor` said "built here: the host doesn't pull over it". - **With the stand-in CLI** (`FCTL_DOCKER_BIN`), the real binary: - `fctl host image pull` printed the output above. - Once the sandbox was gone, `fctl host image pull --json` printed `{"image": "ghcr.io/hyperstructure/sandbox:0.5.0", "instead_of": null, "newer": false, "removed": ["ghcr.io/hyperstructure/sandbox:0.5.0-replaced-0a1d00000000"]}`. - `fctl host run` logged `pulled a newer build of ghcr.io/hyperstructure/sandbox:0.5.0 (Claude Code 2.1.310)`, kept the replaced build while a sandbox used it, and removed `0.4.0`. - `doctor` showed the Claude Code version. - `cargo xtask check --quick` passes, before and after merging `main`. I reran the Podman check on the merged binary, with the same result. - **Not run here:** Docker itself (this sandbox can't run a Docker daemon) and Apple's `container` (that needs a Mac). The commands are the documented ones. I read the behaviors this relies on in their sources: - Docker refuses, without force, to remove an image's last name while a container uses it, in both image stores, and moby's containerd store counts its `moby-dangling@…` records as that same name (`daemon/images/image_delete.go`, `daemon/containerd/image_delete.go`). - In apple/container, a container keeps its own clone of the image's filesystem, a pull leaves the old build orphaned until `image delete` cleans up, and `image tag` overwrites an existing name. Trying the first release on a Docker host and on a Mac is the check that's left. Fixes #47 Session: https://app.fctl.sh/s/ses_01M5813Y3P1VY7K0ZSS6PKRS9V --------- Co-authored-by: fctl-sh[bot] <fctl-sh[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finishes #16. A project starts its sessions from an image of its own, or from one that hosts build out of a Dockerfile or devcontainer.json in its repository. Host owners decide which images their machines run and which repositories they build from, and sessions go only to runners that allow their image. The docs now say how to do all of it.
Fixes #16
What changed, by the issue's points
docs/sandbox-images.md). "What an image needs" now listsgh, a non-root user, both architectures and the oldest Claude Code fctl works with. That is 2.1.101, the first release that asks its host for fresh OAuth tokens (oauth_token_refresh;remote_controlcame in 2.1.63). I found both by checking published Claude Code builds, because the changelog doesn't say. fctl-agent now runsclaude --versionbefore it starts Claude, and fails the session with a clear message when Claude Code is missing or older. "What fctl writes into a sandbox" is new. The worked example adds .NET rather than Rust, since the default image has had Rust since feat: add Python 3.15, Rust, Bun and TypeScript to the sandbox image #45.fctl projects setgets--image,--build/--build-context,--setup-script(the script,@FILE, or-for stdin),--env/--unset-env, and--cpus/--memory(defaultfor the runner's own), with--jsonlike the rest.fctl projects showprints them.~/.claude/settings.jsoninstead of replacing the file. It owns the permission mode,attribution,disableClaudeAiConnectorsand the auto-updater. It dropsapiKeyHelper,ANTHROPIC_*and provider variables, andpermissions.disableBypassPermissionsMode. Plugins, marketplaces, hooks and permission rules stay.crates/fctl-host/src/build.rs). The new project settingimage_buildnames a Dockerfile (plus the directory it builds in) or a devcontainer.json.docker buildorcontainer build, one build at a time.build_timeout_minutes(60) andmax_build_mb(20 GiB). It keeps each project's newest build, and starts from the last build while GitHub is unreachable.image,build.dockerfile/context/args/target. It leaves out features and lifecycle commands, and says so in the session's log.host.tomlgetsallowed_images(references,/*prefixes, digest pins) andallowed_builds(owner/name,owner/*).fctl host installgets--allow-imageand--allow-buildfor them.fctl_domain::schedule::pick). A session no runner takes waits and says why ("No runner allows this project's image, …" / "No runner builds this project's image from …").docs/security.mdsays that whoever sets the image, the build or the setup script can read the credentials of the people who start sessions.fctl admin runnersandfctl host statusshow each runner's image and what else it runs and builds.docker login,container registry login), and the docs say so. fctl stores no registry credentials.ADR 0017 records the decisions. One worth a look:
allowed_imagesunset still means any image, so upgrading no host stops a project's sessions.Compatibility
ProjectSettings.image_buildis always serialized, asnullwhen there is none.HostHello.allowed_images/allowed_buildsandRunner.image/allowed_*are skipped when empty.image_build.image_buildkeep a project's build, unless they choose an image instead (settings::keep_unsent).How I verified it
cargo xtask checkpasses (fmt, generated types, Biome and TypeScript, Clippy native and wasm, all tests, the web build). New unit tests cover:keep_unsentand build validation;docker. It covers: build once, reuse, no rebuild after an unrelated commit, rebuild after a change in the build directory or the Dockerfile, removal of the old build, fallback while GitHub is down, a failing build, the size cap, and a devcontainerimage. It also caught the token going to the archive's redirect origin, which is fixed.cargo xtask dev --no-host, two hosts I registered with the new flags: a process-driver host allowingghcr.io/acme/*, and a Docker-driver host behind a stand-indockerthat builds fromacme/*):other/toolwaited ("No runner builds this project's image from other/tool"); a build fromacme/appwent to the building host, and stopped there only because the dev stack has no GitHub App;RUST_LOGstayed,apiKeyHelperandANTHROPIC_BASE_URLwere gone;fctl admin runnersandfctl host statusread as described.profile.dsnippet set the PATH for login shells.allowed_imagesallowed that tag pinned to any digest;.jsonbuild file was quadratic;docker.iomatched nothing;Not verified: a real
docker buildor Applecontainer build. This sandbox has neither runtime, so the builder commands,container image inspectsizes andcontainer image deleteare untested on real runtimes. A first build on a Docker host and on a Mac host is worth doing before relying on it.Not in this change
For a person to decide; I can open issues for them:
Other repositories
Screenshots
Taken at 809b7b2 against
cargo xtask devwith two dev hosts, 1440 px wide unless noted. Later commits don't change these screens.Settings → Sandbox, a project that builds its image. Before, the form has no way to show a build; after, the Image choice, the file and the directory it builds in.
The three choices, an image from a registry with its hint, and the dark theme at 390 px:
Admin → Runners. Each runner now says its image, what else it runs and what it builds.
Session: https://app.fctl.sh/s/ses_01M57YPG1ATK9ARAXTZG43G51J