diff --git a/.claude/skills/change-the-image/SKILL.md b/.claude/skills/change-the-image/SKILL.md index a748122..7e32bb4 100644 --- a/.claude/skills/change-the-image/SKILL.md +++ b/.claude/skills/change-the-image/SKILL.md @@ -1,6 +1,6 @@ --- name: change-the-image -description: Add, upgrade or remove a tool in the sandbox image, or change its layout. Covers where versions live, both architectures, the Lima VM that provision.sh also provisions, what fctl and derived images rely on, the image's size, and verifying a change with a local build or the pull request's Image workflow. Use for any change to Dockerfile, provision.sh or check.sh. +description: Add, upgrade or remove a tool in the sandbox image, or change its layout. Covers where versions live, both architectures, the Lima VM that provision.sh also provisions, what fctl and derived images rely on, the image's size, and verifying a change with a local build or the pull request's Image and Develop fctl workflows. Use for any change to Dockerfile, provision.sh, check.sh or check-fctl.sh. --- # Changing the image @@ -44,11 +44,13 @@ Removing a tool breaks images and sessions that use it: a breaking change - **Two architectures.** Every download works on amd64 and arm64: pick the file with `dpkg --print-architecture` (`amd64`, `arm64`) or `uname -m` (`x86_64`, `aarch64`). -- **The Lima VM.** fctl's Lima driver runs `provision.sh` in a Debian 12 VM - (Lima's cloud image, not `bookworm-slim`) as root, with `AGENT_USER` set - to the VM's user, who already exists and may not be uid 1000. Use - `$AGENT_USER` and `$AGENT_HOME`, never `agent` or `/home/agent`, and don't - rely on what only the Dockerfile sets. +- **The Lima VM.** fctl's Lima driver runs `provision.sh` in a Debian 13 VM + (Lima's cloud image, not `trixie-slim`), or a Debian 12 one with Lima + before 2.0 and on hosts of fctl 0.14 and earlier, so keep it working on + both. It runs as root, with `AGENT_USER` set to the VM's user, who + already exists and may not be uid 1000. Use `$AGENT_USER` and + `$AGENT_HOME`, never `agent` or `/home/agent`, and don't rely on what + only the Dockerfile sets. - **Login shells.** Setup scripts and the Lima driver run `bash -l`, which reads `/etc/profile.d/fctl.sh`. Keep its PATH and the Dockerfile's `ENV PATH` the same. @@ -86,7 +88,10 @@ change the next time they pull the image. - **In an fctl session** (no Docker): open the pull request. The Image workflow builds both architectures with the latest fctl and Claude Code and runs `check.sh` in each: `gh pr checks N --watch --fail-fast`, then - the run's summary for the sizes. + the run's summary for the sizes. The Develop fctl workflow runs + `check-fctl.sh` in each too: fctl's own checks, sessions and screenshots, + as sessions working on fctl do them, with fctl's branch of the same name + when there is one. - **With fctl:** on a host, `docker build -t sandbox:dev .` and set it as a project's image (Settings → Sandbox → Image), or `fctl host image build --source .` from a checkout, which builds it under diff --git a/.claude/skills/commit/SKILL.md b/.claude/skills/commit/SKILL.md index 99fb5cf..04fad83 100644 --- a/.claude/skills/commit/SKILL.md +++ b/.claude/skills/commit/SKILL.md @@ -54,8 +54,8 @@ to the fctl session that made the change is welcome. 1. Branch from an up-to-date `main`: `/` (`feat/postgres`), or `fctl/-` when an fctl session makes the branch. 2. Commit as you go; the commits are squashed, so the title is what lasts. -3. Run `shellcheck provision.sh check.sh` and `actionlint` if you have them; - the Lint workflow runs both anyway. +3. Run `shellcheck provision.sh check.sh check-fctl.sh` and `actionlint` if + you have them; the Lint workflow runs both anyway. 4. Open the pull request with a conventional title and a description. One change per pull request. 5. Wait for the checks in the same turn and fix failures (below). @@ -75,6 +75,14 @@ gh pr checks N --watch --fail-fast - **Image** builds the pull request's image for amd64 and arm64 with the latest fctl and Claude Code and runs `check.sh` in each. It runs when the image's files or the workflow change, and takes a few minutes. +- **Develop fctl** builds the pull request's image for amd64 and arm64 and + runs `check-fctl.sh` in each: fctl's checks, a session through its local + stack and one through its standalone control plane, and screenshots, with + hyperstructure/fctl's branch of the pull request's name when there is one, + else its `main`. It runs when the image's files, the script or the + workflow change, and takes about a quarter of an hour. A step that the pull + request doesn't touch can fail because of fctl's `main`: the log names + the step. - **Lint** runs ShellCheck and actionlint. - **Skills** validates every skill in `.claude/skills` against the [Agent Skills specification](https://agentskills.io/specification). It diff --git a/.github/workflows/fctl.yml b/.github/workflows/fctl.yml new file mode 100644 index 0000000..8ab82c9 --- /dev/null +++ b/.github/workflows/fctl.yml @@ -0,0 +1,65 @@ +# Sessions that work on fctl itself build, test and run it in the image. On +# pull requests that change the image, this builds it for arm64 and amd64 and +# runs check-fctl.sh in each: what fctl's CI checks, a session through its +# local stack and one through its standalone control plane, an npm install +# that must leave its lockfile as it is, and screenshots of its web app with +# Playwright, which it keeps as an artifact. It checks hyperstructure/fctl's +# branch of the pull request's name when there is one (an fctl session that +# changes both repositories names their branches alike), else main; +# `gh workflow run fctl.yml -f ref=…` checks another branch with main's image. +name: Develop fctl + +on: + pull_request: + paths: + - Dockerfile + - provision.sh + - check-fctl.sh + - .dockerignore + - .github/workflows/fctl.yml + workflow_dispatch: + inputs: + ref: + description: The branch of hyperstructure/fctl to check (default main) + required: false + +permissions: + contents: read + +concurrency: + group: fctl-${{ github.ref }} + cancel-in-progress: true + +jobs: + develop: + name: Develop fctl (${{ matrix.arch }}) + strategy: + fail-fast: false + matrix: + include: + - arch: amd64 + runner: ubuntu-24.04 + - arch: arm64 + runner: ubuntu-24.04-arm + runs-on: ${{ matrix.runner }} + timeout-minutes: 90 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Build the image + run: docker build --pull --tag sandbox:fctl . + # As fctl hosts start sandboxes: with an init, 4 CPUs and 8 GB. + - name: Develop fctl in it + env: + FCTL_REF: ${{ inputs.ref || github.head_ref || 'main' }} + run: | + mkdir -p "$RUNNER_TEMP/screenshots" + chmod 777 "$RUNNER_TEMP/screenshots" + docker run --rm -i --init --cpus 4 --memory 8g -e FCTL_REF -e OUT=/tmp/screenshots \ + -v "$RUNNER_TEMP/screenshots:/tmp/screenshots" sandbox:fctl bash -s < check-fctl.sh + - if: always() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: screenshots-${{ matrix.arch }} + path: ${{ runner.temp }}/screenshots + if-no-files-found: ignore + retention-days: 14 diff --git a/.github/workflows/lint.yml b/.github/workflows/lint.yml index 1a67b2d..55829be 100644 --- a/.github/workflows/lint.yml +++ b/.github/workflows/lint.yml @@ -24,7 +24,7 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - name: ShellCheck - run: shellcheck --version && shellcheck provision.sh check.sh + run: shellcheck --version && shellcheck provision.sh check.sh check-fctl.sh - name: actionlint run: | archive="actionlint_${ACTIONLINT_VERSION}_linux_amd64.tar.gz" diff --git a/AGENTS.md b/AGENTS.md index 3d89ab0..cccf0ca 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,7 +15,9 @@ to its code in hyperstructure/fctl. | `Dockerfile` | The image: runs `provision.sh`, then sets the user and the PATH. | | `provision.sh` | Installs everything, as root. The versions are at the top. | | `check.sh` | Checks a built image for the layout fctl and derived images rely on. | +| `check-fctl.sh` | Develops fctl in a built image, as sessions working on hyperstructure/fctl do. | | `.github/workflows/image.yml` | Builds both architectures, checks them and publishes them. | +| `.github/workflows/fctl.yml` | On pull requests, runs `check-fctl.sh` in both architectures' builds. | | `.github/workflows/lint.yml`, `commits.yml`, `skills.yml` | ShellCheck and actionlint; pull request titles; the skills, against the Agent Skills specification. | | `.claude/skills/` | How to change the image, publish it, and commit. | @@ -25,13 +27,15 @@ to its code in hyperstructure/fctl. docker build -t sandbox:dev . # your architecture docker build -t sandbox:dev --build-arg FCTL_VERSION=0.3.0 . # with another fctl docker run --rm -i sandbox:dev sh -s < check.sh # check the layout -shellcheck provision.sh check.sh +docker run --rm -i --init sandbox:dev bash -s < check-fctl.sh # develop fctl in it +shellcheck provision.sh check.sh check-fctl.sh actionlint # the workflows ``` fctl sessions have no Docker. There, a pull request's Image workflow is the build: it builds amd64 and arm64 on runners of their own and runs `check.sh` -in each. Watch it with `gh pr checks N --watch --fail-fast`. +in each, and its Develop fctl workflow runs `check-fctl.sh` in each, which +takes longer. Watch them with `gh pr checks N --watch --fail-fast`. ## What fctl relies on @@ -49,10 +53,11 @@ pull request. - **The files fctl downloads.** `fctl host image build` downloads `main` as an archive and builds `Dockerfile` with the repository's root as the context, passing `FCTL_VERSION` and `FCTL_DOWNLOAD` as build arguments. - With Lima, it copies `provision.sh` into a Debian 12 VM and runs it as + With Lima, it copies `provision.sh` into a Debian 13 VM (Debian 12 with + Lima before 2.0, and on hosts of fctl 0.14 and earlier) and runs it as root with `AGENT_USER` set to the VM's user and the same variables. Keep both files at the root, under these names, self-contained, and working on - Debian 12 with a user other than `agent`. + Debian 13 and 12 with a user other than `agent`. - **The tags**: `` for every fctl release, `.` and `latest`. Hosts pull their own version's tag, again every 6 hours for newer builds, and fall back to `latest`. They remove other versions' @@ -80,14 +85,20 @@ hyperstructure/fctl) and, when hosts must adapt, an fctl release first. - Rust is the release fctl builds with (`rust-toolchain.toml` in hyperstructure/fctl), so sessions working on fctl have nothing to download. Bump the two together. +- fctl's own development works in the image: sessions that work on + hyperstructure/fctl run its checks, its local stack, its standalone + control plane and its screenshots there. The Develop fctl workflow checks + that on pull requests with `check-fctl.sh`; when it fails, its log says + which step, and whether the image or fctl's `main` changed decides where + the fix goes. - Prefer Debian's packages; otherwise the project's own installer or release archive, pinned to a version. Remove apt's lists and other caches in the same step. - Mind the size: hosts download the image for every fctl release. Say in the pull request what a change adds; the Image workflow's summary lists each architecture's size. -- `provision.sh` is bash with `set -euo pipefail`; `check.sh` is POSIX sh. - Both pass ShellCheck. +- `provision.sh` and `check-fctl.sh` are bash with `set -euo pipefail`; + `check.sh` is POSIX sh. All three pass ShellCheck. - Workflows pin actions to a commit SHA with the version in a comment, and give each job only the permissions it needs. - Copy is plain and specific, in sentence case. diff --git a/Dockerfile b/Dockerfile index fb4da31..6ba0daf 100644 --- a/Dockerfile +++ b/Dockerfile @@ -5,7 +5,7 @@ # # docker build -t sandbox:dev . # fctl host image build # on an fctl host, with its driver -FROM debian:bookworm-slim +FROM debian:trixie-slim # The fctl release to install and where releases come from; `fctl host # image build` passes its own version and download_base. Empty: the latest, diff --git a/README.md b/README.md index 3e3abfa..9330dca 100644 --- a/README.md +++ b/README.md @@ -16,16 +16,17 @@ their own copy instead). It also works as a base for images with more tools ## What's inside -- Debian 12 (bookworm), with `LANG=C.UTF-8` and `TZ=UTC`. +- Debian 13 (trixie), with `LANG=C.UTF-8` and `TZ=UTC`. - Claude Code, the native build, in `~/.local/bin`: the newest release when the image was built. - fctl, the release the tag names, in `/usr/local/bin`. Agents name and describe their session with `fctl edit`. - Python 3.15.0 as `python3` and `python` for every user (uv's build of - CPython, in `/opt/uv/python`), and uv 0.13.1. -- Rust 1.99.0 through rustup, with rustfmt, Clippy and the + CPython, in `/opt/uv/python`), and uv 0.13.4. +- Rust 1.99.1 through rustup, with rustfmt, Clippy and the `wasm32-unknown-unknown` target. -- Node.js 22 with npm and corepack, TypeScript 7.0.2 (`tsc`) and Bun 1.4.3. +- Node.js 26 with npm and corepack 0.36.0 (with its yarn and pnpm shims), + TypeScript 7.0.2 (`tsc`) and Bun 1.4.3. - build-essential (gcc, make), cmake, pkg-config and OpenSSL's headers. - git, the GitHub CLI, ripgrep, fd, jq, ShellCheck, sqlite3, curl, wget, rsync, lsof, fuser, tree, file, unzip, zip, xz and zstd. @@ -90,7 +91,7 @@ it. `cargo`, `rustup` and `bun` as `agent`. - **Python** is marked externally managed: packages go into a virtual environment (`uv venv`, `python3 -m venv`) or come with `uv run --with`. - Debian's Python 3.11, which Node.js depends on, stays in `/usr/bin`, after + Debian's Python 3.13, which Node.js depends on, stays in `/usr/bin`, after `/usr/local/bin` on the PATH. - **npm** leaves out its funding and audit notices and its update check (`/usr/etc/npmrc`); `npm audit` still runs when asked. @@ -119,6 +120,7 @@ Apple silicon hosts need arm64 images. ```sh docker build -t sandbox . docker run --rm -i sandbox sh -s < check.sh +docker run --rm -i --init sandbox bash -s < check-fctl.sh # fctl's development in it, 10 minutes or so ``` `--build-arg FCTL_VERSION=0.4.0` picks the fctl release (default: the @@ -133,6 +135,16 @@ publishes it: each architecture builds on a runner of its own and is checked with `check.sh` before the two are tagged as one image. Pull requests that change the image build and check both without publishing. +Sessions that work on fctl itself build, test and run it in the image, so +pull requests that change it also develop fctl in both architectures' builds +(the Develop fctl workflow, [`.github/workflows/fctl.yml`](.github/workflows/fctl.yml)): +[`check-fctl.sh`](check-fctl.sh) runs what fctl's CI checks, a session +through its local stack and one through its standalone control plane, an +`npm install` that must leave its lockfile as it is, and screenshots of its +web app with Playwright. It checks hyperstructure/fctl's branch of the pull +request's name when there is one (an fctl session that changes both +repositories names their branches alike), and `main` otherwise. + ## Questions and changes Open an issue for a tool the image lacks or something that doesn't work in diff --git a/check-fctl.sh b/check-fctl.sh new file mode 100755 index 0000000..198c1d0 --- /dev/null +++ b/check-fctl.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# Checks that fctl's own development works in an image built from this +# repository, as sessions that work on hyperstructure/fctl do it: what its CI +# checks (`cargo xtask check`), a session through its local stack (`cargo +# xtask dev --smoke`) and through its standalone control plane (`cargo xtask +# serve`), an `npm install` that leaves its lockfile alone, and screenshots of +# its web app with Playwright, as its screenshots skill takes them. Runs +# inside the image as its user; the Develop fctl workflow runs it on pull +# requests that change the image: +# +# docker run --rm -i --init sandbox:dev bash -s < check-fctl.sh +# +# FCTL_REF is the branch of hyperstructure/fctl to check (default, and when +# there is no such branch: main), FCTL_DIR where to clone it (default +# ~/workspace/fctl), and OUT, when set, a directory for the screenshots. +set -euo pipefail + +fail() { + echo "check-fctl.sh: $*" >&2 + exit 1 +} + +# All of it runs once bash has read it, with nothing on stdin: the script +# itself comes on stdin, and the tools it runs mustn't read the rest. +main() { + local repo=https://github.com/hyperstructure/fctl ref=${FCTL_REF:-main} + if ! [[ $ref =~ ^[A-Za-z0-9._/-]+$ ]] || ! git ls-remote --exit-code --heads "$repo" "$ref" >/dev/null; then + ref=main + fi + local dir=${FCTL_DIR:-$HOME/workspace/fctl} + git clone --quiet --depth 1 --branch "$ref" "$repo" "$dir" + cd "$dir" + echo "Developing hyperstructure/fctl $ref ($(git rev-parse --short HEAD)) with $(node --version), $(rustc --version)" + + cargo xtask check + cargo xtask dev --smoke + + # Adding a package rewrites the lockfile with the image's npm. + npm install --no-audit --no-fund + git diff --quiet package-lock.json || + fail "npm $(npm --version) rewrites fctl's package-lock.json: $(git diff --stat package-lock.json)" + + FCTL_SETUP_TOKEN=dev-setup-token cargo xtask serve >"$HOME/serve.log" 2>&1 & + server=$! + trap stop_server EXIT + local deadline=$((SECONDS + 1200)) + until curl -fsS -o /dev/null http://localhost:8787/api/v1/health; do + kill -0 "$server" 2>/dev/null || { cat "$HOME/serve.log"; fail "cargo xtask serve stopped"; } + [ "$SECONDS" -lt "$deadline" ] || { cat "$HOME/serve.log"; fail "cargo xtask serve didn't answer"; } + sleep 5 + done + cargo xtask dev --smoke --server http://localhost:8787 + + mkdir -p "$HOME/screenshots" + cd "$HOME/screenshots" + npm init -y >/dev/null + npm install --no-audit --no-fund playwright + npx playwright install --with-deps chromium + shoot >shoot.mjs + node shoot.mjs http://localhost:8787/ home-light.png + node shoot.mjs http://localhost:8787/ home-dark-390.png dark 390x844 + for png in home-light.png home-dark-390.png; do + [ "$(wc -c <"$png")" -gt 10000 ] || fail "$png is too small to show the web app" + done + if [ -n "${OUT:-}" ]; then cp home-*.png "$OUT/"; fi + echo "fctl's checks, a session through cargo xtask dev and one through cargo xtask serve, and screenshots work in the image." +} + +stop_server() { + pkill -TERM -f deploy/standalone/server.mjs || true + wait "$server" || true +} + +# fctl's screenshots skill's shoot.mjs: signs in as the dev owner, then takes +# the page in a color scheme and size. +shoot() { + cat <<'EOF' +// node shoot.mjs URL OUT.png [light|dark] [WIDTHxHEIGHT] +import { chromium } from "playwright"; + +const [url, out, colorScheme = "light", size = "1440x900"] = process.argv.slice(2); +const [width, height] = size.split("x").map(Number); +const browser = await chromium.launch(); +const page = await browser.newPage({ colorScheme, reducedMotion: "reduce", viewport: { width, height } }); +await page.addInitScript(() => { + let seed = 42; + Math.random = () => (seed = (seed * 16807) % 2147483647) / 2147483647; +}); +await page.goto(new URL("/login", url).href); +await page.fill("input[type=email]", "dev@fctl.test"); +await page.fill("input[type=password]", "fctl-dev-password"); +await page.click("button[type=submit]"); +await page.waitForURL((u) => u.pathname !== "/login"); +await page.goto(url, { waitUntil: "networkidle" }); +await page.screenshot({ path: out }); +await browser.close(); +EOF +} + +main "$@" /etc/apt/sources.list.d/github-cli.list -# Node.js and TypeScript. npm leaves out its funding and audit notices and -# its update check, which only lengthen agents' output. +# Node.js, corepack and TypeScript. Node.js stopped shipping corepack in 25, +# so it comes from npm, which also puts its yarn and pnpm shims on the PATH. +# npm leaves out its funding and audit notices and its update check, which +# only lengthen agents' output. curl -fsSL "https://deb.nodesource.com/setup_${NODE_MAJOR}.x" | bash - apt-get install -y --no-install-recommends gh nodejs -corepack enable || true npm config set --location=global fund=false audit=false update-notifier=false -npm install -g "typescript@$TYPESCRIPT_VERSION" +npm install -g "corepack@$COREPACK_VERSION" "typescript@$TYPESCRIPT_VERSION" npm cache clean --force # uv, and Python for everyone: uv's build of CPython in /opt/uv/python, as @@ -116,4 +119,5 @@ as_agent 'ln -sf bun "$HOME/.bun/bin/bunx"' # What sessions find on the PATH. as_agent 'claude --version; fctl --version; python3 --version; uv --version - rustc --version; cargo --version; node --version; tsc --version; bun --version' + rustc --version; cargo --version; node --version; corepack --version; tsc --version + bun --version'