[fctl-sh]

fctl host image pull on a 0.4.0 host failed right after the release:

$ fctl host image pull
Error response from daemon: manifest unknown
error: docker pull ghcr.io/hyperstructure/sandbox:0.4.0 failed (exit status: 1)

Hosts on the default image use their own version's tag,
ghcr.io/hyperstructure/sandbox:<version>. hyperstructure/sandbox
publishes that tag within the hour after a release: its Image workflow
checks fctl.sh for new releases every hour (ADR 0012).
v0.4.0 came out at 12:44 UTC. GitHub didn't start the workflow's scheduled
runs, so ghcr.io had only 0.3.0, 0.3 and latest until
someone started the workflow by hand at 13:31. Sessions already handle that gap: they start
from latest and say so in their log. The pull command didn't. Neither did
the host's background pull at startup, which only logged a warning, so the
first session had to wait for latest to download.

What changed

• Driver::pull_or_fallback (the container drivers) pulls an image.
  When that fails and the image isn't on the host either, it pulls the
  fallback instead, because that is what sandboxes start from then. A log
  callback says when it falls back.

• fctl host image pull uses it with the host's fallback: latest for
  the default image, none for an image set in host.toml:

  $ fctl host image pull
  Error response from daemon: manifest unknown
  ghcr.io/hyperstructure/sandbox:0.4.0 can't be pulled; pulling ghcr.io/hyperstructure/sandbox:latest instead
  latest: Pulling from hyperstructure/sandbox
  …
  ✓ Pulled ghcr.io/hyperstructure/sandbox:latest
    Sessions start from it until ghcr.io/hyperstructure/sandbox:0.4.0 can be pulled: a release's image is published within the hour after the release.
  

  With --json, image is the image it pulled, and instead_of names the
  host's image when it fell back (null otherwise):
  {"image": "ghcr.io/hyperstructure/sandbox:latest", "instead_of": "ghcr.io/hyperstructure/sandbox:0.4.0"}.

• The host's pull at startup falls back the same way, but never pulls
  over a latest that is already on the host: the host pulls a tag once.

• fctl host doctor says when sessions start from latest because the
  version's image isn't on the host yet.

• If the version's image is already on the host (built with
  fctl host image build, for example), sessions use it, so a failed pull
  doesn't fall back. It fails as it did before.

• docs/sandbox-images.md says that the host and fctl host image pull
  pull latest until the version's image is published, and that the first
  session after that pulls it.

The latest pulled during the gap stays on the host after sessions move to
the version's image, like other old images (#47).

Why not always pull latest

Hosts pull their own version's tag on purpose (ADR 0012). The image
carries that release's fctl, and a change to the image's layout ships in an
fctl release first, so a host that is a version behind keeps an image it
knows how to use. latest stays what it already was for sessions: the
stand-in until a release's image exists.

Verification

• A unit test, pulling_the_default_image_falls_back_to_the_latest_published_one,
  uses the stand-in CLI from the driver tests. It covers five cases: the
  version's image is published and is pulled; it isn't published, so
  latest is pulled instead and the log says so; a build of it is on the
  host, so nothing falls back; neither image can be pulled, and the error
  names both and suggests fctl host image build; an image from
  host.toml has no fallback.
• End to end against ghcr.io at 13:00 UTC, while :0.4.0 wasn't published
  yet, using a stand-in docker whose pull asks ghcr.io for the tag's
  manifest:
  • fctl host image pull printed the output above and exited 0.
  • --json printed {"image": "ghcr.io/hyperstructure/sandbox:latest", "instead_of": "ghcr.io/hyperstructure/sandbox:0.4.0"}.
  • fctl host doctor then said "sandbox image
    ghcr.io/hyperstructure/sandbox:0.4.0: not here yet; sessions start from
    ghcr.io/hyperstructure/sandbox:latest until it is …".
  • fctl host run, on its first start, logged the failed pull of 0.4.0
    and then pulled latest. On the second start, with latest already
    there, it tried only 0.4.0.
• After :0.4.0 was published at 13:31 UTC, the same stand-in pulled it
  directly: ✓ Pulled ghcr.io/hyperstructure/sandbox:0.4.0, and --json
  printed {"image": "ghcr.io/hyperstructure/sandbox:0.4.0", "instead_of": null}.
• cargo xtask check --quick passes.

Refs #2

Session: https://app.fctl.sh/s/ses_01M57GZ9DXXAGA0CF8QW9DS5XZ