Skip to content

[Homebrew/Build] Stop incomplete VFS images before publication#1068

Draft
brandonpayton wants to merge 12 commits into
mainfrom
emdash/vfs-product-integrity-qk044
Draft

[Homebrew/Build] Stop incomplete VFS images before publication#1068
brandonpayton wants to merge 12 commits into
mainfrom
emdash/vfs-product-integrity-qk044

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 22, 2026

Copy link
Copy Markdown
Member

Why

A VFS image is a product artifact. If its builder cannot read a required source file, runs out of image capacity, drops a required symlink, or cannot copy a declared test fixture, the build must fail before any partial image can be published.

Several image paths checked these conditions only after writing an output, or treated missing inputs as optional. That could produce an incomplete artifact whose missing files were discovered only much later at runtime.

What changes

  • Make the shared host-tree copier propagate read, write, capacity, unsupported-entry, and required-symlink failures. Intentional omissions still use explicit exclusion rules.
  • Require shell-derived images to use the reviewed capacity profile, or an explicitly larger reviewed limit.
  • Pass the expected maximum image size into VFS serialization so an oversized image fails before the output is written; retain the post-write assertion as a second check.
  • Make the Homebrew VFS builder use that pre-write capacity contract.
  • Make MariaDB test packaging require every curated test plus nonempty include and std_data fixture trees instead of silently skipping missing inputs.
  • Correct the curated MariaDB 10.5.28 list to the 184 tests that actually exist in the pinned source archive.
  • Add failure-path coverage that proves read failures, serialization failures, missing sources, missing fixtures, symlink loss, and capacity drift cannot leave a publishable output.
  • Refresh the atomic package projection because the package build inputs changed.

Validation

Validated at exact head bfd033c46c45ca332c98175d062a7073efbb6e83:

  • 97/97 focused VFS, package, capacity, and runner tests passed.
  • The pinned MariaDB 10.5.28 source copied all 184 declared tests and both required fixture trees.
  • Host TypeScript declaration generation passed.
  • Program projection and full source-context checks passed.
  • The exact committed-projection xtask test passed.
  • Main-shell closure checks passed.
  • MariaDB browser-runner shell syntax and git diff --check passed.

A fresh staging package rebuild and browser gate are intentionally delegated to CI.

Stack and rollout

This draft is stacked on #1073, which introduces the atomic package projection used here. It must land after #1073.

Successful source trees still produce complete VFS images through the normal builder. This PR does not change the kernel, libc, guest syscall behavior, or Kandelo ABI.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

Phase B-1 matrix build status — pr-1068-staging

ABI v41. 6 built, 2 failed, 8 total.

Package Arch Status Sha
kandelo-sdk wasm32 built 2f07a46b
mariadb-test wasm32 built 1ec82027
mariadb-vfs wasm32 built 6de85a49
mariadb-vfs wasm64 built b3dc3d08
shell wasm32 built feffb1d8
lamp wasm32 failed (prev e922dc30)
node-vfs wasm32 built e12fabca
wordpress wasm32 failed (prev 6632c846)

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

@brandonpayton
brandonpayton force-pushed the homebrew/language-shell-revision-18-qk044 branch from 7409e99 to 6b7a9f9 Compare July 23, 2026 02:14
Base automatically changed from homebrew/language-shell-revision-18-qk044 to main July 23, 2026 03:51
A package can contain several executable outputs, runtime data files, or both. Publishing those paths one at a time can expose a mixture of builds, and a direct local copy can accidentally follow a fetched mirror symlink into the shared cache.

Validate the complete declared closure before changing its live package directory, stage fetched mirrors beside the destination, and replace the directory as one transaction. Collect direct local builds into create-once session generations with one-shot publication claims, while retaining safe atomic replacement for one-member packages and legacy aliases. Treat outputs plus runtime files as one package identity so executable-plus-runtime packages such as CPython and Erlang use the same contract.
A complete package path could previously mix files from different builds or retarget after validation. Malformed and legacy spellings could bypass the package closure, while the browser development server exposed the entire program cache instead of only the files selected by the resolver.

Project the registry into one closed, cached package model; resolve outputs and runtime files as one verified generation; return canonical member paths; and preserve truthful not-found versus invalid-state errors. Narrow Vite serving to exact resolver-approved regular files so Node.js and browser hosts consume the same package generation without exposing neighboring cache content.
Install the reviewed shell VFS with install-local-artifact under a run-unique generation session, then resolve and compare the canonical installed bytes before browser validation. Strengthen the shell workflow contract so direct local-binaries writes cannot return.
Require shell-derived images to match the reviewed capacity profile as well as its data and inode reserves. Keep a deliberate expected-capacity override for a future larger product profile.

Make host-tree composition fail on every read, unsupported-entry, and VFS-write error; intentional omissions remain explicit excludes. Cover the complete copy-option surface, ENOSPC propagation, capacity drift, and larger-profile escape path.
Validate the serialized image capacity before compression or output writes, require intentional symlink handling, and propagate MariaDB test source failures. Add contract coverage for masked capacity, failed host reads, and shell profile constraints.
Bind the Homebrew main-shell serializer to its encoded capacity contract before output writes. Require every declared MariaDB test and fixture tree instead of preserving best-effort omissions, and remove the stale simple_select entry that the pinned source archive does not contain. Add executable failure-path, selection-parity, and package-input coverage.
Regenerate the atomic program projection after replaying the VFS integrity series. This binds MariaDB's new source-copy input and every VFS package that consumes the hardened shared image helpers to their current cache keys.
@brandonpayton
brandonpayton force-pushed the emdash/vfs-product-integrity-qk044 branch from 5aadb71 to bfd033c Compare July 23, 2026 13:18
@brandonpayton brandonpayton changed the title [Homebrew/Build] Refuse incomplete product VFS images [Homebrew/Build] Stop incomplete VFS images before publication Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant