[Homebrew/Packaging] Make guest brew a reproducible product package#1074
Closed
brandonpayton wants to merge 25 commits into
Closed
[Homebrew/Packaging] Make guest brew a reproducible product package#1074brandonpayton wants to merge 25 commits into
brandonpayton wants to merge 25 commits into
Conversation
A package can contain several executable outputs, runtime data files, or both. Publishing those paths one at a time can expose a mixture of builds, and a direct local copy can accidentally follow a fetched mirror symlink into the shared cache. Validate the complete declared closure before changing its live package directory, stage fetched mirrors beside the destination, and replace the directory as one transaction. Collect direct local builds into create-once session generations with one-shot publication claims, while retaining safe atomic replacement for one-member packages and legacy aliases. Treat outputs plus runtime files as one package identity so executable-plus-runtime packages such as CPython and Erlang use the same contract.
A complete package path could previously mix files from different builds or retarget after validation. Malformed and legacy spellings could bypass the package closure, while the browser development server exposed the entire program cache instead of only the files selected by the resolver. Project the registry into one closed, cached package model; resolve outputs and runtime files as one verified generation; return canonical member paths; and preserve truthful not-found versus invalid-state errors. Narrow Vite serving to exact resolver-approved regular files so Node.js and browser hosts consume the same package generation without exposing neighboring cache content.
Install the reviewed shell VFS with install-local-artifact under a run-unique generation session, then resolve and compare the canonical installed bytes before browser validation. Strengthen the shell workflow contract so direct local-binaries writes cannot return.
brandonpayton
marked this pull request as draft
July 23, 2026 13:16
Phase B-1 matrix build status —
|
| Package | Arch | Status | Sha |
|---|---|---|---|
| homebrew-bootstrap | wasm32 | built | 5abead30 |
| kandelo-sdk | wasm32 | built | 2f07a46b |
| mariadb-test | wasm32 | built | 1ec82027 |
| mariadb-vfs | wasm32 | built | 6de85a49 |
| mariadb-vfs | wasm64 | built | b3dc3d08 |
| redis-vfs | wasm32 | built | bca44df6 |
| shell | wasm32 | built | feffb1d8 |
| lamp | wasm32 | built | b1b73fc8 |
| nginx-php-vfs | wasm32 | built | 7a0a085c |
| nginx-vfs | wasm32 | built | 952fec76 |
| node-vfs | wasm32 | built | e12fabca |
| wordpress | wasm32 | built | b02121f9 |
Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.
Require shell-derived images to match the reviewed capacity profile as well as its data and inode reserves. Keep a deliberate expected-capacity override for a future larger product profile. Make host-tree composition fail on every read, unsupported-entry, and VFS-write error; intentional omissions remain explicit excludes. Cover the complete copy-option surface, ENOSPC propagation, capacity drift, and larger-profile escape path.
Validate the serialized image capacity before compression or output writes, require intentional symlink handling, and propagate MariaDB test source failures. Add contract coverage for masked capacity, failed host reads, and shell profile constraints.
Bind the Homebrew main-shell serializer to its encoded capacity contract before output writes. Require every declared MariaDB test and fixture tree instead of preserving best-effort omissions, and remove the stale simple_select entry that the pinned source archive does not contain. Add executable failure-path, selection-parity, and package-input coverage.
Regenerate the atomic program projection after replaying the VFS integrity series. This binds MariaDB's new source-copy input and every VFS package that consumes the hardened shared image helpers to their current cache keys.
Allow the static Formula planner to recognize three canonical native-tool Requirement classes without evaluating Formula source. Preserve the existing sealed homebrew/core identity plan, and reject unknown, dynamic, forged, unloaded, or test-only Requirement declarations.
Bind allowlisted Requirement classes, Formula identities, sentinel executables, and tags into a closed schema-4 host dependency plan. Validate that plan in every producer and consumer, reconstruct only matched build-only Homebrew dependencies for Superenv, and expose sealed test tools through the normal Formula test lifecycle. Cover malformed plans and evaluated-object drift, plus an actual install/test against the exact pinned Homebrew source.
Provision an isolated Bundler copy before the publisher boundary, then seal it and run the real pinned Homebrew install and test commands behind an OS-enforced network sandbox. Probe that boundary with a reachable control socket and fail if the gem tree changes or Bundler activity appears.
Carry the dev-shell PATH explicitly through the passwordless-sudo network namespace used by GitHub runners. This keeps sealed native Requirement sentinels visible after sudo applies secure_path, while retaining the OS-enforced offline boundary.
Document the closed schema-4 native Requirement contract, the offline pinned publisher lifecycle, and the remaining canonical-package activation gate. Keep the trusted publisher Homebrew revision distinct from the dedicated guest homebrew-bootstrap package so staging URLs are not mistaken for durable consumer inputs.
Generate the real pinned Homebrew lifecycle plan through the same static Formula parser and closed-schema validator used by publication before staging it for Build and test. Bound every host dependency list to the platform's 128-entry limit in both validation layers, and cover oversized plans so protected control data cannot grow beyond the reviewed graph contract.
Make product WordPress and LAMP builds resolve only their SHA-pinned source archives instead of invoking the unpacked local-demo setup path. Centralize the reviewed WordPress core copy policy, materialize the SQLite plugin from its separate pinned source, and keep every unrelated source symlink fail-closed. Cover the package entrypoint boundary, local-demo setup ownership, exact exclusions, plugin guest placement, unexpected symlinks, and refreshed package identities.
brandonpayton
force-pushed
the
integration/homebrew-bootstrap-after-atomic-qk044
branch
from
July 23, 2026 14:30
714e74b to
2e52d12
Compare
brandonpayton
changed the title
brandonpayton
marked this pull request as ready for review
July 23, 2026 14:31
This was referenced Jul 23, 2026
Declare nginx, nginx+PHP, and Redis composite images as complete package generations with exact dependencies and cache inputs. This prevents their browser mirrors from bypassing the strict package-generation contract. Add a source-level ownership audit for every composite VFS wrapper so a future image cannot call the package installer without a manifest, build sidecar, exact source artifact, canonical mirror path, and explicit non-Wasm policy.
Declare the shell-owned inputs used by nginx-derived images and narrow the nginx-only and Redis host inputs to the resolver/VFS boundaries they actually consume. Keep nginx+PHP broad because opcache prewarming boots the complete host runtime. Extend the cache-domain test with an inline explanation of that distinction so future maintainers cannot silently over- or under-invalidate these images.
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Kandelo needs a reproducible copy of upstream Homebrew in order to run the normal
brewcommand inside a guest. That copy was previously produced only as a side effect of a larger image build, so it had no independent package identity or durable publication path.The publisher also has tools that are needed only on the native CI host. Homebrew calls these tools
Requirements. They must be available while a Formula is built without becoming dependencies that a Kandelo guest tries to install.Finally, a VFS image is a product artifact. Missing pinned source files, dropped symlinks, failed writes, or exhausted image capacity must stop publication instead of producing a partial image.
What changes
Package guest Homebrew
homebrew-bootstrapas a normal Kandelo program package.Model publisher-only tools correctly
Requirementclasses as native publisher inputs instead of guest runtime dependencies.Refuse incomplete VFS products
setup.shalias available only for local demos, exclude that exact alias from product source copying, and reject every unrelated symlink.Preserve the execution plan
Validation
Validated on exact #1073 head
665ac0b0fec7ddd3fffe5f951c5abed50d6432b3and current combined head2e52d12520699230dd479bbc7b73ede0eee76f92:git diff --checkpassed.Fresh combined CI is now the integration proof. After #1073 merges, this branch will be rebased onto its exact merge commit and revalidated before merge.
Stack and boundaries
This PR is stacked on #1073 and must land after it. It combines and supersedes the implementation in draft PR #1068 while preserving its commits and review history.
It changes packaging, publication, product-image build validation, tests, and documentation. It does not change the kernel, libc, Kandelo ABI, or generic VFS runtime behavior, and it does not yet switch the main shell image.