[Packaging] Resolve every package from one verified build#1073
Conversation
Phase B-1 matrix build status —
|
| Package | Arch | Status | Sha |
|---|---|---|---|
| kandelo-sdk | wasm32 | built | 58303f53 |
| mariadb-test | wasm32 | built | 4aa87217 |
| mariadb-vfs | wasm32 | built | 54750c03 |
| mariadb-vfs | wasm64 | built | e4d4ac50 |
| redis-vfs | wasm32 | built | 527cb563 |
| shell | wasm32 | built | 8f31a830 |
| lamp | wasm32 | built | c613db2f |
| nginx-php-vfs | wasm32 | built | 5e700750 |
| nginx-vfs | wasm32 | built | 1538a047 |
| node-vfs | wasm32 | built | 2cbc8fc1 |
| wordpress | wasm32 | built | bcd426e3 |
Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.
Exact staging follow-up (
|
A package can contain several executable outputs, runtime data files, or both. Publishing those paths one at a time can expose a mixture of builds, and a direct local copy can accidentally follow a fetched mirror symlink into the shared cache. Validate the complete declared closure before changing its live package directory, stage fetched mirrors beside the destination, and replace the directory as one transaction. Collect direct local builds into create-once session generations with one-shot publication claims, while retaining safe atomic replacement for one-member packages and legacy aliases. Treat outputs plus runtime files as one package identity so executable-plus-runtime packages such as CPython and Erlang use the same contract.
A complete package path could previously mix files from different builds or retarget after validation. Malformed and legacy spellings could bypass the package closure, while the browser development server exposed the entire program cache instead of only the files selected by the resolver. Project the registry into one closed, cached package model; resolve outputs and runtime files as one verified generation; return canonical member paths; and preserve truthful not-found versus invalid-state errors. Narrow Vite serving to exact resolver-approved regular files so Node.js and browser hosts consume the same package generation without exposing neighboring cache content.
Install the reviewed shell VFS with install-local-artifact under a run-unique generation session, then resolve and compare the canonical installed bytes before browser validation. Strengthen the shell workflow contract so direct local-binaries writes cannot return.
Declare nginx, nginx+PHP, and Redis composite images as complete package generations with exact dependencies and cache inputs. This prevents their browser mirrors from bypassing the strict package-generation contract. Add a source-level ownership audit for every composite VFS wrapper so a future image cannot call the package installer without a manifest, build sidecar, exact source artifact, canonical mirror path, and explicit non-Wasm policy.
Declare the shell-owned inputs used by nginx-derived images and narrow the nginx-only and Redis host inputs to the resolver/VFS boundaries they actually consume. Keep nginx+PHP broad because opcache prewarming boots the complete host runtime. Extend the cache-domain test with an inline explanation of that distinction so future maintainers cannot silently over- or under-invalidate these images.
Build the synthetic package projection through xtask so the boundary test exercises the same manifest, cache-key, and source-freshness contract as production. Keep generated imports outside source-scanned paths and assemble the authored fixture string so interrupted tests and static scanners cannot create false package dependencies.
Keep package generation checks fail-closed while avoiding one canonical source-projection subprocess per optional command. Delay the example runner's package probes until after CLI input validation, and cover the independent-batch and complete-closure boundaries.
Regenerate the authoritative package projection on the exact post-#1078 base so composite VFS consumers use cache identities derived from the current source closure.
2d8108b to
099828f
Compare
|
prepare-merge: test-gate passed against the synthetic PR merge and sealed |
Why
Some Kandelo packages contain several files that must come from the same build. Python, for example, needs both an executable and runtime data. Mixing those files across builds can produce subtle startup failures.
This PR makes one package build an indivisible generation. Consumers receive the complete, verified generation or a clear error; they never assemble a package from unrelated files.
What changes
program-packages.jsoncontract shared by Rust, Node.js, browser tooling, shell scripts, external registries, and the installed host package.Portable CI workspaces
Staging uncovered a real handoff bug: an archive step flattened trusted package symlinks into ordinary files and erased their generation identity.
The fix now:
The same transport path is used by staging, prepare-merge, and force-rebuild workflows.
Browser package safety
The browser discovers its full authored package graph once and verifies that graph as a batch. This avoids repeating an expensive source check for every import while preserving package-generation integrity.
@binaries/...imports use the same canonical resolver.The composite
nginx-vfs,nginx-php-vfs, andredis-vfsimages now also own proper package manifests instead of using anonymous package-shaped mirror paths.Validation
Current exact base:
a112375f50ecb967c8d9a23e9627f343235b78bdCurrent exact head:
7382ec93b61d0891e4543114393d062ab2544d7aLocal validation on the current tree:
asctime/1-1pass in 2.34 seconds with zero timeouts.git diff --checkpassed.Exact GitHub validation:
This PR will not merge until those exact-head workflows and prepare-merge are green.
Not activated here
This is packaging infrastructure for the remaining Homebrew migration. It does not switch the main shell image, add
brewinside the guest, retirepackages/registry, change the kernel or ABI, or implement bottle-declared composable VFS packages.