Skip to content

[VFS] Verify lazy package downloads before trusting their final URLs#1072

Open
brandonpayton wants to merge 2 commits into
mainfrom
vfs/verified-closed-package-sources-qk044
Open

[VFS] Verify lazy package downloads before trusting their final URLs#1072
brandonpayton wants to merge 2 commits into
mainfrom
vfs/verified-closed-package-sources-qk044

Conversation

@brandonpayton

@brandonpayton brandonpayton commented Jul 23, 2026

Copy link
Copy Markdown
Member

Why

A browser acceptance run may need to download a Homebrew package tree from a local or staging URL while the VFS image keeps its final immutable release URL. The temporary source is only a way to move bytes; it must never become trusted package identity.

The loader also needs one failure rule for the whole batch. If one download fails or the caller cancels, queued work must stop, active responses must close, and the original failure must remain visible.

What changes

  • Validate and snapshot the complete source list before making any request.
  • Require canonical source URLs and canonical final HTTPS URLs.
  • Fetch with credentials omitted and redirects rejected.
  • Bound the number of entries, total declared bytes, and concurrent downloads.
  • Stream exactly the declared decoded byte count and verify SHA-256 before binding bytes to the final URL.
  • Treat the decoded stream as authoritative instead of wire Content-Length, which can describe compressed bytes.
  • On the first failure or caller abort, stop dequeuing, cancel every active fetch/body, wait for cleanup, and return that exact first reason.
  • Redact source query strings from loader-generated errors.
  • Export the browser-safe VFS API for the revision-19 Homebrew shell activation.

This PR adds the verified transport primitive only. It does not change the kernel, ABI, package format, or current shell demo.

Evidence

Exact final head fc556b20553dfc566cd3168d9d5a04154dfe2edc, based on main 2b77bfd2107ebd8bce77eaed0cc8767246396830:

  • Focused closed-lazy-asset host suite: 54/54 passed
  • Native Chromium transport proof: 1/1 passed
    • decoded gzip size and digest
    • omitted cookies
    • redirect rejection with no target request
    • overflow cancellation and transport close
    • broken-stream rejection
    • caller abort with exact reason and transport close
  • Host typecheck/declaration build: passed
  • Host production build: passed; only existing CommonJS import.meta warnings
  • Rebase range-diff: both commits patch-identical
  • git diff --check: passed

The normal pull-request CI run is now in progress.

@brandonpayton
brandonpayton marked this pull request as ready for review July 23, 2026 05:15
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

Phase B-1 matrix build status — pr-1072-staging

ABI v41. 4 built, 0 failed, 4 total.

Package Arch Status Sha
shell wasm32 built d761658f
lamp wasm32 built e6fed502
node-vfs wasm32 built 7b7e43e9
wordpress wasm32 built 7dab6109

Auto-generated; replaced on each push. Raw data in the publish-status workflow artifact.

Pre-publication browser checks may read an artifact from a local or staging source while the VFS stores its final immutable HTTPS URL. Callers previously had to fetch and assemble that binding themselves, which made it too easy to associate incomplete or changed bytes with the trusted URL.

Add a bounded loader that validates the complete source list before I/O, omits credentials, rejects redirects, streams exactly the declared length, verifies SHA-256, and only then returns canonical closed-asset bindings. Export the primitive and cover invalid identities, truncation, overflow, digest changes, aggregate limits, concurrency, and result ordering.
Abort peer fetches and wait for their response bodies to close when one verified source fails or the caller cancels. Validate and snapshot the full manifest before I/O, keep decoded bytes as the size authority, redact source queries from loader errors, and reject noncanonical URL fragments.

Cover the contract with focused host tests and a real Chromium transport test for gzip decoding, omitted credentials, redirects, bounded streams, stream failure, and caller abort.
@brandonpayton
brandonpayton force-pushed the vfs/verified-closed-package-sources-qk044 branch from f503fe0 to fc556b2 Compare July 23, 2026 06:07
@brandonpayton brandonpayton changed the title [VFS] Verify lazy package bytes before trusting their final URL [VFS] Verify lazy package downloads before trusting their final URLs Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant