Skip to content

Speed up sparse delta base materialization#186

Merged
peters merged 1 commit into
mainfrom
issue-185-delta-materialization
Jun 28, 2026
Merged

Speed up sparse delta base materialization#186
peters merged 1 commit into
mainfrom
issue-185-delta-materialization

Conversation

@peters

@peters peters commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Prefer the active installed app as the base archive for sparse/file-ops delta materialization.
  • Keep byte-exact delta formats on the release-graph restore path.
  • Emit specific apply-phase status labels for base restore, target delta apply, rebuilt package write, and extraction.
  • Retry with the release-graph base if an installed-app base produces an invalid target archive, preserving the previous successful path before falling back to a full package.

Fixes #185.

Performance Notes

Benchmarked locally with a real large downstream app cache and release index, using a temp store/cache and excluding the warmed current full archive where noted.

  • Cold long-chain release-graph restore: exceeded 12 minutes and was terminated at 720 seconds.
  • Bounded release-graph restore from the latest cached full checkpoint plus two sparse deltas: 117,371 ms for a 344,062,822 byte rebuilt full archive.
  • Installed-app synthesis from the active app tree: 18,024 ms for a 317,602,882 byte archive.
  • The live installed archive did not have a compressed SHA match with the release index, so the implementation keeps the safe release-graph retry when target delta verification fails.

Validation

  • ./scripts/sync-surge-core-vendor.sh --check
  • ./scripts/check-version-sync.sh
  • cargo fmt --all -- --check
  • RUSTFLAGS="-D warnings" cargo test -p surge-core test_download_and_apply_delta -- --nocapture
  • RUSTFLAGS="-D warnings" cargo test --workspace
    • First sandboxed run failed only at commands::lock::tests::acquire_and_release_use_manifest_lock_server with Operation not permitted while binding a local listener.
    • Reran the same command with elevated permissions: passed.
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo clippy --workspace --lib --bins --examples -- -D warnings -D clippy::unwrap_used -D clippy::expect_used
  • cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic
  • dotnet format dotnet/Surge.slnx --verify-no-changes
  • dotnet test dotnet/Surge.slnx --configuration Release

@peters peters force-pushed the issue-185-delta-materialization branch from 643ba10 to c4ae970 Compare June 28, 2026 08:30
@peters peters merged commit d3e2ee9 into main Jun 28, 2026
12 checks passed
@peters peters deleted the issue-185-delta-materialization branch June 28, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update appears stuck at package apply started during long delta materialization

1 participant