From 58cc183a61d681ff1cabc8f7e444529f09ee1f99 Mon Sep 17 00:00:00 2001 From: Matthew James Briggs Date: Sun, 23 Aug 2026 15:44:17 +0000 Subject: [PATCH 1/4] docs: fix release-process design flaws found in review Restructure the release design so the metadata commit B lives on a machine-owned release branch instead of merging to main, pin build provenance in a manifest committed inside B, and close the smaller gaps: version as a dispatch input, actions: read permissions, checksum re-verification at publish, and per-target artifact names. Section 13 logs each of the seven flaws and its fix. --- docs/ai/design/release-process.md | 282 +++++++++++++++++++++--------- 1 file changed, 199 insertions(+), 83 deletions(-) diff --git a/docs/ai/design/release-process.md b/docs/ai/design/release-process.md index 0a60d3c0e..bfefac821 100644 --- a/docs/ai/design/release-process.md +++ b/docs/ai/design/release-process.md @@ -2,8 +2,9 @@ Status: PROPOSED. This is the design record for `mx`'s GitHub Actions release system. It was produced from a deep-research pass on how C++/Swift libraries do continuous delivery on GitHub, -followed by an owner interview. The decisions below are settled; the workflows themselves are -follow-up work. +followed by an owner interview, then revised after a design review found mechanical flaws in the +first draft (§13 records each flaw and its fix). The decisions below are settled; the workflows +themselves are follow-up work. ## 1. Goal @@ -13,8 +14,9 @@ silent rebuild from a different tree. Swift (a binary XCFramework consumed via S first release target. Conan is a planned second target; the architecture must not be Swift-only. The design deliberately assembles this from GitHub primitives rather than copying a precedent: -the deep-research pass found no real-world project running this exact "build a SHA, then a tag -verifies that same SHA" two-phase pattern. Most projects do single-phase (tag-triggered build +the deep-research pass found no real-world project running this exact two-phase pattern (phase 1 +builds a chosen SHA and writes a tag-ready release commit; phase 2's tag push re-verifies that +commit from primary sources and publishes). Most projects do single-phase (tag-triggered build + publish, or a `workflow_dispatch` version-bump). The two-phase split exists here to make the "published bytes == validated bytes" invariant airtight. @@ -23,43 +25,66 @@ verifies that same SHA" two-phase pattern. Most projects do single-phase (tag-tr The entire release is four manual actions. Everything else is automation reacting to events. 1. **(Optional) Open and merge a PR** for any real work the release needs. → commit `A` on `main`. -2. **Run `release-build`** (a `workflow_dispatch`), passing SHA `A`. -3. **Review and merge the auto-generated `release-prep` PR** the build opened. → merge commit `B`. +2. **Run `release-build`** (a `workflow_dispatch`), passing SHA `A` and the version + `MAJOR.MINOR.PATCH`. +3. **Review release commit `B`** — the tip of branch `release-prep/v`, which the build + pushed. The run summary prints `B`'s SHA and links the `A...B` compare view; the diff is + release metadata only. 4. **Push a tag** (`vMAJOR.MINOR.PATCH`) pointing at `B`. -No other human step exists. In particular there is no "click publish on a draft," no manual -checksum edit, and no reviewer-approval gate beyond the tag push itself — pushing the tag *is* -the approval. +No other human step exists. In particular there is no PR to merge (§3 explains why `B` never +lands on `main`), no "click publish on a draft," no manual checksum edit, and no +reviewer-approval gate beyond the tag push itself — pushing the tag *is* the approval, and the +review in step 3 is what the tag pusher is vouching for. `release-build` can be triggered three interchangeable ways, since `workflow_dispatch` is one -API event: the GitHub Actions UI ("Run workflow" → `sha` field), the `gh` CLI -(`gh workflow run release-build.yml -f sha=<40-hex>`), or the GitHub MCP tool -(`actions_run_trigger`, `method: run_workflow`, `inputs: { sha }`). In all three, the dispatch -`ref` selects only *which copy of the workflow YAML runs* (normally `main`); the commit actually -built is the `sha` input, which the job checks out explicitly. +API event: the GitHub Actions UI ("Run workflow" → `sha` and `version` fields), the `gh` CLI +(`gh workflow run release-build.yml -f sha=<40-hex> -f version=1.4.0`), or the GitHub MCP tool +(`actions_run_trigger`, `method: run_workflow`, `inputs: { sha, version }`). In all three, the +dispatch `ref` selects only *which copy of the workflow YAML runs* (normally `main`); the commit +actually built is the `sha` input, which the job checks out explicitly. -## 3. The checksum constraint (why `B` exists) +## 3. The checksum constraint (why `B` exists — and why it is not on `main`) SwiftPM resolves a version by checking out the *tagged commit* and reading `Package.swift` as it exists there. For binary-release mode to work for a consumer depending on `from: "1.4.0"`, the `Package.swift` at the `v1.4.0` commit must already contain the correct -`binaryTarget(url:checksum:)`. The URL is predictable ahead of time (it is a fixed function of -the tag name and a fixed asset filename). The **checksum is not** — it can only be computed after -the XCFramework is built. +`binaryTarget(url:checksum:)`. The URL is predictable ahead of time — it is a fixed function of +the tag name and a fixed asset filename, which is exactly why the version must be supplied at +build time (§7). The **checksum is not** — it can only be computed after the XCFramework is +built. A commit cannot contain the hash of its own future build output; that is a property of content hashing, not a fixable limitation. Therefore *some* commit must exist between "the SHA the owner picked" (`A`) and "the SHA that gets tagged" (`B`), carrying the checksum that `A`'s build -produced. That commit is `B`, introduced by the `release-prep` PR. - -**All such release-metadata edits are bundled into the single `release-prep` PR.** If a future -target (Conan) has an analogous "the tagged commit must already carry a computed value" gotcha, -its fix goes into the same PR diff — not a second PR. The `release-prep` PR is the one and only -place release metadata is written back into the tree. - -Because `B` differs from `A` only in release-metadata files (never `src/`), `A`'s build output is -byte-identical to what `B` would produce. The system relies on this: it does **not** rebuild at -`B`; it re-associates `A`'s already-built artifacts with `B` (§5). +produced. That commit is `B`: a single commit whose parent is `A`, created and pushed by +`release-build` on the machine-owned branch `release-prep/v`. + +**`B` is never merged into `main`.** Two independent reasons force this: + +- *The tagged manifest and `main`'s manifest need opposite defaults.* A consumer's SwiftPM + evaluates `Package.swift` in the consumer's own environment; there is no way to make every + downstream consumer set an environment variable, so the tagged commit must resolve to the + `binaryTarget` **by default**. Meanwhile `main` must default to a source build for + sibling-checkout development and the non-Apple CI consumers. One file cannot default both ways + at once, so the binary-default manifest may only ever exist on release commits (§8). +- *Merging would break the artifact-reuse premise.* If anything landed on `main` between `A` and + the merge, the merge commit's tree would be "current `main` + metadata," not "`A` + metadata," + and reusing `A`'s build output for it would be unsound. A merge-based design silently requires + `main` to be frozen for the duration of every release; building `B` directly on top of `A` on + its own branch removes that requirement entirely — `main` can move freely at any time. + +Being off-`main` is invisible to consumers: SwiftPM resolves versions from tags alone, and a git +tag keeps `B` permanently reachable even if the release branch is later deleted. + +**All release-metadata edits from all targets are bundled into the single commit `B`.** If a +future target (Conan) has an analogous "the tagged commit must already carry a computed value" +gotcha, its files go into the same commit — not a second commit. `B` is the one and only place +release metadata is written into a tree. + +Because `B` differs from `A` only in declared release-metadata files (never `src/`), `A`'s build +output is byte-identical to what a build at `B` would produce. The system relies on this: it +does **not** rebuild at `B`; it publishes `A`'s already-built artifacts for `B` (§5). ## 4. Workflows @@ -67,11 +92,17 @@ All new, all separate from `ci.yaml`. Names are indicative. | File | Trigger | Responsibility | |---|---|---| -| `release-build.yml` | `workflow_dispatch`, input `sha` (required; validated against `^[0-9a-f]{40}$`) | Checks out `sha` explicitly. Calls one reusable target workflow per release target. Uploads each target's artifacts (`retention-days: 90`). Opens the `release-prep` PR (branch `release-prep/`, label `release-prep`) bundling every target's metadata patch. | -| `_release-target-swift.yml` | `workflow_call` | Builds three slices via CMake + an iOS toolchain file: macOS universal (`arm64`+`x86_64`), iOS device (`arm64`), iOS simulator (`arm64`+`x86_64`). Stitches them with `xcodebuild -create-xcframework`, zips, computes the SHA256 via `swift package compute-checksum`. Outputs: artifact name, checksum, the `Package.swift` patch, and the list of files it is allowed to touch. | -| `_release-target-conan.yml` | `workflow_call` | *Future.* Same input/output contract as the Swift target (SHA in; artifact + metadata patch + touched-file list out). Not built now; its existence in this table is the proof the abstraction is not Swift-only. | -| `release-prep-merged.yml` | `pull_request: closed`, filtered to `merged == true` **and** label `release-prep` | Parses `A` from the branch name. Hard-fails unless the `A..B` diff touches *only* the union of files each target declared. Downloads `release-build`'s artifact for `A` by run ID and **re-uploads it as this run's own artifact**, now keyed to `B`. This run's success is the validated record for `B`. | -| `release-publish.yml` | `push: tags: ["v*"]` | Resolves the tag's commit SHA. Queries the Actions API for the most recent **successful `release-prep-merged` run** with that `head_sha`. Hard-fails if none. Downloads that run's re-uploaded artifact and publishes a GitHub Release immediately (not draft), `generate_release_notes: true`, attaching the zip and its `.sha256`. | +| `release-build.yml` | `workflow_dispatch`, inputs `sha` (required; validated against `^[0-9a-f]{40}$`) and `version` (required; validated against `^\d+\.\d+\.\d+$`) | Checks out `sha` explicitly. Calls one reusable target workflow per release target. Uploads each target's artifacts (`retention-days: 90`). A final prep job collects every target's metadata files, writes `release-manifest.json` (§5), commits `B` on `release-prep/v` (force-push; the branch is machine-owned), and prints `B`'s SHA plus the `A...B` compare link in the run summary. | +| `_release-target-swift.yml` | `workflow_call` | Builds three slices via CMake + an iOS toolchain file: macOS universal (`arm64`+`x86_64`), iOS device (`arm64`), iOS simulator (`arm64`+`x86_64`). Stitches them with `xcodebuild -create-xcframework`, zips, computes the SHA256 via `swift package compute-checksum`. Outputs: artifact name, checksum, the complete release-mode `Package.swift` content (§8), and the list of paths it is allowed to occupy in `B`. | +| `_release-target-conan.yml` | `workflow_call` | *Future.* Same input/output contract as the Swift target (SHA + version in; artifact + metadata files + allowed-path list out). Not built now; its existence in this table is the proof the abstraction is not Swift-only. | +| `release-publish.yml` | `push: tags: ["v*"]` | Resolves the tag to `B`, reads `release-manifest.json` from `B`'s tree, and re-verifies every claim in it (§5). Hard-fails on any mismatch. Otherwise downloads the pinned run's artifacts and publishes a GitHub Release immediately (not draft), `generate_release_notes: true`, attaching each target's archive and its `.sha256`. | + +The first draft had a fourth workflow, `release-prep-merged.yml`, reacting to the merge of a +release-prep PR; with no PR and no merge, it no longer exists. + +One bootstrap consequence of the `push: tags` trigger: GitHub runs the workflow YAML as it +exists *at the tagged commit* — i.e. `B`, whose tree is `A` plus metadata. So only SHAs that +already contain `release-publish.yml` are releasable. Acceptable; no escape hatch needed. ## 5. The SHA-binding invariant @@ -80,36 +111,76 @@ enforce this: - **No rebuild.** `release-publish` never compiles anything. It fetches an already-built artifact by workflow-run ID and publishes it verbatim. This removes rebuild drift as a possible failure - mode entirely. (Build-provenance attestation — `actions/attest-build-provenance`, SLSA/Sigstore - — was evaluated and rejected for this role: the research found no confirmed mechanism by which - it proves phase 2's artifacts equal phase 1's. Artifact re-download by run ID is the reliable - mechanism.) -- **The lookup key is `head_sha`, and the run queried is `release-prep-merged`, not - `release-build`.** This is the subtle part: `release-build` ran against `A`, but the commit that - gets tagged is `B`. Only `release-prep-merged` ever ran against `B`. Its success — gated on the - `A..B` diff being release-metadata-only — *is* the authoritative "this SHA is validated" record. - No separate database, manifest, or attestation is needed; the Actions run history is the source - of truth, and it cannot be forged by the workflow itself. - -If a SHA has multiple successful `release-prep-merged` runs (e.g. a re-run), the **most recent** -one is authoritative. + mode entirely. (Build-provenance attestation — `actions/attest-build-provenance`, + SLSA/Sigstore — was evaluated and rejected for this role: the research found no confirmed + mechanism by which it proves phase 2's artifacts equal phase 1's. Artifact download by run ID + is the reliable mechanism.) +- **`B` carries its own provenance, and publish re-derives every claim from primary sources.** + `release-manifest.json`, committed in `B` by the prep job, records: the version, the base SHA + `A`, the `release-build` **run ID** (`github.run_id` is available to a run while it executes, + so the run that builds the artifacts can pin itself into `B`), each target's artifact name and + checksum, and the union of paths the targets declared they may touch. `release-publish` trusts + none of it blindly; at tag time it hard-fails unless **all** of the following hold: + 1. the tag name equals `v` + the manifest's version (so the asset URL baked into + `Package.swift` can never desync from the tag actually pushed); + 2. `B` has exactly one parent and it is the manifest's `A`; + 3. `A` is an ancestor of `main` (releases come from mainline history only); + 4. the `A..B` diff touches only the declared paths — additionally clamped by a static ceiling + hardcoded in `release-publish.yml` itself (release metadata paths only; never `src/`, + never `.github/`); + 5. the pinned run exists **in this repository**, is a run of `release-build.yml`, and + completed successfully; + 6. its artifacts download, and a freshly recomputed SHA-256 of each archive equals both the + manifest's checksum and the checksum string inside `B`'s `Package.swift` (for the Swift + target). For a zip, `swift package compute-checksum` *is* the SHA-256 of the file, so plain + `sha256sum` on the publish runner reproduces it. + +Why not query the Actions run history by commit, as the first draft did? Two API facts make that +unsound: for a `pull_request`-triggered run, the Actions API reports `head_sha` as the *PR +branch head*, never the merge commit — so the draft's lookup ("the validating run whose +`head_sha` is the tagged SHA") could never match anything; and a `workflow_dispatch` run cannot +be located by its `sha` *input* at all (inputs are not queryable, and such a run's `head_sha` is +the dispatch ref's head, not the input). Pinning the run ID inside `B` solves both discovery +problems with no heuristics — including the draft's "most recent successful run wins" +tie-break, which is gone: re-dispatching the same version force-pushes a fresh `B`, and the tag +pins exactly one. + +Residual trust, stated honestly: someone with push access could hand-craft a `B` (including, +since tag-push workflows execute the YAML at the tagged commit, a `B` that tampers with +`release-publish.yml` itself — which check 4 forbids, but a tampered workflow would not enforce +check 4). Checks 1–6 make such a commit fail loudly against any honest workflow copy; what +closes the loop is that nothing publishes until an admin reviews the `A...B` diff (human step 3, +where a workflow edit is glaring) and pushes the protected tag. That is the same trust anchor +the first draft placed in "review and merge the PR," relocated, not weakened. ## 6. Artifacts and retention -`release-build` uploads, per target, as a single named artifact (`mx-release-`): the built -archive (the XCFramework zip), its `.sha256` file, and a small `manifest.json` recording the SHA -and the platforms/arches built. Retention is set explicitly to **`retention-days: 90`** rather -than inherited from the repo default, so the window is a documented contract. +`release-build` uploads, per target, a single named artifact **`mx-release--`** +(e.g. `mx-release-swift-`; the target name is in the key so a future Conan target cannot +collide with Swift's): the built archive (the XCFramework zip), its `.sha256` file, and a small +manifest fragment recording the SHA and the platforms/arches built. Retention is set explicitly +to **`retention-days: 90`** rather than inherited from the repo default, so the window is a +documented contract. -`release-prep-merged` re-uploads that artifact under `B`, starting a fresh 90-day clock keyed to -the taggable commit. 90 days from build to tag is more than sufficient for this project's cadence. +The first draft re-uploaded artifacts under a second workflow run to re-key them to `B` on a +fresh 90-day clock; with the run ID pinned in `B`'s manifest there is nothing to re-key, and a +single window remains: **the tag must be pushed within 90 days of the build.** That is more than +sufficient for this project's cadence; expiry recovery is §11. ## 7. Versioning and tags -The **git tag is the sole source of truth** for the version. Format `vMAJOR.MINOR.PATCH` (e.g. -`v1.4.0`). There is no `VERSION` file and no version field in `Package.swift` (SwiftPM does not -need one). `release-publish` derives the version by stripping the `v` from `github.ref_name`. A -future Conan target consumes that same string. +The version is chosen once, at `release-build` dispatch, as a required input. It has to be: the +release-asset URL is a function of the tag name, and it is baked into `B`'s `Package.swift`, +which must exist *before* the tag does. (The first draft called the tag "the sole source of +truth" while also requiring the URL to be written at build time, with no version input anywhere +— a contradiction; no step in its flow ever supplied the version to the thing writing the URL.) + +The tag remains the **approval**: nothing publishes until `v` is pushed at `B`, and +`release-publish` hard-fails unless the tag name matches the manifest's version exactly (§5, +check 1), so the tag and the baked URL cannot desync. Format `vMAJOR.MINOR.PATCH` (e.g. +`v1.4.0`). There is still no `VERSION` file and no version field in `Package.swift`; +`release-publish` derives the version string it passes to release notes (and a future Conan +target) from `github.ref_name`. Release tags are protected by a repository ruleset on pattern `v*`: **Restrict updates** and **Restrict deletions**, with repo admins on the bypass list. This is complementary to — not a @@ -138,10 +209,23 @@ is no vendor-neutral tool that writes that bundle format correctly. Shape: Platform/arch scope for v1: **macOS** (`arm64`+`x86_64`), **iOS device** (`arm64`), **iOS simulator** (`arm64`+`x86_64`). tvOS, watchOS, and visionOS are explicitly out of scope for v1. -`Package.swift` today has a stubbed `MX_BINARY_RELEASE` branch that `fatalError`s. This system -lights it up: in binary-release mode it becomes -`.binaryTarget(name: "Mx", url: , checksum: )`. The URL/checksum are -exactly what the `release-prep` PR writes into the tree at `B`. +Two `Package.swift` manifests exist in this design, and only one ever lives on `main`: + +- **`main`'s manifest** stays a source-only package: sibling-checkout development + (`.package(path:)`) and any non-Apple consumption keep working with zero setup. The env-gated + `MX_BINARY_RELEASE` arm that `fatalError`s today is superseded by this design and should be + deleted when the workflows land — `main` never switches to binary mode, so the arm is dead + code under the final scheme. +- **The release manifest**, generated by `_release-target-swift.yml` from a template checked in + at `A` (`.github/release/Package.swift.template`, with the URL and checksum interpolated) and + existing only in `B`: `.binaryTarget(name: "Mx", url: , checksum: + )` is the **default**, so a `from: "1.4.0"` consumer gets the binary with no + environment setup — the fix for the first draft's fatal assumption that consumers could be + asked to set an environment variable. The source target remains reachable from the release + manifest as the automatic arm on non-Apple hosts (where an XCFramework `binaryTarget` cannot + resolve; the manifest is Swift code evaluated on the consumer's machine, so `#if os(...)` + handles this) and behind an explicit `MX_SOURCE_BUILD=1` opt-out for anyone who wants to + compile a tagged release from source on a Mac. ## 9. The release-target contract (Conan honesty) @@ -149,42 +233,44 @@ Each release target is a reusable workflow (`on: workflow_call`) — reusable wo composite actions, because only reusable workflows can contain multiple jobs (a build matrix plus packaging). The shared contract that keeps the system multi-target: -- **Inputs:** the commit `sha` to build. -- **Outputs:** (a) a named artifact containing the publishable archive(s) + checksums + a - manifest; (b) a metadata patch to apply to the tree in the `release-prep` PR; (c) the list of - files that patch is allowed to touch (used by `release-prep-merged` to bound the `A..B` diff). +- **Inputs:** the commit `sha` to build and the `version` being released. +- **Outputs:** (a) a named artifact `mx-release--` containing the publishable + archive(s) + checksums + a manifest fragment; (b) the exact release-metadata file contents to + commit in `B`; (c) the list of paths those files may occupy — the target's contribution to the + allowed-path set that `release-publish` enforces on the `A..B` diff. -`release-build` fans out across targets and merges every target's metadata patch into the one -`release-prep` PR; `release-prep-merged` validates the diff against the union of all targets' -touched-file lists; `release-publish` publishes every target's artifact under the one tag. Adding -Conan is: write `_release-target-conan.yml` to that contract, register it in the fan-out. No -change to the publish or validation logic. Conan's actual recipe/upload mechanics are deliberately -left unspecified here. +`release-build` fans out across targets and its prep job merges every target's files into the +one commit `B`; `release-publish` validates the diff against the union of all targets' declared +paths (clamped by its static ceiling, §5) and publishes every target's artifact under the one +tag. Adding Conan is: write `_release-target-conan.yml` to that contract, register it in the +fan-out, and add its metadata paths to the publish ceiling. No other change to the publish or +validation logic. Conan's actual recipe/upload mechanics are deliberately left unspecified here. ## 10. Token permissions (least privilege, per job) | Job | Permissions | |---|---| | `_release-target-*` build jobs | `contents: read` | -| `release-build` PR-opening job | `contents: write`, `pull-requests: write` | -| `release-prep-merged` | `contents: read` (re-uploads an artifact; does not write to `main`) | -| `release-publish` | `contents: write` (the only job that can create a Release) | +| `release-build` prep job | `contents: write` (pushes the `release-prep/v*` branch; no PR is opened, so no `pull-requests` scope) | +| `release-publish` | `contents: write` (the only job that can create a Release), `actions: read` (reads the pinned run's metadata and downloads its artifacts — cross-run artifact access requires it; the first draft's table omitted `actions: read` entirely, so both of its artifact-reading jobs would have been denied by the API) | No workflow requests broad repo-wide write. The single job that can mint a public Release is isolated behind the tag-push trigger. ## 11. Failure modes -- **`release-publish` finds no validated record for the tagged SHA** (wrong SHA tagged, the - `release-prep` PR was closed-not-merged or lost its label, or the artifact expired past 90 - days): **hard-fail, loudly, no fallback and no auto-rebuild.** The message directs the owner to - verify the `release-prep` PR merged with its label intact, and to recover by deleting the tag, - fixing the cause, and re-pushing. Publishing is all-or-nothing; a partial publish is never - emitted. -- **`release-prep-merged` finds the `A..B` diff touches files outside the declared set:** - hard-fail. This is the guard that keeps `B`'s tree equal to `A`'s for build purposes; if it - trips, the artifact reuse would be unsound, so the release is blocked rather than published from - mismatched bytes. +- **Any `release-publish` verification failure** (§5: missing or malformed manifest — including + a tag pushed at `A` or any non-release commit; tag/version mismatch; parentage, ancestry, or + diff violation; pinned run missing, failed, or foreign; artifact expired past 90 days; + checksum mismatch): **hard-fail, loudly, no fallback and no auto-rebuild.** Publishing is + all-or-nothing; a partial publish is never emitted. The failure message names the check that + tripped and directs the owner to recover by deleting the tag, fixing the cause (re-dispatching + `release-build` when the artifact expired or the run was bad — this produces a fresh `B` to + tag), and re-pushing. +- **Commits landing on `main` between `A` and the tag:** harmless. `B` sits directly on `A` on + its own branch, so mainline activity cannot leak into the release tree. (In the first draft + this was a silent freeze requirement that would have hard-failed any release with a concurrent + merge.) - **No manual override / `force_rebuild` escape hatch exists**, by design — it would silently reintroduce the rebuild-drift risk the whole two-phase split exists to prevent. @@ -196,3 +282,33 @@ isolated behind the tag-push trigger. if `mx` later ships a signed executable. - Auto-generated release notes are GitHub's default categorization of merged PRs since the prior tag; a curated `CHANGELOG.md`-driven flow is not adopted for v1. + +## 13. Revision log + +**2026-08-23 — design review of the first draft found seven flaws; all are fixed above.** + +1. *The `head_sha` lookup could never work.* A `pull_request: closed`-triggered run reports the + PR branch head as its `head_sha`, never the merge commit, so `release-publish` querying for a + validating run keyed to the tagged SHA would find nothing, every time. → Run-history queries + are gone; `B` pins its builder's run ID in `release-manifest.json` and publish re-verifies + from primary sources (§5). +2. *The version was unknown when the asset URL had to be written.* The tag was "the sole source + of truth," chosen at step 4, yet the URL derived from it was needed at step 2. → `version` is + a required `release-build` input; publish enforces tag == manifest version (§7). +3. *Consumers could never reach binary mode.* The env-gated `MX_BINARY_RELEASE` arm is + unreachable for downstream SwiftPM consumers, who cannot be made to set environment + variables. → The release manifest, existing only at `B`, is binary-by-default; `main` stays + source-only (§3, §8). +4. *The merge-based flow silently required a frozen `main`.* Any commit landing between `A` and + the release-prep merge made the merged tree diverge from `A` + metadata. → `B` is a single + commit atop `A` on a release branch, never merged (§3). +5. *The permissions table omitted `actions: read`,* which cross-run artifact download requires; + both artifact-reading jobs would have 403'd. → Added (§10). +6. *Nothing specified how the `release-build` run ID was discovered* (dispatch inputs are not + queryable via the API). → The run pins its own `github.run_id` into `B`'s manifest (§5). +7. *The written checksum was never verified against the artifact,* so a corrupted or tampered + checksum would ship a release every consumer fails to resolve. → Publish recomputes the + SHA-256 and requires bytes == manifest == `Package.swift` (§5, check 6). + +Also: per-target artifact names now include the target (`mx-release--`) so a second +target cannot collide with Swift's artifact key (§6). From 585fe0029562f96c317de27e1d2ccd3a55800b0a Mon Sep 17 00:00:00 2001 From: Matthew James Briggs Date: Sun, 23 Aug 2026 15:52:29 +0000 Subject: [PATCH 2/4] docs: put release tags on main in the release design Owner requirement: tagged release commits must be part of main's history. The release PR returns with two machine-written commits -- B (binary manifest + provenance, the tag target) and R (restores the source manifest) -- so a merge-commit merge places B in main's ancestry while every main tip stays a source package. B's parent is still A, preserving the no-frozen-main and single-binary-commit properties from the previous revision. Publish now also enforces that B is an ancestor of main before releasing. --- docs/ai/design/release-process.md | 195 +++++++++++++++++++----------- 1 file changed, 126 insertions(+), 69 deletions(-) diff --git a/docs/ai/design/release-process.md b/docs/ai/design/release-process.md index bfefac821..0d96c3013 100644 --- a/docs/ai/design/release-process.md +++ b/docs/ai/design/release-process.md @@ -2,9 +2,10 @@ Status: PROPOSED. This is the design record for `mx`'s GitHub Actions release system. It was produced from a deep-research pass on how C++/Swift libraries do continuous delivery on GitHub, -followed by an owner interview, then revised after a design review found mechanical flaws in the -first draft (§13 records each flaw and its fix). The decisions below are settled; the workflows -themselves are follow-up work. +followed by an owner interview, then revised twice: once after a design review found mechanical +flaws in the first draft, and once for the owner requirement that release tags live on `main` +(§13 records each change and why). The decisions below are settled; the workflows themselves are +follow-up work. ## 1. Goal @@ -12,6 +13,8 @@ Ship binary releases of `mx` for multiple package ecosystems from a chosen commi provable guarantee that what gets published is exactly what was built and validated — never a silent rebuild from a different tree. Swift (a binary XCFramework consumed via SwiftPM) is the first release target. Conan is a planned second target; the architecture must not be Swift-only. +Release tags are part of `main`'s history (owner requirement): every published version is +reachable from `main`, so `git log`, `git describe`, and `git tag --merged main` all see it. The design deliberately assembles this from GitHub primitives rather than copying a precedent: the deep-research pass found no real-world project running this exact two-phase pattern (phase 1 @@ -27,15 +30,16 @@ The entire release is four manual actions. Everything else is automation reactin 1. **(Optional) Open and merge a PR** for any real work the release needs. → commit `A` on `main`. 2. **Run `release-build`** (a `workflow_dispatch`), passing SHA `A` and the version `MAJOR.MINOR.PATCH`. -3. **Review release commit `B`** — the tip of branch `release-prep/v`, which the build - pushed. The run summary prints `B`'s SHA and links the `A...B` compare view; the diff is - release metadata only. -4. **Push a tag** (`vMAJOR.MINOR.PATCH`) pointing at `B`. +3. **Review and merge the auto-opened release PR** (branch `release-prep/v`), using + **"Create a merge commit"** — not squash or rebase. §3 explains why the merge method matters; + §5 enforces it. +4. **Push a tag** (`vMAJOR.MINOR.PATCH`) pointing at commit `B` — the first of the PR's two + commits, whose SHA the run summary and the PR body both print alongside the exact + `git tag`/`git push` commands. -No other human step exists. In particular there is no PR to merge (§3 explains why `B` never -lands on `main`), no "click publish on a draft," no manual checksum edit, and no -reviewer-approval gate beyond the tag push itself — pushing the tag *is* the approval, and the -review in step 3 is what the tag pusher is vouching for. +No other human step exists. In particular there is no "click publish on a draft," no manual +checksum edit, and no reviewer-approval gate beyond the tag push itself — pushing the tag *is* +the approval, and the review in step 3 is what the tag pusher is vouching for. `release-build` can be triggered three interchangeable ways, since `workflow_dispatch` is one API event: the GitHub Actions UI ("Run workflow" → `sha` and `version` fields), the `gh` CLI @@ -44,7 +48,7 @@ API event: the GitHub Actions UI ("Run workflow" → `sha` and `version` fields) dispatch `ref` selects only *which copy of the workflow YAML runs* (normally `main`); the commit actually built is the `sha` input, which the job checks out explicitly. -## 3. The checksum constraint (why `B` exists — and why it is not on `main`) +## 3. The checksum constraint (why `B` exists, and where it sits) SwiftPM resolves a version by checking out the *tagged commit* and reading `Package.swift` as it exists there. For binary-release mode to work for a consumer depending on `from: "1.4.0"`, the @@ -57,29 +61,50 @@ built. A commit cannot contain the hash of its own future build output; that is a property of content hashing, not a fixable limitation. Therefore *some* commit must exist between "the SHA the owner picked" (`A`) and "the SHA that gets tagged" (`B`), carrying the checksum that `A`'s build -produced. That commit is `B`: a single commit whose parent is `A`, created and pushed by -`release-build` on the machine-owned branch `release-prep/v`. - -**`B` is never merged into `main`.** Two independent reasons force this: - -- *The tagged manifest and `main`'s manifest need opposite defaults.* A consumer's SwiftPM - evaluates `Package.swift` in the consumer's own environment; there is no way to make every - downstream consumer set an environment variable, so the tagged commit must resolve to the - `binaryTarget` **by default**. Meanwhile `main` must default to a source build for - sibling-checkout development and the non-Apple CI consumers. One file cannot default both ways - at once, so the binary-default manifest may only ever exist on release commits (§8). -- *Merging would break the artifact-reuse premise.* If anything landed on `main` between `A` and - the merge, the merge commit's tree would be "current `main` + metadata," not "`A` + metadata," - and reusing `A`'s build output for it would be unsound. A merge-based design silently requires - `main` to be frozen for the duration of every release; building `B` directly on top of `A` on - its own branch removes that requirement entirely — `main` can move freely at any time. - -Being off-`main` is invisible to consumers: SwiftPM resolves versions from tags alone, and a git -tag keeps `B` permanently reachable even if the release branch is later deleted. +produced. + +**The shape that puts `B` on `main` without putting the binary manifest on `main`:** the release +PR that `release-build` opens carries exactly two machine-written commits on branch +`release-prep/v`, based on `A`: + +- **`B`** (parent: `A`) — writes the binary-default release `Package.swift` (§8) and + `release-manifest.json` (§5). This is the commit the tag points at. +- **`R`** (parent: `B`) — restores `Package.swift` to `A`'s copy verbatim + (`git checkout A -- Package.swift`; purely mechanical, no target involvement), keeping + `release-manifest.json` as a visible release record on `main`. + +Merging the PR with a merge commit `M` makes `B` part of `main`'s history — the same way every +commit of every merged feature branch is on `main` — while `main`'s *tree* changes only by the +release-record file. This resolves the conflict that sank both earlier drafts: a consumer's +SwiftPM evaluates `Package.swift` in the consumer's own environment, and there is no way to make +every downstream consumer set an environment variable, so the tagged commit must resolve to the +`binaryTarget` **by default**; meanwhile every tip of `main` must default to a source build for +sibling-checkout development and the non-Apple CI consumers. One file cannot default both ways +at one commit — so the binary default exists at exactly one commit, `B`, and `R` closes it out +before the branch meets `main`. + +Two properties fall out of `B` having parent `A` (rather than being the merge or squash commit, +as in the first draft): + +- *No frozen `main`.* Whatever lands on `main` between `A` and the merge, `B`'s tree is still + exactly "`A` + declared release metadata," so reusing `A`'s build artifacts for `B` stays + sound. Concurrent merges can at most conflict on `release-manifest.json`, which nothing else + touches. (The first draft tagged the merge commit itself, which silently required `main` to be + frozen for the duration of every release.) +- *The PR's net diff is tiny.* GitHub's "Files changed" view (merge-base `A` to head `R`) shows + only `release-manifest.json`; the binary manifest appears in the commits tab at `B`, which the + run summary and PR body link directly for review of the URL/checksum lines. + +**The merge method is load-bearing.** Only a true merge commit preserves `B` in `main`'s +ancestry. A squash or rebase merge copies the net diff instead — harmless to `main`'s tree (the +net diff is the record file), but `B` would then be reachable only via the tag, violating the +tags-on-`main` requirement. `release-publish` hard-fails in that case (§5, check 3); recovery is +`git merge -s ours release-prep/v` pushed to `main` — a history-only merge whose tree +change is nil, since `main` already carries the net diff — then re-push the tag. **All release-metadata edits from all targets are bundled into the single commit `B`.** If a future target (Conan) has an analogous "the tagged commit must already carry a computed value" -gotcha, its files go into the same commit — not a second commit. `B` is the one and only place +gotcha, its files go into `B` too — not a second commit or PR. `B` is the one and only place release metadata is written into a tree. Because `B` differs from `A` only in declared release-metadata files (never `src/`), `A`'s build @@ -92,13 +117,14 @@ All new, all separate from `ci.yaml`. Names are indicative. | File | Trigger | Responsibility | |---|---|---| -| `release-build.yml` | `workflow_dispatch`, inputs `sha` (required; validated against `^[0-9a-f]{40}$`) and `version` (required; validated against `^\d+\.\d+\.\d+$`) | Checks out `sha` explicitly. Calls one reusable target workflow per release target. Uploads each target's artifacts (`retention-days: 90`). A final prep job collects every target's metadata files, writes `release-manifest.json` (§5), commits `B` on `release-prep/v` (force-push; the branch is machine-owned), and prints `B`'s SHA plus the `A...B` compare link in the run summary. | +| `release-build.yml` | `workflow_dispatch`, inputs `sha` (required; validated against `^[0-9a-f]{40}$`) and `version` (required; validated against `^\d+\.\d+\.\d+$`) | Checks out `sha` explicitly. Calls one reusable target workflow per release target. Uploads each target's artifacts (`retention-days: 90`). A final prep job collects every target's metadata files, writes `release-manifest.json` (§5), commits `B` then `R` on `release-prep/v` (force-push; the branch is machine-owned), opens the release PR, and prints `B`'s SHA, the tag commands, and the `A...B` compare link in the run summary and the PR body. | | `_release-target-swift.yml` | `workflow_call` | Builds three slices via CMake + an iOS toolchain file: macOS universal (`arm64`+`x86_64`), iOS device (`arm64`), iOS simulator (`arm64`+`x86_64`). Stitches them with `xcodebuild -create-xcframework`, zips, computes the SHA256 via `swift package compute-checksum`. Outputs: artifact name, checksum, the complete release-mode `Package.swift` content (§8), and the list of paths it is allowed to occupy in `B`. | | `_release-target-conan.yml` | `workflow_call` | *Future.* Same input/output contract as the Swift target (SHA + version in; artifact + metadata files + allowed-path list out). Not built now; its existence in this table is the proof the abstraction is not Swift-only. | | `release-publish.yml` | `push: tags: ["v*"]` | Resolves the tag to `B`, reads `release-manifest.json` from `B`'s tree, and re-verifies every claim in it (§5). Hard-fails on any mismatch. Otherwise downloads the pinned run's artifacts and publishes a GitHub Release immediately (not draft), `generate_release_notes: true`, attaching each target's archive and its `.sha256`. | -The first draft had a fourth workflow, `release-prep-merged.yml`, reacting to the merge of a -release-prep PR; with no PR and no merge, it no longer exists. +No workflow reacts to the release PR's merge; merging only places `B` into `main`'s history, and +all validation happens at tag time. (The first draft had a `release-prep-merged.yml` for this; +it is gone, along with its label machinery.) One bootstrap consequence of the `push: tags` trigger: GitHub runs the workflow YAML as it exists *at the tagged commit* — i.e. `B`, whose tree is `A` plus metadata. So only SHAs that @@ -123,8 +149,13 @@ enforce this: none of it blindly; at tag time it hard-fails unless **all** of the following hold: 1. the tag name equals `v` + the manifest's version (so the asset URL baked into `Package.swift` can never desync from the tag actually pushed); - 2. `B` has exactly one parent and it is the manifest's `A`; - 3. `A` is an ancestor of `main` (releases come from mainline history only); + 2. the tagged commit has exactly one parent and it is the manifest's `A` — which also catches + tagging the wrong commit: `R` fails it (its parent is `B`), as does the merge commit (two + parents), and `A` itself has no manifest at all; + 3. `A` is an ancestor of `main` (releases come from mainline history only) **and `B` is an + ancestor of `main`** — the tags-on-`main` guarantee, which requires the release PR to have + been merged, with a true merge commit, before the tag is pushed (§3 gives the recovery when + it was squashed); 4. the `A..B` diff touches only the declared paths — additionally clamped by a static ceiling hardcoded in `release-publish.yml` itself (release metadata paths only; never `src/`, never `.github/`); @@ -135,6 +166,9 @@ enforce this: target). For a zip, `swift package compute-checksum` *is* the SHA-256 of the file, so plain `sha256sum` on the publish runner reproduces it. +`R` and the merge commit are deliberately *not* validated at publish: they affect only `main`'s +tree, which the PR review in step 3 governs. + Why not query the Actions run history by commit, as the first draft did? Two API facts make that unsound: for a `pull_request`-triggered run, the Actions API reports `head_sha` as the *PR branch head*, never the merge commit — so the draft's lookup ("the validating run whose @@ -142,16 +176,16 @@ branch head*, never the merge commit — so the draft's lookup ("the validating be located by its `sha` *input* at all (inputs are not queryable, and such a run's `head_sha` is the dispatch ref's head, not the input). Pinning the run ID inside `B` solves both discovery problems with no heuristics — including the draft's "most recent successful run wins" -tie-break, which is gone: re-dispatching the same version force-pushes a fresh `B`, and the tag -pins exactly one. +tie-break, which is gone: re-dispatching the same version force-pushes a fresh branch, and the +tag pins exactly one `B`. Residual trust, stated honestly: someone with push access could hand-craft a `B` (including, since tag-push workflows execute the YAML at the tagged commit, a `B` that tampers with `release-publish.yml` itself — which check 4 forbids, but a tampered workflow would not enforce check 4). Checks 1–6 make such a commit fail loudly against any honest workflow copy; what -closes the loop is that nothing publishes until an admin reviews the `A...B` diff (human step 3, -where a workflow edit is glaring) and pushes the protected tag. That is the same trust anchor -the first draft placed in "review and merge the PR," relocated, not weakened. +closes the loop is that nothing publishes until an admin reviews the release PR (step 3, where a +workflow edit is glaring) and pushes the protected tag. That is the same trust anchor the first +draft placed in "review and merge the PR" — unchanged, since this design kept that step. ## 6. Artifacts and retention @@ -188,6 +222,9 @@ substitute for — the tag-push-is-approval decision: it prevents *post-hoc tamp force-push retargeting `v1.4.0` at a different commit after it already shipped), which would desync the public release from what the tag claims. It does not add a pre-publish review gate. +Once the PR is merged and the tag is pushed, the `release-prep/v` branch can be +deleted: `B` stays reachable from both `main` and the tag. + ## 8. Build mechanics (Swift target) `mx` builds with CMake/Make, not an Xcode project, and the design keeps it that way. Compilation @@ -209,16 +246,18 @@ is no vendor-neutral tool that writes that bundle format correctly. Shape: Platform/arch scope for v1: **macOS** (`arm64`+`x86_64`), **iOS device** (`arm64`), **iOS simulator** (`arm64`+`x86_64`). tvOS, watchOS, and visionOS are explicitly out of scope for v1. -Two `Package.swift` manifests exist in this design, and only one ever lives on `main`: +Two `Package.swift` manifests exist in this design, and the binary one lives at exactly one +commit per release: - **`main`'s manifest** stays a source-only package: sibling-checkout development - (`.package(path:)`) and any non-Apple consumption keep working with zero setup. The env-gated - `MX_BINARY_RELEASE` arm that `fatalError`s today is superseded by this design and should be - deleted when the workflows land — `main` never switches to binary mode, so the arm is dead - code under the final scheme. + (`.package(path:)`) and any non-Apple consumption keep working with zero setup, at every + commit of `main` (`R` guarantees this — the binary manifest never survives to a `main` tip). + The env-gated `MX_BINARY_RELEASE` arm that `fatalError`s today is superseded by this design + and should be deleted when the workflows land — `main` never switches to binary mode, so the + arm is dead code under the final scheme. - **The release manifest**, generated by `_release-target-swift.yml` from a template checked in at `A` (`.github/release/Package.swift.template`, with the URL and checksum interpolated) and - existing only in `B`: `.binaryTarget(name: "Mx", url: , checksum: + existing only at `B`: `.binaryTarget(name: "Mx", url: , checksum: )` is the **default**, so a `from: "1.4.0"` consumer gets the binary with no environment setup — the fix for the first draft's fatal assumption that consumers could be asked to set an environment variable. The source target remains reachable from the release @@ -240,18 +279,19 @@ packaging). The shared contract that keeps the system multi-target: allowed-path set that `release-publish` enforces on the `A..B` diff. `release-build` fans out across targets and its prep job merges every target's files into the -one commit `B`; `release-publish` validates the diff against the union of all targets' declared -paths (clamped by its static ceiling, §5) and publishes every target's artifact under the one -tag. Adding Conan is: write `_release-target-conan.yml` to that contract, register it in the -fan-out, and add its metadata paths to the publish ceiling. No other change to the publish or -validation logic. Conan's actual recipe/upload mechanics are deliberately left unspecified here. +one commit `B` (and mechanically reverts the tree-affecting ones in `R`); `release-publish` +validates the diff against the union of all targets' declared paths (clamped by its static +ceiling, §5) and publishes every target's artifact under the one tag. Adding Conan is: write +`_release-target-conan.yml` to that contract, register it in the fan-out, and add its metadata +paths to the publish ceiling. No other change to the publish or validation logic. Conan's actual +recipe/upload mechanics are deliberately left unspecified here. ## 10. Token permissions (least privilege, per job) | Job | Permissions | |---|---| | `_release-target-*` build jobs | `contents: read` | -| `release-build` prep job | `contents: write` (pushes the `release-prep/v*` branch; no PR is opened, so no `pull-requests` scope) | +| `release-build` prep job | `contents: write` (pushes the `release-prep/v*` branch), `pull-requests: write` (opens the release PR) | | `release-publish` | `contents: write` (the only job that can create a Release), `actions: read` (reads the pinned run's metadata and downloads its artifacts — cross-run artifact access requires it; the first draft's table omitted `actions: read` entirely, so both of its artifact-reading jobs would have been denied by the API) | No workflow requests broad repo-wide write. The single job that can mint a public Release is @@ -260,17 +300,20 @@ isolated behind the tag-push trigger. ## 11. Failure modes - **Any `release-publish` verification failure** (§5: missing or malformed manifest — including - a tag pushed at `A` or any non-release commit; tag/version mismatch; parentage, ancestry, or - diff violation; pinned run missing, failed, or foreign; artifact expired past 90 days; - checksum mismatch): **hard-fail, loudly, no fallback and no auto-rebuild.** Publishing is - all-or-nothing; a partial publish is never emitted. The failure message names the check that - tripped and directs the owner to recover by deleting the tag, fixing the cause (re-dispatching - `release-build` when the artifact expired or the run was bad — this produces a fresh `B` to - tag), and re-pushing. -- **Commits landing on `main` between `A` and the tag:** harmless. `B` sits directly on `A` on - its own branch, so mainline activity cannot leak into the release tree. (In the first draft - this was a silent freeze requirement that would have hard-failed any release with a concurrent - merge.) + a tag pushed at `A` or any non-release commit; tag/version mismatch; parentage violation — + including tagging `R` or the merge commit by mistake; `B` not in `main`'s ancestry — the PR + unmerged, or squash/rebase-merged; diff violation; pinned run missing, failed, or foreign; + artifact expired past 90 days; checksum mismatch): **hard-fail, loudly, no fallback and no + auto-rebuild.** Publishing is all-or-nothing; a partial publish is never emitted. The failure + message names the check that tripped and directs the owner to recover by deleting the tag, + fixing the cause (merging the PR when it wasn't; the `-s ours` history-only merge when it was + squashed, §3; re-dispatching `release-build` when the artifact expired or the run was bad — + producing a fresh PR and a fresh `B` to tag), and re-pushing. +- **Commits landing on `main` between `A` and the tag:** harmless. `B` sits directly on `A`, so + mainline activity cannot leak into the tagged tree; at worst the release PR needs a trivial + conflict resolution in `release-manifest.json`, which nothing else touches. (In the first + draft this was a silent freeze requirement that would have hard-failed any release with a + concurrent merge.) - **No manual override / `force_rebuild` escape hatch exists**, by design — it would silently reintroduce the rebuild-drift risk the whole two-phase split exists to prevent. @@ -285,7 +328,20 @@ isolated behind the tag-push trigger. ## 13. Revision log -**2026-08-23 — design review of the first draft found seven flaws; all are fixed above.** +**2026-08-23 (b) — owner requirement: tags live on `main`.** The first revision fixed the +"binary-default vs source-default" conflict and the frozen-`main` problem by tagging a commit on +a never-merged release branch. The owner requires release tags to be part of `main`'s history. +Resolution: the release PR returns, now carrying two machine-written commits — `B` (binary +manifest + provenance record; the tag target) and `R` (restores the source manifest) — so a +merge-commit merge places `B` in `main`'s ancestry while every `main` tip stays a source +package (§3). Both original fixes survive: the binary default still exists at exactly one +commit, and `B`'s parent is still `A`, so nothing about `main`'s movement affects the tagged +tree. New consequences, both enforced by §5 check 3: the release PR must be merged with a true +merge commit (squash/rebase would strand `B` outside `main`'s history; recovery documented in +§3), and the tag can only be pushed after the merge. `release-build` regains +`pull-requests: write`. + +**2026-08-23 (a) — design review of the first draft found seven flaws; all remain fixed above.** 1. *The `head_sha` lookup could never work.* A `pull_request: closed`-triggered run reports the PR branch head as its `head_sha`, never the merge commit, so `release-publish` querying for a @@ -298,10 +354,11 @@ isolated behind the tag-push trigger. 3. *Consumers could never reach binary mode.* The env-gated `MX_BINARY_RELEASE` arm is unreachable for downstream SwiftPM consumers, who cannot be made to set environment variables. → The release manifest, existing only at `B`, is binary-by-default; `main` stays - source-only (§3, §8). + source-only at every tip (§3, §8). 4. *The merge-based flow silently required a frozen `main`.* Any commit landing between `A` and - the release-prep merge made the merged tree diverge from `A` + metadata. → `B` is a single - commit atop `A` on a release branch, never merged (§3). + the release-prep merge made the merged tree diverge from `A` + metadata. → The tagged commit + is `B`, whose parent is always `A`; the merge commit is never tagged, so `main`'s movement + is irrelevant to the tagged tree (§3). 5. *The permissions table omitted `actions: read`,* which cross-run artifact download requires; both artifact-reading jobs would have 403'd. → Added (§10). 6. *Nothing specified how the `release-build` run ID was discovered* (dispatch inputs are not From 65f432a7fab133019d6192894da0f48c0f9531b9 Mon Sep 17 00:00:00 2001 From: Matthew James Briggs Date: Sun, 23 Aug 2026 16:15:21 +0000 Subject: [PATCH 3/4] feat: implement the binary release system (release-build + release-publish) The workflows specified by docs/ai/design/release-process.md: - _release-target-swift.yml: builds the Mx XCFramework from commit A (three CMake+clang slices merged with libtool, stitched by -create-xcframework), checksums it, and emits the release artifact plus the metadata (interpolated release Package.swift + contract fragment) that the prep job commits. - release-build.yml: workflow_dispatch (sha + version); fans out to the target, then writes release-manifest.json, commits B (binary manifest + provenance) and R (source-manifest restore) on release-prep/v, and opens the release PR with the tag instructions. - release-publish.yml: on v* tag push, re-verifies every manifest claim from primary sources (checks 1-6: tag/version match, B sits directly on A, A and B ancestors of main, diff within the declared paths under a hardcoded ceiling, the pinned run is a successful release-build in this repo, recomputed sha256 == manifest == Package.swift) and publishes the already-built artifacts. - .github/release/Package.swift.template: the binary-by-default release manifest with non-Apple source fallback and MX_SOURCE_BUILD opt-out. Package.swift drops the dead MX_BINARY_RELEASE fatalError arm (main never switches to binary mode under the final scheme), and the design doc is synced with the implementation (revision 13c records the drift). --- .github/release/Package.swift.template | 64 ++++++ .github/workflows/_release-target-swift.yml | 175 ++++++++++++++ .github/workflows/ci.yaml | 3 +- .github/workflows/release-build.yml | 238 ++++++++++++++++++++ .github/workflows/release-publish.yml | 190 ++++++++++++++++ Package.swift | 73 +++--- docs/ai/design/release-process.md | 56 ++++- 7 files changed, 748 insertions(+), 51 deletions(-) create mode 100644 .github/release/Package.swift.template create mode 100644 .github/workflows/_release-target-swift.yml create mode 100644 .github/workflows/release-build.yml create mode 100644 .github/workflows/release-publish.yml diff --git a/.github/release/Package.swift.template b/.github/release/Package.swift.template new file mode 100644 index 000000000..87b841b7e --- /dev/null +++ b/.github/release/Package.swift.template @@ -0,0 +1,64 @@ +// swift-tools-version: 5.9 + +// TEMPLATE for the machine-generated release manifest -- the Package.swift that +// exists only at a tagged release commit ("B" in docs/ai/design/release-process.md). +// The _release-target-swift workflow interpolates the three MX_RELEASE +// placeholders (version, url, checksum) and the release-build prep job commits +// the result as B; commit R then restores the source manifest, so every tip of +// main keeps the source-only Package.swift. +// +// KEEP IN SYNC: mxSourceTarget below must stay byte-for-byte identical to the +// target in the repository's Package.swift, so that a release built from source +// (non-Apple hosts, or MX_SOURCE_BUILD=1) is the same build a sibling checkout +// gets. When Package.swift's target changes, change this copy too. + +import PackageDescription + +// Release: __MX_RELEASE_VERSION__ +// +// SwiftPM evaluates this manifest on the consumer's machine, so the selection +// below runs there: +// - Apple hosts get the published prebuilt XCFramework by default -- no +// environment setup. Setting MX_SOURCE_BUILD=1 opts a Mac into compiling +// the tagged release from source instead. +// - Non-Apple hosts always compile from source; an XCFramework binaryTarget +// cannot resolve there. + +let mxSourceTarget: Target = .target( + name: "Mx", + path: "src", + exclude: [ + "private/cpul", + "private/mxtest", + "private/mx/examples", + ], + publicHeadersPath: "include", + cxxSettings: [ + .headerSearchPath("private"), + .headerSearchPath("private/pugixml"), + ] +) + +let mxTarget: Target +#if os(macOS) +if Context.environment["MX_SOURCE_BUILD"] != nil { + mxTarget = mxSourceTarget +} else { + mxTarget = .binaryTarget( + name: "Mx", + url: "__MX_RELEASE_URL__", + checksum: "__MX_RELEASE_CHECKSUM__" + ) +} +#else +mxTarget = mxSourceTarget +#endif + +let package = Package( + name: "mx", + products: [ + .library(name: "Mx", targets: ["Mx"]), + ], + targets: [mxTarget], + cxxLanguageStandard: .cxx20 +) diff --git a/.github/workflows/_release-target-swift.yml b/.github/workflows/_release-target-swift.yml new file mode 100644 index 000000000..1e40b0a75 --- /dev/null +++ b/.github/workflows/_release-target-swift.yml @@ -0,0 +1,175 @@ +# The Swift release target (docs/ai/design/release-process.md §8-9): builds the +# Mx XCFramework from commit A and emits the two artifacts the release system +# consumes -- the publishable archive and the metadata the prep job commits as B. +# Called only from release-build.yml (workflow_call); never triggered directly. +name: _release-target-swift + +on: + workflow_call: + inputs: + sha: + description: Full 40-hex SHA of the commit to build (commit A). + required: true + type: string + version: + description: The MAJOR.MINOR.PATCH being released (no leading v). + required: true + type: string + +jobs: + xcframework: + runs-on: macos-latest + permissions: + contents: read + env: + ARCHIVE: Mx.xcframework.zip + steps: + - uses: actions/checkout@v5 + with: + ref: ${{ inputs.sha }} + + - name: Toolchain versions + run: | + sw_vers + xcodebuild -version + cmake --version | head -n 1 + swift --version + + # Three static-library slices via CMake's built-in Apple cross-compilation + # support (CMAKE_SYSTEM_NAME=iOS plus per-slice sysroot/archs); CMake drives + # clang directly -- no .xcodeproj, no third-party toolchain file. Each slice + # builds only the `mx` target and its dependencies (mx_core, pugixml), then + # libtool merges the three .a files so each XCFramework slice carries a + # single library. Deployment floors: macOS 11.0 (first arm64 macOS), + # iOS 13.0 -- raise deliberately, never by accident. + - name: Build slices (macOS universal, iOS device, iOS simulator) + run: | + set -euxo pipefail + build_slice() { + slice="$1"; shift + cmake -S . -B "build/release-$slice" -DCMAKE_BUILD_TYPE=Release "$@" + cmake --build "build/release-$slice" --target mx --parallel 4 + libtool -static -o "build/release-$slice/libMxAll.a" \ + "build/release-$slice/libmx.a" \ + "build/release-$slice/libmx_core.a" \ + "build/release-$slice/libpugixml.a" + } + build_slice macos \ + -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" \ + -DCMAKE_OSX_DEPLOYMENT_TARGET=11.0 + build_slice ios \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=iphoneos \ + -DCMAKE_OSX_ARCHITECTURES=arm64 \ + -DCMAKE_OSX_DEPLOYMENT_TARGET=13.0 \ + -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY + build_slice iossim \ + -DCMAKE_SYSTEM_NAME=iOS \ + -DCMAKE_OSX_SYSROOT=iphonesimulator \ + -DCMAKE_OSX_ARCHITECTURES="arm64;x86_64" \ + -DCMAKE_OSX_DEPLOYMENT_TARGET=13.0 \ + -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY + + # xcodebuild -create-xcframework is the one Apple-tool call: nothing else + # writes the bundle format correctly. Headers are the public mx::api + # surface (src/include), shipped per slice; no module map -- consumers are + # C++/ObjC++ and include mx/api/... via the header search path. + - name: Assemble Mx.xcframework + run: | + set -euxo pipefail + mkdir -p stage/headers out/dist + cp -R src/include/. stage/headers/ + xcodebuild -create-xcframework \ + -library build/release-macos/libMxAll.a -headers stage/headers \ + -library build/release-ios/libMxAll.a -headers stage/headers \ + -library build/release-iossim/libMxAll.a -headers stage/headers \ + -output out/Mx.xcframework + # ditto --keepParent puts the .xcframework bundle at the zip root, the + # layout SwiftPM's binaryTarget expects. + ditto -c -k --keepParent out/Mx.xcframework "out/dist/$ARCHIVE" + + - name: Checksum, release Package.swift, contract fragment + env: + VERSION: ${{ inputs.version }} + SHA: ${{ inputs.sha }} + run: | + set -euxo pipefail + # For a zip, swift package compute-checksum IS the file's SHA-256; + # assert that equivalence here so release-publish can re-verify with + # plain sha256sum (design §5, check 6). + CHECKSUM=$(swift package compute-checksum "out/dist/$ARCHIVE") + SHA256=$(shasum -a 256 "out/dist/$ARCHIVE" | cut -d' ' -f1) + if [ "$CHECKSUM" != "$SHA256" ]; then + echo "::error::swift package compute-checksum ($CHECKSUM) != sha256 ($SHA256)" + exit 1 + fi + printf '%s %s\n' "$CHECKSUM" "$ARCHIVE" > "out/dist/$ARCHIVE.sha256" + + # The release-mode Package.swift: the template at A, with the asset URL + # (a pure function of the tag name, which is why version is a build + # input) and the computed checksum interpolated. + URL="https://github.com/$GITHUB_REPOSITORY/releases/download/v$VERSION/$ARCHIVE" + TEMPLATE=.github/release/Package.swift.template + for placeholder in __MX_RELEASE_URL__ __MX_RELEASE_CHECKSUM__ __MX_RELEASE_VERSION__; do + if ! grep -q "$placeholder" "$TEMPLATE"; then + echo "::error::$TEMPLATE lost placeholder $placeholder" + exit 1 + fi + done + mkdir -p out/metadata/tree + sed -e "s|__MX_RELEASE_URL__|$URL|g" \ + -e "s|__MX_RELEASE_CHECKSUM__|$CHECKSUM|g" \ + -e "s|__MX_RELEASE_VERSION__|$VERSION|g" \ + "$TEMPLATE" > out/metadata/tree/Package.swift + if grep -q '__MX_RELEASE' out/metadata/tree/Package.swift; then + echo '::error::placeholders survived interpolation' + exit 1 + fi + + # Evaluate the generated manifest exactly as a consumer's SwiftPM will + # (dump-package loads the manifest; it fetches and builds nothing). + mkdir -p "$RUNNER_TEMP/manifest-check" + cp out/metadata/tree/Package.swift "$RUNNER_TEMP/manifest-check/" + (cd "$RUNNER_TEMP/manifest-check" && swift package dump-package > /dev/null) + + # The target's contract fragment (design §9): identity, artifact, + # checksum, platforms, and the paths this target may occupy in B. A + # copy ships in both artifacts. + jq -n \ + --arg version "$VERSION" \ + --arg base_sha "$SHA" \ + --arg artifact "mx-release-swift-$SHA" \ + --arg archive "$ARCHIVE" \ + --arg checksum "$CHECKSUM" \ + '{target: "swift", + version: $version, + base_sha: $base_sha, + artifact: $artifact, + archive: $archive, + checksum: $checksum, + platforms: ["macos-arm64", "macos-x86_64", "ios-arm64", + "ios-simulator-arm64", "ios-simulator-x86_64"], + allowed_paths: ["Package.swift"]}' \ + > out/metadata/fragment.json + cp out/metadata/fragment.json out/dist/fragment.json + + # The publishable artifact (design §6): archive + .sha256 + fragment. + # retention-days is the documented 90-day contract: the tag must be pushed + # while this artifact still exists. + - name: Upload release artifact + uses: actions/upload-artifact@v4 + with: + name: mx-release-swift-${{ inputs.sha }} + path: out/dist/ + retention-days: 90 + if-no-files-found: error + + # The prep-job channel (design §9, outputs b and c): the exact tree files + # for commit B plus the contract fragment. + - name: Upload metadata artifact + uses: actions/upload-artifact@v4 + with: + name: mx-release-swift-metadata-${{ inputs.sha }} + path: out/metadata/ + retention-days: 90 + if-no-files-found: error diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 368846a16..562e0ec8f 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -239,7 +239,8 @@ jobs: # Proves mx compiles as a Swift package under AppleClang/SPM on every PR -- # the path komp takes when it depends on a sibling mx via `.package(path:)`. - # Runs no test suites; the binary xcframework release arm is follow-up work. + # Runs no test suites; the binary xcframework ships from release-build.yml + # (docs/ai/design/release-process.md), not from CI. # # SwiftPM has no compiler-launcher hook, so incremental builds come from caching # its .build tree: llbuild tracks inputs by content signature, so a restored diff --git a/.github/workflows/release-build.yml b/.github/workflows/release-build.yml new file mode 100644 index 000000000..296c02778 --- /dev/null +++ b/.github/workflows/release-build.yml @@ -0,0 +1,238 @@ +# Phase 1 of the release system (docs/ai/design/release-process.md): build a +# chosen commit A for every release target, then open the release PR whose first +# commit, B, is the tag target. Nothing here publishes; phase 2 is +# release-publish.yml, triggered by the tag push after the PR merges. +name: release-build + +on: + workflow_dispatch: + inputs: + sha: + description: Full 40-hex SHA of the main commit to release (commit A). + required: true + type: string + version: + description: Version to release, MAJOR.MINOR.PATCH (no leading v). + required: true + type: string + +# One release-build per version at a time: a re-dispatch of the same version +# queues behind the in-flight run, then force-pushes a fresh release branch. +concurrency: + group: release-build-${{ github.event.inputs.version }} + +permissions: {} + +jobs: + validate: + name: Validate inputs + runs-on: ubuntu-latest + permissions: {} + steps: + - name: Validate sha and version + env: + SHA: ${{ inputs.sha }} + VERSION: ${{ inputs.version }} + run: | + set -euo pipefail + [[ "$SHA" =~ ^[0-9a-f]{40}$ ]] \ + || { echo "::error::sha must be the full 40-character lowercase hex SHA, got '$SHA'"; exit 1; } + [[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] \ + || { echo "::error::version must be MAJOR.MINOR.PATCH with no leading v, got '$VERSION'"; exit 1; } + + # The target fan-out (design §9): one reusable workflow per release target. + # Adding a target = add a job like this one, and extend the hardcoded path + # ceiling in release-publish.yml with its declared metadata paths. + swift: + name: Swift target + needs: validate + permissions: + contents: read + uses: ./.github/workflows/_release-target-swift.yml + with: + sha: ${{ inputs.sha }} + version: ${{ inputs.version }} + + # Collect every target's metadata, write release-manifest.json, commit B + # (release metadata on top of A) then R (restore the source Package.swift), + # force-push release-prep/v, and open the release PR. + prep: + name: Release PR + needs: [validate, swift] + runs-on: ubuntu-latest + permissions: + contents: write # push the release-prep/v* branch + pull-requests: write # open the release PR + env: + SHA: ${{ inputs.sha }} + VERSION: ${{ inputs.version }} + GH_TOKEN: ${{ github.token }} + steps: + - uses: actions/checkout@v5 + with: + ref: ${{ inputs.sha }} + fetch-depth: 0 + + - name: Download target metadata + uses: actions/download-artifact@v4 + with: + pattern: mx-release-*-metadata-${{ inputs.sha }} + path: ${{ runner.temp }}/release-metadata + + - name: Write release-manifest.json, commit B and R, push the branch + run: | + set -euo pipefail + shopt -s nullglob + + if ! git merge-base --is-ancestor "$SHA" origin/main; then + echo "::warning::$SHA is not an ancestor of main yet; release-publish will refuse the tag until it is (design §5, check 3)." + fi + + FRAGMENTS=("$RUNNER_TEMP"/release-metadata/*/fragment.json) + [ "${#FRAGMENTS[@]}" -gt 0 ] || { echo "::error::no target fragments were downloaded"; exit 1; } + + # Defense against artifact mixups: every fragment must be for this + # exact sha and version. + for f in "${FRAGMENTS[@]}"; do + jq -e --arg sha "$SHA" --arg version "$VERSION" \ + '.base_sha == $sha and .version == $version' "$f" > /dev/null \ + || { echo "::error::fragment $f is not for v$VERSION at $SHA"; exit 1; } + done + + # Lay each target's tree files into the checkout (for Swift: the + # binary-default release Package.swift). + for d in "$RUNNER_TEMP"/release-metadata/*/; do + if [ -d "$d/tree" ]; then cp -R "$d/tree/." .; fi + done + + # release-manifest.json (design §5): version, base SHA A, this run's + # ID (the run pins itself into B -- dispatch inputs are not queryable + # later), every target's fragment, and the union of paths B may touch. + jq -s \ + --arg version "$VERSION" \ + --arg base_sha "$SHA" \ + --argjson run_id "$GITHUB_RUN_ID" \ + '{version: $version, + base_sha: $base_sha, + run_id: $run_id, + targets: ., + allowed_paths: ((map(.allowed_paths[]) + ["release-manifest.json"]) | unique)}' \ + "${FRAGMENTS[@]}" > release-manifest.json + + # Commit B: exactly the declared release-metadata files, nothing else. + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + mapfile -t ALLOWED < <(jq -r '.allowed_paths[]' release-manifest.json) + git add -- "${ALLOWED[@]}" + mapfile -t DIRTY < <(git status --porcelain | sed 's/^...//') + for path in "${DIRTY[@]}"; do + ok=false + for allowed in "${ALLOWED[@]}"; do + if [ "$path" = "$allowed" ]; then ok=true; fi + done + if ! $ok; then + echo "::error::unexpected change outside the declared release paths: $path" + exit 1 + fi + done + if git diff --cached --quiet -- Package.swift; then + echo "::error::the release Package.swift is identical to A's -- interpolation failed" + exit 1 + fi + git commit -m "release: v$VERSION binary release manifest" + B=$(git rev-parse HEAD) + + # Commit R: restore A's Package.swift verbatim (purely mechanical) so + # the merge leaves every tip of main source-only, keeping + # release-manifest.json as the on-main release record. + git checkout "$SHA" -- Package.swift + git commit -m "release: v$VERSION restore the source manifest" + + BRANCH="release-prep/v$VERSION" + git push --force origin "HEAD:refs/heads/$BRANCH" + { + echo "B=$B" + echo "BRANCH=$BRANCH" + } >> "$GITHUB_ENV" + + - name: Open or refresh the release PR + run: | + set -euo pipefail + cat > "$RUNNER_TEMP/body.md" <<'BODY' + Machine-generated release PR for **v__VERSION__**, base commit A = `__A__`, + opened by [release-build run __RUN_ID__](__RUN_URL__). Process: + `docs/ai/design/release-process.md`. + + Two machine-written commits: + + 1. `__B__` -- **commit B, the tag target**: the binary-default release + `Package.swift` (asset URL + checksum of the built XCFramework) plus + `release-manifest.json`, the provenance record that release-publish + re-verifies at tag time. Review it here: __COMPARE__ + 2. commit R -- restores `Package.swift` to A's copy verbatim, so every tip + of `main` stays a source-only package. The PR's net diff is + `release-manifest.json` only. + + To ship v__VERSION__: + + 1. Review B via the compare link above (the URL and checksum lines). + 2. Merge this PR using **"Create a merge commit"** -- not squash, not + rebase. Only a true merge preserves B in main's ancestry; publish + refuses the tag otherwise. If it was squashed or rebased anyway, recover + with a history-only merge: + `git fetch origin && git checkout main && git pull && git merge -s ours origin/release-prep/v__VERSION__ && git push origin main`, + then push the tag. + 3. After the merge, push the tag at B -- the tag push is the approval and + the only trigger for publishing: + + ``` + git fetch origin + git tag v__VERSION__ __B__ + git push origin v__VERSION__ + ``` + + Do not tag before merging: publish requires B to be an ancestor of main. + release-publish then re-verifies everything in `release-manifest.json` + (checks 1-6) and publishes the GitHub Release with the already-built, + already-checksummed artifacts. No rebuild. + + Notes: + + - CI does not run on this PR: it was pushed with the workflow token, + whose events GitHub never lets trigger other workflows. CI validated + A on its own PR, and release-build built and checksummed A. + - The tag must be pushed within 90 days of the build (artifact + retention); after that, re-dispatch release-build for a fresh B. + - This branch can be deleted once the tag is pushed; B stays reachable + from main and from the tag. + BODY + COMPARE="https://github.com/$GITHUB_REPOSITORY/compare/$SHA...$B" + RUN_URL="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID" + sed -i \ + -e "s|__VERSION__|$VERSION|g" \ + -e "s|__A__|$SHA|g" \ + -e "s|__B__|$B|g" \ + -e "s|__RUN_ID__|$GITHUB_RUN_ID|g" \ + -e "s|__RUN_URL__|$RUN_URL|g" \ + -e "s|__COMPARE__|$COMPARE|g" \ + "$RUNNER_TEMP/body.md" + + TITLE="release: v$VERSION" + existing=$(gh pr list --repo "$GITHUB_REPOSITORY" --head "$BRANCH" --base main \ + --state open --json number --jq '.[0].number // empty') + if [ -n "$existing" ]; then + gh pr edit "$existing" --repo "$GITHUB_REPOSITORY" \ + --title "$TITLE" --body-file "$RUNNER_TEMP/body.md" + PR_URL=$(gh pr view "$existing" --repo "$GITHUB_REPOSITORY" --json url --jq .url) + else + PR_URL=$(gh pr create --repo "$GITHUB_REPOSITORY" --base main --head "$BRANCH" \ + --title "$TITLE" --body-file "$RUNNER_TEMP/body.md") + fi + + { + echo "## release: v$VERSION" + echo + echo "Release PR: $PR_URL" + echo + cat "$RUNNER_TEMP/body.md" + } >> "$GITHUB_STEP_SUMMARY" diff --git a/.github/workflows/release-publish.yml b/.github/workflows/release-publish.yml new file mode 100644 index 000000000..8dde1e340 --- /dev/null +++ b/.github/workflows/release-publish.yml @@ -0,0 +1,190 @@ +# Phase 2 of the release system (docs/ai/design/release-process.md §5): a v* tag +# push re-verifies commit B's provenance record against primary sources and +# publishes the already-built artifacts. Publishing never rebuilds anything: the +# bytes attached to the Release are the bytes release-build validated, fetched +# from that run by the run ID pinned inside B. Any check failing means no +# release, loudly (design §11) -- recovery is always: delete the tag, fix the +# cause, re-push. +# +# GitHub runs this file as it exists at the tagged commit, so only commits whose +# trees already carry release-publish.yml are releasable (design §4). +name: release-publish + +on: + push: + tags: ["v*"] + +concurrency: + group: release-publish-${{ github.ref_name }} + +permissions: {} + +jobs: + verify-and-publish: + runs-on: ubuntu-latest + permissions: + contents: write # create the GitHub Release + actions: read # read the pinned run's metadata + download its artifacts + env: + GH_TOKEN: ${{ github.token }} + TAG: ${{ github.ref_name }} + steps: + - uses: actions/checkout@v5 + with: + fetch-depth: 0 + + - name: Verify the release commit (checks 1-4) + run: | + set -euo pipefail + fail() { + echo "::error::release-publish: $*. Nothing was published. Recover by deleting the tag (git push origin :refs/tags/$TAG), fixing the cause, and re-pushing (docs/ai/design/release-process.md, section 11)." + exit 1 + } + B=$(git rev-parse "refs/tags/$TAG^{commit}") + + # The provenance record must exist in B's tree; a tag at A or any + # other non-release commit dies here, and R or the merge commit die + # at check 2. + git show "$B:release-manifest.json" > manifest.json 2>/dev/null \ + || fail "no release-manifest.json in the tagged commit's tree ($B) -- this tag does not point at a release commit B" + jq -e '(.version | type == "string") and (.base_sha | type == "string") + and (.run_id | type == "number") + and (.targets | type == "array" and length > 0) + and (.allowed_paths | type == "array" and length > 0)' \ + manifest.json > /dev/null \ + || fail "release-manifest.json at $B is malformed" + + VERSION=$(jq -r .version manifest.json) + A=$(jq -r .base_sha manifest.json) + RUN_ID=$(jq -r .run_id manifest.json) + + # Check 1: the tag matches the version whose asset URL was baked into + # B -- otherwise the published URL could never resolve. + [ "$TAG" = "v$VERSION" ] \ + || fail "check 1: tag $TAG does not equal v$VERSION from the manifest" + + # Check 2: B sits directly on A -- exactly one parent, and it is A. + # Catches tagging R (parent is B) or the merge commit (two parents). + PARENTS=$(git rev-list --parents -n 1 "$B" | cut -d' ' -f2-) + [ "$PARENTS" = "$A" ] \ + || fail "check 2: the tagged commit must have exactly one parent, the manifest's base $A; found '$PARENTS' (was R or the merge commit tagged by mistake?)" + + # Check 3: releases come from mainline history, and the tag must be on + # main -- i.e. the release PR merged, with a true merge commit. + git merge-base --is-ancestor "$A" origin/main \ + || fail "check 3: base $A is not an ancestor of main" + git merge-base --is-ancestor "$B" origin/main \ + || fail "check 3: $B is not an ancestor of main. Merge the release PR with 'Create a merge commit' before tagging. If it was squash- or rebase-merged, recover with: git merge -s ours origin/release-prep/v$VERSION && git push origin main -- then re-push the tag" + + # Check 4: the A..B diff touches only the manifest's declared paths, + # clamped by this static ceiling. The ceiling is hardcoded here on + # purpose: no target may ever declare src/ or .github/ (or anything + # else) into a release commit. + CEILING=("Package.swift" "release-manifest.json") + mapfile -t DECLARED < <(jq -r '.allowed_paths[]' manifest.json) + mapfile -t CHANGED < <(git diff --name-only "$A" "$B") + [ "${#CHANGED[@]}" -gt 0 ] \ + || fail "check 4: B is tree-identical to A -- not a release commit" + for path in "${CHANGED[@]}"; do + ok=false + for allowed in "${DECLARED[@]}"; do + if [ "$path" = "$allowed" ]; then ok=true; fi + done + $ok || fail "check 4: $path changed between A and B but is not in the manifest's allowed_paths" + ok=false + for allowed in "${CEILING[@]}"; do + if [ "$path" = "$allowed" ]; then ok=true; fi + done + $ok || fail "check 4: $path changed between A and B but is outside release-publish's hardcoded path ceiling" + done + + { + echo "B=$B" + echo "A=$A" + echo "VERSION=$VERSION" + echo "RUN_ID=$RUN_ID" + } >> "$GITHUB_ENV" + + - name: Verify the pinned build run (check 5) + run: | + set -euo pipefail + fail() { + echo "::error::release-publish: $*. Nothing was published. Recover by deleting the tag (git push origin :refs/tags/$TAG), fixing the cause, and re-pushing (docs/ai/design/release-process.md, section 11)." + exit 1 + } + # The API path scopes the lookup to this repository, so a run ID from + # some other repo simply does not resolve here. + RUN_JSON=$(gh api "repos/$GITHUB_REPOSITORY/actions/runs/$RUN_ID") \ + || fail "check 5: run $RUN_ID does not exist in $GITHUB_REPOSITORY" + RUN_PATH=$(jq -r .path <<< "$RUN_JSON") + RUN_CONCLUSION=$(jq -r .conclusion <<< "$RUN_JSON") + [ "$RUN_PATH" = ".github/workflows/release-build.yml" ] \ + || fail "check 5: run $RUN_ID is a run of '$RUN_PATH', not release-build.yml" + [ "$RUN_CONCLUSION" = "success" ] \ + || fail "check 5: run $RUN_ID concluded '$RUN_CONCLUSION', not success" + + - name: Verify the artifacts byte-for-byte (check 6) + run: | + set -euo pipefail + fail() { + echo "::error::release-publish: $*. Nothing was published. Recover by deleting the tag (git push origin :refs/tags/$TAG), fixing the cause, and re-pushing (docs/ai/design/release-process.md, section 11)." + exit 1 + } + : > assets.txt + while IFS= read -r target_json; do + target=$(jq -r .target <<< "$target_json") + artifact=$(jq -r .artifact <<< "$target_json") + archive=$(jq -r .archive <<< "$target_json") + checksum=$(jq -r .checksum <<< "$target_json") + + gh run download "$RUN_ID" --repo "$GITHUB_REPOSITORY" \ + --name "$artifact" --dir "dist/$target" \ + || fail "check 6: artifact $artifact of run $RUN_ID would not download -- expired (the 90-day retention window) or missing; re-dispatch release-build for a fresh build and tag its fresh B" + [ -f "dist/$target/$archive" ] \ + || fail "check 6: artifact $artifact does not contain $archive" + + # The recomputed bytes must equal the manifest's claim and the + # .sha256 file that ships next to the archive. + actual=$(sha256sum "dist/$target/$archive" | cut -d' ' -f1) + [ "$actual" = "$checksum" ] \ + || fail "check 6: recomputed sha256 of $archive is $actual but the manifest says $checksum" + filed=$(cut -d' ' -f1 "dist/$target/$archive.sha256") + [ "$filed" = "$checksum" ] \ + || fail "check 6: $archive.sha256 says $filed but the manifest says $checksum" + + # Swift only: the checksum and asset URL baked into B's + # Package.swift are what every consumer will resolve against; they + # must match the artifact and this tag exactly. + if [ "$target" = "swift" ]; then + git show "$B:Package.swift" > package-swift-at-B + baked=$(sed -n 's/.*checksum: "\([0-9a-f]\{64\}\)".*/\1/p' package-swift-at-B | head -n 1) + [ "$baked" = "$checksum" ] \ + || fail "check 6: the checksum in B's Package.swift ('$baked') does not match the artifact's sha256 ($checksum)" + url="https://github.com/$GITHUB_REPOSITORY/releases/download/$TAG/$archive" + grep -qF "\"$url\"" package-swift-at-B \ + || fail "check 6: B's Package.swift does not carry the asset URL this release will serve ($url)" + fi + + printf '%s\n' "dist/$target/$archive" "dist/$target/$archive.sha256" >> assets.txt + done < <(jq -c '.targets[]' manifest.json) + + - name: Publish the GitHub Release + run: | + set -euo pipefail + fail() { + echo "::error::release-publish: $*. Nothing was published. Recover by deleting the tag (git push origin :refs/tags/$TAG), fixing the cause, and re-pushing (docs/ai/design/release-process.md, section 11)." + exit 1 + } + if gh release view "$TAG" --repo "$GITHUB_REPOSITORY" > /dev/null 2>&1; then + fail "a release named $TAG already exists; delete it along with the tag before re-releasing" + fi + mapfile -t ASSETS < assets.txt + gh release create "$TAG" --repo "$GITHUB_REPOSITORY" \ + --verify-tag --title "$TAG" --generate-notes "${ASSETS[@]}" + { + echo "## $TAG published" + echo + echo "All six checks passed against release-manifest.json at \`$B\` (base \`$A\`, release-build run $RUN_ID). Attached:" + echo + printf -- '- %s\n' "${ASSETS[@]}" + } >> "$GITHUB_STEP_SUMMARY" diff --git a/Package.swift b/Package.swift index ff0263ca7..0703375f3 100644 --- a/Package.swift +++ b/Package.swift @@ -1,47 +1,42 @@ // swift-tools-version: 5.9 import PackageDescription -// mx is consumed as a Swift package in one of two modes: +// This is mx's SOURCE manifest: `Mx` compiles from this checkout, so sibling +// development (a consumer using `.package(path:)`) needs no environment setup, +// and non-Apple hosts can build. Every tip of main carries this manifest. // -// default -> `Mx` is a source library compiled from this checkout, -// for sibling-checkout development (a consumer using -// `.package(path:)`). This is the default, so consumers -// need no environment setup -- just a checkout. -// MX_BINARY_RELEASE -> `Mx` is the published xcframework binary release. That -// pipeline is follow-up work and is not wired up yet, so -// this arm fails loudly rather than silently resolving to -// a nonexistent artifact. -let useBinaryRelease = Context.environment["MX_BINARY_RELEASE"] != nil +// Released versions resolve differently: a consumer depending on +// `from: "X.Y.Z"` checks out the tagged release commit, which carries a +// machine-generated binary-default manifest (an XCFramework binaryTarget) +// interpolated from .github/release/Package.swift.template by the +// release-build workflow. That manifest exists only at tagged release commits, +// never at a tip of main. See docs/ai/design/release-process.md. +// +// KEEP IN SYNC: the target below must stay byte-for-byte identical to +// mxSourceTarget in .github/release/Package.swift.template, so a release built +// from source is the same build a sibling checkout gets. -let mxTarget: Target -if useBinaryRelease { - fatalError( - "The Mx binary release is not published yet. Build from a source " - + "checkout (the default); the binary-release arm is follow-up work." - ) -} else { - // SPM globs every C++ translation unit under `src` (all of which live in - // `src/private`), minus the Catch2 runner, the test suites, and the example - // programs -- the only files carrying their own main(). The public surface - // is the mx::api headers under `src/include`; `src/private` is added to the - // internal header search path so the model can include `mx/core/...`; and - // `src/private/pugixml` is added so sources can use the canonical - // `#include "pugixml.hpp"` form that the CMake build also exposes. - mxTarget = .target( - name: "Mx", - path: "src", - exclude: [ - "private/cpul", - "private/mxtest", - "private/mx/examples", - ], - publicHeadersPath: "include", - cxxSettings: [ - .headerSearchPath("private"), - .headerSearchPath("private/pugixml"), - ] - ) -} +// SPM globs every C++ translation unit under `src` (all of which live in +// `src/private`), minus the Catch2 runner, the test suites, and the example +// programs -- the only files carrying their own main(). The public surface +// is the mx::api headers under `src/include`; `src/private` is added to the +// internal header search path so the model can include `mx/core/...`; and +// `src/private/pugixml` is added so sources can use the canonical +// `#include "pugixml.hpp"` form that the CMake build also exposes. +let mxTarget: Target = .target( + name: "Mx", + path: "src", + exclude: [ + "private/cpul", + "private/mxtest", + "private/mx/examples", + ], + publicHeadersPath: "include", + cxxSettings: [ + .headerSearchPath("private"), + .headerSearchPath("private/pugixml"), + ] +) let package = Package( name: "mx", diff --git a/docs/ai/design/release-process.md b/docs/ai/design/release-process.md index 0d96c3013..5f23a8954 100644 --- a/docs/ai/design/release-process.md +++ b/docs/ai/design/release-process.md @@ -1,11 +1,11 @@ # Release process design -Status: PROPOSED. This is the design record for `mx`'s GitHub Actions release system. It was +Status: IMPLEMENTED. This is the design record for `mx`'s GitHub Actions release system. It was produced from a deep-research pass on how C++/Swift libraries do continuous delivery on GitHub, followed by an owner interview, then revised twice: once after a design review found mechanical flaws in the first draft, and once for the owner requirement that release tags live on `main` -(§13 records each change and why). The decisions below are settled; the workflows themselves are -follow-up work. +(§13 records each change and why). The workflows in §4 exist and this document is kept in sync +with them; §13's implementation entry records where the build drifted from the design text. ## 1. Goal @@ -126,6 +126,20 @@ No workflow reacts to the release PR's merge; merging only places `B` into `main all validation happens at tag time. (The first draft had a `release-prep-merged.yml` for this; it is gone, along with its label machinery.) +**Implementation notes.** Each target hands its results to the prep job through a companion +artifact, `mx-release--metadata-`: a `tree/` directory holding the exact files to +commit in `B` (for Swift, the interpolated release `Package.swift`) plus `fragment.json` (target +name, artifact name, archive filename, checksum, platforms, allowed paths) — the concrete form +of the §9 contract's outputs (b) and (c). The prep job merges every target's `tree/`, builds +`release-manifest.json` from the fragments, and refuses to commit if anything outside the +declared paths is dirty. The prep job pushes the branch and opens the PR with the workflow's own +`GITHUB_TOKEN`, and GitHub suppresses workflow triggers for events created with that token — so +**the release PR runs no CI**. That is acceptable by design: CI validated `A` on its own PR, +`release-build` built and checksummed `A`, and the release PR's content is machine-written +metadata. Two repository settings are prerequisites: **"Allow GitHub Actions to create and +approve pull requests"** (Settings → Actions → General) must be on, and **"Create a merge +commit"** must be an allowed merge method (§3). + One bootstrap consequence of the `push: tags` trigger: GitHub runs the workflow YAML as it exists *at the tagged commit* — i.e. `B`, whose tree is `A` plus metadata. So only SHAs that already contain `release-publish.yml` are releasable. Acceptable; no escape hatch needed. @@ -228,10 +242,16 @@ deleted: `B` stays reachable from both `main` and the tag. ## 8. Build mechanics (Swift target) `mx` builds with CMake/Make, not an Xcode project, and the design keeps it that way. Compilation -uses CMake with an iOS toolchain file (e.g. `leetal/ios-cmake`) invoked once per slice; CMake -drives `clang`/`clang++` **directly** (the same compiler the existing native `macos` CI job -uses) — no `.xcodeproj` is generated and no `xcodebuild` runs for *compilation*. The toolchain -file only points CMake at the correct Apple SDK sysroot and target triple per slice. +uses CMake's built-in Apple cross-compilation support, invoked once per slice — +`CMAKE_SYSTEM_NAME=iOS` plus the slice's `CMAKE_OSX_SYSROOT`/`CMAKE_OSX_ARCHITECTURES` (the +design anticipated a third-party toolchain file, e.g. `leetal/ios-cmake`; none turned out to be +needed). CMake drives `clang`/`clang++` **directly** (the same compiler the existing native +`macos` CI job uses) — no `.xcodeproj` is generated and no `xcodebuild` runs for *compilation*. +Each slice builds the `mx` CMake target and its dependencies (`mx_core`, `pugixml`) and merges +the three static libraries into one with `libtool -static`, so each XCFramework slice carries a +single library plus the public headers (`src/include`). No module map ships: the consumers are +C++/ObjC++ and include `mx/api/...` via the header search path SwiftPM derives from the bundle; +Swift-`import` support would be additive later. Deployment floors are macOS 11.0 and iOS 13.0. The single unavoidable Apple-tool call is the final `xcodebuild -create-xcframework`, used purely to assemble the three static-lib slices plus headers into a valid `.xcframework` bundle — there @@ -252,9 +272,9 @@ commit per release: - **`main`'s manifest** stays a source-only package: sibling-checkout development (`.package(path:)`) and any non-Apple consumption keep working with zero setup, at every commit of `main` (`R` guarantees this — the binary manifest never survives to a `main` tip). - The env-gated `MX_BINARY_RELEASE` arm that `fatalError`s today is superseded by this design - and should be deleted when the workflows land — `main` never switches to binary mode, so the - arm is dead code under the final scheme. + The env-gated `MX_BINARY_RELEASE` arm that used to `fatalError` was superseded by this design + and was deleted when the workflows landed — `main` never switches to binary mode, so the arm + was dead code under the final scheme. - **The release manifest**, generated by `_release-target-swift.yml` from a template checked in at `A` (`.github/release/Package.swift.template`, with the URL and checksum interpolated) and existing only at `B`: `.binaryTarget(name: "Mx", url: , checksum: @@ -264,7 +284,11 @@ commit per release: manifest as the automatic arm on non-Apple hosts (where an XCFramework `binaryTarget` cannot resolve; the manifest is Swift code evaluated on the consumer's machine, so `#if os(...)` handles this) and behind an explicit `MX_SOURCE_BUILD=1` opt-out for anyone who wants to - compile a tagged release from source on a Mac. + compile a tagged release from source on a Mac. The template's source arms carry a copy of the + source target, so the template must be kept in sync with `Package.swift` whenever the target + changes — both files carry a KEEP IN SYNC comment pointing at each other, the release PR + review is the human check, and the build evaluates the interpolated manifest with + `swift package dump-package` before committing it. ## 9. The release-target contract (Conan honesty) @@ -328,6 +352,16 @@ isolated behind the tag-push trigger. ## 13. Revision log +**2026-08-23 (c) — implemented.** The workflows in §4, the template in §8, and the +`Package.swift` cleanup shipped in one commit. Implementation drift from the design text, each +recorded in place above: CMake's built-in `CMAKE_SYSTEM_NAME=iOS` support replaced the +anticipated third-party toolchain file, and each slice's three static libraries are merged with +`libtool -static` (§8); the §9 contract's outputs (b) and (c) are concretely a per-target +`mx-release--metadata-` artifact consumed by the prep job (§4); the release PR runs +no CI because it is pushed with the workflow token (§4); and two repository settings became +explicit prerequisites — Actions may create PRs, and "Create a merge commit" is an allowed merge +method (§4). + **2026-08-23 (b) — owner requirement: tags live on `main`.** The first revision fixed the "binary-default vs source-default" conflict and the frozen-`main` problem by tagging a commit on a never-merged release branch. The owner requires release tags to be part of `main`'s history. From be3709468d8a4a7422b44db8fecd88d8bb0ae9fe Mon Sep 17 00:00:00 2001 From: Matthew James Briggs Date: Sun, 23 Aug 2026 18:40:17 +0000 Subject: [PATCH 4/4] docs: drop a banned phrase from the release design doc mx-comments flags "load-bearing" as an overused, invented phrase. Reword to plain language. --- docs/ai/design/release-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/ai/design/release-process.md b/docs/ai/design/release-process.md index 5f23a8954..a10c4ce7f 100644 --- a/docs/ai/design/release-process.md +++ b/docs/ai/design/release-process.md @@ -95,7 +95,7 @@ as in the first draft): only `release-manifest.json`; the binary manifest appears in the commits tab at `B`, which the run summary and PR body link directly for review of the URL/checksum lines. -**The merge method is load-bearing.** Only a true merge commit preserves `B` in `main`'s +**The merge method matters.** Only a true merge commit preserves `B` in `main`'s ancestry. A squash or rebase merge copies the net diff instead — harmless to `main`'s tree (the net diff is the record file), but `B` would then be reachable only via the tag, violating the tags-on-`main` requirement. `release-publish` hard-fails in that case (§5, check 3); recovery is