Repository navigation
ci: run tests and release on merge to stable through the shared workflows - #120
singlerider wants to merge 15 commits into
Conversation
Move PyPI (maturin) and crates.io publishing out of python-tests.yml and rust-tests.yml into publish.yml, triggered only by v* tags. - version job fails unless the tag equals v + the Cargo.toml version - test workflows run via workflow_call; every build job gates every registry upload so a release is never partial - drop --skip-existing and registry already-published probes - pyproject reads its version from Cargo via dynamic = ["version"] - concurrency groups use a fixed per-file prefix since github.workflow is the caller's name under workflow_call - pin every action to a full SHA, replace -latest runners Part of blacklanternsecurity/bbot-enterprise#139 Part of blacklanternsecurity/bbot-enterprise#149 Part of blacklanternsecurity/bbot-enterprise#156
The publish workflow now refuses any tag that is not vMAJOR.MINOR.PATCH or vMAJOR.MINOR.PATCH-rc.N before comparing it to Cargo.toml, so the org tag convention is enforced rather than implied. PyPI upload drops PYPI_API_TOKEN and relies on trusted publishing via the job's id-token permission. Registry jobs run in the release environment so the publish gate applies. A final job creates the GitHub Release for the tag in the same run and attaches an SPDX JSON SBOM from anchore/sbom-action, so a tag and its release always exist together. CLA calls the shared reusable workflow instead of carrying a fork.
Governance declares one gated environment named pypi across published repos. crates and docker jobs stay in release.
rust-tests.yml, python-tests.yml and the inline wheel matrix, crates upload and release steps duplicated what every PyO3 repo carries, and the wheel builds hardcoded the interpreter list. tests.yml now calls the shared rust-tests.yml and python-tests.yml in blacklanternsecurity/CLA with scripts/build-openssl.sh as setup. publish.yml calls release-check.yml, maturin-wheels.yml (Linux families only, the container OpenSSL build now lives in scripts/wheel-container-setup.sh), crates.yml and publish.yml, keeping only the trusted-publishing PyPI upload at top level. pyproject drops managed = false and adds maturin to the dev group so the shared workflow can uv sync and maturin develop. release.toml is removed: cargo-release competed with the one release cut tool. README examples are ruff formatted, which ruff format --check enforces.
Completes the previous commit, which removed the old test workflows but missed the files replacing them. tests.yml calls the shared rust and python test workflows with scripts/build-openssl.sh as setup. publish.yml calls release-check, maturin-wheels (Linux families, container OpenSSL build in scripts/wheel-container-setup.sh), crates and publish, keeping only the trusted-publishing PyPI upload at top level. pyproject lets uv manage the project and adds maturin to the dev group. README examples are ruff formatted.
Releases are a pushed tag cut with the shared release.sh, which checks the tag against the manifest. State that here instead of copying the script.
en0f
left a comment
There was a problem hiding this comment.
-
PRs into stable will be stuck. The default ruleset on stable requires checks named rust-tests and python-tests (3.10) through (3.14). This PR deletes the workflows that produce those checks. The new shared workflows report under different names (rust / passed, python / passed, python / test (3.x), and so on). Once this reaches dev, the next dev → stable PR will wait forever on checks that never run. Before merging, change the required checks in that ruleset to rust / passed and python / passed. The dev ruleset has no required checks, so this PR itself can merge.
-
The pins point at a branch that hasn't been merged. publish.yml and tests.yml pin blacklanternsecurity/CLA@e532142…, a commit on feat/shared-workflows. That CLA PR is still open. If it's squash-merged and the branch is deleted, that commit may stop being fetchable, and tests and releases would fail to resolve the workflows. Merge the CLA PR first, then update the pins (and the README link) to the matching commit on CLA main.
Check before the first tag: the PyPI job now publishes with trusted publishing (environment: pypi) instead of PYPI_API_TOKEN. Unless PyPI already has a trusted publisher registered for blacklanternsecurity/blasthttp, workflow publish.yml, environment pypi, the first release will fail at the PyPI step. I can't see PyPI settings from here, so I couldn't confirm it.
liquidsec
left a comment
There was a problem hiding this comment.
Please do not remove the rust tests here. They have highly customized tests that will not apply to any other repository (for example, custom openssl build)
Restore rust-tests.yml from dev and drop the shared rust job from tests.yml. The Rust job builds a custom OpenSSL via scripts/build-openssl.sh, caches it, and runs fmt, clippy and tests in a way that is specific to this repo. The rust-tests job name is kept since it is a required status check. The rust-publish job stays removed: crates.io publishing now runs only on version tags via publish.yml.
rust-tests.yml is now callable and publish.yml runs it alongside tests.yml before building wheels, so a tag cannot release a crate or wheel whose Rust tests, clippy, or fmt fail. The concurrency group is named explicitly and only cancels on pull requests, matching tests.yml. Inherited github.workflow would resolve to the caller's name and collide with the publish run's own group.
CLA#3 was squash-merged into main. The pinned workflow files and scripts/release.sh are byte-identical at de82726743e51ac51a1be31196d893312a9fea59.
en0f
left a comment
There was a problem hiding this comment.
I found two blocking issues: one is already live, the other will hit as soon as Python 3.15.0 ships. Everything else checks out.
-
PRs into stable will hang (still open from your earlier review). The default ruleset on stable still requires python-tests (3.10) through (3.14). This PR deletes python-tests.yml, and the shared workflow reports those results under different names (python / test (3.x) and python / passed), so the old check names never appear. Every dev → stable PR would wait on them forever. rust-tests is fine because the repo's own workflow is kept. Fix: swap those five required checks for python / passed before this lands.
-
Python 3.15 will break PR CI and the wheel builds.
How: The shared python-versions.yml picks the Python versions from requires-python = ">=3.10" and whatever versions uv can download. Once uv ships 3.15.0 final, 3.15 joins both the test matrix and the wheel -i list.
Why it fails: The locked PyO3 (pyo3-ffi 0.27.2) caps CPython at 3.14. Both old workflows set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 to get past that cap, and the new calls don't pass it.
What breaks: python / test (3.15) fails, which fails python / passed on every PR, and the publish run fails at wheels.
Timing: It isn't failing yet, because the latest uv (0.12.23) still lists only 3.15.0rc3, which the shared script skips.
Fix: In tests.yml and in the wheels job of publish.yml, pass env: '{"PYO3_USE_ABI3_FORWARD_COMPATIBILITY": "1"}'. Bumping PyO3 to a release that supports 3.15 would also work.
The locked pyo3-ffi 0.27.2 caps CPython at 3.14. The removed workflows set PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1, and the shared python-tests and maturin-wheels callers did not, so 3.15 would fail the test matrix and the wheel builds once uv ships it. Pass the variable through the env input of both calls.
en0f
left a comment
There was a problem hiding this comment.
Approved, but @singlerider see comment on repo settings:
PRs into stable will hang. I checked the live default ruleset on stable. It still requires these checks:
rust-tests (still produced, because the repo keeps its own workflow)
python-tests (3.10) through python-tests (3.14)
This PR deletes python-tests.yml. The shared workflow reports under python / test (3.x) and python / passed instead. Once this reaches dev, the next dev → stable PR will wait forever on five checks that never run. The fix is in repo settings, not the code: swap those five required checks for python / passed before or right as this merges. The dev ruleset has no required checks, so this PR itself can merge.
The shared workflow pins pointed at de82726 (and the CLA caller at 4f85d3c where it still lagged). CLA main is now 40c6e18, which adds the optional registry login for image SBOMs. Every CLA reference, including the release.sh link, now resolves on CLA main at one commit. Refs blacklanternsecurity/bbot-enterprise#139
liquidsec
left a comment
There was a problem hiding this comment.
This will mean anyone with write access can publish. This needs to be discussed.
Follows .github-config#17 through CLA#11: publish.yml runs on push to the trunk instead of a v* tag, publishes in the release environment a bbot-leads member approves, re-runs safely (PyPI skip-existing, crates.io only when the version is new), and leaves the tag and the GitHub release to the bls-release App at the end of the run. The pypi environment and the long-lived registry token are gone.
|
@en0f @liquidsec reworked to the approved publishing policy (.github-config#17), through CLA#11. This answers "anyone with write access can publish" and the review and pipeline gates:
|
pjhartlieb
left a comment
There was a problem hiding this comment.
@singlerider, I reviewed this at 349e58d5 (based on current dev, a9c04545) against the .github-config#16 spec part B, the publishing policy, and CLA#11 at 4894c4ad. The inputs match CLA#11's interface.
As in blastdns#139, the PyPI job only downloads, attests and publishes, and crates.io goes through CLA's crates.yml with the OpenSSL setup script. The old publishers are removed: python-tests.yml (MATURIN_PYPI_TOKEN upload at line 270) is deleted, and rust-publish (cargo publish --token at line 80) is gone from rust-tests.yml. Deleting release.toml (cargo-release, which pushed v{{version}} tags) is right too. rust-tests still reports under its live name.
High
H1. The release App key can't come from the release environment (C1).
publish.yml:73 and :79 pass vars.RELEASE_APP_ID / secrets.RELEASE_APP_KEY from a uses: job, which can't hold environment:. Neither exists, and the spec's names are BLS_RELEASE_APP_ID / BLS_RELEASE_PRIVATE_KEY, held in release.
Requested: once CLA#11 reads them inside its environment job, drop release-app-id and the secrets: block.
H2. The Python required checks on stable will never report.
Ruleset 14658129 requires python-tests (3.11)–(3.15) and rust-tests. This head reports rust-tests but python / test (3.10)…(3.15) for Python.
Requested: change the ruleset to the new Python contexts before the promotion (coordinated with .github-config#1/#2), or keep the live names.
Medium
M1. rust-tests.yml runs in the release workflow unpinned and with default permissions.
rust-tests.yml:18, :21, :27 and :36 use actions/checkout@v7, setup-rust-toolchain@v2, actions/cache@v6 and setup-python@v7, and the file has no permissions: block. publish.yml:23–26 now calls it in the release run. The description's "grep … @[0-9a-f]{40} → (none)" doesn't hold at this head.
Requested: pin all four to full SHAs and add permissions: contents: read. Optionally, add persist-credentials: false on the checkout.
M2. crates.io publishes in parallel with PyPI. crates (publish.yml:37–43) needs only wheels.
Requested: crates: needs: [check, pypi].
M3. Version state (C3).
dev is 0.10.0 and stable is 0.10.1, both already on PyPI and crates.io with no v* release.
Requested: the promotion should carry 0.10.2 or later.
M4. Stale description and pin (C2, C4, C5).
The description says human-pushed tags and links release.sh, and its admin step registers PyPI against pypi.
Requested: change the environment to release, re-pin to the CLA#11 merge commit (tests.yml:18 and cla.yml are on 40c6e18), and hold the promotion until release is protected.
Low
- L1. Duplicate test runs.
tests.ymlandrust-tests.ymlstill trigger onpush: stable, with groupstests-${{ github.ref }}andrust-tests-${{ github.ref }}, the same groups as the calls frompublish.yml(C7). - L2.
cla.ymlinherits every secret underpull_request_target. PassAPP_ID/APP_PRIVATE_KEYby name (C6). - L3. Policy link. The README links
.github-config/blob/stable/docs/policy/publishing.md, which is in a private repo and isn't merged yet (.github-config#17). The link won't resolve for public readers.
Verdict
Request changes: H1 (depends on CLA#11), H2 before the promotion, and M1–M4.
Next: we're making these changes ourselves in a follow-up PR, which will be linked here when it opens. This review is the record of what that PR addresses.
en0f
left a comment
There was a problem hiding this comment.
The Publish workflow won’t start: the crates job doesn’t grant id-token: write
publish.yml sets permissions: contents: read for the whole workflow. The crates job calls the shared crates.yml without a permissions: block of its own, so it only gets contents: read.
The shared crates.yml (CLA at 4894c4a) needs id-token: write on its job, because crates.io trusted publishing uses an OIDC login (crates-io-auth-action).
A called workflow can’t get more permissions than its caller gives it. GitHub checks this before any job runs, so every push to stable gives the whole Publish run a startup failure. The error is “nested job ‘crates’ is requesting ‘id-token: write’, but is only allowed ‘id-token: none’”. Nothing publishes, and even the check job never runs.
The PR checks can’t catch this, because publish.yml only runs on a push to stable.
Fix:
yaml
crates:
needs: [check, wheels]
if: needs.check.outputs.release == 'true'
permissions:
contents: read
id-token: write
uses: blacklanternsecurity/CLA@4894c4a...
# Conflicts: # .github/workflows/python-tests.yml # pyproject.toml
- publish: publish crates.io after PyPI, granting the id-token the shared crates.yml needs, so the workflow no longer fails at startup; pass no App credentials - rust-tests: pin every action to a SHA, declare contents: read, and keep the token out of .git/config - tests, rust-tests: drop the stable push trigger and separate the concurrency groups from the publish calls; keep reporting stable's python-tests (3.11) to (3.15) contexts until the ruleset moves - cla: pass the CLA App credentials by name instead of secrets: inherit - stable-guard: refuse a pull request into stable whose version or dependencies cannot release - README: link the public org release standard - bump to 0.10.2: 0.10.1 is on PyPI and crates.io with no v0.10.1 tag - re-pin every CLA reference to CLA#11
en0f
left a comment
There was a problem hiding this comment.
The shared workflows are pinned to a CLA PR that hasn’t merged. Every uses: line (in cla.yml, dependabot-auto-merge.yml, publish.yml, tests.yml and stable-guard.yml) points at 07e14bc. That commit is on CLA#11, which is still Open. The PR description itself says to re-pin to CLA#11’s merge commit before this merges. So the order is: merge CLA#11, re-pin all five files, then merge this.
pjhartlieb
left a comment
There was a problem hiding this comment.
@singlerider, re-reviewed at 7968f0a2.
From the 2026-10-21 review
Status at 7968f0a2 |
|
|---|---|
| H1 App credentials | Fixed. No App lines in callers; publish.yml:68-78 |
H2 required checks on stable |
Fixed. tests.yml:21-33 reports python-tests (3.11)–(3.15); rust-tests.yml:15 keeps rust-tests |
M1 rust-tests.yml pins and scope |
Fixed. rust-tests.yml:7-8 contents: read, :18-38 SHA pins, :20 persist-credentials: false, PR and workflow_call only |
| M2 crates after PyPI | Fixed. publish.yml:38, with id-token: write at :40-42 |
| M3 version state | Fixed. Cargo.toml 0.10.2: no tag, not on PyPI or crates.io. It releases only when promoted to stable |
| M4 description, pin | Description fixed. Pin: see 1 below |
| L1 duplicate runs | Fixed. No test workflow triggers on push: stable |
L2 cla.yml inherit |
Fixed. cla.yml:16-18 |
| L3 policy link | Fixed. README.md:620 links the public .github standard, which resolves anonymously |
The wheel targets (publish.yml:33) cover all 10 platforms that 0.10.1 shipped. PYO3_USE_ABI3_FORWARD_COMPATIBILITY reaches both the tests and the wheels.
Still open
- Pin. Every CLA ref is CLA#11's unmerged head
07e14bcb. Re-pin to the CLA merge commit before this merges. dependabot-auto-merge.ymlis still ondev(R11). Delete it in this PR.publish.yml:7-8iscontents: read. Make itpermissions: {}and grantcontents: readoncheck,tests,rust-testsandwheels.- Release docs: a release prompts for
releaseapproval four times: release-check'scredentialsjob,pypi,cratesand the release.
Reproduce
TMPDIR=<scratch dir> bash verify-blasthttp-120.sh 7968f0a22ed4ec7a7c90132446c55ead84483d0d
# expect: SUMMARY repo=blasthttp pr=120 sha=7968f0a2... pass=24 fail=2 (P13, P14 = items 3, 2)
bash verify-blasthttp-120.sh 349e58d5af6900b67592f3d171676bc50e2ac1c7
# negative control, the head reviewed 2026-10-21: pass=12 fail=14 (P02 P04 P05 P06 P07 P08 P11 P12 P13 P14 P15.stable P16 H01 H04)
It needs git, an authenticated gh (GET calls only), uv and curl. It downloads actionlint 1.7.12 and checks it against the release checksums.
verify-blasthttp-120.sh
#!/usr/bin/env bash
# verify-blasthttp-120.sh <sha>
#
# Read-only re-check of blacklanternsecurity/blasthttp#120 (publish on merge to stable)
# against the 2026-10-21 review findings and go-live plan item A4 (+ R6, R11).
# Clean blobless clone of the repo at <sha>; GitHub reads are `gh api --method GET`
# only; PyPI and crates.io reads are anonymous GETs. Nothing is written anywhere
# except a temp dir under $TMPDIR (removed on exit).
#
# Prints PASS/FAIL/INFO per check and a final SUMMARY line; exits 1 on any FAIL.
# Needs: git, gh (authenticated), uv, curl, shasum or sha256sum.
# Env: CLA_SHA (expected CLA pin, default the CLA#11 head reviewed 2026-10-23).
set -uo pipefail
OWNER=blacklanternsecurity
REPO=blasthttp
PR=120
SHA=${1:?usage: $0 <commit-sha>}
CLA_SHA=${CLA_SHA:-07e14bcbba5e737e1b29bd9e3f9128fbf2483be9}
ACTIONLINT_VERSION=1.7.12
WORK=$(mktemp -d "${TMPDIR:-$PWD}/verify-$REPO-$PR.XXXXXX")
trap 'rm -rf "$WORK"' EXIT
echo "verify $OWNER/$REPO#$PR at $SHA (CLA_SHA=$CLA_SHA) $(date -u +%Y-%m-%dT%H:%M:%SZ)"
git clone --quiet --filter=blob:none --no-checkout "https://github.com/$OWNER/$REPO.git" "$WORK/repo" || { echo "FAIL clone"; echo "SUMMARY repo=$REPO sha=$SHA pass=0 fail=1"; exit 1; }
git -C "$WORK/repo" fetch --quiet origin "$SHA" 2>/dev/null \
|| git -C "$WORK/repo" fetch --quiet origin "+refs/pull/$PR/head:refs/remotes/origin/pr"
git -C "$WORK/repo" -c advice.detachedHead=false checkout --quiet "$SHA" || { echo "FAIL checkout $SHA"; echo "SUMMARY repo=$REPO sha=$SHA pass=0 fail=1"; exit 1; }
# actionlint, downloaded from the release and verified against the release's checksums file
case "$(uname -s)_$(uname -m)" in
Darwin_arm64) PLAT=darwin_arm64 ;; Darwin_x86_64) PLAT=darwin_amd64 ;;
Linux_x86_64) PLAT=linux_amd64 ;; Linux_aarch64|Linux_arm64) PLAT=linux_arm64 ;;
*) PLAT="" ;;
esac
ACTIONLINT=""
if [ -n "$PLAT" ]; then
base="https://github.com/rhysd/actionlint/releases/download/v$ACTIONLINT_VERSION"
tgz="actionlint_${ACTIONLINT_VERSION}_${PLAT}.tar.gz"
sums="actionlint_${ACTIONLINT_VERSION}_checksums.txt"
mkdir -p "$WORK/al"
if curl -sSfL -o "$WORK/al/$tgz" "$base/$tgz" && curl -sSfL -o "$WORK/al/$sums" "$base/$sums"; then
want=$(grep " $tgz\$" "$WORK/al/$sums" | cut -d' ' -f1)
got=$( (shasum -a 256 "$WORK/al/$tgz" 2>/dev/null || sha256sum "$WORK/al/$tgz") | cut -d' ' -f1)
if [ -n "$want" ] && [ "$want" = "$got" ]; then
tar -xzf "$WORK/al/$tgz" -C "$WORK/al" actionlint && ACTIONLINT="$WORK/al/actionlint"
echo "INFO actionlint $ACTIONLINT_VERSION $PLAT sha256 $got matches the release checksums file"
else
echo "INFO actionlint sha256 mismatch (want=$want got=$got); not used"
fi
fi
fi
cat > "$WORK/check.py" <<'PY'
import base64, json, os, re, subprocess, sys, tomllib, urllib.request, urllib.error
import yaml
owner, repo, pr, sha, cla_sha, root, actionlint = sys.argv[1:8]
wfdir = os.path.join(root, ".github", "workflows")
results = []
def out(status, cid, desc, detail=""):
results.append(status)
print(f"{status} {cid} {desc}" + (f" -- {detail}" if detail else ""))
def check(cid, desc, ok, detail=""):
out("PASS" if ok else "FAIL", cid, desc, detail)
def info(cid, desc, detail=""):
print(f"INFO {cid} {desc}" + (f" -- {detail}" if detail else ""))
def gh(path):
cmd = ["gh", "api", "--method", "GET", path]
p = subprocess.run(cmd, capture_output=True, text=True)
return p.returncode, p.stdout
def http_status(url):
req = urllib.request.Request(url, headers={"User-Agent": f"{owner}-{repo}-pr-review-readonly"})
try:
with urllib.request.urlopen(req, timeout=30) as r:
return r.status, r.read()
except urllib.error.HTTPError as e:
return e.code, b""
def load(path):
with open(path) as f:
return yaml.safe_load(f)
def triggers(wf):
on = wf.get("on", wf.get(True))
if isinstance(on, str):
return {on: None}
if isinstance(on, list):
return {k: None for k in on}
return on or {}
files = sorted(f for f in os.listdir(wfdir) if f.endswith((".yml", ".yaml"))) if os.path.isdir(wfdir) else []
wfs = {f: load(os.path.join(wfdir, f)) for f in files}
texts = {f: open(os.path.join(wfdir, f)).read() for f in files}
pub = wfs.get("publish.yml")
jobs = (pub or {}).get("jobs", {}) or {}
def needs(job):
n = (jobs.get(job) or {}).get("needs", [])
return [n] if isinstance(n, str) else list(n)
def job_using(fragment):
return [j for j, d in jobs.items() if fragment in str((d or {}).get("uses", ""))]
# P01 trigger
t = triggers(pub or {})
push = t.get("push") or {}
check("P01", "publish.yml triggers only on push to stable",
pub is not None and set(t) == {"push"} and push.get("branches") == ["stable"] and not push.get("tags"),
f"on={json.dumps(t, default=str)}")
# P02 job order: tests -> wheels -> pypi -> crates -> release
wheels = job_using("maturin-wheels.yml") or [j for j in jobs if j == "wheels"]
crates = job_using("CLA/.github/workflows/crates.yml") or [j for j in jobs if j == "crates"]
rel = job_using("CLA/.github/workflows/publish.yml")
pypi = [j for j, d in jobs.items() if "pypa/gh-action-pypi-publish" in json.dumps(d) or "\"command\": \"upload\"" in json.dumps(d)]
local_tests = [j for j, d in jobs.items() if str((d or {}).get("uses", "")).startswith("./")]
order = []
if wheels and pypi and crates and rel:
w, p, c, r = wheels[0], pypi[0], crates[0], rel[0]
order = [set(local_tests) <= set(needs(w)) and bool(local_tests), w in needs(p), p in needs(c), {c, p} <= set(needs(r))]
check("P02", "job order tests -> wheels -> pypi -> crates -> release",
bool(order) and all(order),
f"tests={local_tests} wheels.needs={needs(wheels[0]) if wheels else None} pypi.needs={needs(pypi[0]) if pypi else None} "
f"crates.needs={needs(crates[0]) if crates else None} release.needs={needs(rel[0]) if rel else None}")
# P03 pypi job
pj = jobs.get(pypi[0]) if pypi else {}
steps = json.dumps((pj or {}).get("steps", []))
check("P03", "pypi job: environment release, id-token: write, skip-existing",
bool(pj) and pj.get("environment") == "release" and (pj.get("permissions") or {}).get("id-token") == "write"
and re.search(r'"skip-existing": (true|"true")|--skip-existing', steps) is not None,
f"environment={pj.get('environment') if pj else None} permissions={pj.get('permissions') if pj else None}")
# P04 crates via CLA crates.yml with id-token
cj = jobs.get(crates[0]) if crates else {}
check("P04", "crates job calls CLA crates.yml and grants id-token: write",
bool(cj) and "CLA/.github/workflows/crates.yml" in str(cj.get("uses", "")) and (cj.get("permissions") or {}).get("id-token") == "write",
f"permissions={cj.get('permissions') if cj else None}")
alltext = "\n".join(f"{f}:{i+1}:{l}" for f, s in texts.items() for i, l in enumerate(s.splitlines()))
def hits(rx):
return [l for l in alltext.splitlines() if re.search(rx, l.split(":", 2)[2])]
# P05 no App credential lines in callers
h = hits(r"release-app-|RELEASE_APP_|BLS_RELEASE_")
check("P05", "no release App credential lines in callers (CLA reads them in `release`)", not h, "; ".join(h[:4]))
# P06 no secrets: inherit
h = hits(r"secrets:\s*inherit")
check("P06", "no `secrets: inherit` in any workflow", not h, "; ".join(h[:4]))
# P07 SHA pins
bad = []
for l in alltext.splitlines():
m = re.search(r"^\s*(?:-\s+)?uses:\s*([^\s#]+)", l.split(":", 2)[2])
if m and not m.group(1).startswith("./") and not re.search(r"@[0-9a-f]{40}$", m.group(1)):
bad.append(l.strip())
check("P07", "every `uses:` is local or pinned to a full SHA", not bad, "; ".join(bad[:4]))
# P08 / P09 CLA pins
cla_refs = sorted({m.group(1, 2) for m in re.finditer(r"blacklanternsecurity/CLA/(\.github/workflows/[\w.-]+)@([0-9a-f]{7,40})", alltext)})
pins = sorted({s for _, s in cla_refs})
check("P08", f"every CLA reference pinned to CLA_SHA {cla_sha[:8]}", pins == [cla_sha], f"pins={[p[:8] for p in pins]}")
missing = []
for path, s in cla_refs:
rc, _ = gh(f"repos/blacklanternsecurity/CLA/contents/{path}?ref={s}")
if rc != 0:
missing.append(f"{path}@{s[:8]}")
check("P09", "every referenced CLA workflow exists at its pin", bool(cla_refs) and not missing,
f"{len(cla_refs)} refs" + (f"; missing {missing}" if missing else ""))
# P10 old publish paths
h = hits(r"PYPI_TOKEN|PYPI_API_TOKEN|MATURIN_PYPI_TOKEN|secrets\.CARGO_REGISTRY_TOKEN|cargo publish .*--token|maturin publish|twine upload|gh release create|git push .*--tags|git tag ")
extra = [f for f in ("release.toml", "release.sh") if os.path.exists(os.path.join(root, f))]
check("P10", "no old publish path in any workflow (stored tokens, cargo publish --token, maturin publish, tag pushes)", not h and not extra, "; ".join(h[:4] + extra))
# P11 no other workflow runs on push to stable (duplicate test runs, C7)
dup = []
for f, wf in wfs.items():
if f == "publish.yml":
continue
tt = triggers(wf)
if "push" in tt:
b = (tt.get("push") or {}).get("branches")
if b is None or "stable" in b or any("*" in x for x in b):
dup.append(f)
check("P11", "no workflow besides publish.yml triggers on push to stable", not dup, f"{dup}")
# P12 explicit top-level permissions (R6)
noperm = [f for f, wf in wfs.items() if "permissions" not in wf]
check("P12", "every workflow declares top-level permissions (R6)", not noperm, f"{noperm}")
# P13 permissions: {} on publish.yml (A4)
check("P13", "publish.yml top-level `permissions: {}` (A4)", pub is not None and pub.get("permissions") == {},
f"permissions={pub.get('permissions') if pub else None}")
# P14 dependabot-auto-merge removed from dev (R11)
dam = [f for f in files if "dependabot-auto-merge" in f]
check("P14", "dependabot-auto-merge workflow removed (R11)", not dam, f"{dam}")
# P15 live required checks reported at this SHA
p = subprocess.run(["gh", "api", "--method", "GET", "--paginate", f"repos/{owner}/{repo}/commits/{sha}/check-runs?per_page=100",
"--jq", ".check_runs[] | {name, status, conclusion, app: {id: .app.id}}"], capture_output=True, text=True)
runs = [json.loads(l) for l in p.stdout.splitlines() if l.strip()]
reported = {(r["name"], (r.get("app") or {}).get("id")) for r in runs}
for branch in ("dev", "stable"):
rc, body = gh(f"repos/{owner}/{repo}/rules/branches/{branch}")
req = []
for rule in json.loads(body or "[]"):
if rule.get("type") == "required_status_checks":
p = rule["parameters"]
req += [(c["context"], c.get("integration_id"), rule.get("ruleset_id"), p.get("strict_required_status_checks_policy")) for c in p["required_status_checks"]]
miss = [c for c, i, _, _ in req if not any(n == c and (i is None or a == i) for n, a in reported)]
strict = sorted({s for *_, s in req})
check(f"P15.{branch}", f"every live `{branch}` required check is reported at this SHA",
not miss, f"required={len(req)} strict={strict} missing={miss}")
# P16 version not already published (C3)
cargo = tomllib.load(open(os.path.join(root, "Cargo.toml"), "rb"))
pyproj = tomllib.load(open(os.path.join(root, "pyproject.toml"), "rb"))
ver = cargo["package"]["version"]
crate = cargo["package"]["name"]
pyname = pyproj.get("project", {}).get("name")
pyver = pyproj.get("project", {}).get("version")
rc_tag, _ = gh(f"repos/{owner}/{repo}/git/ref/tags/v{ver}")
s_pypi, _ = http_status(f"https://pypi.org/pypi/{pyname}/{ver}/json")
s_crates, _ = http_status(f"https://crates.io/api/v1/crates/{crate}/{ver}")
check("P16", f"manifest version {ver} is unreleased (no v{ver} tag, not on PyPI or crates.io)",
rc_tag != 0 and s_pypi == 404 and s_crates == 404 and (pyver in (None, ver)),
f"tag_exists={rc_tag == 0} pypi={s_pypi} crates={s_crates} pyproject.version={pyver}")
# P17 actionlint
if actionlint:
p = subprocess.run([actionlint, "-shellcheck=", "-pyflakes=", "-no-color"], cwd=root, capture_output=True, text=True)
check("P17", "actionlint 1.7.12 clean (shellcheck/pyflakes integration off)", p.returncode == 0, (p.stdout + p.stderr).strip().replace("\n", " | ")[:600])
else:
out("FAIL", "P17", "actionlint 1.7.12 clean", "actionlint unavailable or checksum mismatch")
# P18 CI at this SHA
bad = sorted({r["name"] for r in runs if r.get("status") == "completed" and r.get("conclusion") not in ("success", "skipped", "neutral")})
pend = sorted({r["name"] for r in runs if r.get("status") != "completed"})
check("P18", "every check run at this SHA concluded success/skipped/neutral", bool(runs) and not bad and not pend,
f"runs={len(runs)} failing={bad} pending={pend}")
# W: wheel matrix through CLA maturin-wheels.yml, compared with the latest PyPI release
wj = jobs.get(wheels[0]) if wheels else None
via_cla = bool(wj) and "CLA/.github/workflows/maturin-wheels.yml" in str(wj.get("uses", ""))
check("W00", "wheels built by CLA maturin-wheels.yml", via_cla, f"wheels job={wheels[0] if wheels else None}")
s, raw = http_status(f"https://pypi.org/pypi/{pyname}/json")
latest = json.loads(raw) if s == 200 else {}
wheels_pypi = [u["filename"] for u in latest.get("urls", []) if u["filename"].endswith(".whl")]
arch = {"x86_64": "x86_64", "i686": "x86", "aarch64": "aarch64", "armv7l": "armv7", "s390x": "s390x", "ppc64le": "ppc64le",
"amd64": "x64", "win32": "x86", "arm64": "aarch64"}
def fam(plat):
a = next((v for k, v in arch.items() if plat.endswith(k)), plat)
for prefix, family in (("musllinux", "musllinux"), ("manylinux", "linux"), ("linux", "linux"), ("macosx", "macos"), ("win", "windows")):
if plat.startswith(prefix):
return family, a
return "other", plat
shipped = set()
interp = set()
for fn in wheels_pypi:
parts = fn[:-4].split("-")
for plat in parts[4].split("."):
f, a = fam(plat)
shipped.add((f, a))
interp.add(f"{parts[2]}-{parts[3]}")
if via_cla:
m = re.search(r"maturin-wheels\.yml@([0-9a-f]{40})", str(wj["uses"]))
rc, src = gh(f"repos/blacklanternsecurity/CLA/contents/.github/workflows/maturin-wheels.yml?ref={m.group(1)}") if m else (1, "")
clawf = yaml.safe_load(base64.b64decode(json.loads(src)["content"]).decode()) if rc == 0 else {}
default = clawf.get("on", clawf.get(True))["workflow_call"]["inputs"]["targets"]["default"]
targets = json.loads((wj.get("with") or {}).get("targets") or default)
built = {(f, a) for f, al in targets.items() for a in al}
gap = sorted(shipped - built)
check("W01", f"platforms built cover every platform of {pyname} {latest.get('info', {}).get('version')} on PyPI",
not gap, f"shipped={len(shipped)} built={len(built)} not_built={gap}")
rc2, src2 = gh(f"repos/blacklanternsecurity/CLA/contents/scripts/manifest.py?ref={m.group(1)}")
mp = base64.b64decode(json.loads(src2)["content"]).decode() if rc2 == 0 else ""
cpython_default_only = 'build["implementation"] != "cpython"' in mp and 'build["variant"] != "default"' in mp
dropped = sorted(i for i in interp if not re.fullmatch(r"cp3(\d+)-cp3\1", i)) if cpython_default_only else []
check("W02", "every interpreter ABI on the latest PyPI release is still built (CLA builds released CPython, default variant, only)",
not dropped, f"pypi_abis={sorted(interp)} not_built={dropped}")
else:
out("FAIL", "W01", "platform coverage", "depends on W00")
out("FAIL", "W02", "interpreter coverage", "depends on W00")
# repo-specific checks
extra = os.environ.get("EXTRA_CHECKS", "")
if "blasthttp" in extra:
rt = wfs.get("rust-tests.yml") or {}
rtt = texts.get("rust-tests.yml", "")
unp = [u for u in re.findall(r"(?m)^\s*(?:-\s+)?uses:\s*([^\s#]+)", rtt) if not re.search(r"@[0-9a-f]{40}$", u)]
tt = triggers(rt)
check("H01", "rust-tests.yml: SHA-pinned, contents: read, no push trigger, persist-credentials: false",
bool(rt) and not unp and rt.get("permissions") == {"contents": "read"} and "push" not in tt and "persist-credentials: false" in rtt,
f"unpinned={unp} permissions={rt.get('permissions')} triggers={list(tt)}")
ok = "rust-tests" in jobs and "rust-tests" in needs(wheels[0] if wheels else "")
check("H02", "publish.yml runs rust-tests.yml before the wheels", ok, f"wheels.needs={needs(wheels[0]) if wheels else None}")
env_tests = "PYO3_USE_ABI3_FORWARD_COMPATIBILITY" in texts.get("tests.yml", "")
env_wheels = "PYO3_USE_ABI3_FORWARD_COMPATIBILITY" in json.dumps(wj or {})
check("H03", "PYO3_USE_ABI3_FORWARD_COMPATIBILITY passed to the shared tests and wheels (en0f)", env_tests and env_wheels,
f"tests={env_tests} wheels={env_wheels}")
readme = open(os.path.join(root, "README.md")).read()
links = sorted(set(re.findall(r"https://github\.com/blacklanternsecurity/\.github[\w.-]*/blob/[^\s)]+", readme)))
st = {l: http_status(l)[0] for l in links}
check("H04", "README release-policy link resolves for an anonymous reader", bool(links) and all(v == 200 for v in st.values()), f"{st}")
n_pass, n_fail = results.count("PASS"), results.count("FAIL")
print(f"SUMMARY repo={repo} pr={pr} sha={sha} pass={n_pass} fail={n_fail}")
sys.exit(1 if n_fail else 0)
PY
echo "INFO env release environment exists: $(gh api --method GET "repos/$OWNER/$REPO/environments" --jq '[.environments[].name] | index("release") != null' 2>/dev/null || echo unknown)"
EXTRA_CHECKS="blasthttp" uv run --quiet --no-project --python 3.12 --with pyyaml==6.0.2 python -I "$WORK/check.py" \
"$OWNER" "$REPO" "$PR" "$SHA" "$CLA_SHA" "$WORK/repo" "$ACTIONLINT"
Summary
Moves blasthttp CI onto the org's shared workflows. A release is a reviewed merge into
stablethat bumps the version, approved at thereleaseenvironment.Shared workflows
Calls the reusable workflows in blacklanternsecurity/CLA, pinned to CLA#11 at 07e14bcbba5e737e1b29bd9e3f9128fbf2483be9. Re-pin to its merge commit before this merges.
tests.yml(pull_request, workflow_call) calls CLA'spython-tests.yml. Untilstable's ruleset moves, arequiredjob reportspython-tests (3.11)to(3.15)from the real result.rust-tests.ymlkeeps its liverust-testsname, now with every action SHA-pinned,contents: read, andpersist-credentials: false. Neither triggers onpush: stableany more, and their concurrency groups are separate from the calls inpublish.yml.publish.yml(push tostable) callsrelease-check.yml, both test workflows,maturin-wheels.yml(linux and musllinux, vendored OpenSSL), publishes to PyPI with OIDC inrelease, then crates.io throughcrates.ymlwith theid-token: writeit needs (without it the whole run failed at startup), thenpublish.yml(tag and GitHub release with SBOM and wheels, created last). No caller passes App credentials.stable-guard.yml:release readyon pull requests intostablerefuses a non-MAJOR.MINOR.PATCHversion and git or URL dependencies.cla.ymlpassesAPP_ID/APP_PRIVATE_KEYby name. No workflow usessecrets: inherit.v0.10.1tag.pyproject.tomlreads it from Cargo.toml.release.toml(cargo-release, which pushedv{{version}}tags) removed. README## Releasinglinks the public org release standard.dev(AGPL-3.0, Python 3.15 support).Before promoting to
stablereleaseenvironment with reviewers and astable-only deployment policy, holdingBLS_RELEASE_APP_IDandBLS_RELEASE_PRIVATE_KEY.publish.yml, environmentrelease. Then deletePYPI_API_TOKEN.python / passed,rust-testsandrelease ready, then drop therequiredjob.Validation
Refs blacklanternsecurity/bbot-enterprise#139, blacklanternsecurity/bbot-enterprise#140, blacklanternsecurity/bbot-enterprise#149, blacklanternsecurity/bbot-enterprise#155, blacklanternsecurity/bbot-enterprise#156 (blasthttp part)