fix(install): authenticate api.github.com call to avoid 60/hr rate limit - #1157
Conversation
All three installers called api.github.com/repos/.../releases/latest unauthenticated to resolve the latest release tag. GitHub caps unauthenticated REST API requests at 60/hour per source IP (shared across NAT/CGNAT/corporate proxy users), causing opaque 'Failed to fetch latest version' errors. Each installer now detects a token (precedence: GITHUB_TOKEN env > GH_TOKEN env > 'gh auth token') and attaches it as an Authorization: Bearer header to ONLY the version-resolution API call. Falls back to anonymous (unchanged behavior) when no token is available. Release downloads and git clone are left unauthenticated - they are not subject to the 60/hr REST limit. Token variables are cleared from memory immediately after the API call (defense in depth). install.ps1 wraps the API call in try/catch to surface a friendlier error instead of a raw terminating exception under $ErrorActionPreference=Stop. Docs updated with a rate-limited-IP install section and a troubleshooting entry. Parity + negative-assertion tests added. Closes backnotprop#1156
|
Thanks for the fast follow-up. Reviewed all three scripts plus the tests. The token handling is scoped correctly: the header only ever reaches the hardcoded api.github.com URL, downloads stay anonymous, and the tripwire tests lock that in. Docs read well too. One real problem found while testing, and it blocks merge as-is: A stale or invalid token now breaks installs that work today. I verified against the live API: a bogus Bearer token gets a 401, curl -f fails, and the installer exits with the same opaque error this PR is trying to eliminate. Expired GITHUB_TOKENs linger in CI images, dotfiles, and direnv setups all the time. Those users currently install fine anonymously; with this change they fail. Fix: if the authenticated call fails, retry once without the header before giving up. Roughly, in install.sh: latest_tag=$(curl -fsSL "${GH_AUTH_HEADER[@]}" "$api_url" | grep '"tag_name"' | cut -d'"' -f4) || true
if [ -z "$latest_tag" ] && [ ${#GH_AUTH_HEADER[@]} -gt 0 ]; then
latest_tag=$(curl -fsSL "$api_url" | grep '"tag_name"' | cut -d'"' -f4)
fiwith the same retry in install.ps1 (catch, then retry with empty headers) and install.cmd. That keeps the upgrade strictly additive: a good token raises the limit, a bad token costs one extra request, nobody is worse off. Two smaller notes:
Happy to merge once the anonymous fallback is in. |
Addresses review feedback on backnotprop#1157: a stale/revoked token (expired GITHUB_TOKEN lingering in CI images, dotfiles, direnv) gets a 401 and would break an install that works fine anonymously today. Each installer now retries the api.github.com version-resolution call without the auth header when the authenticated attempt fails: - install.sh: `|| true` + `[ -z "$latest_tag" ]` gate - install.ps1: try/catch, retries with no -Headers on catch - install.cmd: ERRORLEVEL check, retries without !GH_AUTH_HEADER! A bad token costs one extra request; nobody is worse off than today. New test asserts the retry path exists in all three installers (89 pass). Also corrects the AI disclosure: the assistant was opencode, not Claude.
|
Thanks for the thorough review - the stale-token regression is a real gap, good catch. Pushed the anonymous fallback to all three installers:
Verified: valid token succeeds first try (tag v0.25.1, no retry); bogus token 401s, falls back to anonymous. (My test IP had already burned its anonymous 60/hr quota during this session, so the anonymous retry 403'd locally - environmental, not a code issue; the retry path itself fired correctly.) On the Windows note: I don't have Windows access either, so install.cmd remains source-scan-only verified. The quoting pattern and the new ERRORLEVEL-based retry follow the existing cmd idioms in the file. If CI runs the install.cmd path on Windows that would be the real confirmation; otherwise a maintainer with Windows could spot-check. Test count is now 89 (added a retry-path parity test). |
|
Verified the update end to end:
CI approved and running. This is good to go from my side; final merge call is the maintainer's, likely in the morning (US time). The hosted copies at plannotator.ai are synced separately after merge, so the fix goes live with that step. Thanks for turning this around quickly. |
|
A deeper second-pass review found real issues, so this is not merge-ready yet after all. Findings, worst first: Blocking
Security (both cmd-only, both one-line fixes)
Should fix
The scope/provenance side came back fully clean: diff touches exactly the stated files, every URL verified character by character, no non-ASCII tricks, no new hosts or subprocesses beyond Sorry for the two-round review; the cmd items and the 401-only retry are the substance. Happy to re-verify quickly once pushed. |
Errorlevel check adjacency in install.cmd, delayed-expansion token reads, token charset guard, PATH-resolved gh with --hostname github.com, 401-only anonymous retry in install.sh and install.ps1, preserved original errors in install.ps1, honest cleanup comments, non-vacuous test assertions, and doc corrections (1000/hr Actions token, zero-scope guidance, -Version for PowerShell).
|
Heads up @tbontb-iaq: rather than another review round trip, we pushed the fixes directly to your branch as 02a0ddd (maintainer edits are enabled on the PR). It addresses everything from the second review: errorlevel check adjacency in install.cmd, delayed-expansion token reads plus a charset allowlist, PATH-resolved gh invoked with --hostname github.com, 401-only anonymous retry in install.sh and install.ps1 (cmd keeps the simpler retry, with a comment), preserved original error messages in install.ps1, honest cleanup comments, non-vacuous test assertions, and the doc corrections (1,000/hr for the Actions token, zero-scope guidance, -Version for PowerShell). Verified: 90/90 install tests pass, macOS bash 3.2 parses and runs the new block correctly (bogus token falls back on 401 and resolves; no token resolves in one call; network failure does not retry). install.cmd is verified by careful read only, since we cannot execute cmd here, so if you have a Windows box handy, one run each with and without GITHUB_TOKEN set would be a welcome final check. Please pull before making further changes to avoid a divergent branch. Thanks again for the contribution and the quick turnarounds. |
Summary
Fixes #1156.
All three installers (
install.sh,install.ps1,install.cmd) calledapi.github.com/repos/.../releases/latestunauthenticated to resolve the latest release tag. GitHub caps unauthenticated REST API requests at 60/hour per source IP (shared across NAT/CGNAT/corporate proxy users, not per user), causing opaqueFailed to fetch latest versionerrors on shared egress IPs and during repeated/debug runs within an hour.Fix
Each installer now detects a token and attaches it as an
Authorization: Bearerheader to only the version-resolution API call:GITHUB_TOKENenv varGH_TOKENenv vargh auth token(ifghCLI is installed and authenticated)When no token is available, the header is omitted and behavior is unchanged (anonymous, 60/hr) - fully backward compatible. With a token the limit rises to 5000/hour.
Scope of auth: only the
api.github.comversion-resolution call is authenticated. Release-asset downloads (github.com/.../releases/download/...) andgit cloneare intentionally left unauthenticated - they are not subject to the 60/hr REST limit, and authing large binary downloads would expose the token in the process argument list for the full download duration.Hardening (from pre-PR self-review)
_gh_token/GH_AUTH_HEADERin sh,GH_TOKEN_VAL/GH_AUTH_HEADERin cmd,$ghToken/$ghHeadersin ps1) are cleared from memory immediately after the API call - they are not needed for the download/clone phases. Defense in depth against environment leakage.install.ps1wraps theInvoke-RestMethodcall intry/catch, surfacing a friendlier error (with a pointer to Installer fails behind rate-limited IPs: api.github.com called without auth (60 req/hr) #1156) instead of a raw terminating exception under$ErrorActionPreference = "Stop"(the existing anonymous path had this latent issue too; the PR widens the failure surface since token-bearing calls can 401/403).Known limitation
The token is passed to
curlvia-Hininstall.shandinstall.cmd(PowerShell uses a request-object hashtable, which is not affected). This means the token is briefly visible in the process argument list (/proc/<pid>/cmdlineon Linux,pson macOS) for the duration of the single API call (<1s). This matches the approach used byrustup,homebrew, and other installers. A more robust fix would usecurl --configwith a0600temp file, but that adds complexity (temp-file lifecycle,mktempfailure underset -e, and is impractical in batch forinstall.cmd). If maintainers prefer the--configapproach for the POSIX installer, I'm happy to follow up.Docs
installation.md: new "Installing behind a rate-limited IP"<details>section.troubleshooting.md: new "Installer fails with 'Failed to fetch latest version'" entry with the token precedence and a link to Installer fails behind rate-limited IPs: api.github.com called without auth (60 req/hr) #1156.Tests
New tests in
install.test.ts(install shared behaviordescribe block):GITHUB_TOKEN/GH_TOKEN, use a Bearer scheme, fall back togh auth token, and attach the header to theapi.github.comcall (with per-installer mechanism assertions: array splat for sh, hashtable for ps1, delayed-expansion arg for cmd).All 88 tests pass (
bun test scripts/install.test.ts).Verification
Out of scope
apps/marketing/public/install.shandinstall.ps1are git-tracked symlinks toscripts/install.*- no separate edit needed.install.cmdhas no public mirror (served differently).AI assistance disclosure
This pull request was developed with the assistance of an AI coding agent (opencode). The issue investigation, code changes across all three installers, the test additions, and the self-review were produced collaboratively with the AI. I reviewed and verified every change before submitting, including running the full test suite (89 pass) and live smoke tests of the token-detection and anonymous-fallback logic (valid token succeeds first try; bad token 401s and retries anonymously).