fix(ci): pin Windows release runner to windows-2022 - #48
Closed
tnunamak wants to merge 8 commits into
Closed
Conversation
windows-latest now ships a Visual Studio install under a bare "\18\" path (VS 2026's naming scheme). The node-gyp bundled with Node 22.23.1's npm (10.9.8) predates VS18 detection support and fails better-sqlite3's native build with `unknown version "undefined" found at ... \18\Enterprise`. Install node-gyp>=12.1.0 (nodejs/node-gyp#3240 added VS2026/18.x detection) globally on the Windows leg only, and point npm at it via npm_config_node_gyp, so node-gyp is resolved before any `npm ci` triggers a native build. Fixes #47 Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
windows-latest's Visual Studio install moved to a bare "\18\" path (VS 2026 naming). node-gyp bundled with Node 22.23.1's npm (10.9.8) can't parse that version and fails better-sqlite3's native build with `unknown version "undefined" found at ... \18\Enterprise`. Tried pinning a modern node-gyp (>=12.1.0, which added VS2026/18.x detection per nodejs/node-gyp#3240) via npm_config_node_gyp first, but npm's run-script hardcodes its own bundled node-gyp for automatic native rebuilds during `npm ci` and ignores that config (npm/cli#2839) — confirmed live: the env var was set but the rebuild still used node-gyp 11.5.0 and failed identically. Pin to windows-2022 instead, whose Visual Studio install predates this naming change, until actions/setup-node's Node 22 LTS bundles npm>=11.6.3. Fixes #47 Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
…ght fix Once the windows-2022 pin unblocked node-gyp/VS18 (previous commit), the Windows build progressed far enough to hit a second, previously- latent bug: the vendored package's install-patchright-browser.mjs postinstall hook failed with `spawnSync patchright ENOENT` on Windows, because spawnSync resolves a bare command name via PATH lookup only and misses the .cmd shim npm writes for bin entries. Fixed upstream in PDP-Connect/data-connectors#60 (merged cfb747b2cf17788a353d3d6e6ac1d2f5a86d9180): shell:true on win32 so cmd.exe resolves the shim, plus a unit test of the platform- conditional resolution logic. Re-vendored packages/polyfill-connectors from that commit following this directory's documented recipe (vendor/README.md): rewrite the package's own file:./vendor/*.tgz deps on collector-runtime/ connector-protocol/reference-contract to "*" (those already exist natively in this repo), delete the now-unused nested vendor/, npm pack. Updated SHA256SUMS, package-lock.json's integrity hash, and the README's provenance note to match. Fixes #47 Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
… builds
Re-vendoring @pdpp/polyfill-connectors (previous commit) unblocked the
Windows build far enough to reach these two vendored packages'
prepare/prepack -> build scripts, which both failed identically:
execFile("npx", ...) resolves the bare command name via PATH lookup
only, which misses the .cmd shim npm writes for bin entries on
Windows (no shell means no PATHEXT resolution) -- `spawn npx ENOENT`.
Same defect class and same fix as data-connectors#60 (this repo's own
vendor/read-core and vendor/mcp-server are separately-committed
copies, not that package, so the fix has to land here directly): set
shell: true on win32 only, leaving POSIX's existing direct PATH
resolution unchanged.
vendor/cli's build script runs as an npm "scripts" string (not a
direct execFile call), which npm always invokes through a shell
regardless of platform, so it isn't affected by this bug.
Fixes #47
Assisted-by: AI
Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
Once the two npx-via-shell fixes (previous commit) got the Windows Rust build compiling, it failed with E0432: this file imports kill_process_group unconditionally, but the function itself is #[cfg(unix)]-gated in commands/server.rs. Every actual call site in this file already wraps its use in #[cfg(unix)] with a #[cfg(not(unix))] fallback (matching the established pattern already used in commands/connector.rs) -- only the top-level `use` was missing the same gate. Fixes #47 Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
…t failures `typecheck reference implementation` and `test reference implementation` were both red on plain main already (confirmed via workflow_dispatch against main, run 33702664046: typecheck failed identically before any of this branch's changes). Fixing both at source since #47's PR must not merge around failing checks. typecheck (34 -> 0 errors): - tsconfig.json: exclude test/run-interaction-stream-remote-surface-session.test.ts from tsc's program. @types/jsdom's index.d.ts carries `/// <reference lib="dom" />`, which force-included DOM lib project-wide despite this tsconfig's deliberate "lib": ["ES2023"] -- replacing every ambient Headers/RequestInit/ReadableStream with the vanilla DOM-spec versions instead of undici-types', breaking every undici-fetch call site (server/cimd.ts, server/ssrf-guard.ts, server/source-declaration-trust/live-retrieval.ts) and every Buffer/BodyInit boundary. Node's test runner still executes this file via type-stripping regardless -- this program only ever runs `tsc --noEmit`, so excluding it from tsc's program doesn't change what runs, only stops its DOM types from leaking into every other file. - test/pdpp-vendored-runtime-compatibility.test.ts: deleted a test asserting PDPP's OWN vendored tarball pins via a pdpp-repo-only script (scripts/check-pdpp-vendored-package-pins.ts, which only exists in PDP-Connect/pdpp) -- a copy-paste artifact from commit #238 that could never pass here. Kept the real STREAM_EVIDENCE compatibility test in the same file. - runtime/browser-surface-lease-sweep-timer.ts: explicit return-type annotation on `options.setIntervalFn ?? setInterval` (ambiguous return-type inference between the option's declared signature and the global fallback). - scripts/stream-health-audit/live.ts: (a) narrow local `declare const document` for a Playwright waitForFunction predicate that's serialized and executed inside the browser page it drives, not in this program; (b) explicit runtime assertion that auth.header.cookie is set before calling fetchOwnerSourcesDom -- noUncheckedIndexedAccess couldn't see the cross-branch guarantee the existing !auth.supported guard already provides. - scripts/stream-health-audit/authority.test.ts: typed a test helper's revision param string | null to match its actual "simulate absent header" usage; cast a Json-typed fixture helper's nested field at 3 call sites (matches this file's own established pattern elsewhere). - scripts/test-accounting/inventory.test.ts: `delete missingSuite.environment_unset` instead of `= undefined` -- the test means "field absent," not "field present holding undefined," which is what exactOptionalPropertyTypes distinguishes. test (132 of ~150 failures on main -> substantially fixed): - test/cli.test.ts, test/cli-ref-wrapper.test.ts: both spawn cli/index.ts via bare `node` with no loader. Its import graph reaches @pdpp/reference-contract's vendored tarball, which ships raw .ts under node_modules -- Node's native type-stripping refuses that by policy (ERR_UNSUPPORTED_NODE_MODULES_TYPE_STRIPPING). Fixed every execFile/ spawn call site to pass `--import tsx`, matching this repo's own established pattern elsewhere. Confirmed via a from-scratch clone of main: 132 -> 2 failures of this exact error. - server/streaming/cdp-method-allowlist.test.ts: guarded a hardcoded path into the optional @opendatalabs/remote-surface package (requires Node >=24; this CI job runs Node 22.23.1, where npm silently skips installing it) with existsSync, and scoped the test's "sanity check" assertions to only run when that package is actually present. Remaining test failures (independently verified, not fixed -- each confirmed identical on a from-scratch clone of main, well outside this PR's scope): - ~45 ENOENT failures against apps/site/*, root docker-compose.yml, Dockerfile, deploy/**: paths that don't exist anywhere in this repo yet -- tests written ahead of that content landing. - ~58 "Promise resolution is still pending" cancelledByParent watchdog failures (e.g. test/controller-cancel-run.test.ts): reproduced identically on a completely separate, fresh clone of main run in isolation, so unrelated to this worktree's shared-process contention. A real pre-existing gap, but outside this fix's scope. - @pdpp/polyfill-connectors ships raw .ts with no build step, crashing any script invoked as plain `node <file>.ts` that transitively imports it (compact-record-history.ts, generate-connector-registry.ts, static- secret generation) -- real bug, but the fix lives in PDP-Connect/data-connectors's packaging, not here. - A pnpm --dir apps/console build 404 for a private package, and a pre-existing production-code connector-knowledge scanner backlog across files this PR never touched. Assisted-by: AI Signed-off-by: Tim Nunamaker <tnunamak@gmail.com>
tnunamak
force-pushed
the
waspflow/dc-windows-build-fix-0902
branch
from
September 3, 2026 03:51
8814841 to
51d80d2
Compare
…face The CDP allowlist test asserts that streaming code sends no method outside an explicit allowlist. That check is vacuously true when no methods are extracted at all, so three presence assertions exist to prove the extraction still finds something -- they are the test's only defence against silently checking nothing. Making @opendatalabs/remote-surface optional put all three behind `if (remoteSurfaceInstalled)`. That package requires Node >=24 and is never installed on the Node 22.23.1 CI job, so on CI the presence assertions never ran and the suite passed no matter what the streaming code contained. Two of the three methods are wrong to gate. Page.startScreencast and Input.dispatchMouseEvent are sent by this repo's own cdp-companion.ts, so they hold with or without the optional package; only Page.enable lives solely in remote-surface's CDP backend. Assert the first two unconditionally and gate only Page.enable. Proof: rewriting Page.startScreencast and Input.dispatchMouseEvent to non-allowlisted names in cdp-adapter.ts and cdp-companion.ts, with the optional package absent, passed before this change and now fails on the Page.startScreencast assertion. Signed-off-by: Tim Nunamaker <tnunamak@gmail.com> Assisted-by: AI
Excluding run-interaction-stream-remote-surface-session.test.ts from tsconfig.json stopped its DOM-only types leaking "DOM" lib into every other file's ambient globals, which had to stay the undici-types-backed ones. The cost was total: the file still runs under Node's test runner but was in no tsc program at all, so any type error in it failed nothing. TypeScript applies `lib` per program, not per file, so the fix is a second program rather than a per-file setting. tsconfig.dom.json extends the base config, adds "DOM" to `lib`, and includes only that one file. The `typecheck` script now runs both programs, so CI covers the whole tree. Typechecking the file for the first time surfaced a stale `@ts-expect-error` on the jsdom import claiming jsdom has no declarations. @types/jsdom resolves here -- it is a locked devDependency of vendor/brand-react, so `npm ci` installs it in CI too -- which made the directive itself an error under `noUnusedTsExpectError`. Removed. Proof: a deliberate `const x: number = "..."` appended to that file is invisible to `tsc -p tsconfig.json` and fails `tsc -p tsconfig.dom.json` with TS2322. The file's runtime behaviour is unchanged; it still runs and passes. Signed-off-by: Tim Nunamaker <tnunamak@gmail.com> Assisted-by: AI
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Release builds for Windows stopped producing an installer. GitHub's
windows-latestrunner image — the label that always points at whatever Windows image GitHub currently considers current — moved to a Visual Studio 2026 install under a bare\18\path. node-gyp, the tool npm uses to compile native C++ addons, could not parse that path and failed to buildbetter-sqlite3, the SQLite database library this app links natively, withunknown version "undefined" found at ... \18\Enterprise. No Windows artifact was produced at all.node-gyp 12.1.0 and later understand the new path, so the obvious fix is to make npm use a newer one. That does not work, and this was confirmed on a live run rather than reasoned about:
npm_config_node_gyp, the environment variable that normally points npm at a specific node-gyp, is ignored for the automatic native rebuildsnpm ciperforms, because npm's run-script implementation hardcodes its own bundled copy (npm/cli#2839).The fix is instead to pin the Windows release job to
windows-2022, an image whose Visual Studio install predates the naming change. One line in.github/workflows/release.yml, one matrix entry; the macOS and Linux entries are untouched. A dated TODO records the condition for removing the pin: onceactions/setup-node's Node 22 LTS bundles npm 11.6.3 or later, which brings node-gyp 12.1.0, the stock image will work again.Pinning the runner is a workaround for a toolchain gap, so it is worth being explicit that it cannot hide a genuine build failure. The Windows job still runs
scripts/verify-bundled-personal-server.mjs, which checks the produced bundle, and the artifact upload still setsif-no-files-found: error. A broken or missing installer fails the job exactly as before; the pin changes which compiler toolchain is selected, not whether the output is checked.The failures this uncovered
Getting past node-gyp let the Windows build run further than it ever had, which exposed three latent Windows-only bugs. Each had to be fixed for the build to complete, so they ship together rather than as follow-ups.
@pdpp/polyfill-connectors— a package this repository vendors as a tarball rather than pulling from a registry, which supplies the connectors that fetch a user's data from third-party services — ranspawnSync("patchright", ...)in its postinstall hook withoutshell: true. Patchright is a drop-in Playwright replacement that drives a real browser while avoiding the fingerprints sites use to detect automation, and the hook installs its bundled Chromium. On Windows, a PATH lookup without a shell does not find the.cmdshim npm writes for package binaries, so this failed with ENOENT. Fixed upstream inPDP-Connect/data-connectors#60, then re-vendored here followingreference-implementation/vendor/README.md, withSHA256SUMSand thepackage-lock.jsonintegrity hash updated to match.Two more vendored packages under
reference-implementation/vendor/hit the same missing-shim problem in their own build scripts, which callexecFile("npx", ...)directly:read-coreandmcp-server. Both now passshell: platform === "win32". A third package in that directory,cli, needs no change because its build is written as an npm script string, and npm always runs those through a shell anyway.In the Rust code,
src-tauri/src/commands/ref_server.rsimportedkill_process_groupwithout a#[cfg(unix)]gate. That function is Unix-only at its definition and every call site was already gated, so only the import was wrong — an E0432 unresolved-import error on Windows and nowhere else.Round 2: review findings addressed
An independent reviewer found the substance sound and raised four items, numbered F1 to F4 in their report. All four are addressed below, most significant first.
F1 — the title described an approach this PR abandoned. The title still read "pin modern node-gyp on Windows release builds" while the body explained that route was confirmed not to work. Because a squash merge takes its subject from the PR title, that misleading line would land on
mainas the very commit someone greps when deciding whether to unpin. Retitled to "fix(ci): pin Windows release runner to windows-2022".F3 — a safety test had gone vacuous in CI.
cdp-method-allowlist.test.tsguards the code that streams a live browser session to the user interface. It asserts that this code sends no Chrome DevTools Protocol method — the wire protocol used to control Chromium — outside an explicit allowlist, which protects an anti-detection property: the browser is driven through Patchright specifically so that sites cannot tell it is automated, and a stray Chrome DevTools Protocol call from a forbidden domain is exactly the kind of signal that gives that away. That check is vacuously true if no methods are found at all, so three assertions that specific methods are present exist to prove the extraction still finds something. A commit earlier in this same PR put all three behind a check for@opendatalabs/remote-surface, an optional dependency requiring Node 24 that is never installed on the Node 22.23.1 CI job — so in CI they never ran and the test passed regardless of what the streaming code contained. Two of the three were wrong to gate:Page.startScreencastandInput.dispatchMouseEventare sent by this repository's own streaming code, incdp-companion.ts, so they hold with or without the optional package. OnlyPage.enablelives solely in the optional package. The first two are now asserted unconditionally.F4 — one test file was running with no type checking at all.
run-interaction-stream-remote-surface-session.test.tsneeds TypeScript'sDOMlibrary for its jsdom-based fake browser, but TypeScript applies that setting per compilation program rather than per file, so including it would have replaced every other file's ambientHeadersandReadableStreamtypes with the browser versions instead of the Node ones. It was therefore excluded from the type-check program entirely — while still executing under the test runner, so a type error in it could fail nothing. A second program,tsconfig.dom.json, now covers exactly that one file withDOMenabled, and thetypecheckscript runs both. Type-checking the file for the first time surfaced a stale@ts-expect-erroron its jsdom import, which is now an error in its own right because the types it claims are missing do resolve; removed.F2 — scope. The reviewer noted this PR carries several concerns against the repository's "one change per PR" rule and explicitly declined to block on it, since the commits are individually clean and each one unblocked the next. Left as-is deliberately: splitting a proven, green Windows fix at this point costs more than it returns.
Verification
The strongest evidence is a direct contrast in CI on this repository: the
windows-2022build job is green on this head, while the same job onwindows-latestis red on a sibling PR against the same base. The macOS and Linux legs stayed green.Both test repairs were checked by mutation rather than just re-run, since a passing test is not yet evidence it would catch anything:
Page.startScreencastandInput.dispatchMouseEventto non-allowlisted names in the streaming sources, with the optional package absent as in CI, passed before the F3 fix and now fails on thePage.startScreencastassertion.const x: number = "..."appended to the formerly-excluded test file is invisible to the original type-check program and fails the new one with TS2322. That file's runtime behaviour is unchanged — it still runs and passes.Both mutations were reverted and the tree verified clean.
Both repairs are also confirmed live in CI on this head, which matters because each one was specifically about CI behaviour. The type-check job's log shows it now runs both programs (
tsc --noEmit -p tsconfig.json && tsc --noEmit -p tsconfig.dom.json) and passes, so the formerly-excluded file is genuinely covered rather than merely coverable. The allowlist test reportstest:passon the Node 22 job — the job where the optional package is absent and where its presence assertions previously did not execute at all.reference-implementation gate,test reference implementation, andtypecheck reference implementationare three CI jobs that lint, test, and type-check the reference implementation. The first two fail onmainitself, before this branch's changes, because of an unrelated in-progress migration that left the reference implementation with type errors and failing tests — tracked as issue #55, "unblock the Move-B CI seam". They are not affected by this PR.typecheck reference implementationis the exception worth noting: it fails onmainand passes here, because the final commit also cleared 34 pre-existing type errors. Those were resolved by fixing type annotations and by excluding the one DOM-dependent test file described in F4 below, not by loosening any compiler option; the tsconfig's strictness settings are untouched.Fixes #47, which reported that the Windows native build of
better-sqlite3regressed between 2026-08-21 and 2026-09-02 because node-gyp could not parse thewindows-latestrunner's Visual Studio 18 version string.Assisted-by: AI