Skip to content

ci: plugin rebuild loop finishes instead of hanging on --watch - #370

Merged
foxnne merged 1 commit into
mainfrom
ci/plugin-loop-hang
Oct 13, 2026
Merged

foxnne merged 1 commit into
mainfrom
ci/plugin-loop-hang

Conversation

@foxnne

@foxnne foxnne commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Plugin rebuild loop / Measure (Linux) had never completed: every run (38083453064, 38077858541, 38012636910, 37975678599, 37969247976) was cancelled at GitHub's 1 h job limit, which put a red X on unrelated PRs touching sdk/plugin_sdk.zig.

Why it ran for an hour

In the cancelled runs, the builds took seconds (6–18 s cold), but every --watch -fincremental edit printed timeout no rebuild within 300 s. Before that, the wait for the watcher's first pass also ran its full 300 s without saying so. That's about 20 minutes per backend × 6 optimize/backend combinations.

The watcher wasn't slow, it was dead. bench.py never checked it and deleted watch.log with the scratch profile. With the log kept (this PR's first run), the reason is:

compile exe fizzy-plugin-sdk-meta-Debug Debug native failure
error: thread 2652 panic: REX_GOTPCRELX

With -fincremental on x86_64 Linux, Zig 0.16 links with its own ELF linker. That linker panics on a relocation in the C objects in the plugin's build graph (stb, freetype, tree-sitter), so the build runner aborts on the first pass. That's a limitation in Zig, not in fizzy's build. On macOS (aarch64, LLVM + the system linker) the same commands work: an edit rebuilds in 1–2 s.

What changes

  • scripts/plugin-loop/bench.py
    • A wait for a rebuild also watches the watcher. If it has exited, the wait ends at once.
    • A backend's watch section ends at its first failure instead of waiting out each remaining edit. An edit gets 60 s (was 300). The first pass, a cold build, keeps 300.
    • On failure it prints the last 40 lines of the watch log and adds a failed row to the step summary with the exit code and the first error: line.
  • .github/workflows/plugin-loop.yml: the job's timeout-minutes is 25 (was 60) and the Measure step gets its own 20. A hang the bench can't see now ends in 20 minutes with GitHub naming the step, against a normal run of about 6.

Measured (run 38247684393, zig 0.16.0, ubuntu-latest)

--watch -fincremental
Debug, default / llvm / self-hosted failed: panic: REX_GOTPCRELX (first pass)
ReleaseFast, default 16.6 / 16.3 / 16.7 s per edit
ReleaseFast, llvm 17.4 / 18.8 / 18.8 s per edit
ReleaseFast, self-hosted failed: panic: REX_GOTPCRELX (first pass)

Measure step: 5 min 20 s; job about 6 min, green. What incremental compilation buys on Linux is now measurable on a newer Zig with zig_version = master from Actions > Run workflow.

Verification

  • CI run 38247684393 on this branch: Measure (Linux) completed successfully in about 6 min, with the failed rows and watch-log tails above in its output.
  • Locally (macOS, zig 0.16.0): scripts/plugin-loop/bench.py --optimize Debug --backend default --watch 2 finished in 29 s with both watch edits timed (1.2 s, 2.4 s).
  • Workflow YAML parses and bench.py compiles. No Zig sources changed, so the zig build gates are unaffected.

🤖 Generated with Claude Code

The Measure (Linux) job never finished: every run was cancelled at the 1 h job limit. Under
`--watch -fincremental` on x86_64 Linux, Zig 0.16 links with its own ELF linker, which panics
(`REX_GOTPCRELX`) on the C objects in the plugin's build graph. The watcher aborted on its first
pass, and bench.py, never checking it, waited 300 s for the first pass and 300 s per edit that
nobody would build: about 20 minutes per backend, six backends.

bench.py now ends a wait as soon as the watcher exits, stops a backend's watch section at its
first timeout (60 s per edit, down from 300), and prints the watch log's tail with a `failed` row
carrying the error. The job's limit is 25 minutes and the Measure step's 20, against a measured
run of about 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@foxnne
foxnne force-pushed the ci/plugin-loop-hang branch from cf26cdc to 8cc81e6 Compare October 13, 2026 13:30
@foxnne
foxnne marked this pull request as ready for review October 13, 2026 13:31
@foxnne
foxnne merged commit 9b11973 into main Oct 13, 2026
9 checks passed
@foxnne
foxnne deleted the ci/plugin-loop-hang branch October 13, 2026 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant