Skip to content

Add corpus regeneration diff CI - #284

Open
sirreal wants to merge 4 commits into
masterfrom
add-corpus-diff-ci
Open

Add corpus regeneration diff CI#284
sirreal wants to merge 4 commits into
masterfrom
add-corpus-diff-ci

Conversation

@sirreal

@sirreal sirreal commented Aug 18, 2026

Copy link
Copy Markdown
Member

Implements the corpus-diff check that the #262 and #270 review rounds relied on manually: parse a pinned corpus of WordPress core source (wp-includes from the WordPress 6.8 tag) with the parser at the PR's merge base and at its head, normalize both JSON outputs with prep-diff.php, and diff. Every hunk in the diff is a behavior change the PR makes — intended and explained, or a regression. This check has repeatedly caught regressions no unit test covered.

Decisions taken (also documented in the workflow header):

  • Non-blocking to start. The job always succeeds; the diff is uploaded as an artifact and the job summary reports the hunk/line count (or 0 hunks). Make it blocking with an escape hatch (label or committed expected diff) once the signal has proven trustworthy.
  • The head checkout's tooling drives both sides. tools/export-corpus.php takes the parser root as a parameter, so one copy exports through the base checkout and the head checkout; prep-diff.php from head normalizes both. Tooling changes therefore never masquerade as parser changes; a change to prep-diff.php itself shows its normalization effect in the diff and is reviewed like any other change.
  • PHP 7.4 — the supported floor, matching the oldest runtime in the unit-test matrix.
  • Corpus scope: wp-includes only for now (~1000 files, a few minutes to parse). wp-admin/includes can be added later for hook/uses coverage at the cost of runtime.
  • Separate workflow from unit tests: different failure semantics.

Determinism: pinned corpus tag (cached by tag), sorted file list via the existing get_wp_files(), LC_ALL=C, the same PHP binary for both sides, JSON_PRETTY_PRINT on both. The export script routes PHP diagnostics to stderr so stdout stays pure JSON (vendored code emits deprecation notices on newer PHP).

Smoke-tested locally: exporting tests/source through tools/export-corpus.php and piping through prep-diff.php produces clean normalized JSON.

Design doc and manual procedure this implements were written during the #262/#270 review work.

Update

Corpus pinned to WordPress 7.0.4. 7.0.4 is the latest WordPress release; pinning the point release rather than the 7.0 major tag tracks the patches that land in wp-includes, so the corpus matches what is shipped. WordPress 7.0 also raised core's minimum PHP to 7.4, exactly this job's PHP floor. Verified locally against wp-includes@7.0.4: 1039 PHP files, all present in the export, no parse errors, ~49 MB of JSON — both of the job's guards (>= 500 files, >= 1 MB of JSON) still hold.

Manual procedure documented in the README. A new "Corpus diff" section records what the check is, how to run it locally with the repo's own tools/export-corpus.php and prep-diff.php (merge-base worktree, export both sides over the same corpus, normalize, diff), and the tooling policy the workflow header already states: the head checkout's tooling drives both sides.

🤖 Generated with Claude Code

7.0.4 is the latest WordPress release. Pinning the point release rather
than the 7.0 major tag tracks the patches to wp-includes, so the corpus
matches what is shipped.

WordPress 7.0 raised core's minimum PHP to 7.4, which is exactly this
job's PHP floor.

Verified against wp-includes@7.0.4: 1039 PHP files, all of them present
in the export, no parse errors, ~49 MB of JSON. Both of the job's guards
(>= 500 files, >= 1 MB of JSON) still hold.
Records what the corpus-diff check is, how to run it by hand with the
repo's own tools/export-corpus.php and prep-diff.php, and the policy the
workflow already states: the head checkout's tooling drives both sides.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant