Skip to content

ci: discover mzcompose compositions via Git in the lint check - #38778

Open
antiguru wants to merge 1 commit into
MaterializeInc:mainfrom
antiguru:mzcompose-lint-git-discovery
Open

ci: discover mzcompose compositions via Git in the lint check#38778
antiguru wants to merge 1 commit into
MaterializeInc:mainfrom
antiguru:mzcompose-lint-git-discovery

Conversation

@antiguru

Copy link
Copy Markdown
Member

The check_all_files_referenced_in_ci check in check-mzcompose-files.sh listed compositions with a bare find . -name mzcompose.py, which descends into gitignored directories. A checkout nested inside the repository, such as a Git worktree or a build directory, therefore contributes its own copies of every composition. The exclusion list did not suppress them, because each entry is anchored at the repository root and the nested copies live under a different prefix. The check then reported cli, monitoring, canary-environment, mzcompose_examples and get-cloud-hostname as unreferenced, plus any composition that exists in the nested checkout's revision but not in the current pipeline templates. CI is unaffected, since its checkout holds no nested repository, but the check cannot pass locally for anyone who keeps a worktree inside the repository.

This replaces the find with git_files from shlib.bash, the same helper and the same '**/mzcompose.py' pattern that check-python-files.sh already applies to mzcompose files. Git resolves the gitignore rules, so the check sees exactly the compositions that mzbuild.Repository discovers through git.expand_globs when it builds them. The exclusion list moves into an array that filters the result by exact path, which keeps the per-entry comments and drops the ./ prefix the find predicates needed.

One behavior difference follows from the helper. git_files reports tracked files only, so a newly created composition is checked once it is staged rather than as soon as the file exists. CI always operates on a tracked checkout, so coverage there does not change.

The change is confined to a CI lint check, so it adds no tests and needs no release note.

🤖 Generated with Claude Code

The `check_all_files_referenced_in_ci` check listed compositions with a
bare `find . -name mzcompose.py`, which descends into gitignored
directories. A checkout nested inside the repository, such as a Git
worktree or a build directory, therefore contributes its own copies of
every composition. The exclusion list did not suppress them because each
entry is anchored at the repository root, so the check reported `cli`,
`monitoring`, `canary-environment`, `mzcompose_examples` and
`get-cloud-hostname` as unreferenced, plus any composition that exists in
the nested checkout's revision but not in the current pipeline templates.

Use `git_files` from `shlib.bash` instead, the same helper and pattern
that `check-python-files.sh` already uses for mzcompose files. It relies
on Git to resolve gitignore rules, so it sees exactly the compositions
that `mzbuild.Repository` discovers when it builds them. The exclusion
list moves into an array that filters the result by exact path.

One behavior difference follows from the helper: `git_files` reports
tracked files only, so a newly created composition is checked once it is
staged rather than as soon as the file exists. CI always operates on a
tracked checkout, so coverage there is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@antiguru
antiguru requested a review from a team as a code owner September 11, 2026 09:48
@antiguru
antiguru requested a review from bosconi September 11, 2026 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant