Repository navigation
Conversation
With `--experimental_remote_repo_contents_cache`, a module extension or repository rule that reads a Label makes Bazel copy that whole repo to disk. The `python` extension and the `pythons_hub` repo read `runtimes_manifest.txt` this way, so every build materializes rules_python. This loads the same text from `runtimes_manifest_workspace.bzl` (`MANIFEST_TEXT`), which `runtimes_manifest_workspace_sync_test` already keeps identical to the `.txt`. The archive-flavor test passes its manifest through a new testing-only `_default_runtime_manifest` argument, and the now-unused default manifest mock is removed.
|
Hm, interesting, and a bit confusing. (1) How can the rules_python bzlmod extension be processed if rules_python isn't available? (2) Does this effectively mean bzlmod extensions can't read any files without incurring a download of an entire repository? |
|
(1) Bazel has a virtual file system that will download the remote files on demand when loading bzl files. All the starlark can work without materializing the whole thing. This is a decent change on its own merits, but we can close in favor of a ctx.read(label) change later after my Bazel PR merges if you prefer. |
|
I brought this up in #bzlmod on bazel slack: https://bazelbuild.slack.com/archives/C014RARENH0/p1792553506777879 It turns out bazel will prefetch all Does changing If not, then: (1) Whats the minimum version of bazel to get the above behavior? Basically we're going to drop bazel 7 support when bazel 10 comes out, so I'd rather not add it just for bazel 7. (2) Some variant of this PR SGTM. In that case, the main change in this PR to make is removing the dead code and renaming as appropriate -- runtimes_manifest.txt essentially becomes unused, and runtimes_manifest_workspace.bzl becomes a misnomer. But either way, there's no avoiding the fetch of the content: either bazel prefetches it due to the .bzl extension, or it gets fetched later when read() is called. |
With
--experimental_remote_repo_contents_cache, a module extension orrepository rule that reads a Label makes Bazel copy that whole repo to disk.
The
pythonextension and thepythons_hubrepo readruntimes_manifest.txtthis way, so every build materializes rules_python.This loads the same text from
runtimes_manifest_workspace.bzl(
MANIFEST_TEXT), whichruntimes_manifest_workspace_sync_testalready keepsidentical to the
.txt.Before:
DEBUG: Materializing remote repo @@rules_python+on every build.After: no materialization; the same 741
@pythons_hubtargets and anidentical
versions.bzl. Tested with Bazel 10.0.0rc1 and a remote cache,querying
@pythons_hub//:allfrom a fresh output base after a warm-up run.Tests: the mock
module_ctxcan't read real files, which is why theextension tests had a fake default manifest. Since the manifest is now loaded
rather than read, they use the real one, and the fake is removed. The
archive-flavor test passes its own manifest through a new testing-only
_default_runtime_manifestargument.