mdbook: flatten input paths to support cross-package/repository dependencies

[mgeisler]

This is a follow-up to #4027. While #4027 enabled the staging of files from external repositories, mdbook still requires all configuration, source files, and themes to sit in a contiguous directory structure.

This change implements path flattening for both build and serve actions:

• Build Action: Strips repository and package prefixes during workdir staging in mdbook.bzl.
• Serve Action: Implements conditional staging in server.rs. If files are split across repositories, it flattens them into a temporary directory using symlinks to preserve live-reloading. Standard books run in-place for maximum plugin compatibility.

Includes a new integration test in test/flattening that verifies resolution across three different packages, and unit tests in private/server.rs for the staging logic.

---

The changes here were made with Gemini and tested by me on a Bazel migration for Comprehensive Rust in google/comprehensive-rust#3197 and later PRs. I understand the changes, but I am not a deep expert on Bazel and its directory structure so I don't know if there is a more idiomatic way to do this. Existing tests (not yet released) were adjusted to fit with the new more flexible staging of files.

[UebelAndre]

Looks like CI is failing

[mgeisler]

Looks like CI is failing

Thanks, should be fixed now!

[UebelAndre]

Instead of having this rule strip repository and package prefixes (which would require files to either be in the root of an external repository or a matching package?) why not require that users have a well structured set of inputs? For example, if you want an external file to appear in docs, my expectation is that one would use copy_file to select the external file and output something that mdbook would now see?

[mgeisler]

why not require that users have a well structured set of inputs? For example, if you want an external file to appear in docs, my expectation is that one would use copy_file to select the external file and output something that mdbook would now see?

In Comprehensive Rust, I have 500+ Markdown files which come from an external repository — and if I understand correctly, I would need to copy them file-by-file.

However, it occurred to me that I can modify my repository rule to generate the whole thing in the shape I want: Mardown files as well as the book.toml file.

This seems to work, so we can close this again! Thanks for the review and the question so I could look at this again.

[UebelAndre]

In Comprehensive Rust, I have 500+ Markdown files which come from an external repository — and if I understand correctly, I would need to copy them file-by-file.

However, it occurred to me that I can modify my repository rule to generate the whole thing in the shape I want: Mardown files as well as the book.toml file.

This seems to work, so we can close this again! Thanks for the review and the question so I could look at this again.

Happy to continue to discuss! I asked this question because I have already dealt with similar things; which we even use within this repo:

rules_rust/docs/BUILD.bazel

Lines 227 to 258 in 71bdd24

	stardoc(
	name = "cargo_settings",
	out = "src/cargo_settings.md",
	func_template = "cargo_settings_func.vm",
	input = "@rules_rust//cargo/settings:settings.bzl",
	deps = ["@rules_rust//cargo:bzl_lib"],
	)
	mdbook(
	name = "book",
	srcs = glob(["src/**/*.md"]) + [
	":cargo",
	":cargo_settings",
	":crate_universe_bzlmod",
	":crate_universe_workspace",
	":providers",
	":rust",
	":rust_bindgen",
	":rust_bzlmod",
	":rust_clippy",
	":rust_doc",
	":rust_fmt",
	":rust_mdbook",
	":rust_prost",
	":rust_pyo3",
	":rust_settings",
	":rust_toolchains",
	":rust_unpretty",
	":rust_wasm_bindgen",
	],
	book = "book.toml",
	)

So my expectation was always that sources for the mdbook need to have rlocationpaths that would result in a valid source tree. We currently have regression testing to ensure generated files work but was concerned external might not? I've also encountered issues where mdbook is happy to silently ignore certain inputs like plugins so your resulting book doesn't look like what you expect. Would love to see improvements there. Best of luck to you in building your big mdbook!