Skip to content

feat: generate compile database without building - #387

Merged
Sunrisepeak merged 17 commits into
mcpp-community:mainfrom
wellwei:codex/configure-only-cdb
Aug 10, 2026
Merged

feat: generate compile database without building#387
Sunrisepeak merged 17 commits into
mcpp-community:mainfrom
wellwei:codex/configure-only-cdb

Conversation

@wellwei

@wellwei wellwei commented Aug 8, 2026

Copy link
Copy Markdown
Member

Summary

  • add mcpp build --configure-only, reusing the real build plan and existing build selectors while skipping ordinary compilation, linking, BMI cache population, and build-success cache writes
  • include tests/**/*.cpp, matching [build].flags, and test dev-dependencies in the generated CDB; stage only std and already-cached dependency BMIs needed by language tooling
  • publish compile_commands.json atomically across platforms, preserve last-known-good files and CDB symlinks, and keep normal build/test publication failures non-fatal
  • document the trust and side-effect boundary: configure-only may still execute build.mcpp, resolve/install dependencies or toolchains, and update lock/resolution metadata

This is the independently agreed A part from #379. It intentionally does not add JSON/NDJSON, ide subcommands, snapshots, mcpp.wire, invalidatedBy, or new selector semantics. It does not close the RFC.

Core impact

  • shared test target discovery is extracted from mcpp test; existing list/build/run behavior remains covered by unit and E2E tests
  • the Ninja backend gains an opt-in requireCompileDatabase flag; default build/test behavior remains warning-only
  • CDB publication changes from direct truncation to same-directory temporary write plus atomic replacement; concurrent writers remain documented last-writer-wins

Test plan

  • fresh self-hosted mcpp build --no-color --no-cache on macOS ARM64
  • fresh mcpp test --no-color: 70 passed, 0 failed
  • tests/e2e/202_configure_only_cdb.sh: syntax-error source, test/dev-dependency flags, workspace fan-out, -p, no build artifacts/cache, publication failure preservation
  • adjacent E2E: 01, 18, 35, 76, 77, 90, 157, 159
  • unit coverage for test discovery, configure prerequisites, atomic replacement, unchanged mtime, symlink preservation, unreadable/failed destinations

Refs #379
Refs mcpp-community/mcpp-vscode#5

Comment thread .agents/docs/2026-08-08-configure-only-cdb-implementation-plan.md
wellwei and others added 13 commits August 10, 2026 11:28
… replace

- publish_compile_commands: treat a missing CDB (symlink_status type ==
  not_found) as the normal first-build case instead of a fatal error.
  is_symlink sets ec on missing paths on every standard library (generic
  ENOENT on libstdc++/libc++, system ERROR_FILE_NOT_FOUND on MSVC), so no
  compile_commands.json was ever written on fresh workspaces and all
  CDB-dependent e2e tests failed.
- close the existing-CDB ifstream before the atomic replace so Windows
  MoveFileExW can replace the destination (open handles cause Access
  denied / sharing violation).
- unit tests: escape Windows backslash paths in the entry() JSON helper,
  write the mtime fixture in binary mode so on-disk bytes match on
  Windows, and add a regression test for publishing with no prior CDB.
Windows CDBs store file paths with backslashes, so `grep 'src/main.cpp'`
never matched and the test failed on the Windows e2e shard once the CDB
was actually being generated. Match either separator with `[\/]`.
Windows CDBs store paths with JSON-escaped backslashes (src\\main.cpp),
so one-separator patterns still missed. Allow one or more separators.
@wellwei
wellwei force-pushed the codex/configure-only-cdb branch from 9e6c8f3 to c35f1b4 Compare August 10, 2026 06:03
wellwei and others added 3 commits August 10, 2026 14:23
…g at its graph

The backend writes build.ninja before it honors dryRun, so `--configure-only`
rewrites the very file the P0 fast path replays — and a configure plan's graph
is not a normal build's graph: it carries the test targets and dev-dependencies,
so its `default` line names the TEST binaries and does not contain the package's
own target at all.

`target/.build_cache` was left untouched, and the fast path decides freshness by
comparing build.ninja's mtime against the SOURCES, never against the graph. So on
any already-built project:

    mcpp build                  # default bin/app
    mcpp build --configure-only # default bin/smoke   <- same build.ninja
    mcpp build                  # fast path -> links bin/smoke, never bin/app,
                                #              prints `Finished dev in 0.04s`

which is the routine case, not a corner one: the whole point of the flag is that
an editor runs it continuously.

Drop the fast-path entry for the build dir whose graph was rewritten, before the
backend runs so a failed configure cannot leave the stale claim standing either.
Scoped to that one outputDir: other (target, profile, cache mode) triples own
different build dirs and keep their entries.

`write_build_cache`'s serializer is split out so the invalidation path rewrites
the file through the same spelling rather than a second one.

The e2e must NOT delete the built binary before the second build — a missing
output makes ninja fail in a way the fast path reads as a stale graph and falls
back to a full prepare, which hides the defect. Verified red (links `smoke`)
before the fix and green after.
@Sunrisepeak

Copy link
Copy Markdown
Member

Review:架构对,一个真缺陷已修并入本 PR

从架构 / 稳定性 / 简洁性 / 跨平台 / 可观测性 / 测试覆盖六个方向核过。结论是方案选对了:复用真实 prepare_build() + 真实 BuildPlan + Ninja dry-run,编译参数只有一条推导路径;把 run_tests() 的发现逻辑抽成 mcpp.build.test_targets 顺手把 fs::relative 换成 lexically_relative(前者解析符号链接,正是 #344 那类静默错位的来源);CDB 原子发布把「发布失败」和「丢掉上一份可用 CDB」解耦。这些都不需要改。

下面是实测发现的问题,已经在本 PR 上改完并验证。

一、真缺陷:configure-only 会把构建快路径指向自己的图(已修)

backend 在 opts.dryRun 早退之前就写了 build.ninjaninja_backend.cppm:1543),而 configure 的 plan 带着 test targets 和 dev-dependencies —— 它的 default 行是测试二进制,包自己的 target 根本不在图里。target/.build_cache 没动过,而快路径判新鲜只比 build.ninja 与源码的 mtime,从不看图本身。于是在任何已经构建过的工程上:

$ mcpp build                    # default bin/repro
$ mcpp build --configure-only   # default bin/smoke   <- 同一个 build.ninja
$ mcpp build
    Finished dev in 0.04s
$ ls target/*/*/bin
repro  smoke                     # 链了测试,没链 target,还报了成功

这不是边角情形:这个 flag 的全部意义就是编辑器会不停地跑它。

修法是在 backend 跑之前丢掉指向该构建目录的快路径条目(configure.cppm),失败的 configure 也不会留下过期声明。只作用于被重写的那一个 outputDir,别的 (target, profile, cache mode) 三元组各有自己的构建目录,实测 dev+release 两条条目只掉 dev 那条。write_build_cache 的序列化拆了出来,让失效路径走同一份拼写而不是第二份。

顺带记一条:mcpp test 有同形的老洞(mcpp build; mcpp test; mcpp build 同样会重放测试图)。那是既存缺陷、不同命令,没有夹带进本 PR,另开。

二、E2E 编号撞车(已修)

202_configure_only_cdb.sh 与 main 上的 202_machine_output_contract.sh 同号,且 main 已经排到 210。改名 211_configure_only_cdb.sh

三、新加的守卫差点是假绿(已修)

我给 e2e 补的快路径断言最初写成「先 rm 掉产物再 build」,在没有修复的二进制上也是绿的 —— 产物缺失会让 ninja 以「图过期」的样子失败,快路径于是回退到完整 prepare,恰好把被测缺陷盖住了。改成不删产物、直接断言「第二次 build 之后不得出现测试二进制」,并实测过 red(链出 smoke)→ green。脚本里写了注释说明为什么不能删。

另外把首次调用的 $MCPP 补上引号(其余调用都带;179_spaced_paths.sh 存在就是因为路径带空格是真会发生的),workspace fan-out 的报错补上 member 名(普通路径有 error: {member}: {msg},configure 分支原来把 member 丢了)。

四、补 CHANGELOG

feat: 按仓库惯例进 CHANGELOG。写在在飞的 [2026.8.10.1] 段里(最新 tag 还是 v2026.8.8.4,该段尚未发布),不构成发版。

五、核过没问题的部分

  • replace_file:POSIX rename / Windows MoveFileExW(MOVEFILE_REPLACE_EXISTING),从不先删目标;临时文件是 publishPath 的同目录兄弟,所以 rename 不跨文件系统;sharing violation 有界退避(50/150/450ms)。Windows 上「替换前先关掉读句柄」也处理了。
  • 符号链接形态的 CDB:解析到链接目标再替换,链接本身保住。
  • staged_std_compat_bmi_path() 在 registry 里对非 MSVC 一律走 clang 分支(pcm.cache),乍看会让 GCC 落错目录 —— 实际不可达:compatCommands 只对 MSVC/clang 非空(stdmod.cppm:342),GCC 的 plan.stdCompatBmiPath 恒为空。
  • cached BMI 的落盘名与 backend 的 bmi_path() 逐字一致(traits.bmiDir + ':'→'-' + traits.bmiExt)。
  • publish_compile_commands 里 merge 前的 existing == fresh 快退不是死代码:单条目工程上 plan 序就是排序序,会命中,省一次 parse+merge;且该分支下 existing 恒等于 plan 条目集,跳过 prune 不会漏删。

六、验证

本机 x86_64-linux-gnu:unit 76 个测试二进制全过;e2e 211(new) / 01 / 76 / 77 / 157 / 159 / 90 全 OK。red→green 见上。

🤖 Generated with Claude Code

…uced

Windows links `fastpath.exe`; the new fast-path section hardcoded the POSIX
spelling and failed at the baseline check before reaching the assertion it
exists for (e2e 2/2 windows). The suffix is a host constant, so take it off the
artifact `find` actually returned rather than branching on the platform.
@Sunrisepeak
Sunrisepeak merged commit 3f237ed into mcpp-community:main Aug 10, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants