Skip to content

fix: 2026.8.8.1 的 #352 修复对真实 subos 完全不生效 —— wire format 读错了 - #377

Closed
speak-agent wants to merge 3 commits into
mainfrom
fix/fast-run-stale-cache-subos
Closed

fix: 2026.8.8.1 的 #352 修复对真实 subos 完全不生效 —— wire format 读错了#377
speak-agent wants to merge 3 commits into
mainfrom
fix/fast-run-stale-cache-subos

Conversation

@speak-agent

@speak-agent speak-agent commented Aug 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

2026.8.8.1 里的 #352 修复,对着真实的 subos 一个变量都不应用。 这个 PR 有三处修复,按严重度排列。

① wire format 我读错了 —— 功能完全不工作

盘上真实的 envs 是以 binding 为键的对象:

"envs": { "mesa@25.0.7.1": [ {"var":…,"op":…,"value":…}, … ], … }

我的 reader 期望的是数组 [{binding, decls}] ⇒ is_array() 为假 ⇒ 循环一次都不执行。对着一台 NVIDIA 机器上真实的 subos,发布版的行为是:

LIBGL_DRIVERS_PATH=(unset)
__EGL_VENDOR_LIBRARY_DIRS=(unset)
XDG_DATA_DIRS=/usr/share/ubuntu:…      ← 宿主的,没被碰过

10 个单测和 1 条 e2e 一路全绿,因为每一份 fixture 都是我按同一个臆想手写的。两边一致,两边都错。

根因不是笔误,是方法:用与解析器同源的理解去造 fixture,抓不出对 wire format 的误解;只有取自写入方的 fixture 能抓。所以新增 RealXlingsCapture —— 真实 xlings 输出的逐字副本,只调了换行宽度。

parser 现在是从 xlings 自己的 reader 誊写的,不是照着它建模。这顺带抓到第二处分歧:xlings 会丢弃 op 不是 set/prepend 的声明,而我原来任何 op 都收、非 set 一律当 prepend。比写入方更宽容的读取方,迟早会应用一个写入方本想拒绝的东西。

② 升级后复用升级前的缓存

快路径的 identity 是 profile / cache mode / resource 列表;fingerprint 校验拿缓存条目和它自己比。没有一项会注意到这条目是另一个 mcpp 写的,所以升级后的 mcpp 会一直复用升级前的构建。

reader 的注释声称它把旧缓存当 miss —— 那个检查从来没写过。注释描述了代码没有的行为,比漏掉更糟。

实测(2026.8.7.1 → 2026.8.8.1 真实升级):环境从没到达。修复后第 1 次 run 走全路径重建,之后快路径正常。

空的 subosDir 不能代表「早于这个字段」 —— 系统工具链合法地没有 subos —— 所以「这一行存在过没有」单独记录。

③ e2e 的 fixture 也是那个臆想格式

修好 reader 之后它正确地失败了,这正是这次修复可信的理由。注释里写明了这是 wire format 不是写法偏好 —— 臆想的那个形状看起来更整齐,一定会有人想「简化」回去。

验证

  • 单测 62/62,含 RealXlingsCapture
  • e2e 200_subos_env_reaches_program:冷路径 / 缓存快路径 / 旧缓存降级 / 「无声明时不凭空发明」四条
  • 对着真实 xlings install graphics 产出的 subos,三个变量全部到位,共享的 EGL vendor 目录去重成一条

Refs #352

@speak-agent
fix(run): an upgraded mcpp replayed a pre-upgrade cache with no subos…
378b69f 
… env

2026.8.8.1 taught `mcpp run` to hand a program its subos's environment, and
recorded which subos in the build cache so the fast path could do it too. What
it did not do is notice a cache entry written before that field existed. The
reader's own comment claimed it did -- "absent ⇒ the run fast path treats the
entry as a miss" -- and no such check was ever written. A comment describing
behaviour the code does not have is worse than the missing check.

The consequence is the whole #352 fix quietly not applying to anyone who
upgrades, and not just for one run: the fast path's identity is the profile,
the cache mode and the resource list, and its fingerprint check compares a
cached entry against ITSELF. Nothing there notices that a different mcpp wrote
the entry, so the pre-upgrade build is replayed until something else happens to
invalidate it.

Measured on a real upgrade from 2026.8.7.1 rather than reasoned about: build
with the old binary, run with the new one, and the environment never arrives.

An empty subosDir cannot stand in for "predates the field" -- a system
toolchain outside the xpkgs store legitimately has no subos -- so presence is
tracked separately. One rebuild after upgrading, and the field is there.

The e2e ages a cache by stripping the line, which is exactly what an older
mcpp's cache looks like, and asserts three things: the fast path is refused,
the rebuild applies the environment, and the field is recorded so the next run
does not repeat the work.
@speak-agent
fix(xlings): read the subos_info wire format xlings actually writes
56e580f 
2026.8.8.1 shipped a reader for a format that does not exist. `envs` is an
OBJECT keyed by binding:

    "envs": { "mesa@25.0.7.1": [ {"var":…,"op":…,"value":…}, … ], … }

The reader expected an array of {binding, decls} objects, so `is_array()` was
false, the loop never ran, and every variable came back unset. Against a real
subos on an NVIDIA host the released build applies nothing at all --
LIBGL_DRIVERS_PATH unset, __EGL_VENDOR_LIBRARY_DIRS unset, XDG_DATA_DIRS still
the host's. The whole #352 fix is inert.

Ten unit tests and an e2e passed the whole way, because every fixture was
hand-written in the same invented shape the parser expected. Both sides agreed
and both were wrong. A fixture composed from the same understanding as the
parser cannot catch a misunderstanding of the wire format; only one taken from
the writer can. RealXlingsCapture is that -- a verbatim copy of what a real
xlings wrote, reformatted for width and nothing else.

The parser is now transcribed from xlings's own reader rather than modelled on
it, which also caught a second divergence: xlings drops a declaration whose op
is neither "set" nor "prepend", and this accepted any op and treated everything
non-"set" as a prepend. A reader more permissive than its writer eventually
applies something the writer meant to reject.

Verified the way it should have been the first time: the fixed binary against
the subos a real `xlings install graphics` produced, with all three variables
arriving and the shared EGL vendor directory de-duplicated to one entry.
@speak-agent
test(e2e): the fixture was the invented wire format too
5a86eac 
The e2e wrote its subos manifest in the same shape the broken reader expected,
so it kept passing while the feature did nothing against a real subos. Fixing
the reader made it fail, which is the correct outcome and the reason the fix
is trustworthy.

The comment now says the structure is a wire format rather than a convenience,
because the invented shape is the tidier-looking of the two and someone will
want to "simplify" it back.
@speak-agent speak-agent changed the title fix(run): 升级后的 mcpp 会复用升级前的缓存,subos 环境静默不生效 fix: 2026.8.8.1 的 #352 修复对真实 subos 完全不生效 —— wire format 读错了 Aug 7, 2026
@speak-agent

speak-agent commented Aug 8, 2026

Copy link
Copy Markdown
Member Author

关闭,内容并入一个覆盖三份设计文档的跨仓实现。

本 PR 的两处修复不作废,会原样带进新 PR:

  • subos_info wire format 读错(对真实 subos 完全不生效)
  • 升级后复用升级前缓存

关闭的原因是范围:.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md 定下的架构(编译器当能力用、权威=subos runtime、删掉 fixup_gcc_specs)会改动同一批文件,分成两个 PR 只会让 review 看两遍同一片代码。

@speak-agent speak-agent closed this Aug 8, 2026
@speak-agent speak-agent mentioned this pull request Aug 8, 2026
Merged
Sunrisepeak pushed a commit that referenced this pull request Aug 8, 2026
@speak-agent
docs: the design doc this branch cites, plus what implementing it cor…
7e3fd66 
…rected

The doc lived only on the branch of the closed PR #377, so every reference to
it from code comments, docs, and the implementation plan pointed at a file the
repository did not contain.

New §9 records what landing it changed about the design itself. §9.1 is the
one that matters: the compatibility fallback described in §3.5 reads gcc's
specs and clang's cfg -- files the same change stops writing. On an existing
machine they are still there; on a fresh install they are not, and nothing
downstream says so. That asymmetry is why local verification was green while
CI was not, and it generalises past this change: when removing a mechanism,
enumerate who reads its output.
speak-agent added a commit that referenced this pull request Aug 8, 2026
@speak-agent
feat: 编译器是能力,不是配置 —— payload 版本权威 + specs 污染 + xlings pin (2026.8.8.2) (
fdad165 
#378)

一个 payload 交付两样可分离的东西:编译的**能力**,以及关于如何链接的**主张**。mcpp 要前者,后者自己给 —— 链接行才是构建决策该待的地方,因为它是逐次构建而变的那一个。四个看起来无关的缺陷收敛到这一条上。

设计:`.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md`(含 §9 实施回写)。替代已关闭的 #377。

## 1. 产物加载哪个 glibc,由权威决定,不再靠目录顺序

旧规则向目录问「那个 glibc」并取 `readdir` 的第一项。只装一个时它永远正确,所以从没有东西逼它正确 —— 而一条带 `xim:glibc@>=2.38` 的依赖就足以装进第二个。mcpp-index 上真实发生过:编译侧取 2.44,产物 interpreter(冻结在 gcc specs 里)仍是 2.39,二进制引用 `GLIBC_2.42` 却跑在没有它的运行时上,**报错落在与那条依赖无关的包上**。

解析顺序:`[xlings] subos` 自述 → 活动 subos 自述 → 兼容记录(specs / cfg / **PT_INTERP**)→ 单例 payload。计入工具链指纹(11 字段)。

两条判据是落地时被 CI 逼出来的,已写回设计:

- **没得选就不算猜。** 恰好一个 glibc payload 时没有选择可言,拒答等于拒答一个只有一个答案的问题。两个及以上才沉默 —— 那才是事故的形状。
- **化石不算权威。** `specs`/`.cfg`/`PT_INTERP` 都是过去某刻的记录,payload 被换掉时没人回头改。一条记录只有在它指的东西还在,才算数。subos 自己的声明同样适用。

把「查不到」也当成「有歧义」的代价是产物掉到宿主 loader —— 比猜错版本更糟,因为它连沙箱都出去了。

## 2. 不再改写 GCC 的 `specs`

旧重写用单路径 needle 配双路径 replacement,每个跑过它的 home 都漏下一条:一台开发机产出的**每个** gcc 产物里都带 **68 条**陈旧 `RUNPATH`,全指向已删除的 `mktemp` 目录。没有东西发现它们,因为一条死 RUNPATH 只多花一点搜索时间。

改为 `-specs=` 一份逐构建生成的干净文件(`-dumpspecs` 取**内建** `*link:`,去掉 loader/rpath)。不带 `+` 的 `-specs=` 是**替换**,payload 分毫未动。逐构建、不需写权限,继承来的只读 payload 也能用。

删掉 `*link:` 也删掉了它提供的东西,因此 loader 与每条 rpath 由 mcpp 显式写在链接行上 —— 包括**编译器自己的 `lib64`**(`libgcc_s.so.1` 在那)。

## 3. 落后于 pin 的 vendored xlings 会被替换 —— 但只在替换品确实更新时

一个 home 会永远留着第一次获取的 xlings(实测 2026.8.2.1 对 pin 2026.8.6.3)。`subos_info` 是 2026.8.5.1 才有的,所以那台机器上 subos 的自述一直被太旧的客户端丢弃,#352 的修复无论 mcpp 多新都无法生效。

修复的第一版直接删了重取,把 2026.8.2.1 换成系统的 **0.4.51** —— 更旧、且同样缺那个特性。现在先给替换品定价再动手。

## 4. `--sysroot` 的判据从「存在」改为「归属」

gcc 把 `--sysroot=<...>/.xlings/subos/default` 当字符串烙进去,而一台机器上有很多同名目录 —— 烙入的路径经常**存在、却属于另一个 checkout**。实测:本仓库的构建解析到了无关仓库下的 sysroot。归属先于可用性。

---

## 验证

| | |
|---|---|
| 单测 | **67/67**(新增 5 个文件、28 条断言) |
| 本地全量 e2e | **186 passed / 6 failed / 8 skipped**,6 个失败全在已记录基线内(03/09/20/33/59/98),其中 03/09/20/155/184 逐一用**已发布的 2026.8.6.3** 复现确认;`178` 反而转绿 |
| CI | **18/18 全绿** |
| 版本 | 2026.8.8.2;xlings pin → 2026.8.7.1(索引最新;上游 2026.8.8.1 未收录,pin 过去会让 CI 装不到) |

runtime binding 四条来源逐条实测:新建自述 subos、注入 `glibc@2.99`(不存在)、挪开 `specs` **且**把 gcc 自身 `PT_INTERP` 改指宿主、三条来源全打掉只剩单例。

## 这一轮真正的教训

**十轮 CI 抓出 11 个缺陷,其中 8 个在本地定向子集全绿时不可见。** 形状高度一致:修补落在控制流或本机状态到不了的地方。

- 兼容回落读的是**本 PR 让 mcpp 不再写**的文件 —— 每台老机器都还留着它,所以本机永远绿
- 归属谓词加在 `probe_sysroot` **第一步就 return 掉**的函数里
- `--no-cache` 删 `target/`,带走了我放进去的 spec;e2e 201 只跑暖构建,**那个专为此机制存在的测试看不见它**
- 我给 201 加的 PayloadFirst 分支,**把修复拿掉后照样通过** —— 本机 gcc 的 baked sysroot 指向另一个仓库且存在,把它拉回了 Sysroot 模式。守卫因此改成**指名模式**的单测,并逐一验证过红→绿
- 判据本身也错过:**「跑不跑得起来」在任何有宿主工具链的机器上都是 yes**,哪怕产物正在偷用宿主的库。换成「是否从 `/lib`、`/usr/lib` 加载任何库」
- 最后一条是**误报**:`ldd` 用宿主 loader 内省,总把 payload 解释器报成解析到宿主 —— 一个会拦下正确工作的检查,代价一样大

以及一个低级错误:`mcpp.toml` 的 `[toolchain] default` 被本地试验写成 llvm,`git add -A` 一起提交,五个 job 连带失败。**多个 job 同时开始失败,几乎总是指向一处共同输入被改动。**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@speak-agent