test(upgrade): real-machine 0.4.69 → latest upgrade simulation - #434
Closed
Sunrisepeak wants to merge 5 commits into
Closed
test(upgrade): real-machine 0.4.69 → latest upgrade simulation#434Sunrisepeak wants to merge 5 commits into
Sunrisepeak wants to merge 5 commits into
Conversation
Every upgrade check so far has run against a synthesized legacy home. That keeps missing the failures that matter, because those come from what 0.4.69 actually wrote -- recipe versions that have since changed, host detection that used to be wrong, payload layouts that have since moved. A field report on 2026-07-28 showed an llvm@20.1.7 recorded with backslash paths and .exe aliases on Linux. The upgraded client could neither reinstall it nor remove it. No synthesized fixture had produced that shape. This installs the real 0.4.69 release into an isolated HOME, installs the core packages through it, runs the real self update, then drives the commands a user would drive afterwards -- recording every exit code rather than aborting on the first failure. No product change: this only observes.
This branch exists to produce one table. The simulation installs a real 0.4.69 release, populates it with real packages, upgrades it and then drives the upgraded client -- several GB and many minutes per platform. The normal build/test matrix running alongside it adds contention and noise to a PR that is never going to be merged, so it is removed here rather than gated. The harness is made portable in the process: - platform/arch/asset/exe derived from uname once, at the top - Windows keeps its inherited environment (env -i loses SYSTEMROOT and processes stop starting for reasons unrelated to xlings); only HOME, USERPROFILE and PATH are overridden, and XLINGS_HOME is unset - millisecond clock via python3: date +%s%3N is a GNU extension that prints a literal %3N on macOS instead of failing - gcc is linux-only in the index, so the package set is per-platform; llvm is published for all three and carries the two-version role everywhere And it stops trusting exit codes. The first run reported 23/26 green, including "remove llvm" at 166ms and "install llvm" at 333ms -- durations that cannot contain the work they claim. rc=0 from this CLI means nothing raised, not that anything happened, so state changes are asserted separately (is it still listed, does the shim report the version that was switched to). The job stays green when the simulation finds blockers: blockers are the output, not the failure. Count and full step table go to the job summary, transcript and per-step logs upload as an artifact.
Sunrisepeak
force-pushed
the
test/legacy-upgrade-simulation
branch
from
July 27, 2026 20:38
b08fd24 to
5118116
Compare
The first 3-platform run reported 19 blockers on each of macOS and Windows.
None were about xlings: every step failed in under 100ms because the harness
never started. Two causes.
macOS ships bash 3.2, where expanding an EMPTY array under `set -u` is itself
an unbound-variable error -- so "${PROXY_ENV[@]}" killed every invocation on a
runner that has no proxy set. The array is now seeded with one inert entry.
On Windows the release is a .zip and unzip has no --strip-components, so the
extract lifted the contents of the top-level directory with `mv "$top"/*`.
That glob skips dotfiles, and the release package's marker is `.xlings.json`
-- so `self install` reported "cannot detect source package directory" with
the binary sitting right there. The extracted directory is now used in place.
Also: a bootstrap that produces no client binary stops the run instead of
letting the remaining phases file 15 rc=127 "blockers". A findings table that
is mostly the harness failing is a table nobody reads. Blockers keep the job
green; a broken harness now makes it red, which is the right way round.
What the simulation actually says, and what to build from it. Two of the three axes already hold and were measured, not assumed: nothing that worked stops working (gcc 15.1.0 <-> 16.1.0 still switches, confirmed through the shim's own --version), and the upgrade takes 3.8s on a runner / 16.8s on a home connection. The gap is entirely the third axis -- the user has to know an upgrade exists. Seven findings, of which three reproduce on linux, macOS and Windows alike: - self update upgrades the binary and leaves .xlings.json:version at the old value forever -- and that field is what any "you are behind" notification would compare against - a package with a non-latest version installed reports as NOT installed (catalog computes installedness against latest only), so llvm@20.1.7 is installed, active, running, and invisible to list/info - install <bare> pins to the active version, so the same binary says "installed no" and "20.1.7 is already installed" about the same package in the same second -- two resolution rules, neither wrong under its own - nothing moves a package forward: install pins, self update is client-only, nothing reports a newer version exists - self doctor exits 1 on a freshly upgraded, untouched home, over a headers- only package classified as a broken payload, with a remedy that reinstalls the same payload and reproduces the same finding The shape underneath: three stores answer "what is installed" and each command asks whichever one it happens to reach for. So the ordering matters -- fixing the notification first produces a notification that is wrong on upgraded machines about packages it thinks are not installed. D1 -> D3 -> D2. The field dead end (llvm recorded with backslash paths and .exe aliases on Linux, neither reinstallable nor removable) did NOT reproduce: 0.4.69 plus the current index writes clean records. Reproducing it needs the old index too, which is the next scenario. Its provenance cannot be recreated at all -- detect_platform() is a compile-time #if -- which makes it a recovery problem, not a prevention one. Deliberately not fixed here.
It should -- but not as an "upgrade" module. The tempting shape owns the new behaviour: check for a newer release, print the notice, offer `xlings upgrade`. That shape fails predictably. It becomes a FOURTH consumer of three stores that already disagree, and inherits every contradiction: a notification built on today's `installed` would announce upgrades for packages it simultaneously reports as not installed. The module worth building is the missing one, not the requested one -- a single authority for "what is installed, and what is current". That question has 67 direct askers today (14 Config::versions(), 23 match.installed, 21 workspace, 9 get_active_version), each reaching for whichever store is nearest. Every finding in the report is that module being absent: B2 is `installed` computed against latest only, B3/B5 are `resolved` having two definitions, B1 is `client_version` read from a file nobody updates. With it, the notification and `xlings upgrade` are thin consumers -- a comparison and a plan -- rather than subsystems with their own view. Two things stay out on purpose: doctor's misclassification of headers-only packages is a classification bug in one file and should not wait behind an architectural change, and orphan shims belong to removal. Also records how to land it without a big bang: build it as a pure reader first (no behaviour change, fully unit-testable), point info/list at it, then install/remove, then the self update write, then build on top. Each step flips a named assertion in the simulation -- the module is done when the blocker count is zero, not when it compiles.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Temporary/draft. Observation + design only — no product change in this PR.
Result
Three platforms, full simulation, same three findings on all of them:
self updateleaves the recorded version atv0.4.69llvm@20.1.7invisible after removing the latest versioninstall llvmdoes not bring22.1.8backself doctorexits 1 on a healthy freshly-upgraded homeDesign doc:
.agents/docs/2026-07-28-seamless-upgrade-design.md.Why a real simulation
Every seamless-upgrade check so far ran against a synthesized legacy home — a
state file hand-written to look like 0.4.69's output. That keeps missing the
failures that matter, because those come from what 0.4.69 actually wrote.
So: install the real 0.4.69 release into an isolated
HOME, install the realpackages through it (
gcc@15.1.0+gcc@16.1.0,llvm@20.1.7+llvm@22.1.8,mcpp), run the realxlings self update, then drive what a user drives thenext day.
The findings
Two of the three axes already hold, measured not assumed. Nothing that
worked stops working —
gcc 15.1.0 ↔ 16.1.0still switches, confirmed throughthe shim's own
--version, on a home built by 0.4.69. Andself updatecosts3.8 s on a runner / 16.8 s on a home connection.
The gap is entirely 不需要动作:
self updateupgrades the binary and leaves.xlings.json:versionatv0.4.69forever. That field is written only byself install(
install.cppm:577) and read back by it (:425) — and it is exactly what a"you are behind" notification would compare against.
A package with a non-latest version installed reports as not installed.
catalog.cppm:333tests<store>/<pkg>/<latest>. So withllvm@20.1.7installed, active and running:
info llvm→installed no, andlist --allomits it entirely.
install <bare>pins to the active version. Composed with the above, thesame binary says both of these in the same second:
Neither is wrong under its own resolution rule. There are two rules.
Nothing moves a package forward.
installpins,self updateisclient-only, nothing reports that a newer version exists (Nothing ever tells a user a newer xlings exists #425).
self doctorexits 1 on a freshly-upgraded, untouched home, overlinux-headers@5.11.1— a headers-only package classified as a brokenpayload, with
→ run xlings install linux-headers@5.11.1as the remedy,which reinstalls the same payload and reproduces the same finding. Doctor
already has the right concept (
ⓘ release anchor) and applies it toglibcand
zlibin the same report.Underneath, one shape: three stores answer "what is installed", and each
command asks whichever one it reaches for. Which is why order matters —
fixing the notification first yields a notification that is wrong on upgraded
machines about packages it thinks are not installed.
The field dead end did NOT reproduce
The 2026-07-28 report —
llvm@20.1.7recorded with backslash paths and.exealiases on Linux, neither reinstallable (
xvm-legacy-payload-mismatch) norremovable (
recipe removal target is outside the owned selection) — does notoccur with 0.4.69 + the current index. 0.4.69 on Linux writes clean
forward-slash paths and non-
.exealiases. Reproducing it needs the oldindex too; that is the next scenario.
Its provenance cannot be recreated by today's code at all:
detect_platform()is a compile-time
#if defined(__linux__), so a Linux binary cannot select theWindows
xpmtable. That makes it a recovery problem, not a preventionone. Deliberately not fixed here.
Harness notes
set -e— a failing step is the data; the first blocker must not hidethe rest
remove llvmat 166 ms andinstall llvmat 333 ms — durations that cannotcontain the work they claim.
rc=0means nothing raised. State changes areasserted separately.
XLINGS_HOMEis never set, so home derivation is exercised rather thanbypassed
Windows, all of them the harness failing to start (bash 3.2 treating an
empty array expansion as unbound under
set -u;mv "$top"/*skipping therelease package's
.xlings.jsonmarker). Both fixed. A broken bootstrap nowstops the run instead of filing 15 rc=127 findings.
broken harness makes it red
CI on this branch
Every other workflow is deleted here on purpose: this branch exists to produce
one table, and the normal matrix only adds contention to a PR that will not be
merged.