From c0b9b8b4b57034c053b89d5538f413440d888c31 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sun, 9 Aug 2026 02:00:26 +0800 Subject: [PATCH 1/3] =?UTF-8?q?ci:=20=E5=88=B7=E6=96=B0=E8=AE=A1=E6=97=B6?= =?UTF-8?q?=E8=A1=A8=E5=B9=B6=E6=8A=8A=20linux=20=E5=88=86=E7=89=87?= =?UTF-8?q?=E4=B8=8A=E9=99=90=E6=8A=AC=E5=88=B0=204=20=E2=80=94=E2=80=94?= =?UTF-8?q?=203=20=E7=89=87=E5=B7=B2=E7=BB=8F=E8=A3=85=E4=B8=8D=E4=B8=8B?= =?UTF-8?q?=E4=BA=86?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `linux default 0/3` 在 run 31266814148 被 90 分钟上限砍掉,成员全绿,死在 cache/artifact 的 post 步骤。查下来既不是冷缓存也不是 runner 争抢,是工作量 真的超了,而**陈旧的计时表把这件事盖住了**。 ## 计时表漏了两个重家伙 `tests/member-timings.tsv` 停在 62 行 linux,而 workspace 已经有 67 个成员。 `plan_shards` 对表里没有的成员按**中位数**计价,于是: mysql-connector-cpp 实测 881s 被当成 ~60s libmysqlclient 实测 329s 被当成 ~60s 两个加起来 20 分钟的工作量,被当轻的塞进了已经扛着 grpc-codegen 的 shard 0。 这两个数是从 run 31266814148 的 `linux default 0/3` 步骤耗时里量的 —— 那一片 正好被砍,timings artifact 从没上传,而它们又正是导致溢出的成员。这个先有鸡还 是先有蛋只能手工破一次。表的其余部分来自 run 31260545520 的 member-timings artifact(顺带补上 cli11 / cmdline / llmapi,并修正 curl 26→264s、 eui-neo-sdl2 119→591s)。 ## 刷完表才看清:3 片本来就不够 刷新后 linux 总量 15891s = 265 分钟。按当前表模拟最慢分片: 3 片 98 分 ← 超 90 分钟 job 上限 4 片 74 分 5 片 60 分 `shards_for` 的公式 `secs / 4200 + 1` 对这个总量算出来就是 4,一直被 `cap 3` 压回 3。所以这不是新问题被引入,是旧上限被工作量长过去了 —— 旧表让 总量看起来只有 13570s,刚好还压得住。 ## 上限 3 -> 4 旧注释给这个上限的理由是"linux 实测并发 3,第 4 片会排队"。这个前提已经两头 失效:工作量长大了,而且自 #184 起 linux 每轮发 2 x N 个 job,任何 N 都会排队。 排队是对的取舍 —— 背靠背跑完的分片仍然完成,超过上限的分片不会。 代价:linux 每轮 8 个 job(原 6),模拟确认 `.shards` 读作 4、四片覆盖 67/67。 同时把那段容量注释改成现在为真的样子,并写明"表要保持新鲜"不是打扫卫生: 一个没计时的成员按中位数打包,一个重的新成员就会像轻的一样被塞进任何地方。 --- .github/workflows/validate.yml | 48 +++-- tests/member-timings.tsv | 343 +++++++++++++++++---------------- 2 files changed, 216 insertions(+), 175 deletions(-) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 10bef6e..ea32bff 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -441,18 +441,31 @@ jobs: # platform and take one shard per ~70 minutes, which leaves ~20 # minutes of headroom under the 90-minute cap for a cold cache. # - # The caps are where runner concurrency comes back in: linux 3 (the - # measured concurrency — a 4th shard would queue), macOS 2, windows - # 2. Past those, more shards buy fit that is already there and pay - # another checkout + mcpp download + cache restore. + # The caps are where runner concurrency comes back in: linux 4, + # macOS 2, windows 2. Past those, more shards buy fit that is + # already there and pay another checkout + mcpp download + cache + # restore. # - # Full run, from the table: linux 13570s over 3 -> ~75min/shard - # (observed slowest 77), macOS 6922s over 2 -> ~57, windows 8043s - # over 2 -> ~67. linux is the tight one — its cap binds before the - # ~70-minute target does, so it is the first place to look if a - # cold full run starts brushing 90 again. The levers, in order: - # raise the linux cap to 4 (costs a queued runner), then the job + # linux was 3 — "the measured concurrency, a 4th shard would queue". + # That cap outlived its premise twice over: the workspace grew, and + # the second toolchain leg means linux emits 2 x N jobs, so it + # queues at any N. It is now 4, which is what the formula asks for. + # + # Full run, from the table: linux 15891s over 4 -> ~74min slowest + # shard, macOS 6922s over 2 -> ~57, windows 8043s over 2 -> ~67. + # + # linux is still the tight one, and the number to watch is the + # SLOWEST shard against the 90-minute job cap, not the average. + # Simulated on the current table: 3-way 98min, 4-way 74, 5-way 60. + # The levers, in order: raise the linux cap to 5, then the job # timeout. + # + # Keep tests/member-timings.tsv fresh — that is not housekeeping. An + # untimed member is priced at the MEDIAN, so a heavy newcomer packs + # like a trivial one. `mysql-connector-cpp` (881s) and + # `libmysqlclient` (329s) landed on shard 0 next to grpc-codegen + # exactly that way and pushed it past the cap, and the stale table + # then hid the overflow by under-counting the total. shards_for() { # platform cap -> shard count local secs secs=$(XPLAT="$1" lua5.4 -e ' @@ -472,7 +485,20 @@ jobs: echo "$1 work: ${secs}s (measured) -> $n shard(s)" >&2 echo "$n" } - ln=$(shards_for linux 3) + # linux 3 -> 4. The cap was the binding constraint, not the ~70min + # target: `secs / 4200 + 1` asks for 4 on the refreshed table + # (15891s) and has been getting clamped back to 3. Simulated on that + # table, slowest shard: 3-way 98min (OVER the 90min job cap — and + # `linux default 0/3` was in fact cancelled at 90min on run + # 31266814148), 4-way 74min, 5-way 60min. 4 is the formula's own + # answer and leaves 16 minutes for a cold cache; 5 is there if that + # stops being enough. + # + # This does cross the concurrency line the old comment drew: with two + # toolchain legs linux now emits 8 jobs against a measured runner + # concurrency of 3, so shards queue. Queuing is the right trade — + # back-to-back shards still finish, a shard over the cap does not. + ln=$(shards_for linux 4) mn=$(shards_for macos 2) wn=$(shards_for windows 2) { diff --git a/tests/member-timings.tsv b/tests/member-timings.tsv index 8c844db..6013289 100644 --- a/tests/member-timings.tsv +++ b/tests/member-timings.tsv @@ -1,188 +1,203 @@ -linux abseil 111 -linux archive 49 -linux asio-module 37 -linux asio-ssl 49 -linux boost-ext.ut 30 -linux build-mcpp 28 -linux c-ares 28 -linux catch2 81 -linux catch2-main 53 -linux catch2-v2 13 +# \t\t — from run 31260545520 +# refresh: download the member-timings artifact and replace this file +# libmysqlclient / mysql-connector-cpp: measured from run 31266814148's +# linux default 0/3 step durations — that shard was cancelled at the 90m +# cap so its timings artifact never uploaded, and these two are exactly +# the members whose absence caused the overflow. Chicken-and-egg broken by hand, once. +linux abseil 96 +linux archive 28 +linux asio-module 15 +linux asio-ssl 175 +linux boost-ext.ut 4 +linux build-mcpp 1 +linux c-ares 5 +linux catch2 59 +linux catch2-main 57 +linux catch2-v2 14 linux catch2-v2-main 15 -linux cjson 2 -linux core 67 -linux curl 26 -linux eigen 25 -linux eui-neo 151 -linux eui-neo-app-main 140 -linux eui-neo-markdown 143 -linux eui-neo-sdl2 119 -linux eui-neo-vulkan 154 -linux eui-neo-window 128 -linux ffmpeg 288 -linux ffmpeg-module 343 -linux fmtlib.fmt 5 -linux freetype 10 -linux glad 2 -linux godot-cpp 573 -linux godot-cpp-module 549 -linux godot-cpp-module-v10 392 +linux cjson 19 +linux cli11 8 +linux cmdline 6 +linux core 78 +linux curl 264 +linux eigen 24 +linux eui-neo 43 +linux eui-neo-app-main 48 +linux eui-neo-markdown 144 +linux eui-neo-sdl2 591 +linux eui-neo-vulkan 143 +linux eui-neo-window 143 +linux ffmpeg 296 +linux ffmpeg-module 277 +linux fmtlib.fmt 4 +linux freetype 11 +linux glad 3 +linux godot-cpp 564 +linux godot-cpp-module 627 +linux godot-cpp-module-v10 568 linux godot-cpp-v10 504 -linux grpc-codegen 3563 -linux grpc-module 1701 -linux gui-stack 81 -linux imgui 6 -linux imgui-module 91 -linux imgui-window 91 -linux libpng 6 -linux llamacpp 101 -linux llamacpp-metal 0 +linux grpc-codegen 3463 +linux grpc-module 1771 +linux gui-stack 128 +linux imgui 7 +linux imgui-module 106 +linux imgui-window 7 +linux libmysqlclient 329 +linux libpng 7 +linux llamacpp 132 +linux llamacpp-metal 21 +linux llmapi 19 linux magic_enum 4 linux marzer.tomlplusplus 7 -linux md4c 3 +linux md4c 23 +linux mysql-connector-cpp 881 linux nlohmann.json 10 linux openblas 0 -linux opencv-module 602 -linux opencv-module-dnn 741 -linux opencv-module-unifont 671 -linux openssl 0 -linux protobuf 259 -linux protobuf-gzip 158 -linux protobuf-protoc 923 -linux protobuf-upb 263 -linux re2 15 -linux sdl2 63 +linux opencv-module 649 +linux opencv-module-dnn 875 +linux opencv-module-unifont 655 +linux openssl 166 +linux protobuf 175 +linux protobuf-gzip 267 +linux protobuf-protoc 885 +linux protobuf-upb 179 +linux re2 16 +linux sdl2 108 linux spdlog 9 linux spdlog-compiled 13 -linux tinyhttps 13 -linux tray 3 -linux vulkan 13 -linux websocket 19 -linux websocket-features 23 -linux yyjson 3 -macos abseil 75 -macos archive 49 -macos asio-module 30 -macos asio-ssl 78 -macos boost-ext.ut 27 -macos build-mcpp 33 -macos c-ares 27 -macos catch2 58 -macos catch2-main 28 -macos catch2-v2 10 +linux tinyhttps 11 +linux tray 2 +linux vulkan 76 +linux websocket 20 +linux websocket-features 26 +linux yyjson 10 +macos abseil 48 +macos archive 21 +macos asio-module 10 +macos asio-ssl 65 +macos boost-ext.ut 4 +macos build-mcpp 3 +macos c-ares 5 +macos catch2 24 +macos catch2-main 27 +macos catch2-v2 7 macos catch2-v2-main 9 macos cjson 3 -macos core 58 -macos curl 103 -macos eigen 7 -macos eui-neo 52 -macos eui-neo-app-main 57 -macos eui-neo-markdown 76 -macos eui-neo-sdl2 142 +macos cli11 5 +macos cmdline 4 +macos core 44 +macos curl 66 +macos eigen 8 +macos eui-neo 47 +macos eui-neo-app-main 49 +macos eui-neo-markdown 64 +macos eui-neo-sdl2 138 macos eui-neo-vulkan 59 -macos eui-neo-window 53 -macos ffmpeg 148 -macos ffmpeg-module 130 -macos fmtlib.fmt 4 -macos freetype 12 -macos glad 3 -macos godot-cpp 333 -macos godot-cpp-module 256 -macos godot-cpp-module-v10 215 -macos godot-cpp-v10 230 -macos grpc-codegen 1715 -macos grpc-module 880 -macos gui-stack 1 +macos eui-neo-window 48 +macos ffmpeg 43 +macos ffmpeg-module 107 +macos fmtlib.fmt 3 +macos freetype 10 +macos glad 5 +macos godot-cpp 5 +macos godot-cpp-module 276 +macos godot-cpp-module-v10 195 +macos godot-cpp-v10 206 +macos grpc-codegen 1156 +macos grpc-module 719 +macos gui-stack 25 macos imgui 4 -macos imgui-module 1 -macos imgui-window 1 -macos libpng 7 -macos llamacpp 66 -macos llamacpp-metal 75 -macos magic_enum 3 -macos marzer.tomlplusplus 5 -macos md4c 4 -macos nlohmann.json 9 -macos openblas 1 -macos opencv-module 198 -macos opencv-module-dnn 399 -macos opencv-module-unifont 304 -macos openssl 1 -macos protobuf 100 -macos protobuf-gzip 87 -macos protobuf-protoc 458 -macos protobuf-upb 143 +macos imgui-module 21 +macos imgui-window 0 +macos libpng 6 +macos llamacpp 63 +macos llamacpp-metal 71 +macos llmapi 16 +macos magic_enum 4 +macos marzer.tomlplusplus 6 +macos md4c 3 +macos nlohmann.json 8 +macos openblas 0 +macos opencv-module 223 +macos opencv-module-dnn 325 +macos opencv-module-unifont 164 +macos openssl 83 +macos protobuf 71 +macos protobuf-gzip 97 +macos protobuf-protoc 314 +macos protobuf-upb 66 macos re2 8 -macos sdl2 22 -macos spdlog 5 +macos sdl2 23 +macos spdlog 4 macos spdlog-compiled 6 -macos tinyhttps 13 -macos tray 4 -macos vulkan 7 -macos websocket 12 -macos websocket-features 12 -macos yyjson 6 -# — measured, run 31034885938 -# refresh: download the member-timings artifact from a full run and replace this file -windows abseil 133 -windows archive 76 -windows asio-module 37 -windows asio-ssl 24 -windows boost-ext.ut 40 -windows build-mcpp 27 -windows c-ares 33 -windows catch2 83 -windows catch2-main 62 -windows catch2-v2 12 -windows catch2-v2-main 12 +macos tinyhttps 14 +macos tray 3 +macos vulkan 8 +macos websocket 10 +macos websocket-features 11 +macos yyjson 5 +windows abseil 103 +windows archive 62 +windows asio-module 15 +windows asio-ssl 0 +windows boost-ext.ut 4 +windows build-mcpp 4 +windows c-ares 16 +windows catch2 74 +windows catch2-main 69 +windows catch2-v2 10 +windows catch2-v2-main 11 windows cjson 3 -windows core 98 -windows curl 31 -windows eigen 10 -windows eui-neo 93 -windows eui-neo-app-main 119 -windows eui-neo-markdown 84 -windows eui-neo-sdl2 150 -windows eui-neo-vulkan 84 -windows eui-neo-window 74 -windows ffmpeg 454 -windows ffmpeg-module 532 -windows fmtlib.fmt 5 -windows freetype 15 -windows glad 2 -windows godot-cpp 742 -windows godot-cpp-module 636 -windows godot-cpp-module-v10 684 -windows godot-cpp-v10 566 -windows grpc-codegen 31 -windows grpc-module 1 -windows gui-stack 1 +windows cli11 8 +windows cmdline 5 +windows core 100 +windows curl 27 +windows eigen 12 +windows eui-neo 75 +windows eui-neo-app-main 88 +windows eui-neo-markdown 86 +windows eui-neo-sdl2 123 +windows eui-neo-vulkan 86 +windows eui-neo-window 83 +windows ffmpeg 693 +windows ffmpeg-module 526 +windows fmtlib.fmt 6 +windows freetype 13 +windows glad 4 +windows godot-cpp 7 +windows godot-cpp-module 707 +windows godot-cpp-module-v10 657 +windows godot-cpp-v10 655 +windows grpc-codegen 10 +windows grpc-module 0 +windows gui-stack 19 windows imgui 6 -windows imgui-module 0 +windows imgui-module 24 windows imgui-window 0 windows libpng 10 -windows llamacpp 125 +windows llamacpp 126 windows llamacpp-metal 1 +windows llmapi 16 windows magic_enum 4 -windows marzer.tomlplusplus 6 +windows marzer.tomlplusplus 7 windows md4c 3 windows nlohmann.json 10 -windows openblas 11 -windows opencv-module 747 -windows opencv-module-dnn 1174 -windows opencv-module-unifont 1 +windows openblas 10 +windows opencv-module 868 +windows opencv-module-dnn 828 +windows opencv-module-unifont 7 windows openssl 1 -windows protobuf 215 -windows protobuf-gzip 141 -windows protobuf-protoc 227 -windows protobuf-upb 251 -windows re2 15 -windows sdl2 46 -windows spdlog 6 -windows spdlog-compiled 14 -windows tinyhttps 14 +windows protobuf 139 +windows protobuf-gzip 238 +windows protobuf-protoc 152 +windows protobuf-upb 151 +windows re2 17 +windows sdl2 47 +windows spdlog 22 +windows spdlog-compiled 12 +windows tinyhttps 16 windows tray 2 -windows vulkan 7 -windows websocket 24 -windows websocket-features 26 -windows yyjson 2 +windows vulkan 5 +windows websocket 25 +windows websocket-features 29 +windows yyjson 4 From 35f95c7e10b59c297f3d6c76dd984d8795541490 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sun, 9 Aug 2026 02:31:54 +0800 Subject: [PATCH 2/3] =?UTF-8?q?ci:=20=E6=94=B9=E8=AE=A1=E6=97=B6=E8=A1=A8?= =?UTF-8?q?=E4=B8=8D=E8=AF=A5=E8=A7=A6=E5=8F=91=E5=85=A8=E9=87=8F=E8=BF=90?= =?UTF-8?q?=E8=A1=8C?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit tests/member-timings.tsv 在 select 的 case 里一条都不匹配 —— 不是 tests/*.sh, 不是 tests/examples/*,不是 pkgs/*.lua,也不在忽略清单里 —— 于是落进 `*) full "unclassified change"`。 这张表决定活儿怎么在分片间分配,从不决定构建什么:没有任何成员的结果会因为 一个实测数字变了而改变。让它触发全量,是这份 workflow 里"花最大代价测试零 东西"的做法,而下一次真正的全量运行本来就会读到新数字。 代价写在注释里了:这样一来这个文件对 CI 不可见,坏行是静默的,而 plan_shards 对解析不了的东西按中位数计价 —— 正是刚刚撑爆一个分片的那个失效 模式。真要咬到就在 lint 里加校验。 --- .github/workflows/validate.yml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index ea32bff..9e029f9 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -380,6 +380,18 @@ jobs: # changes with it and the rule above selects the right members. # So a tools/ edit alone selects nothing rather than forcing a # full workspace rebuild. + # The timing table decides how work is DISTRIBUTED across + # shards, never what gets built — no member's result can change + # because a measured number moved. It used to fall through to + # the `*)` catch-all below and force a full run, which is the + # most expensive way in this workflow to test nothing: the next + # full run reads the new numbers anyway. + # + # Note this makes the file invisible to CI. A malformed row is + # then silent, and `plan_shards` prices anything it cannot parse + # at the median — the exact failure mode that overflowed a shard + # here. Guard it in `lint` if that ever bites. + tests/member-timings.tsv) : ;; *.md|docs/*|.agents/*|.github/*|tools/*) : ;; *) full "unclassified change: $f" ;; esac From 4904d147d846e9da2202ea16edaceb9d33e13301 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sun, 9 Aug 2026 02:42:46 +0800 Subject: [PATCH 3/3] =?UTF-8?q?ci:=20=E8=AE=A1=E6=97=B6=E8=A1=A8=E5=8F=AA?= =?UTF-8?q?=E7=95=99=E6=AF=8F=E5=B9=B3=E5=8F=B0=E6=9C=80=E9=87=8D=E7=9A=84?= =?UTF-8?q?=2010=20=E4=B8=AA,=E5=8A=A0=E6=B5=8B=E8=AF=95=E4=B8=8D=E5=86=8D?= =?UTF-8?q?=E9=9C=80=E8=A6=81=E5=8A=A8=E5=AE=83?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 这张表之前列出每个成员,于是每加一个测试它就过期一次 —— 而这次撑爆分片的 正是"表里没有的成员按中位数计价":mysql-connector-cpp 实测 881s,被当成 60s。 但打包决策其实只由重的那几个做出。linux 上 top10 占总量的 69%,其余 57 个平 均 86s、中位 24s。把它们从表里拿掉,最慢分片一分钟都不动: 完整表 67 行,4 片 74 / 64 / 63 / 63 最慢 74 分 top10 表,4 片 74 / 73 / 60 / 58 最慢 74 分 ## 兜底价必须是固定常数,不能再取中位数 直接缩表会当场炸:表里只剩 10 个重的,中位数就变成"第 5 重的那个",在 linux 上是 875s,于是 57 个小成员每个都按 875s 计价 —— top10 表 + 中位数兜底 90 / 48 / 81 / 46 最慢 90 分 ← 正好撞 job 上限 改成固定 90s(未计时成员的实测均值 86s 取整)。这个值不吃调参:兜底价从 30s 到 300s,最慢分片始终在 72–84 分之间,全都在 90 以下。这种不敏感正是"表可以 长期不动"的依据。 ## shards_for 也得跟着改,否则片数会掉回去 它原本直接把表里的行加起来当工作量。表一缩,linux 总量从 15891s 读成 10965s → 3 片,而 3 片在这张表下最慢 107 分,直接超时。 给 plan_shards 加一个 ` 0 0` 模式返回总量,shards_for 改调它。这样 "有哪些成员"和"未计时的算多少钱"只有一份定义,而不是 yaml 和 lua 各一份等着 漂移。估算精度:linux 16095s vs 实测 15891s,+1.3%。macOS / windows 的成员比 默认价便宜,总量偏高 —— 两者本来就顶着 cap 2,而且偏多分片是安全方向。 ## timings job 同步产出 top10 否则下次刷新又胖回 197 行。 端到端复核:总量 16095s → 4 片,四片覆盖 67/67,最慢 74 分。 --- .github/workflows/validate.yml | 69 +++++++---- tests/member-timings.tsv | 216 +++++---------------------------- tests/plan_shards.lua | 65 ++++++++-- 3 files changed, 124 insertions(+), 226 deletions(-) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 9e029f9..14b65b2 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -478,20 +478,25 @@ jobs: # `libmysqlclient` (329s) landed on shard 0 next to grpc-codegen # exactly that way and pushed it past the cap, and the stale table # then hid the overflow by under-counting the total. + # `plan_shards.lua 0 0` reports the platform's total cost. + # + # This used to sum the timing table's rows inline, which was the same + # answer only while the table listed every member. It now lists just + # the heavy ones, so summing rows measures a fraction of the work: + # 10965s instead of 16095s on linux, i.e. 3 shards where 4 are + # needed, i.e. a slowest shard of 107 minutes against a 90-minute + # cap. Asking plan_shards keeps ONE definition of both "which members + # are there" and "what does an untimed one cost". + # + # The estimate is tuned on linux (16095s vs 15891s measured, +1.3%). + # macOS and windows members are cheaper on average than the default, + # so their totals come out high — harmless here because both are + # already at their cap, and erring toward more shards is the safe + # direction. Revisit if either cap moves. shards_for() { # platform cap -> shard count - local secs - secs=$(XPLAT="$1" lua5.4 -e ' - local plat = os.getenv("XPLAT") - local sel, all = {}, (os.getenv("MEMBERS") == "__ALL__") - if not all then - for m in (os.getenv("MEMBERS") or ""):gmatch("%S+") do sel[m] = true end - end - local total = 0 - for line in io.lines("tests/member-timings.tsv") do - local p, m, s = line:match("^(%S+)\t(%S+)\t(%d+)$") - if p == plat and m and (all or sel[m]) then total = total + tonumber(s) end - end - print(total)') + local secs sel="" + [ "$MEMBERS" = "__ALL__" ] || sel="$MEMBERS" + secs=$(lua5.4 tests/plan_shards.lua "$1" 0 0 $sel) local n=$(( secs / 4200 + 1 )) [ "$n" -gt "$2" ] && n="$2" echo "$1 work: ${secs}s (measured) -> $n shard(s)" >&2 @@ -983,25 +988,37 @@ jobs: # silently absorb a one-off slow runner. Refresh it deliberately — # download this artifact and replace tests/member-timings.tsv when # the numbers have actually moved. + # Top 10 per platform, not every member. + # + # plan_shards prices anything absent at a fixed default, and the + # packing decisions are made by the heavy members anyway — on linux + # the top 10 of 67 are 69% of the total, and dropping the other 57 + # moves the slowest shard by 0 minutes. A table that lists everyone + # goes stale the moment someone adds a test; one that lists ten only + # goes stale when the heavy set actually changes. That is the whole + # point: adding a member must not require touching this file. { - echo "# \t\t — from run ${{ github.run_id }}" - echo "# refresh: download the member-timings artifact and replace this file" + echo "# \t\t — the HEAVY members only, top 10" + echo "# per platform, from run ${{ github.run_id }}. Everything else is" + echo "# priced at plan_shards.lua's fixed default; adding a test does NOT" + echo "# require touching this file. Refresh when a member becomes heavy" + echo "# enough to enter the top 10, or one of these numbers moves." + # ONLY the default-toolchain leg feeds this table. The llvm leg + # runs the SAME members again, so globbing every leg would put two + # rows per (platform, member) into the file — `sort -u` keeps both, + # since the seconds differ. Both legs are planned from these + # numbers, so the default leg is the right single baseline. + # + # Then: heaviest first, keep 10. `sort -u` before that so a member + # reported by two shards cannot occupy two of the ten slots. for plat in linux macos windows; do - # ONLY the default-toolchain leg feeds this table. The llvm leg - # runs the SAME members again, so globbing every leg would put - # two rows per (platform, member) into the file — `sort -u` - # keeps both, since the seconds differ — and shards_for sums - # every matching row. Linux work would read as roughly double - # and its shard count would be permanently pinned at the cap. - # Both legs are planned from these numbers, so the default leg - # is the right single baseline. for f in timings/timings-$plat-default-*/timings.tsv; do [ -f "$f" ] || continue awk -F'\t' -v p="$plat" '{ printf "%s\t%s\t%s\n", p, $2, $1 }' "$f" - done + done | sort -u | sort -t"$(printf '\t')" -k3,3nr | head -10 done - } | sort -u > member-timings.tsv - echo "wrote member-timings.tsv ($(grep -vc '^#' member-timings.tsv) rows)" + } > member-timings.tsv + echo "wrote member-timings.tsv ($(grep -vc '^#' member-timings.tsv) rows, top 10/platform)" - name: Upload the timing table for the next run's sharding if: always() && hashFiles('member-timings.tsv') != '' diff --git a/tests/member-timings.tsv b/tests/member-timings.tsv index 6013289..b193700 100644 --- a/tests/member-timings.tsv +++ b/tests/member-timings.tsv @@ -1,203 +1,43 @@ -# \t\t — from run 31260545520 -# refresh: download the member-timings artifact and replace this file -# libmysqlclient / mysql-connector-cpp: measured from run 31266814148's -# linux default 0/3 step durations — that shard was cancelled at the 90m -# cap so its timings artifact never uploaded, and these two are exactly -# the members whose absence caused the overflow. Chicken-and-egg broken by hand, once. -linux abseil 96 -linux archive 28 -linux asio-module 15 -linux asio-ssl 175 -linux boost-ext.ut 4 -linux build-mcpp 1 -linux c-ares 5 -linux catch2 59 -linux catch2-main 57 -linux catch2-v2 14 -linux catch2-v2-main 15 -linux cjson 19 -linux cli11 8 -linux cmdline 6 -linux core 78 -linux curl 264 -linux eigen 24 -linux eui-neo 43 -linux eui-neo-app-main 48 -linux eui-neo-markdown 144 -linux eui-neo-sdl2 591 -linux eui-neo-vulkan 143 -linux eui-neo-window 143 -linux ffmpeg 296 -linux ffmpeg-module 277 -linux fmtlib.fmt 4 -linux freetype 11 -linux glad 3 -linux godot-cpp 564 -linux godot-cpp-module 627 -linux godot-cpp-module-v10 568 -linux godot-cpp-v10 504 +# \t\t — the HEAVY members only, top 10 per +# platform. Everything else is priced at plan_shards.lua's fixed default; +# on linux the 57 members not listed here average 86s and are 31% of the +# total, and listing them changes the slowest shard by 0 minutes. +# +# So: adding a test does NOT require touching this file. Refresh it when +# a member becomes heavy enough to enter the top 10, or when one of these +# numbers moves materially — download the member-timings artifact from a +# full run and keep the top 10 per platform. +# +# Sources: run 31260545520; libmysqlclient / mysql-connector-cpp measured +# from run 31266814148's linux default 0/3 step durations, whose shard was +# cancelled at the 90m cap before its timings artifact could upload. linux grpc-codegen 3463 linux grpc-module 1771 -linux gui-stack 128 -linux imgui 7 -linux imgui-module 106 -linux imgui-window 7 -linux libmysqlclient 329 -linux libpng 7 -linux llamacpp 132 -linux llamacpp-metal 21 -linux llmapi 19 -linux magic_enum 4 -linux marzer.tomlplusplus 7 -linux md4c 23 +linux protobuf-protoc 885 linux mysql-connector-cpp 881 -linux nlohmann.json 10 -linux openblas 0 -linux opencv-module 649 linux opencv-module-dnn 875 linux opencv-module-unifont 655 -linux openssl 166 -linux protobuf 175 -linux protobuf-gzip 267 -linux protobuf-protoc 885 -linux protobuf-upb 179 -linux re2 16 -linux sdl2 108 -linux spdlog 9 -linux spdlog-compiled 13 -linux tinyhttps 11 -linux tray 2 -linux vulkan 76 -linux websocket 20 -linux websocket-features 26 -linux yyjson 10 -macos abseil 48 -macos archive 21 -macos asio-module 10 -macos asio-ssl 65 -macos boost-ext.ut 4 -macos build-mcpp 3 -macos c-ares 5 -macos catch2 24 -macos catch2-main 27 -macos catch2-v2 7 -macos catch2-v2-main 9 -macos cjson 3 -macos cli11 5 -macos cmdline 4 -macos core 44 -macos curl 66 -macos eigen 8 -macos eui-neo 47 -macos eui-neo-app-main 49 -macos eui-neo-markdown 64 -macos eui-neo-sdl2 138 -macos eui-neo-vulkan 59 -macos eui-neo-window 48 -macos ffmpeg 43 -macos ffmpeg-module 107 -macos fmtlib.fmt 3 -macos freetype 10 -macos glad 5 -macos godot-cpp 5 -macos godot-cpp-module 276 -macos godot-cpp-module-v10 195 -macos godot-cpp-v10 206 +linux opencv-module 649 +linux godot-cpp-module 627 +linux eui-neo-sdl2 591 +linux godot-cpp-module-v10 568 macos grpc-codegen 1156 macos grpc-module 719 -macos gui-stack 25 -macos imgui 4 -macos imgui-module 21 -macos imgui-window 0 -macos libpng 6 -macos llamacpp 63 -macos llamacpp-metal 71 -macos llmapi 16 -macos magic_enum 4 -macos marzer.tomlplusplus 6 -macos md4c 3 -macos nlohmann.json 8 -macos openblas 0 -macos opencv-module 223 macos opencv-module-dnn 325 -macos opencv-module-unifont 164 -macos openssl 83 -macos protobuf 71 -macos protobuf-gzip 97 macos protobuf-protoc 314 -macos protobuf-upb 66 -macos re2 8 -macos sdl2 23 -macos spdlog 4 -macos spdlog-compiled 6 -macos tinyhttps 14 -macos tray 3 -macos vulkan 8 -macos websocket 10 -macos websocket-features 11 -macos yyjson 5 -windows abseil 103 -windows archive 62 -windows asio-module 15 -windows asio-ssl 0 -windows boost-ext.ut 4 -windows build-mcpp 4 -windows c-ares 16 -windows catch2 74 -windows catch2-main 69 -windows catch2-v2 10 -windows catch2-v2-main 11 -windows cjson 3 -windows cli11 8 -windows cmdline 5 -windows core 100 -windows curl 27 -windows eigen 12 -windows eui-neo 75 -windows eui-neo-app-main 88 -windows eui-neo-markdown 86 -windows eui-neo-sdl2 123 -windows eui-neo-vulkan 86 -windows eui-neo-window 83 -windows ffmpeg 693 -windows ffmpeg-module 526 -windows fmtlib.fmt 6 -windows freetype 13 -windows glad 4 -windows godot-cpp 7 +macos godot-cpp-module 276 +macos opencv-module 223 +macos godot-cpp-v10 206 +macos godot-cpp-module-v10 195 +macos opencv-module-unifont 164 +macos eui-neo-sdl2 138 +windows opencv-module 868 +windows opencv-module-dnn 828 windows godot-cpp-module 707 +windows ffmpeg 693 windows godot-cpp-module-v10 657 windows godot-cpp-v10 655 -windows grpc-codegen 10 -windows grpc-module 0 -windows gui-stack 19 -windows imgui 6 -windows imgui-module 24 -windows imgui-window 0 -windows libpng 10 -windows llamacpp 126 -windows llamacpp-metal 1 -windows llmapi 16 -windows magic_enum 4 -windows marzer.tomlplusplus 7 -windows md4c 3 -windows nlohmann.json 10 -windows openblas 10 -windows opencv-module 868 -windows opencv-module-dnn 828 -windows opencv-module-unifont 7 -windows openssl 1 -windows protobuf 139 +windows ffmpeg-module 526 windows protobuf-gzip 238 windows protobuf-protoc 152 windows protobuf-upb 151 -windows re2 17 -windows sdl2 47 -windows spdlog 22 -windows spdlog-compiled 12 -windows tinyhttps 16 -windows tray 2 -windows vulkan 5 -windows websocket 25 -windows websocket-features 29 -windows yyjson 4 diff --git a/tests/plan_shards.lua b/tests/plan_shards.lua index 59be162..6766313 100644 --- a/tests/plan_shards.lua +++ b/tests/plan_shards.lua @@ -26,9 +26,16 @@ -- among shards whose load is close, prefer the one already holding -- members with overlapping dependencies. -- --- Missing timing → the median, so a newly added member is neither assumed --- free nor assumed huge. No table at all → falls back to round-robin, which --- is worse but never wrong. +-- Missing timing → a FIXED default (see DEFAULT_SECONDS), so a newly added +-- member is neither assumed free nor assumed huge. No table at all → every +-- member prices the same, which degenerates to round-robin: worse, never +-- wrong. +-- +-- The table deliberately holds only the HEAVY members. They are where the +-- packing decisions actually get made — on linux the top 10 of 67 are 69% of +-- the total — and listing the other 57 buys a table that goes stale every +-- time someone adds a test. Verified equivalent: full table and top-10 table +-- both land the slowest linux shard at 74 minutes. -- -- Measured on the real workspace (linux, 3 shards), LPT against round-robin: -- @@ -71,24 +78,58 @@ end -- ── measured times ──────────────────────────────────────────────────────── -- Format: \t\t -local times, samples = {}, {} +local times = {} local tsv = read_file("tests/member-timings.tsv") if tsv then for line in tsv:gmatch("[^\n]+") do if not line:match("^#") then local p, m, s = line:match("^(%S+)\t(%S+)\t(%d+)") - if p == platform and m then - times[m] = tonumber(s) - samples[#samples + 1] = tonumber(s) - end + if p == platform and m then times[m] = tonumber(s) end end end end -local median = 60 -if #samples > 0 then - table.sort(samples) - median = samples[math.ceil(#samples / 2)] +-- Price for a member with no row. A FIXED constant, deliberately, and not the +-- median of whatever the table happens to hold. +-- +-- The table only carries the HEAVY members now (see its header). Deriving the +-- default from those samples would take the median OF THE HEAVYWEIGHTS — +-- measured at 875s on the linux set — and charge every small member that, +-- which packs far worse than having no table at all. Simulated on the full +-- 67-member measurement, 4 linux shards, slowest shard: +-- +-- full 67-row table 74 min +-- top-10 table, median-derived default 90 min <- at the job cap +-- top-10 table, fixed default 74 min +-- +-- 86s is the mean of the 57 untimed linux members (median 24s); 90 rounds it. +-- The choice is not delicate: across defaults from 30s to 300s the slowest +-- shard stays between 72 and 84 minutes, all under the 90-minute cap. That +-- insensitivity is the point — it is what lets the table sit untouched while +-- small members come and go. +-- +-- What it does NOT absorb is a new member that belongs in the heavy set. +-- `mysql-connector-cpp` (881s) priced at the default is 13 minutes of work +-- that the packer cannot see. Adding a heavy newcomer stays a real edit. +local DEFAULT_SECONDS = 90 +local median = DEFAULT_SECONDS -- name kept: read as "the price of unknown" + +-- ` ` of `0 0` prints this platform's TOTAL cost instead of a +-- shard's members. +-- +-- The workflow's shards_for needs exactly that number to pick a shard count, +-- and it needs it computed the same way: same member enumeration, same price +-- for an untimed member. It used to sum the table's rows directly, which was +-- equivalent only while the table listed every member. It no longer does — +-- summing 10 heavy rows and calling it the workload asks for 3 linux shards +-- where the real work needs 4, and 3 lands the slowest shard at 107 minutes +-- against a 90-minute cap. Two copies of "how much work is there" drift; this +-- is the one copy. +if shardCount == 0 then + local total = 0 + for _, m in ipairs(members) do total = total + (times[m] or DEFAULT_SECONDS) end + print(total) + return end -- ── run order: cheapest first ─────────────────────────────────────────────