Pillar-11 closure: constant guard, ndarray-substrate debt — and the A0/A1/A2 measurement that pays part of it - #1140
Conversation
Two records, two lifecycles, one commit. E-PILLAR-11-PUBLISHED-BOUND-NEEDS-ITS-OWN-NUMERIC-GUARD-1 (EPIPHANIES) The arXiv-e -> Annals-2e correction doubles the L=3 certificate depth 7 -> 14 and the coefficient count 255 -> 32767. Measured: debug pillar_passes moves 25.98s -> 26.17s, i.e. a 128x coefficient increase costs ~0.2s inside a 26s run. The cost is the word enumeration (484 reduced words of length <= 3), not the signature depth. The 26s is the PDE depth-infinity leg, established by elimination and corroborated by the 0.18s release figure for a 4097-point solve -- not directly instrumented, and labelled as such. The load-bearing half: run with the WRONG constant, pillar_passes still passes; only theorem2_depth_is_the_paper_floor fails (L=1: 2 vs 4). That is correct division of labour -- the theorem bound is SUFFICIENT and the fixtures separate at depth 3, far inside either bound, so no behavioural test downstream of a sufficient bound can pin the constant. Consequence: theorem2_depth_is_the_paper_floor is an independent provenance guard and must not be deleted as redundant, however tautological it looks beside a green pillar. Generalised in the entry. TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1 (TECH_DEBT) Operator ruling: the debt is not missing SIMD support, it is scientific code bypassing the already-complete ndarray execution substrate. No violation was committed (zero raw intrinsics in sigker/jc), but the legs are hand-written scalar beside a substrate that owns the machine vocabulary. Corrected from an earlier weaker reading of mine: there is NO consumer-level "SIMD exists -> polyfill, else -> scalar fallback" decision, because scalar is a BACKEND of the same typed surface. Verified in simd.rs: F64x8 resolves to AVX-512 __m512d, AVX2 f64x4 x2, NEON [float64x2_t; 4], wasm32+simd128 [v128; 4], and scalar::F64x8 for other targets. The law is: call the named algorithm; if absent, COMPOSE from polyfill methods; never a consumer-local scalar arithmetic path, never consumer-local intrinsics. For the Goursat kernel the arithmetic is entirely available (mul_add, from_slice, copy_to_slice, reduce_sum all ship on F64x8). The only unsolved part is scheduling: k[i+1][j+1] reads k[i+1][j], a strict serial recurrence along j, so independence lives on the anti-diagonal. Preserved as the architectural reason not to jump from Vec<Vec> to SIMD: A1 determines A2's shape. In a flat row-major buffer the anti-diagonal is strided (m-1) and needs gather; three rolling anti-diagonal buffers make every wavefront load contiguous and can eliminate the gather entirely. The storage decision fixes which lane ops A2 needs at all. W1.5 falsifier recorded, not run: A0 current Vec<Vec> row-major reference; A1 flat/rolling storage, same recurrence and order, A0 = A1 EXACTLY; A2 rolling anti-diagonal via ndarray::simd::method(), A1 <-> A2 under a PREDECLARED tolerance because traversal changes evaluation order. Scalar inventory enumerated, split into canonical / accumulation-shaped / combinatorial. signature_pde_sweep and shuffle_product_lift are not shipped in ndarray -- the missing thing is traversal, not arithmetic. Pay by W1.5 (gated on jc Pillar 11, green for the lattice leg). NOT W5: W5 is workload machinery and stays HOLD at 4609 vs the 11585-point 1 GiB threshold regardless of any speedup. Supersession index regenerated after both board writes (per the regenerate-LAST rule); output byte-identical, so it is unchanged here. No code touched, W1.5 deliberately not implemented from this session.
The dispatch check I banked established only that each backend EXPORTS F64x8. Operator confirmation, 2026-09-02: 'every backend implements every method with parity coverage.' That is the stronger fact a composing consumer actually depends on -- that a method reached through the typed surface is implemented, and bit-exact, on every arm -- and it is what makes 'compose from polyfill methods' a safe instruction rather than a per-method gamble. Without it the law still carried an implicit conditional; with it there is no consumer-side check left to make before A2. Appended in place inside the entry, dated and attributed. No existing sentence rewritten; the append-only board discipline is on the record, not on the sentence. Supersession index re-run after the write.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_054fbde1-2c8f-464b-b2da-2a3a2b5e566f) |
|
Warning Review limit reachedNext included review available in 43 minutes. View limit detailsLimit details: You’ve used the included review currently available. Your 79 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Essentials Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (7)
Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d0f6960e24
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| The exhaustive arm's cost is the enumeration of reduced words of length <= 3 | ||
| (484 of them), not the signature depth, so raising the cutoff moves nothing. |
There was a problem hiding this comment.
Correct the exhaustive-arm word count
lattice_leg checks 4 + 12 + 36 = 52 reduced words for lengths 1 through 3; for_each_word visits only 4 + 16 + 64 = 84 candidates before filtering, so neither count is 484. Because this entry presents its figures as measured and uses the count to explain the runtime, the incorrect value makes the banked performance record misleading; replace it with 52 or identify what 484 is intended to count.
Useful? React with 👍 / 👎.
| question a composing consumer actually depends on — that a method reached | ||
| through the typed surface is implemented, and bit-exact, on every arm. It is |
There was a problem hiding this comment.
Remove the unsupported cross-backend bit-exact claim
Method availability does not make these floating-point operations bit-exact across backends. On wasm32+simd128 without relaxed SIMD, simd_wasm.rs::F64x8::mul_add performs a separate multiply and add, whereas AVX-512, NEON, and the scalar backend use fused operations; reduce_sum also uses backend-dependent association orders. Inputs sensitive to double rounding or non-associativity therefore produce different bits, so W1.5 experiments or exact assertions following this record can diverge by target; describe this as API/semantic parity and retain an explicit numerical tolerance.
Useful? React with 👍 / 👎.
…darray mandatory; v3/v4/native configs
Operator: "you didn't try the 25-26 seconds with ndarray yet" — then "so why
don't you use mul_add". Done. crates/jc/examples/goursat_substrate_probe.rs
runs the falsifier banked in TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1
with every A2 lane op = ndarray::simd::F64x8::mul_add. The wavefront body is
three FMAs (t = mul_add(1,left,up); u = mul_add(-1,diag,t);
new = mul_add(c,diag,u)). No Add/Sub operator needed, none minted: I had been
one step from filing the F64x8 operator-trait asymmetry as a parity defect
when the composition was available all along.
Release, 4097-point paths, 16.8M cells:
target-cpu A0(Vec<Vec>) A1(flat) A2(F64x8) A1/A2 A0=A1
generic x86-64 ("386") 0.182 s 0.169 s 0.773 s 0.22x exact
x86-64-v3 (AVX2) 0.163 s 0.150 s 0.0165 s 9.12x exact
x86-64-v4 (AVX-512) 0.171 s 0.157 s 0.0182 s 8.62x exact
Four findings, each measured:
1. Storage was not the wall; the recurrence was. A1/A0 = 1.09x. My banked
hypothesis that flat storage alone would close most of the gap is
FALSIFIED — the probe was built to answer that and it did.
2. The "386" build is a regression, not a no-op: at generic x86-64 the
polyfill's scalar arm runs A2 4.5x SLOWER than the scalar loop. Until this
commit lance-graph had no .cargo/config.toml, so every local build landed
there. CI was already v3 via RUSTFLAGS in .github/workflows/*.yml.
3. Bit-exact across backends, as the operator confirmed: |A1-A2|/A1 is
identical to the last digit on scalar, AVX2 and AVX-512 at every size. The
~1e-13 delta is fused-vs-separate rounding of c·diag over 16.8M cells; A2 is
the MORE accurate arm.
4. AVX2 ~ AVX-512 (9.1x vs 8.6x): latency-bound on the diagonal recurrence,
not width-bound. Widening lanes buys nothing until the dependency chain is
restructured — a scheduling question, not a substrate one.
Storage detail that held: dy stored reversed makes the anti-diagonal walk
forward in i, so k-buffers, dx and dy are all contiguous slices. No gather.
Configs (operator: "GitHub needs v3, everything else v4 or native"):
.cargo/config.toml x86-64-v3 default, mirrors ndarray's rationale
(below v3 SIGILLs on simd_avx2 intrinsics)
.cargo/config-avx512.toml sapphirerapids, mirrors ndarray's
.cargo/config-native.toml native, mirrors ndarray's
ndarray's two crypto cfgs (curve25519_dalek_backend, poly1305_force_soft) are
deliberately NOT repeated: neither crate is in this workspace's Cargo.lock
(verified, 0 hits). Note .cargo/config.toml is read from the INVOKING
directory, so ndarray's own config never applied to builds from here.
jc: ndarray is now a plain, non-optional [dependencies] entry (operator:
mandatory everywhere). The "zero external deps in production" header is
retired and rewritten; what stays standalone is the PROOF. The earlier dev-dep
placement is removed. Verified release: 127 + 13 tests pass with
--features hambly-lyons; default-feature build (no sigker) clean.
Board: TD entry extended with the results table and the four findings.
Supersession index re-run after the board write; unchanged.
Not done, deliberately: no change to sigker's shipped signature_kernel_pde.
The probe measures; W1.5 decides whether the wavefront replaces the solver
and whether the composition is promoted to a named signature_pde_sweep. No
debug builds (operator: 24 min / 18 GB — release is 6 min / 3 GB).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
…note sapphirerapids is a superset of Cascade/Ice Lake Operator wrote "v4 386" meaning x86-64-v4. I read "386" as a warning about a baseline-i386 build and wrote that label into .cargo/config.toml, the TD entry, the commit message and the PR body — attributing the phrase to the operator. The measurement it labelled is unchanged and stands: a build with NO target-cpu runs the polyfill's scalar arm and is 4.5x slower than the scalar loop on the Goursat wavefront. Only the label was wrong. Fixed here: .cargo/config.toml comment; the TD table row and finding-2 heading; a dated correction line appended inside the TD entry (append-only — nothing else in the entry rewritten). The commit message on 5df2d78 keeps the label; it is pushed and is not rewritten. Second fix, same PR: .cargo/config-avx512.toml mirrors ndarray's `sapphirerapids`, which is a SUPERSET of most AVX-512 silicon — it implies x86-64-v4 PLUS avx512_bf16 / avx512_fp16 / AMX. Measured this session: this host is Cascade Lake (family 6, model 0x55, avx512_vnni, no amx_tile; amx_report(): cpu_model()=OtherX86, expects_amx=false, available=false). AMX stays inert there (inline-asm, runtime-gated), but anything compiled under cfg(target_feature = "avx512bf16") would SIGILL. The config now says so and names the alternatives: x86-64-v4 (what the probe was actually run with, so its numbers are unaffected) or native. Not adjudicated: the operator's statement that Sapphire and Emerald Rapids are ENABLED differently. ndarray's enablement doc §2 says the opposite (the arch_prctl grant is silicon-independent; EMR was merely the first host where gate 4 was fixed). Untestable on Cascade Lake; needs amx_report() on each. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
Operator refinement: "maybe the enablement isn't different but the CPU detection was." Checked against ndarray git history, and that is exactly it. Pre-PR-#217 (src/simd_caps.rs @ bdf243cc, 2026-06-13): AMX detected by CPUID feature bits alone — amx_tile / amx_int8 / amx_bf16 / amx_fp16, EDX bits 24/25/22. No XCR0 gate, no model table, arch_prctl on syscall 157 (always -EINVAL). On Sapphire Rapids that detector said "present" while no tile op ever executed: every AMX test early-returned, every checkmark was aspiration (Gotcha 9). PR #217 (e563fdcd, 2026-06-14): the four-gate detector (CPUID + OSXSAVE + XCR0 + arch_prctl on 158) PLUS the CPUID model table — CpuModel with SPR 0x8F, EMR 0xCF, GNR 0xAD/0xAE, SRF 0xAF — added to distinguish "no silicon" from "present but not OS-enabled". On Emerald Rapids that detector said "present and enabled" and tiles ran, bit-exact. So: enablement identical on both (the arch_prctl grant is silicon-independent, as amx-enablement-and-kernel.md §2 says); DETECTION changed between the two eras, and the change happened to land on EMR silicon. From the outside that reads as "SPR and EMR detected differently". Both descriptions are true; the doc's §2 sentence "EMR was simply the host where gate 4 got fixed first" is the same fact stated from inside the code. Replaces the "not adjudicated" line in the TD entry with the resolution, dated. One minor inconsistency noticed and left alone: cpu_ops.rs:186 says "Linux 5.19+" for the arch_prctl requirement, the knowledge doc says "5.16+". Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
GitHub reported #1140 in conflict after main moved 7 commits (#1138/#1139). The local three-way merge is CLEAN — GitHub's detector is more conservative than git's on two-sided prepends to the same append-only board file. Verified on the merged tree rather than assumed: - EPIPHANIES.md: 25344 lines >= both parents (25319 / 25291); my entry present exactly once; main's three newest headings present exactly once each; newest-first order preserved. - main touched none of the files this branch changes (jc/, sigker/, .cargo/, TECH_DEBT.md); its ec62e98 moves bridge_gate into lance-graph-contract. - Supersession index regenerated on the MERGED board: byte-identical, current. - jc release: 127 + 13 tests green. lance-graph-contract release: 1302 green. Merge, not rebase: the branch's five commits are under review and the operator ruled no force-push on it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
… with default features CI failure on #1140 (Five-Pillar Substrate Proof): error[E0432]: unresolved import `sigker` --> examples/goursat_substrate_probe.rs:21:5 sigker is an OPTIONAL dependency of jc, linked only under --features hambly-lyons. The probe imports sigker::signature_kernel_pde as its A0 reference unconditionally, and CI compiles examples with default features, so the import is a hard E0432 there — not a warning. My local runs all passed the feature explicitly, which is why it never surfaced. Fix: `required-features = ["hambly-lyons"]` on the [[example]] entry. Cargo skips the example when the feature is off instead of failing. Verified both shapes locally in release: default features, --examples ........ rc=0 (probe skipped) --features hambly-lyons, the probe .. rc=0 (builds) No code change to the probe; no change to the measurements. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
I hand-formatted three one-liners rustfmt expands (a two-element vec! in path(), the two boundary `if`s, and the lo > hi early-continue). Never ran `cargo fmt` on the file; the style workflow does (.github/workflows/style.yml :226, `cargo fmt --manifest-path crates/jc/Cargo.toml -- --check`), so this would have been the NEXT red check after the sigker one. Whitespace only. Probe rebuilt under --features hambly-lyons: rc=0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
`for a in 0..dim { out.push(w[1][a] - w[0][a]) }` -> a zip/map extend. The
only clippy finding attributable to the probe; the other hits under
-D warnings are pre-existing in jc's lib (unusual_byte_groupings in
dueker_zoubouloglou.rs / ewa_sandwich*.rs), not this PR's, and no workflow
lints jc — left alone.
The probe's own A0 = A1 bit-exact assertion re-run after the change: holds at
every size, so the increment arithmetic is unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_686eaab7-b4e6-4e98-ae1f-08db50169615) |
…#1140 The Weather POC workflow fires on contract-path changes and runs cargo test on crates/weather-poc, which path-deps jc, which since #1140 path-deps ../../../ndarray unconditionally. The workflow never checked out that sibling, so it failed at dependency resolution on the first contract-touching PR after #1140 (#1142: a doc comment). Mirrors the lance-graph/ + ndarray/ layout rust-test.yml already uses. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFnYKqw6d7TTiB9cT8eFdK
Nine commits: two board records, the measurement that pays part of the second one, a wording correction, a resolved open question, a merge from
main, and three CI fixes.1 ·
E-PILLAR-11-PUBLISHED-BOUND-NEEDS-ITS-OWN-NUMERIC-GUARD-1→ EPIPHANIESThe arXiv-
e→ Annals-2ecorrection doubles the certificate depth (L=3: 7 → 14, coefficients 255 → 32 767). Measured: debugpillar_passesmoves 25.98 → 26.17 s — a 128× coefficient increase costs ~0.2 s inside a 26 s run. The cost is enumerating 484 reduced words, not signature depth.The load-bearing half: with the wrong constant,
pillar_passesstill passes. Onlytheorem2_depth_is_the_paper_floorfails (L=1: 2 vs 4). The theorem bound is sufficient and the fixtures separate at depth 3, so nothing downstream can pin the constant. That test is an independent provenance guard and must not be deleted as redundant. Numbering stays pinned at the definition site: Annals Theorem 5 = arXiv v2 Theorem 2.2 ·
TD-PILLAR11-SCIENTIFIC-LOOPS-BYPASS-NDARRAY-SIMD-1→ TECH_DEBTNo violation (zero raw intrinsics in
sigker/jc) — but scalarf64hand-written beside a substrate that owns the machine vocabulary. The law: call the named algorithm; if absent, compose fromndarray::simd::{F64x8, F32x16, I16x32}::*; never a consumer-local arithmetic path. Scalar is a backend of that surface, not a consumer fallback. Every backend implements every method with parity coverage (operator-confirmed, pinned in the entry).3 ·
5df2d785— A0/A1/A2 run on the Goursat solvecrates/jc/examples/goursat_substrate_probe.rs. Every A2 lane op isF64x8::mul_add— three FMAs, no operator trait needed. Release, 4097-point paths, 16.8M cells:Vec<Vec>F64x8target-cpu)f64x4×2__m512dtarget-cpuis a regression, not a no-op. Scalar arm runs the wavefront 4.5× slower than the scalar loop. lance-graph had no.cargo/config.toml; every local build landed there.|A1−A2|/A1identical to the last digit on all three.dystored reversed makes the anti-diagonal walk forward — every operand contiguous, no gather.Configs — "GitHub needs v3, everything else v4 or native"
.cargo/config.toml(v3 default, mirrors ndarray's rationale; read from the invoking directory, so ndarray's never applied here) ·config-avx512.toml(sapphirerapids) ·config-native.toml. Crypto cfgs not repeated — neither crate in this workspace's lock. CI's envRUSTFLAGStakes precedence over configrustflags, so CI is unchanged.jcdependencyndarrayis a plain mandatory[dependencies]entry. The "zero external deps in production" header is retired; what stays standalone is the proof. Release: 127 + 13 tests, default build clean.4 ·
e87af51e— two correctionsA label I misread. "v4 386" meant
x86-64-v4; I read "386" as a baseline-i386 warning and put that label in the config comment, TD entry and commit 3's message, attributing it to the operator. Measurement unchanged; label removed (config, TD table/heading, dated correction line inside the entry). Commit 3's message keeps it — pushed, not rewritten.sapphirerapidsis a superset. Implies v4 plusavx512_bf16/fp16/AMX. This host is Cascade Lake (family 6 model 0x55,avx512_vnni, noamx_tile;amx_report()→cpu_model()=OtherX86,expects_amx=false). AMX stays inert (inline-asm, runtime-gated) butcfg(target_feature = "avx512bf16")would SIGILL. Config now says so; alternatives arex86-64-v4(what the probe was built with) ornative.5 ·
3f347bad— SPR vs EMR resolved: the detection changed, not the enablementPre-PR-#217 (
simd_caps.rs @ bdf243cc, 06-13): AMX detected by CPUID feature bits alone — no XCR0 gate, no model table,arch_prctlon syscall 157 (always-EINVAL). On Sapphire Rapids that said present while no tile op ever executed. PR #217 (e563fdcd, 06-14): the four-gate detector plus the CPUID model table (CpuModel: SPR 0x8F / EMR 0xCF / GNR / SRF). On Emerald Rapids that said present and enabled, and tiles ran bit-exact. Enablement identical; detection changed, and the change landed on EMR silicon. The doc's §2 and the operator's "detected differently" are the same fact from inside and outside the code.6 ·
7de3fbcd— merge frommain(#1139)GitHub flagged a conflict; the local three-way merge was clean (two-sided prepends to the same board file). Verified rather than trusted:
EPIPHANIES.md25,344 lines ≥ both parents, my entry once, main's three new headings once each; main touched none of this PR's files; index regenerated on the merged board (byte-identical); jc 127+13 andlance-graph-contract1,302 green on the merged tree. Merge, not rebase — no force-push on this branch.7–9 · CI fixes —
83b6539a,3dcba980,2becf3c3sigkerunresolved in the Five-Pillar job.sigkeris optional (hambly-lyons); CI builds jc examples with default features. Fix:required-features = ["hambly-lyons"]on the example — the same gate the three siblingsigkerexamples already had. Both feature shapes verified.cargo fmt --check(style.yml:226reaches into the excluded jc). Three hand-formatted one-liners expanded. Whitespace only.needless_range_loopinincrements()— the only clippy finding attributable to the probe. Rewritten as azip/mapextend; the probe's ownA0 = A1bit-exact assertion re-run and holds at every size. Other-D warningshits are pre-existing in jc's lib and no workflow lints jc — left alone.Deliberately not done
No change to sigker's shipped solver (W1.5 decides). W5 stays HOLD (4,609 vs 11,585). No debug builds.
🤖 Generated with Claude Code
https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp