Skip to content

Commit 9019417

Browse files
AdaWorldAPIclaude
andauthored
D-LGJ-W8 PR-W8a: contract dep + FixtureClassView + lgj_mask_andnot + lgj_hop (ABI minor 4) (#22)
* Core vertical slice: docs/abi.md contract, native/lgj-abi, Java facade Ships the fully verified core of the Panama x ndarray::simd x Valhalla vertical slice (Phases A-E of the mission plan): - docs/abi.md: the normative Rust<->Java ABI contract, written before either side was implemented so both could be checked against one frozen doc instead of each other. - Five new ndarray::simd primitives (eq_u32_to_mask, gt_i32_to_mask, mask_and/mask_or(_assign), masked_sum_i32), added under ndarray's own W1a consumer contract. - native/lgj-abi: the Rust ABI crate. Generation-checked handle registry, generic SoA fixture, bulk kernels routed exclusively through ndarray::simd, 14-symbol extern "C" surface. 72/72 tests green, clippy/fmt clean, and the registry's core safety check was disable-verified (short-circuited, confirmed exactly the two guarding tests go red, restored). - java/: the Panama membrane (internal/ffm, never exposed publicly) and the public semantic facade (NativePattern/View/Predicate/ Pattern/Mask). 132/132 checks green across 8 suites, including a reflection-enforced ApiSurfaceTest that mechanically proves zero FFM types ever reach a public signature, and a LazinessTest that empirically proves the thesis: building a chain costs zero crossings, evaluating it costs exactly one, independent of row count up to 1,000,000. - .claude/: a 6-agent ensemble, 6 knowledge docs, and a full board (LATEST_STATE/STATUS_BOARD/AGENT_LOG/EPIPHANIES/TECH_DEBT/ISSUES/ PR_ARC_INVENTORY/INTEGRATION_PLANS/CODEX_REVIEW_CHECKLIST), all scoped to this repo's actual seams. A mechanical audit (D-LGJ-AUDIT) found and fixed the one real rule violation before this commit: kernels.rs::simd_popcount was calling the internal ndarray::hpc::bitwise path instead of the sanctioned ndarray::simd re-export. Deliberately NOT included: the Valhalla lab (valhalla-lab/) and the Vector API benchmark harness (bench/) — still in flight, tracked as open STATUS_BOARD.md rows, to land in a follow-up PR once reviewed with the same rigor as this slice. Generated by [Claude Code](https://claude.ai/code) * Valhalla lab: three-truths method, causal isolation, 3 real reproducers Completes D-LGJ-F. One experiment source (src/shared/), compiled twice against real JDKs -- stable JDK 26 GA (record) and the official JEP 401 early-access binary (value record) -- via a self-verifying run.sh that mechanically diffs the two Vocab.java files modulo the 'value' keyword before trusting the A/B is honest. Experiments: IdentityExperiment (semantic truth -- is identity actually unobservable), FootprintExperiment (real per-object/array/field bytes via allocation-delta + JOL where available), FfmAddressingExperiment (is the wrapper free where it touches native memory), ThesisExperiment (the mandatory headline: 65,536 rows as one native lane vs hydrated Java objects, on both platforms). Causal isolation via three additional run.sh passes: escape analysis off, and UseArrayFlattening/UseFieldFlattening toggled independently -- isolates which flag actually drives the measured difference rather than inferring it. Three real Valhalla limitations reproduced and filed under reproducers/, none of which changed the production API: - R1: @NullRestricted field on an identity class is a VerifyError (javac's fault -- no source form expresses the required strict-field init order relative to super()) - R2: array flattening has a hard 8-byte payload cliff, confirmed via -XX:+PrintFlatArrayLayout. LaneId/Ordinal/MaskId (<=8B) flatten; RowRange/Row (16B) do not. This turns "Valhalla helps descriptors, not entities" from a hand-wave into a measured VM cutoff -- and RowRange landing on the wrong side is flagged as the one place the expectation was too optimistic. - R3: the densest null-restricted array form is jdk.internal-only and generics erase flattening entirely; Foo! null-restricted type syntax confirmed not to parse, matching the earlier archaeology finding. One real defect found and fixed before landing: IdentityExperiment and the stable Platform called Class::isValue() directly on four vocabulary types with a comment incorrectly claiming it was "final API on JDK 26" -- it does not exist there at all, confirmed by a real javac failure. Fixed by routing every query through Platform.isValueClass(Class<?>), answered honestly per platform. Generated by [Claude Code](https://claude.ai/code) * Vector API bench: real JMH, cross-checked; the crossing does not always win Completes D-LGJ-G, the mission's mandated "where does execution belong" comparison -- measured, not assumed to favor the Rust crossing. Real JMH 1.37 (fork+warmup+compiler-blackholes confirmed in the log, not a hand-rolled loop -- that lives in valhalla-lab and is labelled as such there). Four components, cost kept strictly separate per the mission brief: A_DowncallOverhead (bare crossing, no work), B_SegmentAccess (raw native-memory read throughput), C_ExecutionBoundary (native fused plan vs Java Vector API vs Java scalar, swept 64 to 4,194,304 rows), E_FusionAndPlanning (fused vs unfused vs the scalar reference kernel vs plan-construction-only, swept 1-8 predicates). 50/50 rows, 0 failures. Data.crossCheck() runs in @setup and throws if the three kernels disagree on count or sum, so a faster-but-wrong Vector kernel could not have won the comparison undetected. The headline complicates the thesis honestly: for a single predicate over one native lane, the Java Vector API -- reading the SAME native MemorySegment zero-copy via IntVector.fromMemorySegment, no byte[], no bounce buffer -- beats the native crossing at EVERY row count tested, 56.4x at small sizes down to 1.3-1.4x at 4M rows. A second crossover is also real: native beats a plain Java scalar loop only past roughly 4,096-16,384 rows. Component E shows why this doesn't overturn the project's premise: SIMD-vs-scalar is the largest lever measured anywhere in this suite (10.8x-31.1x, growing with predicate count), and fused/unfused land within this harness's own ~10% noise floor of each other at 65,536 rows -- the fused plan's real value is the structural one-crossing guarantee (already proven by LazinessTest), not a large measured time saving at this scale. Verdict: the crossing is worth paying for composed, multi-predicate work, not for reading one predicate off one lane, where Java on the same memory is simply faster. RESULTS.md was hand-written from the raw CSV, then independently cross-checked against summarise.sh -- a script the same PR ships that mechanically regenerates every table from results/jmh-results.csv, so a re-run's numbers can never silently drift from a hand-transcribed table. Both productions agreed to 3 decimal places on every cell checked. Generated by [Claude Code](https://claude.ai/code) * Phase I synthesis docs + fusion-sweep 256-row re-run + board hygiene Four synthesis docs close D-LGJ-I: architecture.md (the four layers and what each is provably responsible for), panama.md (manifest-over-header, belt-and-braces ownership, restricted-method discipline), valhalla-lab.md (three-truths synthesis, the 8-byte flattening cliff, zero API adoption), execution-boundary.md (the measured crossover picture + three structural hot-path facts, each checked rather than assumed). The fusion sweep was re-run with a 256-row arm after the first pass's 'fusion does nothing' finding proved true only at 65,536 rows: at 256 rows x 8 predicates unfused/fused reaches 2.99x. RESULTS.md is rewritten from jmh-results-merged.csv; TABLES.md is mechanically generated from the same file so the two cannot drift. MultiLaneColumn (ndarray::simd_soa) evaluated for the fixture kernels and declined on two concrete API mismatches (64-byte-multiple constraint, no u32 lane); earmarked for the future 512-byte row-store slice where it fits by construction. Operator layout reference recorded on the board. PR_ARC_INVENTORY backfilled for merged PRs 1-3; the lapse is owned in the file itself. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #4 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * SoA row store: 512B rows, 32 facet lanes, ABI minor 2 (W1+W2) The lance-graph-shaped substrate, wired end to end. The flat three-lane fixture was always scaffolding (docs/abi.md 10, architecture.md said so from PR #1); this is the layout the stack actually converges on: 64K x 512-byte rows, 32 facet lanes of 16 bytes = 4-byte LE classid + 12-byte payload, the V3 content-blind facet. Rust (native/lgj-abi): - rowstore.rs: one Arc<[u8]>, two readings (row-major chunks and strided facet columns), zero copies, normative SplitMix64 generator. - LGJ_RESOURCE_ROWSTORE + lgj_rowstore_open; facet lanes described through the UNCHANGED LgjLaneDesc (stride_bytes carried this since minor 1); lgj_op_eq_classid produces ordinary masks that compose with the existing algebra; lgj_row_facet_match writes per-row 32-bit facet sets into a caller-owned buffer via MultiLaneColumn (Arc refcount bump, no copy). - byte_len tightened to the exact covered span (len-1)*stride + elem_bytes: a full-stride final window would let Java bound a segment past the allocation's end on a facet lane. - ABI minor 1 -> 2; docs/abi.md gains 11 and its symbol count is corrected (the 14 was drift; the list already enumerated 15, and the real number is now 18 per nm -D). Gates: cargo test 84/84, clippy -D warnings clean, fmt clean, release build exports 18/18 symbols. Both new kernels are parity-checked against independent scalar references over 10 row counts x 2 seeds x 4 facets x 4 needles, then cross-checked a third way against RowStore::classid_at; a two-sided falsifier proves payload bytes never satisfy a classid match and that a real match does fire. Docs: .claude/plans/lgj-soa-substrate-v1.md (W1-W5 waves) + one plan per consumer example (world-trades / bricks-analytics / graph-traversal), .claude/knowledge/soa-row-store-layout.md, and the board triple ledger. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #5 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Knowledge: assess the archived layout-bridge discussion; name W6 The operator's pre-build ChatGPT discussion is assessed once, in .claude/knowledge/prior-art-and-the-layout-bridge-claim.md, so it is never re-mined or cited naively. Verdict: it converged independently on the architecture this repo then built and measured. Kept: the callability-vs-shared-executable-layout positioning, the schema-key-as- join-point extractable (now the named W6 consideration: an explicit schema/classid field on the descriptors when ClassView lands), and the baseline-dependent claims discipline for W5 comparisons. Pinned: its page-descriptor sketch has no liveness story (the registry's whole job), its native-always-wins assumption is measured false (Component C), and its ndarray paragraph describes upstream crates.io ndarray, not the AdaWorldAPI fork whose ndarray::simd polyfill this stack mandates. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Plans: OGAR Machine (exploratory) + lance-graph-hydrate dependency note Captures the operator's second archived context as .claude/plans/ogar-machine-v1.md — a genuinely new workload for the shipped substrate, not convergent confirmation: one row = one machine STATE, control flow as population masks over 64K execution contexts, Ghidra P-code as the normalized guest ISA (repo attached and cloned), differential migration testing (legacy XOR replacement across 65,536 worlds) as the killer demo, Lance as the time machine. Strong claim vs weak claim separated per the discussion's own discipline; gated on W3 + one W5 example + Ghidra archaeology + a tiny falsifiable probe (P-M1). Also records lance-graph #957 (merged: lance-graph-hydrate, the generic SoA->S3->volume->Lance hydration crate minted for consumers to inherit) and #958 (its open hardening fast-follow) in the substrate plan: when this repo's persistence slice arrives, hydration is inherited from lance-graph-hydrate, never re-derived here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #6 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Waves calcified: dispatch maps for every plan; Ghidra plan from real archaeology Operator ruling: calcify, don't execute. .claude/waves/ holds one dispatchable map per plan — README with the standing rules and the verbatim worker preamble, substrate W3+W4 (the only READY wave), three consumer waves stamped DO-NOT-DISPATCH, Ghidra G1+G2, and OGAR-Machine P-M1 (BLOCKED behind a 4-condition gate including an explicit operator go). Each map carries disjoint worker scopes, orchestrator-only steps, exact gate commands, disable-runs, and STOP triggers. ghidra-integration-v1.md is written from archaeology against the real clone, not the sketch: 74 P-code opcodes (CPUI_MAX=75), 12.2 DEV / Java 25+, analyzeHeadless entry, and Ghidra's own PcodeEmulator as the reference-implementation parity oracle (the tesseract-rs method). The ogar-machine plan is cross-updated to cite it. Mapping-time catches that would have burned a dispatch: the graph consumer needs a deliberate edge-bearing generator arm (today's payload is PRNG noise) - a substrate change, flagged in the wave; the hop has a real D1a/D1b design fork with ruling guidance recorded. Muscle memory pinned as E-LGJ-CALCIFY-THEN-DISPATCH-1: the eight earned-this-session rules and the plan->wave->shelf->dispatch rhythm. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #7 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Plan: lance-graph #958 merged (was open at last check) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Java RowStore facade: W3 shipped (185/185, one bug found+fixed) First real dispatch of the calcified wave system (wave-substrate-w3-w4.md Dispatch 1) — 3 Sonnet workers on disjoint scopes, Opus orchestrator integration and central gating, per the standing rules in .claude/waves/README.md. New public surface: RowStore (open/rowCount/isOpen/maskOfFacetClass/ facetMatches/close), FacetMatchView (rowCount/matchesOf/cardinality), FacetId (0..31-checked record) -- zero java.lang.foreign types in any public signature, ApiSurfaceTest passed unmodified. Mask.source() retyped NativePattern -> NativeResource (new minimal interface) so a mask parents onto either a pattern or a row store with the existing algebra unchanged; verified zero call-site breakage before the retype. One real bug caught by the test suite itself: FacetMatchView.rowCount() was missing the closed-store guard its sibling accessors both had -- found by RowStoreLifetimeTest on the first real run, fixed, re-verified. Gate: javac -Xlint:all clean (7 pre-existing [restricted] warnings, 0 new); AllTests 132 -> 185 (+53 checks: 29 parity + 24 lifetime). Both mandated disable-runs ran red-then-green with the exact expected blast radius: (1) Abi.requireMinor inflated by 1 -> exactly the two RowStore suites failed, 8 others stayed green; (2) the generator's a/b draw order swapped -> exactly RowStoreParityTest broke (17/29), the generator-independent RowStoreLifetimeTest stayed green. Board: STATUS_BOARD D-LGJ-W3 DONE, LATEST_STATE, and E-LGJ-WAVE-DISPATCH-VALIDATED-1 -- the wave system's first real dispatch, including an orchestrator-side false alarm (wrong env var name guessed instead of read from source) recorded so it isn't repeated. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #8 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Plan: r2sleigh recorded as third lift path + decompiler candidate Operator-flagged: AdaWorldAPI/r2sleigh (read-only clone verified, HEAD 60942f6) is a Rust workspace lifting Ghidra .sla specs to P-code via libsla, with typed IR, SSA, Z3 symbolic execution, and a P-code-to-C decompiler. The honest FFI fact is pinned: libsla-sys means the SLEIGH runtime underneath is Ghidra's C++ via FFI, not pure Rust -- acceptable on the same lift-time-only footing as running Ghidra itself. G1 gains candidate C (r2sleigh-cli lift, no JVM in the loop, and a STRONGER falsifier: cross-implementation P-code agreement between two independent consumers of one .sla spec); r2dec is named as the engine candidate for the semantic-shim direction; r2sym joins SymbolicSummaryZ3 as branch-population prior art. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Bench Component F: the boundary re-asked on the real row-store layout (W4) One Sonnet worker per wave-substrate-w3-w4.md Dispatch 2, orchestrator- run JMH (9/9 combos), the cross-check discipline intact: both Java facet-match kernels verified row-by-row against the native FacetMatchView in @setup at every row count before anything was timed. The finding: Component C's direction survives, its margin collapses. The Vector API wins the per-row 32-facet strided scan at every row count measured, but by 2.51x / 1.92x / 1.14x (4K / 65K / 1M rows) against C's 56x -- and at 512 MiB traversed all three arms converge on memory bandwidth. More work per byte narrows the boundary exactly as execution-boundary.md predicted; it now records that as measurement. Disclosed, not hidden: the native arm allocates its output segment per call where the Java arms reuse a @setup buffer; facetMatchesInto is the named follow-up if the small-row gap ever matters. Java kernels mirror the Rust chunk algorithm line-for-line (VectorMask.toLong() & 0x1111, same four-term fold) so the comparison is between implementations of ONE algorithm, not two algorithms. Mechanics: summarise.sh gains the F table (and its old 'E/F' section title -- a genuine collision with the new component -- is corrected to 'E'); TABLES.md regenerated from the merged CSV; RESULTS.md gains provenance-table update + full F section; RowStore gains a package-private handle() mirroring NativePattern's for the bench's split-package NativeAccess bridge; main suite re-verified 185/185 against the fresh minor-2 .so in the bench's expected location. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #9 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Parity: the third independent read path (ROW_LAYOUT segment reads) Closes the gap the W3 dispatch honestly flagged: the wave file specified a raw-lane segment-read parity arm that my worker brief dropped. Section added to RowStoreParityTest: every classid of a 1000-row store read DIRECTLY from the raw lane-0 segment, addressed through Layouts.ROW_LAYOUT's own byteOffset arithmetic (sequenceElement + groupElement, not hand-multiplied constants) -- no native kernel, no mask, no FacetMatchView on the path. Three independent routes now reach the same numbers: the native kernels, the pure-Java generator transcription, and the structured-layout segment read. This is also ROW_LAYOUT's first real consumer; before this it was defined and size-checked but read by nothing. Plus the raw lane's own description pinned (byteLength == n*512, contiguous flag set). AllTests 185 -> 188. Also records the operator handoff boundary in the ghidra plan: r2sleigh/ruff/R2IL integration arrives from another session -- this session does not build toward it, and lift-candidate C is frozen until the handoff lands. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #10 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Consumer example: World/Trades — the zero-object fluent domain API (W5a) The One-Billion-Objects poster made runnable, on the shelf-calcified wave map (wave-consumer-trades.md): two Sonnet workers, disjoint scopes, orchestrator-gated. consumers/trades/ is its own compile unit consuming com.adaworldapi.lancegraph exactly as a third-party developer would -- zero new membrane surface, zero core-API changes. Trade is a schema, not an entity: static U32Field VENUE / I32Field PRICE over the existing lanes, venue constants, and a private unconditionally-throwing constructor -- the test forces it accessible via reflection and proves construction STILL fails, plus zero public ctors and zero instance fields by reflection walk. The measured thesis: TradesAllocationTest's steady-state floor is 240 bytes per count() query, IDENTICAL at 64,000 and 1,000,000 rows -- allocation does not scale with rows (the assertion), with a 64 KiB absolute backstop. Laziness holds through the domain vocabulary: 0 crossings composing a 4-predicate Trade chain, exactly 1 at count(). Parity: the fluent chain equals a pure-Java transcribed-generator recomputation at both sizes, anti-vacuity guarded. Disable-run (green-red-green): VENUE pointed at the wrong lane -- the membrane's own LANE_KIND_MISMATCH rejected the misbinding outright, proving the schema binding is checked by the ABI, not trusted. Restored, both suites re-verified (12/12 + 3/3). QUANTITY is honestly absent (the flat fixture has two data lanes); its arrival is the ClassView/W6 slice, stated in Trade's Javadoc rather than faked. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #11 arc entry (post-merge) Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Consumer example: Bricks — mask-first authorization, fail-closed, aggregates only (W5b) The second consumer proof over the unchanged core: authorization is a predicate in the same lazy chain as where(...), composed before execution and evaluated natively — never a Java-side post-filter. Role.EU_ONLY folds REGION.eq(EU) into the plan; DENY_ALL is a genuine impossible predicate (REGION.eq(0xFFFF)) that pays a real crossing and counts zero; a chain that never called authorize() throws UnauthorizedQueryException before any crossing happens (fail-closed, no default-allow path). Aggregate-only egress is structural: every public BricksQuery method returns BricksQuery, long, or Map — no row-shaped public type exists. BricksAuthTest 62/62: generator parity at 1K+64K rows; EU_ONLY equivalence vs GLOBAL+explicit-where; DENY_ALL counts 0 through a real crossing; crossing arithmetic — count()=1, sumBy()=32 (16 groups x 2: plan eval + lgj_reduce_sum_i32), identical at both row counts, which is the thesis (crossings scale with groups, never rows). The measured 32 corrected the worker's "one crossing per group" Javadoc — a real finding about sum-terminal cost, recorded in the doc and asserted in the test. Disable-run: requireAuthorized short-circuited -> exactly the three can-fire fail-closed checks went red (59 green), restored, 62/62. Core suite unaffected (188/188). Zero new membrane surface, per the consumer iron rule. Boards: STATUS_BOARD D-LGJ-W5 bricks DONE; LATEST_STATE dispatch-4 entry (owning that W5a shipped without one). * Board: PR #12 arc entry (post-merge) * Board: record the C-band ruling — the classid domain byte carries ALTITUDE Operator ruling, 2026-08-18: "Java is an entire different layer that's why I chose another higher level." The classid domain byte is stratified by layer, not a flat namespace where placement is mnemonic or next-free. The C-band is the stratum above the Rust substrate: C0 Java/Panama/Valhalla (the membrane, and the FLOOR of that layer), C1 ogar-bricks + Databricks (the analyst estate), C4 Ghidra (a tenant of C0's layer -- Ghidra is itself a JVM application per this repo's own G0 archaeology -- and explosive, for the blast radius of turning any binary into addressable rows). The entry also records, as storno, three of my own proposals the ruling corrects: seating P-code at 0x1718 as an ogar-loco consumer slot (wrong tier -- 0x17 is lance-graph's internal orchestration: elixir-on-rails, rs-graph-llm, Rig marking the replayability boundary), putting P-code at 0x18 beside Blocks (same error one slot over), and proposing a separate substrate/layout-contract domain (not separate -- it is C0's content). Root cause, which recurred three times in one session: clustering by SHAPE (everything becomes (function : value) calls in a 512-byte node) when the real axis is ALTITUDE. Shape-similarity is not domain-identity. What survives: reuse loco's node shape, own your own domain -- loco's own doc says the FunctionBody classid belongs at the substrate and a frontend references it rather than minting its own. Borrowing the container is not joining the domain. One consequence for code here: W6's schema/classid field on LgjResourceInfo/LgjLaneDesc carries a C0 concept; the substrate plan's W6 line now says so. Nothing on the wave list is blocked -- the reservation is OGAR-side and operator-gated. * Board: reconcile Ghidra G1/G2 -- superseded by ruff_r2il, not built Checked what "the other session writing the autoadapting drill-down proposer" actually unblocks here, against the merged PR rather than the summary. AdaWorldAPI/ruff PR #94 shipped crates/ruff_r2il -- a typed intake arm (ore/furnace/slag) reading r2sleigh's R2IL/SSA directly, in-process, ~43s, no JVM roundtrip. Its residual ledger is deliberately left non-empty (B3's own falsifier makes residual == 0 a KILL) for a follow-on pass. That follow-on IS the drill-down proposer: PR2 in the R2IL plan's own wave ladder, reading ResidualLedger::by_address and proposing finer convention rows at each address, converging pass over pass. Not landed yet -- gated on PR1's corpus numbers. PR3 (the classid mint in lance-graph-contract::ogar_codebook, item O5) is gated on PR2. So there is nothing new to consume here today. What there is: ghidra-integration-v1.md's G1 (a bespoke analyzeHeadless lift script) and G2 (a hand-rolled LE image format) are superseded, not merely lower-priority -- the R2IL plan's own stop condition already answers the question those waves existed to answer ("direct r2il/r2ssa consumption solves the upstream seam -- YES, 43s"). Marked superseded in place per that plan's own HANDOFF BOUNDARY note, which asked for exactly this reconciliation once R2IL landed. wave-ogar-machine-pm1.md's gate #3 repointed from "Ghidra G1+G2 merged" to "ruff_r2il PR2+PR3 merged" so the next session checking the gate finds the real dependency instead of a dead one. A separate, independently-found gap flagged (not fixed): lance-graph's ogar_codebook wire-mirror of OGAR's ConceptDomain is already missing Ontology and Blocks (pre-existing drift, not caused here) and will also lack the new C0/C1/C4 domains once PR3 needs to route on them. No code changed. The C-band ruling (OGAR PR #276) is unaffected -- its 0xC4 BinaryLifting fence ("Ghidra and r2sleigh are two consumers of the same SLEIGH specs over ONE vocabulary") is now literally true in code rather than anticipated, since ruff_r2il path-deps r2sleigh directly. Full record: .claude/board/EPIPHANIES.md E-LGJ-GHIDRA-G1-G2-SUPERSEDED-BY-R2IL-1. * Board: ruff #96 is a different arm; read the real staging guide + ran S1 Checked "ruff 96 merged" against the actual PR. It's ruff_python_spo's plain-Python residual ledger (dismech/CURIE-constant harvest, ontology- shaped -- MONDO/KISAO/infores prefixes) -- a sibling drill-loop, but a DIFFERENT crate and DIFFERENT consumer than ruff_r2il, and unrelated to this repo's C-band/Ghidra/JavaRuntime track. Recorded so the two arms aren't confused later just because they share vocabulary ("residual ledger", "drill loop", "proposer"). The genuinely relevant find, unrelated to #96, was already on ruff main: .claude/harvest/r2il/STAGED-CODEGEN-GUIDE.md (commit bbaebda, pushed directly to main between PR #94 and #95), explicitly addressed to "the sibling session ... (the Ghidra console work)" -- this repo, by description. It confirms PR2 (routes -> V3) still hasn't landed and adds what the prior reconciliation lacked: a 5-stage staging order (S1 ledger- read -> S2 ore-join -> S3 additive codegen -> S4 one consumer -> S5 target-profile fork), explicit "do not skip to S3", and a stability table per artifact (FlatFact's payload slots and the placeholder VarnodeFacet classid are NOT stable yet; slag/census/provenance/convention are). Ran S1 -- read-only, no codegen, no PR2 dependency -- against the real in-tree harvest artifacts (gitignored but present in the ruff checkout): B1 conservation PASS (dropped=0, harvested=classified+residual); B2 at 91.30% (inside the declared 90-99% INVESTIGATE band, not a KILL); B3 PASS (43 distinct residual shapes, dominant_share 0.215, every non-trivial bucket carries an address); and the pre-registered 60-80%-classified prediction MISSED at a measured 14.15%, recorded honestly rather than hidden -- which is the point of pre-registering it. Dominant residual is opcode_not_in_convention, expected: pass 1 deliberately classifies only 7 of P-code's 74 opcodes. Corpus is r2sleigh's own e2e stress-test fixtures (143 functions, x86-64), not yet a Ghidra-shaped real binary. No code changed, no wave-gate change -- PR2/PR3 remain unmerged, so wave-ogar-machine-pm1.md's gate #3 stands as previously repointed. Next unblocked step, available whenever there's a reason to spend it: S2 (join ore rows to native addresses), still read-only. * Board: R2IL handshake outcomes + the Valhalla-premise storno (operator-caught) The R2IL session answered all five cross-session questions; outcomes and ownership recorded (mirror sync now owned here; consumers/ghidra is the expected end-state, gated on PR2's layout doc + PR3's classids). The entry's core is a storno of my own handoff premise: 'Valhalla was a laboratory phase, not a door' was right about addressability and wrong about integration -- the shipping descriptor vocabulary is value-record- ready by design (one word per type) and the A/B ran on a real EA build with measured numbers. The ruling (0xC0 = Panama alone) survives on the corrected premise: Valhalla is a designed PROPERTY of the C0 concepts, and properties of concepts do not get domains. OGAR PR #277 (merged) carries the canonical corrected text. * rowstore: generate_with_edges -- the graph-consumer wave's real, measured blocker, cleared Asked what could be built while ruff_r2il's PR2/PR3 are blocked. Nearly dispatched the graph consumer (W5c) on the strength of D1a's mechanism (writable masks, existing facet-match) being sufficient -- claimed twice in earlier turns -- before re-reading wave-consumer-graph.md's own STOP condition in full and catching a real, different blocker: plain RowStore::generate()'s payload is uniform random noise, so a decoded 1-2 hop BFS over it saturates to nearly every row regardless of decode convention -- vacuous under the wave's own anti-vacuity falsifier ("seed/1-hop/2-hop must be three different, non-empty, non-total sizes"). A data-shape problem, not a mechanism problem. Caught before any workers spawned. RowStore::generate_with_edges(n_rows, seed, edge_classid, edge_gate_mask, edge_radius) is the fix -- additive, generate() untouched. Classid assignment is byte-identical to generate() (same SplitMix64 draws, same formula; the classid formula's otherwise-unused high bits of the per-facet `a` draw become an independent sparsity gate, so edge_classid=16, out of range, reproduces generate() exactly -- pinned by test). A sparse, gated subset of edge_classid-matching facets get a bounded local-neighbourhood target row instead of raw noise, which is what keeps a 1-2 hop BFS non-vacuous. Parameters chosen from a real measurement sweep (examples/graph_density_probe.rs), not guessed: a first pass at n_rows= 1000 was too small (avg degree < 1, everything collapsed to zero); widened to n_rows=20_000 for usable numbers, then re-measured at a test-suite-sized n_rows=2000 for the pinned regression: a 10-row seed reaches exactly 19 rows at 1 hop, 29 at 2 hops -- three different, non-empty, non-total sizes, matching the wave's own falsifier shape exactly, pinned as measured_hop_counts_are_three_distinct_non_empty_ non_total_sizes. Two disable-runs: breaking the radius-wrap formula turned exactly the three tests touching the target formula red (transcription test, in-bounds/radius invariant, the pinned regression), leaving the seven tests that don't touch it green. Ignoring the sparsity gate mask turned only the transcription test red -- the in-bounds/radius invariant correctly stayed green, verified as the right outcome rather than a vacuous test: geometry validity is orthogonal to which facets get the treatment, only to the treatment's correctness once applied. wave-consumer-graph.md updated in place: the STOP condition marked RESOLVED with the measured numbers, and the stale "calcify, do not dispatch" header corrected (that gate was already lifted session-wide when W5a/W5b shipped under the identical wording). The graph consumer is now genuinely dispatchable -- not dispatched in this same pass; this change is scoped to the substrate-tier generator only, per the wave file's own rule that a generator extension is not a consumer hack. Gates: lgj-abi 90/90 (was 84, +6 new tests), fmt clean, clippy --all-targets --all-features clean. Board: EPIPHANIES + LATEST_STATE entries prepended in the same commit. * lgj-abi: edge-bearing row store ABI addition (lgj_rowstore_open_with_edges, minor 3) The graph-consumer wave's own STOP-condition-RESOLVED note proved RowStore::generate_with_edges at the Rust generator level but left no membrane path to it: Engine.openRowStore/registry::open_rowstore only ever called plain RowStore::generate, so no ABI symbol existed for Java to reach edge-bearing data at all. Found before dispatching the graph wave's G1/G2 workers, closed as the wave's own D1b rule requires -- growing the membrane is W-tier orchestrator work, not a consumer worker's ad hoc addition. lgj_rowstore_open_with_edges (docs/abi.md ss12, ABI minor 2->3): mirrors lgj_rowstore_open symbol-for-symbol -- same LGJ_RESOURCE_ROWSTORE kind, same lane shape, no new mask op, purely an alternative constructor. Threaded through registry.rs -> exports.rs -> Downcalls/Engine (Abi.requireMinor(3), matching the row store's own minor-2 gate) -> RowStore.openWithEdges. Added a Java-side transcription of the D1a hop mechanism itself (facet-match crossing + raw lane-0 payload decode, zero new ABI op) to RowStoreParityTest, at the exact parameters already pinned as a Rust regression -- it reproduces the identical hop counts (10-row seed -> 19 at 1 hop -> 29 at 2 hops), proving the membrane carries the same edge structure, not merely the same classid stream. Gates: cargo test 93/93 (+3), clippy -D warnings + fmt clean, release build exports the new symbol (nm -D). Java AllTests 194/194 (+6). Two disable-runs, both red-then-green: a classid-not-threaded bug at the registry level, and the Java hop's classid-match condition forced to always skip. Board: STATUS_BOARD D-LGJ-W6, LATEST_STATE dated entry, EPIPHANIES entry recording the gap and its fix. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #14 arc entry (edge-bearing row store ABI addition) Post-merge hygiene commit -- the squash sha (be8fb60) is only knowable after the merge, so this follows PR #14 rather than landing in it. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * RowStore: public per-row payload accessors (classidAt/payloadLow64At/payloadHi32At) The graph-consumer wave's substrate was proven at the Rust generator level (RowStore::generate_with_edges) and the ABI membrane level (lgj_rowstore_open_with_edges, PR #14) -- but working through concretely how consumers/graph's Graph.hop() would decode a matched facet's target row surfaced a third gap: facetMatches returns only a per-row bitset of WHICH facets matched, never the payload bytes, and the only thing that ever read raw row bytes (internal.ffm.Engine.describeLane) is off-limits to a consumer package by ApiSurfaceTest's own design. Zero new ABI surface needed -- these three methods reuse lgj_lane_describe (already ABI minor 1, already a "lifecycle" crossing per abi.md ss6), resolved once per store and cached; every read after is in-process, matching exports.rs's own doctrine: "if Java wants one row it reads the MemorySegment in-process, with no crossing at all." AllTests 204/204 (+10 over 194). RowStoreParityTest reproduces the SAME pinned hop numbers (19 at 1 hop, 29 at 2 hops) a second time, this time through the genuinely public path a real consumer has to use. A redundant closed-store guard (two copies: one on the method that touches the pointer, one on pure arithmetic downstream of it) produced a false-negative disable-run -- 30/30 green under genuinely broken code, masked by Java's receiver-before-argument evaluation order. Caught by asking why the disable didn't fire rather than trusting the green result; de-duplicated to the single correct location; re-ran the same disable-run and confirmed it now goes red-then-green for real. Board: STATUS_BOARD D-LGJ-W7, LATEST_STATE dated entry, EPIPHANIES entry recording both the gap and the self-caught vacuous falsifier. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #16 arc entry (RowStore public per-row payload accessors) Post-merge hygiene commit -- the squash sha (8e4f9aa) is only knowable after the merge. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Wave: graph consumer -- record the D1 mask-write finding before G1/G2 dispatch Orchestrator design ruling, recorded per this wave file's own "Decision D1 resolves BEFORE any worker spawns" requirement: checked whether masks could be built from Java-computed row indices (the scatter half of D1a) and found no public constructor exists -- a fourth core-facade gap in the same shape as D-LGJ-W6/W7. Ruled the hop's row-set currency is a Java-side long[]/Set<Long> for this consumer example rather than building a fifth substrate capability under time pressure; documented why that still satisfies every stated falsifier. Also corrected the crossings instrument reference from internal.ffm.Downcalls.crossings() (off-limits to a consumer) to the public Diagnostics.crossings() before it could mislead a worker into an internal-package import. G1 (traversal facade) and G2 (falsifier tests) already dispatched against this exact content, inlined directly into their prompts. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * consumers/graph: traversal facade + falsifiers (Graph/Edge, GraphHopTest) Two Sonnet workers dispatched per wave-consumer-graph.md, only after the substrate was proven complete at all three levels (generator, ABI membrane, public core facade). G1: Graph/Edge -- immutable chaining over a plain long[] row-index frontier (no native Mask has a public constructor from Java-computed rows; ruled a documented simplification rather than a fourth substrate detour). G2: GraphHopTest -- hop correctness via two independent pure-Java BFS transcriptions, crossings proportional to hops, anti-vacuity, zero-serialization, Edge's reflection guard. Compiled and tested centrally: GraphHopTest 43/43, core suite 204/204 unaffected, trades (12+3) and bricks (62) unaffected. Mandated disable-run (target-decode offset corrupted by +4) went red exactly as required -- the SET-equality check caught it even though the row COUNT coincidentally still matched. One real finding caught and fixed before shipping: G2's crossings test assumed every hop costs an identical number of crossings. Measured directly with a standalone probe: hop 1 on a fresh store costs 2 (the facetMatches crossing plus a one-time RowStore.rawLane() resolution its first payload read triggers), hops 2/3/4 each cost exactly 1, steady-state. Corrected Graph.hop()'s javadoc and GraphHopTest's crossing assertions to state this precisely across three consecutive hops with three different source-row counts, rather than leave the wrong "identical from hop 1" assumption in either place. All three planned consumer examples (trades, bricks, graph) are now DONE. Board: STATUS_BOARD D-LGJ-W5 (graph row), LATEST_STATE dated entry, EPIPHANIES entry, wave file marked DONE. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * Board: PR #18 arc entry (consumers/graph traversal facade + falsifiers) Post-merge hygiene commit -- the squash sha (5d3e694) is only knowable after the merge. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs * D-LGJ-W8 A3 freeze: ratified correction spec v3 + root CLAUDE.md + board storno/row/state (PR-0) The operator's CORRECTION WAVE + RULING CLARIFICATION + A1 ARCHITECTURE RULING (2026-08-18), taken through the full supervision ladder before any implementation: A0 six-lens drift audit (CONFIRMED), A1 combined spec (Part I mask-native correction + Part II 64K parallel-SoA compute/placement/publication, grounded in a 44-finding savant pass and a 37-finding lance-graph spine audit), operator A2 verdict, three adversarial reviewers (1 P0 resolved -- model identifiers de-named to roles in the committed spec; 6 P1 applied: the 8.4 evidence re-scoping, the 9 axis split at the arrival-order leak, F-LAND containment, G2 respecification, the G11 contract-import fence, same-commit board artifacts in the W8a/W8b gate columns; ~20 P2 applied). Spec v3 is RATIFIED and executable. This freeze commit lands, in one commit per board discipline: - .claude/plans/mask-native-navigation-correction-v1.md (spec v3, the full change ledger v1->v2->v2.1->v3 included) - CLAUDE.md (CREATED -- the repo's first root policy guard: the mask-native invariant extended to compute/land/batch, the named import/materialize exceptions, the three-axes model, the GridLake hard gate, the missing-capability STOP rule; deliberately carries no model-policy section) - EPIPHANIES.md storno E-LGJ-ERGONOMICS-MUST-NOT-LEAK-INTO-CURRENCY-1 (corrects Graph.java:18-19's "not a workaround" as target precedent; preserves what PR #18 proved; records the ruling and the guard-test-calcification finding) - STATUS_BOARD.md D-LGJ-W8 row (gate ladder, FREEZE = this commit) - LATEST_STATE.md correction-of-record entry Implementation follows as PR-N (ndarray mask_andnot + simd.rs re-export, merges first), PR-W8a (contract dep + FixtureClassView + lgj_mask_andnot + lgj_hop, ABI minor 4), PR-W8b (Java WideFieldMask + RowStore.hop/importRows + Mask.minus/materializeRows + Graph migration). No implementation work is included here. * Board: PR #20 arc entry (D-LGJ-W8 A3 freeze — spec v3 + root CLAUDE.md + storno) * D-LGJ-W8 PR-W8a: contract dep + FixtureClassView + lgj_mask_andnot + lgj_hop (ABI minor 4) The substrate half of the council-ratified mask-native correction (spec v3 sections 3.2-3.4; PR-N, ndarray #280, merged first): - lance-graph-contract path dep, default-features = false — the RULING's contract inheritance: the law without the engine. Import fence honored: class_view / canonical_node / ontology modules only (gate G11). Cargo.toml's stale "ONLY dependency" comment corrected. - class_view_provider.rs (NEW): FixtureClassView implementing the contract ClassView trait (32 FieldRefs via OnceLock); the named provider seam fns edge_participation/decode_mode; class_id_for as the explicit u32-to-u16 bounds-checked boundary (ISSUES.md ISS-LGJ-CLASSID-WIDTH-PIN). - lgj_mask_andnot: dst = a & !b with the dedup-before-lock aliasing discipline as a 5-branch tree — ANDNOT is non-commutative, so dst==b genuinely differs from dst==a and needs a scratch copy of a (a structural divergence from mask_binop's 4-branch shape, doc'd). Kernel: ndarray::simd::{mask_andnot, mask_andnot_assign} + explicit repairing tail-clear. - lgj_hop: decode-mode fence FIRST (modes != 0 -> new status -14, dst provably untouched); src words snapshotted under a read lock fully released before the dst write lock (dst==src aliasing is deadlock-free by construction, per council S3-4); composition kernel — classid-match through the existing sanctioned eq_u32_strided_to_mask into one reused scratch, scalar decode + scatter only; u64 bounds check BEFORE any usize cast (S3-6); effective participation = facet_mask intersect provider answer. - abi.rs: LGJ_ABI_MINOR 3 -> 4 (dated entry); LGJ_ERR_UNSUPPORTED_ DECODE_MODE = -14. - docs/abi.md: section 13 (both symbols, full semantics); counts 19 -> 21; minor-history subsection; section 12's Java-layer hop composition regraded SUPERSEDED in place (append-only). Gates, run centrally: cargo test 110/110; clippy -D warnings clean; fmt clean; release build exports 21/21 lgj_ symbols (nm -D). Disable-runs, each red-then-green: G6(a) decode offset +4 -> exactly the 10/19/29 fixture-parity test red; G6(b) provider participation forced EMPTY -> fixture parity + provider unit test red; G6(c) tail-clear removed -> the corrupted-operand repair test red; G6(d) mode fence bypassed -> the reserved-mode test red; G6(e) ptr_eq dedup branch bypassed -> the aliasing test DEADLOCKS (60s timeout kill) — the S3-4 deadlock is real, the discipline is load-bearing. Board (same commit): STATUS_BOARD SUBSTRATE flip; LATEST_STATE entry; ISSUES.md ISS-LGJ-CLASSID-WIDTH-PIN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017Pud4qpxFHwqyqDjSabQbs --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
1 parent 92a0e55 commit 9019417

11 files changed

Lines changed: 1370 additions & 9 deletions

File tree

.claude/board/ISSUES.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,18 @@
11
# Issues Log — Open + Resolved (double-entry, append-only)
22

3+
## ISS-LGJ-CLASSID-WIDTH-PIN (2026-08-18) — OPEN, upstream-owned
4+
5+
The lgj wire carries u32 classids (canon 8-hex; `lgj_op_eq_classid`,
6+
`lgj_hop`); the contract's `class_view::ClassId` is u16 while
7+
`rbac::ClassId` is u32 — two widths coexist UPSTREAM in
8+
lance-graph-contract itself. PR-W8a pins the boundary locally:
9+
`class_view_provider::class_id_for` does the explicit u32→u16
10+
bounds-checked conversion (None past `u16::MAX`), with a falsifiable
11+
test pinning the behavior. The capacity tension is an upstream mint
12+
question (named in MedCare-rs commitment #10 and spec NG8) — surfaced
13+
here, deliberately NOT resolved here. Closes when upstream unifies the
14+
width or rules the split permanent.
15+
316
## ISS-LGJ-FANOUT-UNREVIEWED (2026-08-17) — PARTIALLY RESOLVED
417

518
**Resolution for the core (Rust ABI + Java facade), same day:** all three

.claude/board/LATEST_STATE.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,26 @@
1+
## 2026-08-18 — D-LGJ-W8 SUBSTRATE landed: ABI minor 4 (lgj_mask_andnot + lgj_hop), contract dep live, 21 symbols
2+
3+
PR-N (ndarray #280) then PR-W8a, per the spec's merge order. The lgj-abi
4+
crate now depends on `lance-graph-contract` (path, default-features
5+
off — the RULING's one-closure contract inheritance, engine never), with
6+
`FixtureClassView` as the late-bound law provider and `class_id_for` as
7+
the pinned u32→u16 boundary (ISSUES.md ISS-LGJ-CLASSID-WIDTH-PIN).
8+
`lgj_mask_andnot` implements the dedup-before-lock aliasing discipline
9+
with a 5-branch tree (ANDNOT is non-commutative: dst==b needs a scratch
10+
copy of a — a real structural divergence from mask_binop, found during
11+
implementation, doc'd in place); `lgj_hop` is the composition kernel
12+
(existing eq_u32_strided_to_mask for classid-match, scalar decode+
13+
scatter, u64 bounds-check BEFORE cast, snapshot-then-write so dst==src
14+
aliasing is deadlock-free by construction). Gates: 110/110; clippy -D
15+
warnings + fmt; release .so exports 21/21; disable-runs G6(a)-(e) all
16+
red-then-green — (e) proved the deadlock is REAL (60s timeout kill with
17+
the ptr_eq dedup bypassed), the council's S3-4 catch now mechanically
18+
demonstrated. abi.md: §13 added, §12's Java-layer hop composition
19+
marked ⊘ SUPERSEDED (append-only regrade). W8b (Java facade + Graph
20+
migration) dispatched in parallel. Separately: the LOTUS SEAL / FRACTAL
21+
COMMIT FRONTIER research charter opened for lance-graph (operator,
22+
2026-08-18) — research runs there, not here.
23+
124
## 2026-08-18 — D-LGJ-W8 A3 FREEZE: the mask-native correction is ruled, specced, council-ratified; root CLAUDE.md created (PR-0)
225

326
The operator's CORRECTION WAVE + RULING CLARIFICATION + A1 ARCHITECTURE

.claude/board/STATUS_BOARD.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ the policy guard, created at the A3 freeze.
77

88
| D-id | Deliverable | Status |
99
|---|---|---|
10-
| D-LGJ-W8 | Mask-native navigation correction + 64K-compute architecture inheritance. Gate ladder: AUDIT → SPEC → COUNCIL → FREEZE → SUBSTRATE (ndarray PR-N + lgj-abi PR-W8a) → FACADE (PR-W8b) → GRAPH MIGRATION → FALSIFIERS → POLICY | **AUDIT ✓** (A0 6-lens, drift CONFIRMED) → **SPEC ✓** (combined mask + compute, two audit fleets: 44 savant findings + 37 spine findings) → **COUNCIL ✓** (operator A2 verdict + 3 reviewers: 1 P0 resolved — model identifiers de-named from the spec; 6 P1 + ~20 P2 applied; v3 ratified 2026-08-18) → **FREEZE: this commit (PR-0)** → SUBSTRATE/FACADE/MIGRATION queued per spec §3.10 |
10+
| D-LGJ-W8 | Mask-native navigation correction + 64K-compute architecture inheritance. Gate ladder: AUDIT → SPEC → COUNCIL → FREEZE → SUBSTRATE (ndarray PR-N + lgj-abi PR-W8a) → FACADE (PR-W8b) → GRAPH MIGRATION → FALSIFIERS → POLICY | **AUDIT ✓** → **SPEC ✓** → **COUNCIL ✓** (v3 ratified) → **FREEZE ✓** (PR #20, squash `c479f76`) → **SUBSTRATE ✓ 2026-08-18** — PR-N (ndarray #280, `3f3504f`: mask_andnot family + `ndarray::simd` re-export + blackboard W1a-deviation entry; 51/51, vector-path disable-run red-then-green) + PR-W8a (this commit: contract dep `default-features=false`, `FixtureClassView` + provider seam fns + `class_id_for` width pin, `lgj_mask_andnot` with the 5-branch non-commutative aliasing tree, `lgj_hop` with snapshot-then-write + composition kernel + bounds-before-cast, minor 4, status −14, abi.md §13; **110/110**, clippy `-D warnings` + fmt clean, `nm -D` **21/21** symbols, disable-runs **G6(a)-(e) ALL red-then-green** incl. the aliasing DEADLOCK proven by timeout under a bypassed dedup branch, G11 import fence honored: contract imports = class_view/canonical_node/ontology only) → FACADE/MIGRATION (PR-W8b) in flight |
1111

1212

1313

docs/abi.md

Lines changed: 158 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,11 @@ cannot disagree with itself.
6262
The ABI is a **machine membrane**. It is not the product. The product is the Java
6363
semantic API (see `architecture.md`). Therefore:
6464

65-
- It is **small** — currently 19 symbols (minor 3; the "14" this line carried
65+
- It is **small** — currently 21 symbols (minor 4; the "14" this line carried
6666
at minor 1 was arithmetic drift — the §7 list it referred to already
6767
enumerated 15). Growth is a design smell to be argued for, not a default;
68-
minor 2's three additions are argued in §11, minor 3's one addition in §12.
68+
minor 2's three additions are argued in §11, minor 3's one addition in
69+
§12, minor 4's two additions in §13.
6970
- It is **bulk-only**. Every call must be capable of doing work proportional to
7071
`n_rows` (see §6 — the anti-JNI rule).
7172
- It speaks **resource, lane, view, mask, operation, descriptor, status,
@@ -78,7 +79,7 @@ semantic API (see `architecture.md`). Therefore:
7879

7980
```
8081
LGJ_ABI_MAJOR = 0 // incompatible change ⇒ bump; Java refuses to load
81-
LGJ_ABI_MINOR = 3 // additive change ⇒ bump; older Java may still load
82+
LGJ_ABI_MINOR = 4 // additive change ⇒ bump; older Java may still load
8283
LGJ_MAGIC = 0x4C_47_4A_5F_41_42_49_00 // "LGJ_ABI\0" big-endian-read
8384
```
8485

@@ -89,6 +90,14 @@ The magic doubles as an endianness probe: read as a `u64` little-endian it yield
8990
a known constant; anything else means the library was built for a different byte
9091
order and every subsequent read would be garbage.
9192

93+
### Minor version history
94+
95+
- **Minor 2** (2026-08-17) — the SoA row store (§11).
96+
- **Minor 3** (2026-08-18) — the edge-bearing row store (§12).
97+
- **Minor 4** (2026-08-18, D-LGJ-W8) — `lgj_mask_andnot` (mask complement)
98+
and `lgj_hop` (one-hop graph traversal, gated by the
99+
`lance-graph-contract` `ClassView`/`FieldMask` LAW — §13).
100+
92101
## 3. Status codes
93102

94103
Every function returns `i32`. `0` is success; all failures are negative. There
@@ -110,6 +119,7 @@ are no error strings across the membrane and no `errno` dependence.
110119
| `-11` | `EMPTY_PLAN` | a plan with zero ops was submitted |
111120
| `-12` | `ALLOCATION_FAILED` | the allocator refused |
112121
| `-13` | `READ_ONLY` | write attempted against a read-only lane |
122+
| `-14` | `UNSUPPORTED_DECODE_MODE` | `lgj_hop` called with a `decode_mode` this build does not yet implement (§13, ABI minor ≥ 4) |
113123

114124
`INVALID_HANDLE` is deliberately the response to *use-after-close*, not a crash.
115125
See §4.
@@ -269,7 +279,7 @@ predicates or rows are involved. The unfused per-predicate ops are retained only
269279
so the fused path can be benchmarked *against* something and so parity can be
270280
checked predicate-by-predicate.
271281

272-
## 7. The function surface (19 symbols)
282+
## 7. The function surface (21 symbols)
273283

274284
All symbols are prefixed `lgj_`. All return `i32` status except the manifest
275285
getter. `out_*` parameters are written only on `OK`.
@@ -310,10 +320,14 @@ i32 lgj_mask_create(u64 parent, u32 initial, u64* out_handle) // initial: 0=em
310320
i32 lgj_mask_describe(u64 mask, LgjLaneDesc* out) // MASK_WORD lane, WRITABLE
311321
i32 lgj_mask_and(u64 a, u64 b, u64 dst)
312322
i32 lgj_mask_or(u64 a, u64 b, u64 dst)
323+
i32 lgj_mask_andnot(u64 a, u64 b, u64 dst) // ABI minor ≥ 4, see §13
313324
i32 lgj_mask_count(u64 mask, u64* out_count)
314325
```
315326

316-
`dst` may alias `a` or `b`. All three must share the same parent and row count.
327+
`dst` may alias `a` or `b` for `lgj_mask_and`/`lgj_mask_or`. All three must
328+
share the same parent and row count. `lgj_mask_andnot` permits the same
329+
aliasing but is NOT commutative (`a & !b ≠ b & !a`) — its full aliasing and
330+
tail-clearing rules are §13's, not repeated here.
317331

318332
### Bulk predicates (unfused — one predicate per crossing)
319333

@@ -365,6 +379,19 @@ Identical semantics to `lgj_plan_eval` but forced down the scalar reference path
365379
Exists **only** so SIMD-vs-scalar parity is falsifiable *through the membrane*,
366380
which is where the Java tests live. Not for production use.
367381

382+
### Graph traversal (ABI minor ≥ 4)
383+
384+
```
385+
i32 lgj_hop(u64 store, u32 edge_classid, u64 facet_mask, u32 decode_mode,
386+
u64 src_mask, u64 dst_mask)
387+
```
388+
389+
Overwrites `dst_mask` with the one-hop reachable set from `src_mask` over
390+
`store`'s `edge_classid`-matching facets, gated by the `lance-graph-contract`
391+
`ClassView`/`FieldMask` LAW. §13 is the full normative statement (effective
392+
participation, decode modes, the snapshot-then-write aliasing discipline,
393+
bounds-before-cast).
394+
368395
## 8. SIMD provenance
369396

370397
Every kernel behind these symbols routes through **`ndarray::simd`** and nothing
@@ -532,6 +559,17 @@ this generator family).
532559

533560
### A hop, at the Java layer (no new op — composition of existing symbols)
534561

562+
> **⊘ SUPERSEDED (2026-08-18, D-LGJ-W8).** The Java-side composition
563+
> described below is exactly the row-population-hydration shape the
564+
> mask-native navigation correction (`.claude/plans/mask-native-navigation-correction-v1.md`)
565+
> demotes: per-row `payload_lo64` decode + a Java-side scatter loop is a
566+
> materialised-population execution path, not a mask-native one. The
567+
> shipped path — required, not merely preferred (spec §3.8: consumers
568+
> cannot reach the segment surface at all; `ApiSurfaceTest` walls off
569+
> `internal.ffm`) — is the native **§13** `lgj_hop`, ABI minor ≥ 4. This
570+
> subsection is kept, not deleted, as the record of the D1a composition it
571+
> replaces.
572+
535573
A hop is `lgj_row_facet_match(edge_classid)` (§11, unchanged) to find which
536574
of a row's facets carry the edge classid, followed by a **Java-side** decode
537575
of each matched facet's `payload_lo64` as the target row and a scatter into
@@ -546,3 +584,118 @@ The generator's per-facet loop is scalar (SplitMix64 is inherently
546584
sequential per draw); the sparsity gate and target-row arithmetic are cheap
547585
integer ops on that same sequential stream — no new SIMD kernel, matching
548586
`RowStore::generate`'s own scalar generation loop.
587+
588+
## 13. Mask complement + one-hop graph traversal (ABI minor ≥ 4)
589+
590+
Two additions, both consuming the `class_view_provider::edge_participation`
591+
seam (`.claude/plans/mask-native-navigation-correction-v1.md`, D-LGJ-W8,
592+
§3.3): `lgj_mask_andnot` closes a real gap in the mask algebra — no
593+
and-not/complement op existed anywhere in the mask/registry system before
594+
this minor — and `lgj_hop` is the FIRST symbol in this ABI whose semantics
595+
are governed by the `lance-graph-contract` crate rather than by this
596+
crate's fixture alone: a facet only participates in a hop if BOTH the
597+
caller's `facet_mask` and the class's `ClassView`-resolved
598+
`edge_participation` agree it should.
599+
600+
Both are **bulk** (§6): `lgj_mask_andnot` does work ∝ `n_rows / 64` words;
601+
`lgj_hop` does work ∝ `n_rows · popcount(effective participation)` — at
602+
most `n_rows · 32` facet-classid compares, plus one scalar decode+scatter
603+
per matched `(row, facet)` pair.
604+
605+
### Mask complement
606+
607+
```
608+
i32 lgj_mask_andnot(u64 a, u64 b, u64 dst)
609+
```
610+
611+
`dst = a & !b`, word-wise. Same parent/row-count compatibility rule as
612+
`lgj_mask_and`/`lgj_mask_or` (§7): all three masks must share the same
613+
parent and row count, or `MASK_LENGTH_MISMATCH`.
614+
615+
`dst` may alias `a`, `b`, or both — **unlike AND/OR, ANDNOT is not
616+
commutative**, so aliasing `dst == b` is NOT the same case as `dst == a`
617+
with the roles swapped: the kernel snapshots `b`'s value into a scratch
618+
buffer before it is overwritten whenever `dst` aliases `b`, so the result
619+
is always `a & !b` as evaluated BEFORE the call, regardless of which
620+
argument `dst` aliases.
621+
622+
**Tail rule (normative).** Bits at row index `>= n_rows` in the final word
623+
are always zero on return, re-established as a *distinct, defensive* step
624+
after the complement — never merely inherited from well-formed inputs. A
625+
corrupted operand's tail bits (bits set past `n_rows` by an out-of-band
626+
write) are silently REPAIRED, not merely preserved: `!b`'s tail naturally
627+
sets bits wherever `b`'s own tail is zero (the well-formed case), so a
628+
version of this kernel without the explicit clear would leak a corrupted
629+
`a`'s stray tail bits straight into `dst`.
630+
631+
Kernel: `ndarray::simd::{mask_andnot, mask_andnot_assign}` (§8 SIMD
632+
provenance, unchanged rule) — never `ndarray::simd_int_ops` directly.
633+
634+
### One-hop graph traversal
635+
636+
```
637+
i32 lgj_hop(u64 store, u32 edge_classid, u64 facet_mask, u32 decode_mode,
638+
u64 src_mask, u64 dst_mask)
639+
```
640+
641+
`store` must be a `LGJ_RESOURCE_ROWSTORE` (§11); `src_mask` and `dst_mask`
642+
must be row-count-compatible with it — the same row-count-only reading
643+
`lgj_op_eq_classid` already uses (a mask from an equally sized but
644+
distinct resource is accepted; `MASK_LENGTH_MISMATCH` otherwise).
645+
646+
**Semantics.** `dst_mask` is OVERWRITTEN with the one-hop reachable set:
647+
for every row `r` set in `src_mask`, for every facet `f` in the *effective
648+
participation* where `classid(r, f) == edge_classid`, if the selected
649+
decode mode yields a valid target `t < n_rows`, bit `t` is set in the
650+
result.
651+
652+
**Effective participation** is `facet_mask ∩
653+
edge_participation(edge_classid)` — the caller's requested facets narrowed
654+
by what the class's `ClassView` provider
655+
(`class_view_provider::edge_participation`, D-LGJ-W8 §3.3) says this
656+
class's edges actually occupy. `facet_mask` is the wire form of the
657+
contract's `FieldMask`: a `u64` whose bits `>= 32` are ignored (this store
658+
has 32 facets).
659+
660+
**Decode modes.** `decode_mode = 0` is the §12 fixture convention:
661+
`payload_hi32 == 0` marks a structured edge, `payload_lo64` is the LE
662+
target row. Modes `1..=3` are RESERVED — mirroring `EdgeCodecFlavor as u32
663+
+ 1` (`canonical_node.rs`) — and return the new status
664+
`LGJ_ERR_UNSUPPORTED_DECODE_MODE = -14` until real class data lands. This
665+
check runs FIRST, before `store`/`src_mask`/`dst_mask` are even resolved,
666+
so `dst_mask` is provably untouched (`out_*` write-only-on-OK, §7) on a
667+
rejected call regardless of whether the other arguments would themselves
668+
have been valid.
669+
670+
**Aliasing.** `src_mask` is snapshotted (its words copied into an owned
671+
buffer) under a READ lock that is fully RELEASED before `dst_mask`'s
672+
WRITE lock is taken. This is a DIFFERENT discipline from
673+
`lgj_mask_and`/`lgj_mask_or`/`lgj_mask_andnot`'s dedup-before-lock scheme
674+
(registry.rs `lock_masks_ordered`, needed there because those calls hold
675+
two or three mask locks simultaneously): `lgj_hop` never holds more than
676+
one mask lock at a time, so `dst_mask == src_mask` aliasing carries zero
677+
deadlock risk by construction, not by case analysis.
678+
679+
**Bounds-before-cast (normative).** The decoded target `t` is compared
680+
against `n_rows` as a `u64` BEFORE any `t as usize` cast — the ordering is
681+
part of the contract, not an implementation detail, so an out-of-range
682+
`u64` target can never reach an indexing operation.
683+
684+
**Kernel composition.** The classid-match sub-step for each participating
685+
facet routes through the EXISTING sanctioned primitive
686+
(`kernels::simd_rowstore_classid_mask`, `ndarray::simd::eq_u32_strided_to_mask`
687+
— the same kernel `lgj_op_eq_classid` uses, §11) into a scratch word
688+
buffer that is REUSED across every participating facet, never reallocated
689+
per facet. Only the resulting set-bit walk + payload decode + scatter is
690+
scalar: there is no `ndarray::simd` primitive for gather-decode-scatter,
691+
and duplicating the classid compare in scalar Rust would be exactly the
692+
polyfill bypass §8 forbids.
693+
694+
### Bulk-rule conformance (§6, applied)
695+
696+
Neither symbol is lifecycle, and neither is a fixed-cost call: a caller
697+
that doubles `n_rows` observes roughly double the work in both — for
698+
`lgj_mask_andnot`, twice the mask words; for `lgj_hop`, twice the rows
699+
scanned per participating facet. Both therefore satisfy §6's anti-JNI rule
700+
by the same test every existing symbol satisfies it by: work proportional
701+
to `n_rows`, never one crossing per element.

0 commit comments

Comments
 (0)