Skip to content

fix(shape): share gather and tensor contracts - #644

Draft
fhanuman wants to merge 1 commit into
pr/shape-stack-06c-causal-conv-contractsfrom
pr/shape-stack-06-gather-tensor
Draft

fix(shape): share gather and tensor contracts#644
fhanuman wants to merge 1 commit into
pr/shape-stack-06c-causal-conv-contractsfrom
pr/shape-stack-06-gather-tensor

Conversation

@fhanuman

@fhanuman fhanuman commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Why

Gather-family and tensor-shape operations previously constructed and checked result shapes through separate converter, reifier, and verifier logic. That made dimension splicing, permutation validation, packed-int4 interpretation, and imported result-template handling easy to apply inconsistently.

This PR gives those operations shared semantic shape rules while retaining the GatherND DPS-init fallback when runtime tuple width prevents static output-rank inference.

Shape example

The added GatherBlockQuantized reification test records the physical-to-logical packed-int4 shape rule directly in HIP IR:

%result = hip.gather_block_quantized(%ctx)
  ins(%data, %indices, %scales :
      tensor<2048x96xui8>, tensor<8xi64>, tensor<2048x12xf16>)
  outs(%init : tensor<8x192xf16>)
  {bits = 4, block_size = 16, gather_axis = 0, quantize_axis = 1}
  : tensor<8x192xf16>
%c0 = arith.constant 0 : index
%c1 = arith.constant 1 : index
%d0 = tensor.dim %result, %c0 : tensor<8x192xf16>
%d1 = tensor.dim %result, %c1 : tensor<8x192xf16>

The surviving packed axis has physical byte extent 96 and logical output extent 192; the same test suite covers the dynamic form by emitting arith.muli for that doubling.

What changes

  • Share Gather and GatherND dimension-splicing rules across converters, reifiers, and verifiers.
  • Define GatherElements output shape from its indices operand through the named same-shape contract.
  • Validate Transpose permutations through one shared rule before materializing dimensions.
  • Define Resize output N/C from the input and static spatial extents from the imported result template.
  • Define Size as a rank-zero i64 result through a shared semantic rule.
  • Compute GatherBlockQuantized output shapes from the logical dequantized tensor rather than physical packed-byte storage.
  • Account for packed int4 by doubling a surviving quantize-axis extent, keeping converter, lowering, runtime interpretation, and tests aligned.

Stack

Full 31-PR order:

  1. #688 — refactor(hip): externalize constants after ONNX conversion
  2. #724 — feat(shape): preserve symbolic compiler inputs
  3. #639 — refactor(hip): establish shared shape-rule foundation
  4. #681 — fix(shape): canonicalize host reshape provenance
  5. #725 — fix(shape): activate exact broadcasts with symbolic proofs
  6. #640 — fix(hip): share MatMul and Gemm shapes with dynamic batch validation
  7. #734 — fix(shape): share MatMul and Gemm contracts
  8. #641 — fix(hip): unify broadcast and reduction shape contracts
  9. #735 — refactor(conversion): share ONNX reduction conversion
  10. #736 — fix(shape): normalize reduction contracts
  11. #689 — fix(loop): support shape-changing loop carriers
  12. #737 — fix(loop): support shape-changing loop carriers
  13. #642 — feat(hip): add payload-driven shapes and bulk readback
  14. #727 — feat(shape): consume constant payload shapes directly
  15. #728 — feat(shape): add exact Tile and Range destinations
  16. #729 — refactor(slice): share overflow-safe normalization
  17. #731 — feat(slice): lower normalized exact descriptors
  18. #733 — fix(loop): integrate exact payload Slice carriers
  19. #643 — fix(hip): share Conv and Pool shape contracts
  20. #738 — fix(shape): share pooling contracts
  21. #742 — fix(shape): share causal convolution contracts
  22. #644 — fix(hip): share Gather and tensor-shape contracts ← this PR
  23. #739 — fix(shape): share block-quantized Gather contracts
  24. #645 — fix(hip): share attention and normalization shape contracts
  25. #740 — fix(shape): share attention contracts
  26. #741 — fix(shape): share GQA capacity contracts
  27. #743 — fix(gqa): reject unsupported optional features
  28. #646 — refactor(hip): enforce explicit DPS shape contracts locally
  29. #730 — test(hip): audit explicit DPS contract inventory
  30. #732 — test(hip): audit ONNX converter deduplication
  31. #647 — fix(hip): harden shape refinement failure handling

AI assistance

AI tools assisted with the shape-contract refactoring and stack reconstruction. The contributor reviewed and validated the resulting code.

Made-with: Cursor

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

@github-actions github-actions Bot added the large-pr Soft size threshold (>500 LOC or >10 files). Reviewer signal, not a block. label Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 4.8668E-07 64 0
GroupQueryAttention_seq256 25.2366 2621440 0
MatMulNBits_o_seq128 259.906 368640 0
QMoE_seq128 34.957 368640 0

Threshold: 0.01 | Run: 4473 - Commit: 323ec31

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.47 42.08 365 3 1140
GroupQueryAttention_seq128 4291.98 1.69335 12 6 209
matmul_down_seq128 520.02 2.89 69 3 255

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.55 82.92 360 3 15486

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.52 9.29 383 3 15658

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 198.0 78.6 1.23 13.59
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 334.4 40.5 1.12 6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 191 37.2

Run: 4473 - Commit: 323ec31

@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from deb0a3f to 8db3876 Compare August 5, 2026 21:56
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from 8db3876 to 9f06ab6 Compare August 6, 2026 21:00
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from 9f06ab6 to 0858247 Compare August 7, 2026 19:51
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch 2 times, most recently from a0455ce to e337608 Compare August 10, 2026 21:59
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from e337608 to af6a5a5 Compare August 10, 2026 22:29
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from af6a5a5 to 2cb9720 Compare August 10, 2026 23:33
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch 2 times, most recently from 9dd4ecb to ffffe50 Compare August 11, 2026 13:29
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from ffffe50 to 2e812f6 Compare August 11, 2026 14:19
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from 2e812f6 to 943bf82 Compare August 11, 2026 16:25
@fhanuman
fhanuman force-pushed the pr/shape-stack-06-gather-tensor branch from 943bf82 to 78bddcb Compare August 11, 2026 21:22
Unify Gather, GatherND, GatherElements, Transpose, Resize, and Size destination construction, reification, and verification around shared shape rules.

Co-Authored-By: GPT-5.6 Sol <noreply@openai.com>
Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

large-pr Soft size threshold (>500 LOC or >10 files). Reviewer signal, not a block.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant