Skip to content

feat(kernels): add GPU family fatbin support for custom_kernels - #698

Draft
cliu1003 wants to merge 2 commits into
mainfrom
ci/windows_family_archs_build
Draft

feat(kernels): add GPU family fatbin support for custom_kernels#698
cliu1003 wants to merge 2 commits into
mainfrom
ci/windows_family_archs_build

Conversation

@cliu1003

Copy link
Copy Markdown
Contributor

Support gfx115X-all as a multi-slice HIP_ARCHITECTURES family, with LlvmIrJit falling back from per-arch to family DLLs at runtime. Update Windows CI to build with --hip_arch gfx115X-all.

Support gfx115X-all as a multi-slice HIP_ARCHITECTURES family, with LlvmIrJit falling back from per-arch to family DLLs at runtime. Update Windows CI to build with --hip_arch gfx115X-all.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

deps.cmake is included from the top-level CMakeLists.txt, so CMAKE_CURRENT_SOURCE_DIR points at the repo root and the hip_families.cmake lookup fails in CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 4.8668E-07 64 0
GroupQueryAttention_seq256 25.2366 2621440 0
MatMulNBits_o_seq128 259.906 368640 0
QMoE_seq128 34.957 368640 0

Threshold: 0.01 | Run: 4005 - Commit: 663d870

@github-actions

Copy link
Copy Markdown

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.50 6.37 364 3 1242
GroupQueryAttention_seq128 4436.76 1.73626 11 6 312
matmul_down_seq128 526.28 2.47 73 3 354

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.53 45.28 361 3 15590

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.53 9.64 362 3 15759

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 160.1 80.6 1.33 13.56
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 239.9 43.0 1.22 6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 196 42.1

Run: 4005 - Commit: 663d870

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant