Skip to content

[HIP] [Bugfix] topk_gating kernels: trap when vec_size does not divide num_experts - #6323

Open
truong-v wants to merge 3 commits into
ROCm:mainfrom
truong-v:fix/hip-topk-gating-vec-tail
Open

truong-v wants to merge 3 commits into
ROCm:mainfrom
truong-v:fix/hip-topk-gating-vec-tail

Conversation

@truong-v

@truong-v truong-v commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Motivation

Fixes #6317

The top-k selection in topk_gating_kernel and topk_gating_kernel_smem_n scans only full vectors, so with a vec_size that does not divide num_experts the last experts are never selected. Dispatch never does this today, so this PR makes the precondition explicit and fails loudly instead of misrouting.

Technical Details

  • In both kernels, if(num_experts % vec_size != 0) __builtin_trap(); right after num_experts_vec is computed.
  • Adding a scalar tail to the selection loops instead made topk_gating_kernel 2.6-5.8% slower at T <= 1024 for divisible num_experts, so this PR uses the guard.

No test is added, because the condition cannot be produced through aiter.ops.topk.topk_gating.

Test Plan

python op_tests/test_moe_topk_gating.py

Test Result

MI325X (gfx942): all 4560 configurations pass. In a standalone harness with a forced vec_size, all 6 non-divisible cases now trap instead of returning wrong ids. Kernel time is 0.988-1.025x of base (median 1.001x).

Submission Checklist

…erts

The selection loop scans only full vectors, so the last
num_experts % vec_size experts could never be selected. Dispatch never
picks such a vec_size today; make the precondition explicit.
@truong-v
truong-v requested a review from a team October 9, 2026 14:00
@github-actions github-actions Bot changed the title [Bugfix] topk_gating kernels: trap when vec_size does not divide num_experts [HIP] [Bugfix] topk_gating kernels: trap when vec_size does not divide num_experts Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355 Run the full Triton test suite on MI35X, not only the tests the change affects
ci:multi-gpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6323 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

@junhaha666
junhaha666 requested a review from yzhou103 October 23, 2026 09:28

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Issue]: topk_gating kernels never select the last num_experts % vec_size experts (latent)

3 participants