Repository navigation
Conversation
…erts The selection loop scans only full vectors, so the last num_experts % vec_size experts could never be selected. Dispatch never picks such a vec_size today; make the precondition explicit.
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
One backend per PR: PR title tags & labels: |
Motivation
Fixes #6317
The top-k selection in
topk_gating_kernelandtopk_gating_kernel_smem_nscans only full vectors, so with avec_sizethat does not dividenum_expertsthe last experts are never selected. Dispatch never does this today, so this PR makes the precondition explicit and fails loudly instead of misrouting.Technical Details
if(num_experts % vec_size != 0) __builtin_trap();right afternum_experts_vecis computed.topk_gating_kernel2.6-5.8% slower at T <= 1024 for divisiblenum_experts, so this PR uses the guard.No test is added, because the condition cannot be produced through
aiter.ops.topk.topk_gating.Test Plan
Test Result
MI325X (gfx942): all 4560 configurations pass. In a standalone harness with a forced
vec_size, all 6 non-divisible cases now trap instead of returning wrong ids. Kernel time is 0.988-1.025x of base (median 1.001x).Submission Checklist