fix(turbomind): fix dimension mismatch in ApplyTokenBitmaskInplace

[windreamer]

The bug occurs when batch contains requests that have finished generation but are still waiting in the batch for synchronization. In this case, generation_size (number of requests still generating) is less than matchers.size() (total requests in batch).

Root cause:

• In generation.cc Forward(), logits is sliced to match generation_size: env.produce("logits", logits.slice(0, gs));
• But in guided_decoding.cc ApplyMask(), bitmask was sliced using d.matchers.size() instead of the actual logits batch size.

This causes TM_CHECK(logits_shape.first == bitmask_shape.first) to fail in apply_token_bitmask_inplace_cuda.cu when the dimensions don't match.

fix: #4453

[Copilot]

Pull request overview

This PR fixes a Turbomind guided-decoding crash caused by a batch-dimension mismatch between sliced logits (only active/generating requests) and the guided-decoding bitmask (previously sliced using the full batch request count).

Changes:

• Update GuidedDecoding::ApplyMask() to slice the bitmask using logits.shape(0) rather than d.matchers.size().
• Add clarifying comments explaining why logits.shape(0) is the correct dimension to use when some requests in the batch are no longer generating.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.