Skip to content

refactor: remove "auto" option from fused_lm_head_token_chunk_size#2890

Merged
samsja merged 1 commit into
mainfrom
refactor/remove-chunk-loss-auto
Jun 27, 2026
Merged

refactor: remove "auto" option from fused_lm_head_token_chunk_size#2890
samsja merged 1 commit into
mainfrom
refactor/remove-chunk-loss-auto

Conversation

@samsja

@samsja samsja commented Jun 26, 2026

Copy link
Copy Markdown
Member

Summary

Removes the "auto" option from model.fused_lm_head_token_chunk_size. This value previously resolved to 8192 for RL training via the auto_setup_fused_lm_head_token_chunk_size config validator. It was redundant — users can simply set the integer explicitly.

Changes

  • packages/prime-rl-configs/src/prime_rl/configs/trainer.py:
    • Type changed from int | Literal["auto", "disabled"]int | Literal["disabled"]
    • Removed auto_setup_fused_lm_head_token_chunk_size validator (was a no-op for non-"auto" values)
    • Updated docstring to remove "auto" mention
  • docs/scaling.md: Removed the commented # fused_lm_head_token_chunk_size = "auto" example line
  • CHANGELOG.md: Added breaking-change entry

Migration

Users who relied on "auto" should set fused_lm_head_token_chunk_size = 8192 explicitly:

[trainer.model]
fused_lm_head_token_chunk_size = 8192  # was: "auto"

Notes

  • The default remains 1024 (unchanged)
  • "disabled" (vanilla LM head) and explicit integers continue to work as before
  • SFT training still silently disables this (sets to "disabled")
  • No existing config files in the repo used "auto" for this field
  • Follow-up to feat: default ep to auto for MoE models #2868

Note

Low Risk
Small, localized trainer config validation and documentation change with a clear migration path; no runtime training logic beyond dropping the auto-resolve step.

Overview
Breaking config change: model.fused_lm_head_token_chunk_size no longer accepts "auto". The field is now int | Literal["disabled"] only; the default stays 1024.

The auto_setup_fused_lm_head_token_chunk_size validator that rewrote "auto"8192 for RL is removed. Anyone who relied on "auto" should set fused_lm_head_token_chunk_size = 8192 explicitly. "disabled" and explicit integers behave as before; SFT still forces this off.

Docs (scaling.md) drop the commented "auto" example; CHANGELOG documents the migration.

Reviewed by Cursor Bugbot for commit b83e6ab. Bugbot is set up for automated code reviews on this repo. Configure here.

The "auto" value for fused_lm_head_token_chunk_size previously resolved
to 8192 for RL training via a config validator. This was redundant since
users can simply set the integer explicitly. Remove the "auto" option,
its validator (auto_setup_fused_lm_head_token_chunk_size), and update
docs/CHANGELOG accordingly.

The type changes from int | Literal["auto", "disabled"] to
int | Literal["disabled"]. Users who relied on "auto" should set
fused_lm_head_token_chunk_size = 8192 explicitly.
@samsja samsja merged commit bf3d381 into main Jun 27, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants