feat(megatron): support graduated precision boundaries - #980
Conversation
Allow an outer precision override inside Flux sensitive boundaries so recipes can keep the outermost blocks in BF16 while routing adjacent blocks through tensorwise FP8.
Result: agent review blocked before runtime smokeBlocked. The intended Option 8 recipe routes correctly, but the framework API accepts configurations that can silently collapse the graduated layout.
The complete base-to-head diff, surrounding config/provider paths, existing tests, and CI were reviewed. CI is green, but its Torch unit job was skipped. Rerun agent review after these findings are fixed; the PR is not ready for runtime smoke. |
gphuang
left a comment
There was a problem hiding this comment.
Follow-up: re-review confirms API and routing block
Re-reviewed exact head c1e75cf across API behavior, routing, and tests. Findings are unchanged from the previous review: unsupported backend paths silently collapse graduated routing; tw_fp8 and bf16 do not enforce their stated precision semantics; and tests cover only 57 QKV slots rather than all 247 linear slots. The relevant Torch CI job was skipped.
The PR remains draft. Dependent recipe review and the 10-step smoke remain gated until these blockers are fixed.
Reject unsupported precision and backend combinations before model construction, and cover the complete Flux routing contract with negative-path and model-build tests.
Result: agent review approved after fixesReview found that unsupported backend and precision combinations could silently fall back to one backend, while the routing test covered only one linear per block. Fixed in Two independent re-reviews approved the updated diff. The PR stays draft while CI and the pinned GPU test remain open. |
Apply the repository's Black formatting so the reviewed routing fix passes the lint gate.
Merge the native TE-MXFP4 and Flux baseline updates without changing the reviewed local-spec Option 8 routing contract.
Keep post-construction config mutations fail closed instead of silently falling back to homogeneous MXFP4 routing.
Result: reconciled graduated-routing review approvedMerged Primus Independent review found one post-construction fail-open path: disabling sensitive routing while retaining boundary counts silently selected homogeneous MXFP4. Commit All required CI checks and static validation pass. PR 980 remains draft; the 10-step GPU smoke is still pending compute. |
Summary
Supports Task 9 Option 8 for Issue #220.
FluxPretrainTrainer.Defaults remain zero, so existing recipes keep their current routing.
Test plan
{0,56}, FP8{1–3,53–55}, MXFP4{4–52}