Feat/native hf megatron converters - #954
Open
botaohu001 wants to merge 11 commits into
Open
Conversation
…converters Host HF->Megatron checkpoint conversion inside Primus so the native SFT path no longer needs Megatron-Bridge / modelopt / a specific transformers version, and third_party/Megatron-LM stays pristine. - primus/backends/megatron/checkpoint/: single-process converters that build the mcore GPTModel with the same provider training uses and map HF safetensors onto it (Qwen3 dense+MoE, DeepSeek-V2/V3 MLA+MoE, Llama 2/3), validated exactly (0 unexpected / 0 non-_extra_state missing / 0 unconsumed). - checkpoint_convert_patches.py + context.py: a new "convert" lifecycle phase and a phase="convert" patch that no-ops the legacy CUDA fused_kernels build on ROCm, applied in-process by the converter (no submodule edits). Co-authored-by: Cursor <cursoragent@cursor.com>
…osttrain hook - 01_convert_checkpoints.py: for supported families (Qwen3/DeepSeek/Llama), convert natively in-process by default; native_ckpt_convert: false forces the legacy Megatron-Bridge path, true forces native. - megatron_sft_trainer.py: load torch(legacy) checkpoints produced by the native converter (in addition to torch_dist), so the native ckpt loads for LoRA SFT. - logger.py: degrade caller-aware logging to stdout when the logger is not yet initialized (the standalone hook process runs run_patches before setup). Co-authored-by: Cursor <cursoragent@cursor.com>
…emplate Path-neutral template documenting both usage modes: (A) primus-cli direct train posttrain auto-converts from hf_path (native is the default for supported families); (B) convert once via the Primus converter, then point pretrained_checkpoint at it. Co-authored-by: Cursor <cursoragent@cursor.com>
… path
00_install_requirements.sh installs the legacy Megatron-Bridge conversion deps
(transformers==4.57.6 + nvidia-modelopt + onnx), which the default native
(bridge-free) converter does not need. Skip the install when the native path
will run so a stock ROCm image is not forced to re-install those deps (and
clobber the stock transformers):
- honor PRIMUS_SKIP_PIP;
- skip when pretrained_checkpoint is already configured (no conversion);
- skip when the HF model resolves to a supported native family and native
conversion is not explicitly disabled (native_ckpt_convert: false).
The native-vs-bridge decision reuses 01_convert_checkpoints.py's own helpers so
it cannot drift from the actual conversion behaviour.
Co-authored-by: Cursor <cursoragent@cursor.com>
botaohu001
requested review from
Xiaoming-AMD,
limou102 and
wenxie-amd
as code owners
August 6, 2026 11:45
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
- megatron_sft_trainer.py: read latest_checkpointed_iteration.txt via a context manager so the file handle is always closed. - checkpoint_convert_patches.py: make _rocm_safe_load's returns explicit (ROCm path returns None; probe-failure and normal paths return _orig_load(args)) and replace the empty `except: pass` with a logged fallback. Behaviour unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
Replace non-ASCII typographic characters with ASCII equivalents so GitHub no longer flags the files as containing "hidden or bidirectional Unicode text": - native_hf_to_megatron_sft.template.yaml: em-dash / box-drawing -> "-" / "---". - 01_convert_checkpoints.py: "HF -> Megatron" in one log string (was an arrow). No behaviour change. Co-authored-by: Cursor <cursoragent@cursor.com>
…ine) Align the new files' headers with the repo-standard header used across Primus sources (Copyright + "See LICENSE for license information."), removing the per-file "Author:" line. No functional change. Co-authored-by: Cursor <cursoragent@cursor.com>
The Primus-native converter/patch/template files were created in 2026; use "(c) 2026" in their license header to match the repo convention. No functional change. Co-authored-by: Cursor <cursoragent@cursor.com>
Run the repo's pre-commit hooks (isort --profile black, autoflake, black --line-length=110) on the new/changed files so the code-lint CI passes. Pure formatting -- import ordering and reflowing multi-line statements to single lines. No functional change. Co-authored-by: Cursor <cursoragent@cursor.com>
…config smoke test test_example_configs.py (added in #961) loads every YAML under examples/*/configs/ through the real config stack. The native HF->Megatron SFT recipe added here is a copy-and-fill template, not a runnable experiment: its `model` preset is the literal placeholder <YOUR_MODEL_YAML>, so resolving it raises FileNotFoundError by design and can only succeed once a user substitutes the <...> values. List it in NOT_AN_EXPERIMENT, the exemption the test already provides for the lfm2 TE precision fragment, and extend that comment to cover both reasons. The remaining 387 recipes stay covered. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
nvidia-modelopt/ a pinnedtransformers— runs on a stock ROCm image (transformers 4.55, no modelopt), andthird_party/Megatron-LMstays pristine.native_ckpt_convert: falseforces the legacy Megatron-Bridge path.convertlifecycle phase), not a submodule edit.