Retune DSV4 B300 AgentX MTP sweep / 调优 DSV4 B300 AgentX MTP 扫描 - #2536
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
d1b6a57 to
dff6feb
Compare
ivanium
added
full-sweep-fail-fast
agentx
|
Claude finished @ivanium's task in 1m 34s —— View job PR Review
LGTM - no blocking issues found Checks performed:
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — this exact change is already merged to main as commit 804ab8b ("perf(agentx): retune DSV4 B300 MTP sweep"), and the follow-up changelog-link commit dff6feb references this PR.
Extended reasoning...
Overview
This PR retunes the DSV4-Pro B300 AgentX MTP benchmark recipe: bumps the vLLM nightly image tag, reshapes the TP4/TP8/DEP4/DEP8 concurrency grids (dropping the TP4-SimpleCPU arm, adding higher-concurrency DEP8 points), raises DEP4 host-DRAM utilization from 0.80 to 0.95, and unifies --long-prefill-token-threshold 512 across both DP-attention arms. Only configs/nvidia-master.yaml, the corresponding benchmark shell script, and perf-changelog.yaml are touched.
Security risks
None. This is benchmark/perf-sweep configuration for an internal measurement harness — no auth, credential, or user-facing logic is touched.
Level of scrutiny
Low-to-moderate is appropriate: this is a routine performance-tuning PR of the kind that lands regularly in this repo. I verified the DEP4 DRAM-budget comment update is internally consistent with agentic_dram_offload_gb in utils/matrix_logic/generate_sweep_configs.py (utilization scales the total budget linearly, and 1424 GB / 356 GB-per-rank at 0.95 tracks the prior 280 GiB-per-rank figure at 0.80 within expected unit-conversion rounding), and that the new --long-prefill-token-threshold 512 was correctly hoisted out of the DEP8-only branch into the shared DP-attention branch so it now applies to both DEP4 and DEP8 as the description states. All new conc-list values for DP-attention arms satisfy the script's 2*CONC % TP == 0 guard.
Other factors
Notably, git log shows this exact diff already merged to main as commit 804ab8b ("perf(agentx): retune DSV4 B300 MTP sweep"), with a subsequent commit dff6feb linking this PR number into the changelog — indicating the sweep already ran and passed. There is nothing outstanding to flag.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31278845561 |
|
/stage-results 31278845561 |
|
@xinli-sw staged run 31278845561: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-08~r31278845561 This run remains available across future @xinli-sw 已将运行 31278845561 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-08~r31278845561 后续的 |
ivanium
added
full-sweep-fail-fast
full-sweep-enabled
and removed
full-sweep-fail-fast
labels
7299790 to
affc6c3
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31287631879 |
8eadc24 to
64e2316
Compare
64e2316 to
48dbc8d
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31297932579 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31297932579 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31353965724 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31356875713 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31358659497 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31363733422 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31366460093 |
285ea3e to
affc6c3
Compare
Update the vLLM image and concurrency grid, raise the DEP4 host-memory budget, and set long-prefill-token-threshold to 512 for DEP4 and DEP8. 中文：更新 vLLM 镜像与并发配置，提高 DEP4 主机内存预算，并为 DEP4 和 DEP8 将 long-prefill-token-threshold 设为 512。
Move every DEP8 point to vLLM SimpleCPU offload, widen the concurrency grid through c512, and drop DEP4 c40. 中文：将全部 DEP8 配置切换到 vLLM SimpleCPU 卸载，将并发扫描扩展至 c512，并移除 DEP4 c40。
Move TP4 c16 to SimpleCPU offload, skip TP4 c12 and DEP8 c64, and extend the DEP8 sweep through c576. 中文：将 TP4 c16 切换到 SimpleCPU 卸载，移除 TP4 c12 与 DEP8 c64，并将 DEP8 扫描扩展至 c576。
affc6c3 to
63ea0c9
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31415828111 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31415828111 |
|
/stage-results 31415828111 |
|
@ivanium staged run 31415828111: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-10~r31415828111 This run remains available across future @ivanium 已将运行 31415828111 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-10~r31415828111 后续的 |
|
/reuse-sweep-run 31415828111 |
3 participants
Summary
vllm/vllm-openai:nightly-dev-x86_64-cu13.0.1-426e59f.[1, 4]and TP4 c8 GPU-resident; run TP4 c16 with vLLM SimpleCPU offload.[48, 64]and DEP8 SimpleCPU at[128, 256, 384, 512, 576].--long-prefill-token-threshold 512for DEP4 and DEP8. Every offloaded rank receives a 356 GB host-DRAM budget.Validation
python -m pytest utils/matrix_logic/ -v— 224 passed.dsv4-fp4-b300-vllm-agentic-mtprows.bash -nandgit diff --checkpassed.中文说明
vllm/vllm-openai:nightly-dev-x86_64-cu13.0.1-426e59f。[1, 4]与 TP4 c8 保持 GPU 常驻；TP4 c16 使用 vLLM SimpleCPU 卸载。[48, 64]，DEP8 SimpleCPU 使用[128, 256, 384, 512, 576]。--long-prefill-token-threshold 512；每个卸载 rank 分配 356 GB 主机 DRAM。验证
python -m pytest utils/matrix_logic/ -v：224 项测试通过。dsv4-fp4-b300-vllm-agentic-mtp的全部 11 条配置。bash -n与git diff --check均通过。