[KimiK3][AgentX]: GB200 DSpark and Simple CPU KV offload - #2404
Conversation
Add srt-slurm TP, TEP, and DEP AgentX profiles plus GB200 runner routing and recipe overlay plumbing. 中文：新增 Kimi K3 GB200 首日 AgentX 支持，包括 srt-slurm TP、TEP、DEP 配置，以及 GB200 运行器路由和配方覆盖逻辑。
Record the draft pull request URL in the append-only benchmark changelog entry. 中文：在仅追加的基准测试变更日志条目中记录草稿 PR 链接。
中文：将 origin/main 合并到 GB200 Kimi K3 AgentX 分支。
Represent the throughput arm as TP1 x DP16 = EP16 in the master configuration, matching the vLLM recipe. 中文：将吞吐量配置修正为 TP1 × DP16 = EP16，与 vLLM 配方保持一致。
Use the pre-staged /mnt/numa1/models/Kimi-K3 checkpoint on every GB200 compute node while retaining shared-filesystem orchestration and caches. 中文：在每个 GB200 计算节点上使用预置的 /mnt/numa1/models/Kimi-K3 检查点，同时保留共享文件系统上的编排与缓存。
Keep the Day-0 changelog focused on the vLLM recipe performance points and use the model default context length. 中文：精简 Kimi K3 首日性能配置，仅保留与 vLLM 配方一致的性能测试点，并使用模型默认上下文长度。
Move latency and balanced profiles to 16-GPU TP/TEP, use TP2 x DP8 for EP16 throughput, and extend the throughput concurrency sweep through 256. 中文：根据 GB200 HBM 容量调整 Kimi K3 AgentX 扫描。将延迟与均衡配置扩展为 16 GPU TP/TEP，吞吐配置改为 TP2 × DP8（EP16），并将吞吐并发扫描扩展至 256。
The Dynamo vLLM worker entrypoint does not accept vllm serve tool-call or reasoning parser flags; agentic request parsing belongs to the Dynamo frontend. 中文：移除 Dynamo vLLM worker 不支持的 Kimi K3 工具调用与推理解析参数；AgentX 请求解析由 Dynamo frontend 负责。
Remove literal shell placeholders from the vLLM worker environment so Gloo and NCCL use the interfaces selected by the srt-slurm launcher. 中文：移除 vLLM worker 环境中的字面量 shell 占位符，由 srt-slurm launcher 为 Gloo 与 NCCL 选择网络接口。
Patch the pinned agentic srt-slurm checkout to launch one vLLM process per GB200 node, with two local DP ranks and TP2 per rank. Export every per-node metrics endpoint to AIPerf. 中文：修补 AgentX 固定版本的 srt-slurm，使每个 GB200 节点启动一个 vLLM 进程，每个进程承载两个本地 DP rank，并为每个 rank 使用 TP2；同时将所有节点的 metrics endpoint 传给 AIPerf。
Use TP4 x DP4 for the throughput-oriented EP16 profile after TP2 x DP8 exceeded GB200 HBM during MXFP4 weight loading. 中文：TP2 × DP8 在 MXFP4 权重加载阶段超过 GB200 HBM 容量，因此将吞吐优先的 EP16 配置调整为 TP4 × DP4。
Pin a tested srt-slurm revision that natively groups TP GPUs within each DP rank and exports every DP metrics endpoint. Remove the vendored patch and reserve enough HBM for one native 1M-context request. 中文：固定经过测试的 srt-slurm 版本，原生支持每个 DP rank 内的 TP GPU 分组，并导出全部 DP metrics endpoint。删除仓库内补丁，同时为原生 1M 上下文请求预留足够 HBM。
Record the tested TP4 x DP4 = EP16 throughput profile. 中文：记录经过测试的 TP4 x DP4 = EP16 吞吐配置。
Use NVIDIA’s Kimi K3 runtime with native tokenizer, parser, FP8 KV-cache, and FlashInfer support. Keep the model-default 1M context and remove the vLLM-mutating setup script.\n\n中文：改用 NVIDIA 官方 Kimi K3 Dynamo 运行时，启用原生 tokenizer、解析器、FP8 KV 缓存和 FlashInfer 支持。保留模型默认的 1M 上下文，并移除会修改 vLLM 文件的环境配置脚本。
Match identity verification and result metadata to the Dynamo 1.3.0 package reported by the official Kimi K3 runtime image.\n\n中文：将身份校验与结果元数据对齐到官方 Kimi K3 运行时镜像实际报告的 Dynamo 1.3.0 包版本。
Use the official Kimi K3 Dynamo 0.92 GPU memory utilization so native 1M-context MLA prefills retain transient workspace without changing max model length.\n\n中文：采用官方 Kimi K3 Dynamo 配方的 0.92 GPU 显存利用率，为原生 1M 上下文 MLA 预填充保留临时工作区，且不修改最大模型长度。
Apply vLLM EP_SIZE = TP_SIZE × DP_SIZE in the master metadata for the DP1 latency profile.\n\n中文：在 DP1 延迟配置的主配置元数据中应用 vLLM 的 EP_SIZE = TP_SIZE × DP_SIZE 规则。
中文：将 main 最新变更同步到 Kimi K3 GB200 支持分支。
Copy per-concurrency AgentX aggregate JSON files from the compute-visible shared workspace back into the GitHub workspace, and cover success and missing-result behavior with regression tests. 中文：将各并发度的 AgentX 聚合 JSON 从计算节点可见的共享工作区复制回 GitHub 工作区，并通过回归测试覆盖成功复制和结果缺失场景。
Enable AIPerf burst phase starts only for the Kimi K3 GB200 saturation recipes so warmup and profiling reach the requested concurrency within the benchmark window. Inter-turn replay timing remains unchanged. 中文：仅在 Kimi K3 GB200 饱和度测试配置中启用 AIPerf 阶段起始突发，使预热和正式测试能在基准测试窗口内达到目标并发度，同时保持轮次间回放时序不变。
Merge the latest main branch while preserving the Kimi K3 changelog entry at the append-only tail. 中文：合并最新 main 分支，并将 Kimi K3 变更日志条目保留在只追加文件的末尾。
Bring in the merged AgentX v1.0 AIPerf default and current benchmark updates while preserving the Kimi K3 changelog entry at the tail.\n\n中文：合并 AgentX v1.0 AIPerf 默认行为及最新基准测试更新，并保留位于末尾的 Kimi K3 变更日志条目。
Allow the full-context DEP saturation sweep up to 3600 seconds to drain accelerated warmup requests. The grace period exits early when the servers become idle. 中文：将 Kimi K3 DEP 全上下文饱和度测试的加速预热排空上限延长至 3600 秒；服务空闲后会提前结束等待。
中文：将最新 main 分支合并到 Kimi K3 GB200 支持分支。 # Conflicts: # perf-changelog.yaml
Restore the legacy AgentX AIPerf integration with the barrier-aware idle cap, and retain the GB200 Kimi K3 performance changelog entry at the append-only tail. 中文：同步最新 main，恢复带屏障感知空闲上限的旧版 AgentX AIPerf 集成，并将 GB200 Kimi K3 性能变更记录保留在追加式文件末尾。
Add gb200-nv_3 to the GB200 cluster runner pool so the Kimi K3 sweep can use the capacity provisioned for this bring-up. 中文：将 gb200-nv_3 注册到 GB200 集群运行器池，使 Kimi K3 扫描能够使用本次适配新增的容量。
中文：将最新 main 合并到 Kimi K3 GB200 KV 卸载分支，并按追加式规则保留性能变更记录。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30674956007 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30675257102 |
Advance the AgentX v1 harness to abf55f90 so profiling preserves flattened trace timing across warmup handoff and idle watchdogs remain active across dependency barriers. 中文：将 AgentX v1 基准测试工具升级到 abf55f90，使 profiling 在 warmup 交接后保留展平后的 trace 时序，并确保依赖屏障期间 idle watchdog 持续生效。
中文：将最新 origin/main 合并到 GB200 Kimi K3 分支。 # Conflicts: # benchmarks/benchmark_lib.sh # utils/aiperf
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30758333436 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30780993377 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30758333436 |
Keep the main-branch trace idle cap implementation and retain only the distinct live request-error threshold override needed by the GB200 TP16 recipe. 中文：移除重复的轨迹空闲间隔上限逻辑，沿用 main 分支实现；仅保留 GB200 TP16 配方所需的实时请求错误率阈值覆盖。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30780993377 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30781275770 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30781275770 |
|
/stage-results 30781275770 |
|
@cquil11 staged run 30781275770: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 This run remains available across future @cquil11 已将运行 30781275770 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 后续的 |
|
/stage-results 30781275770 |
|
@cquil11 staged run 30781275770: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 This run remains available across future @cquil11 已将运行 30781275770 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 后续的 |
|
/stage-results 30781275770 |
|
@cquil11 staged run 30781275770: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 This run remains available across future @cquil11 已将运行 30781275770 发布到预发布环境：https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-03~r30781275770 后续的 |
|
/reuse-sweep-run 30781275770 |
2 participants
Summary
vllm/vllm-openai:kimi-k3image for every Kimi K3 GB200 AgentX profile.v1.4.0-kimi-k3-dev.1preview from its immutable release commit throughdynamo.install: true.SimpleCPUOffloadConnectoron the TP4 × DP4 / EP16 throughput topology at concurrency 128, 192, 256, and 384.References
Validation
vllm/vllm-openai:kimi-k3, all labeled for speculative decoding.srtctl dry-runon the pinned Kimi K3 srt-slurm revision.python -m pytest utils/matrix_logic/ -v: 224 passed.python -m pytest runners/test_slurm_utils.py -q: 2 passed.perf-changelog.yamlhas one pending Kimi K3 GB200 record, passes the changelog validator, and ends with a newline.The previous 18/18 green full sweep validated the pre-DSpark Dynamo-runtime-image revision and is retained only as historical baseline evidence.