[Klaud Cold][agentic experiment][Variant A] feat: add Kimi-K3 MXFP4 B200 aggregated TP8xPP2 Dynamo-vLLM agentic bring-up / 新增 Kimi-K3 MXFP4 B200 聚合式 TP8xPP2 Dynamo-vLLM 智能体编码基准测试（bring-up）

[functionstackx]

Summary

Bring-up configuration: first Kimi-K3 benchmark config in InferenceX; single-concurrency smoke test, validated via the PR sweep before merge.

Adds an aggregated TP8 × PP2 (plain TP — expert parallelism deliberately OFF) Dynamo-vLLM agentic-coding recipe for Kimi-K3 MXFP4 on B200, following the aggregated tep8pp2 pattern established in #2196 and the srt-slurm v1.0.36 AgentX recipe pattern from #2341/#2302:

• Topology: the native MXFP4 checkpoint (2.8T total params, ~1.4TB of weights) does not fit one 8×B200 node, so TP8 shards attention/dense and PP2 splits the 93 layers across 2 nodes / 16 GPUs (tp*pp/gpus_per_node = 8*2/8 = 2). Aggregated mode (prefill num-worker: 1 + decode num-worker: 0, RECIPES.md §5): a single worker serves both phases, no P/D KV transfer.
• Image: dedicated bring-up image vllm/vllm-openai:kimi-k3 (verified on Docker Hub, amd64 + arm64), with VLLM_ENABLE_K3_LATENT_MOE_TAIL_FUSION=1 and --trust-remote-code --load-format fastsafetensors --moe-backend auto --gpu-memory-utilization 0.95 --no-enable-flashinfer-autotune --reasoning-parser kimi_k3. --enable-auto-tool-choice/--tool-call-parser are not set on the worker: the dynamo-vllm worker entrypoint rejects them (unrecognized arguments — second sweep attempt); chat parsing happens at the dynamo frontend, same convention as the DSv4 GB300 agentic recipes. No explicit max-model-len (vLLM derives the native 1M window from the model config).
• srt-slurm base: the b200-dgxc agentic path now pins upstream NVIDIA/srt-slurm v1.0.36 (mirroring the GB300 migration in Add DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP recipes / 新增 DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP 配置 #2341; validated in [WIP] Test DeepSeek-V4 GB300 Dynamo AgentX recipes / 测试 DeepSeek-V4 GB300 Dynamo AgentX 配置 #2302). The previously cloned cquil11/srt-slurm-nv cam/sa-submission-q2-2026 fork rejected the recipe schema (benchmark.aiperf_server_metrics: Unknown field — first sweep attempt).
• Dynamo: wheel + router pinned to 1.2.1, the combination validated with v1.0.36 in Add DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP recipes / 新增 DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP 配置 #2341's GB300 aggregate AgentX recipes.
• Context: full native 1M max-model-len for the AgentX trace (K3's KDA layers keep per-token KV small; only the 24 gated-MLA layers hold cache); max-num-batched-tokens: 8192 so one long prefill cannot OOM a pipeline stage. Smoke test at concurrency 8 only — the conc curve will be widened once the topology is proven green.
• Launcher (runners/launch_b200-dgxc.sh): adds the kimik3/fp4 model-path mapping (/lustre/fsw/models/Kimi-K3, pre-staged and verified present on the first sweep attempt), overlays the kimi-k3 agentic recipes onto the srt-slurm clone, and adds the agentic default_mounts (/aiperf_mmap_cache, /hf_hub_cache) already used by the GB200/GB300 agentic paths.
• Trace loader: kimik3* already resolves to the default CC-traces weka loader on main (eb30e57).

Files:

• benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k3/agentic/agg-b200-tp8pp2-agentic.yaml (new)
• configs/nvidia-master.yaml: new kimik3-fp4-b200-dynamo-vllm-agentic entry on the cluster:b200-dgxc pool
• runners/launch_b200-dgxc.sh, perf-changelog.yaml, MODELS.md/MODELS_zh.md (PR link on the existing Kimi-K3 row)

Validation:

• python -m pytest utils/matrix_logic/ -v: 224 passed
• generate_sweep_configs.py full-sweep --model-prefix kimik3 yields exactly one agentic matrix entry (conc 8, TP8 PP2 EP1, decode num-worker 0, router 1.2.1, CONFIG_FILE routed to the new recipe)
• YAML parse + bash -n on the launcher pass
• First sweep attempt confirmed the launcher wiring end-to-end (model path resolved, recipe overlay found by srtctl) before failing on the old fork's schema — fixed by the v1.0.36 pin

中文说明

这是 bring-up 配置：InferenceX 中首个 Kimi-K3 基准测试配置；先做单并发冒烟测试，合并前通过 PR 扫描完成集群验证。

为 B200 上的 Kimi-K3 MXFP4 新增聚合式 TP8 × PP2（纯 TP，特意不启用专家并行(EP)）Dynamo-vLLM 智能体编码配方，沿用 #2196 建立的聚合式模式与 #2341/#2302 的 srt-slurm v1.0.36 AgentX 配方模式：

• 拓扑：原生 MXFP4 checkpoint（总参数 2.8T，权重约 1.4TB）无法装入单个 8×B200 节点，因此 TP8 切分注意力/稠密层、PP2 切分 93 层，跨 2 节点 / 16 GPU。聚合模式（预填充 num-worker: 1 + 解码 num-worker: 0）：单个 worker 同时承担预填充与解码，无 P/D KV 传输。
• 镜像：专用 bring-up 镜像 vllm/vllm-openai:kimi-k3（已在 Docker Hub 验证存在），启用 VLLM_ENABLE_K3_LATENT_MOE_TAIL_FUSION=1 及上述 vLLM 启动参数（保留 worker 可接受的 kimi_k3 推理解析器）。--enable-auto-tool-choice/--tool-call-parser 不在 worker 上设置：dynamo-vllm worker 入口不接受这两个参数（第二次扫描报 unrecognized arguments），聊天解析由 dynamo 前端处理，与 DSv4 GB300 智能体配方约定一致；不再显式设置 max-model-len，由 vLLM 从模型配置推导原生 1M 窗口。
• srt-slurm 基座：b200-dgxc 智能体路径改为固定使用上游 NVIDIA/srt-slurm v1.0.36（与 Add DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP recipes / 新增 DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP 配置 #2341 中 GB300 的迁移一致；[WIP] Test DeepSeek-V4 GB300 Dynamo AgentX recipes / 测试 DeepSeek-V4 GB300 Dynamo AgentX 配置 #2302 已验证）。此前克隆的 cquil11/srt-slurm-nv 分支拒绝配方字段（首次扫描报 benchmark.aiperf_server_metrics: Unknown field）。
• Dynamo：wheel 与 router 固定为 1.2.1，即 Add DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP recipes / 新增 DeepSeek-V4 GB300 Dynamo-vLLM AgentX MTP 配置 #2341 GB300 聚合式 AgentX 配方中与 v1.0.36 搭配验证过的版本组合。
• 上下文：AgentX 轨迹回放使用原生 1M max-model-len；max-num-batched-tokens: 8192 防止超长预填充撑爆流水线阶段。仅并发 8 冒烟测试——拓扑验证通过后再扩展并发曲线。
• 启动器（runners/launch_b200-dgxc.sh）：新增 kimik3/fp4 模型路径映射（模型已预置于 /lustre/fsw/models/Kimi-K3，首次扫描已确认存在），将 kimi-k3 智能体配方覆盖到 srt-slurm 克隆中，并补充 GB200/GB300 智能体路径已使用的缓存挂载。

已完成验证：utils/matrix_logic 224 项测试通过；扫描生成器输出恰为一个智能体矩阵条目（conc 8）；YAML 解析与启动器 bash -n 通过；首次扫描已端到端验证启动器接线（模型路径解析、配方覆盖生效），仅在旧分支 schema 校验处失败，已通过固定 v1.0.36 修复。

Related experiments

Three sibling [agentic experiment] PRs race the same recipe with different worker parser flags; whichever goes green first merges and the others close:

• Variant A (this PR): --dyn-tool-call-parser kimi_k3 --reasoning-parser kimi_k3 --dyn-reasoning-parser kimi_k3
• Variant B: --tool-call-parser kimi_k3 --reasoning-parser kimi_k3 — [Klaud Cold][agentic experiment][Variant B] Kimi-K3 B200 agg TP8xPP2 agentic — vLLM tool-call-parser flags / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——vLLM tool-call-parser 参数 #2357
• Variant C: no parser flags on the worker — [Klaud Cold][agentic experiment][Variant C] Kimi-K3 B200 agg TP8xPP2 agentic — no parser flags / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——不带解析器参数 #2358
• Variant D: direct vllm serve (srt-slurm PR 278 + multinode patch), no Dynamo — [Klaud Cold][agentic experiment][Variant D] Kimi-K3 B200 agg TP8xPP2 agentic — direct vllm serve (srt-slurm PR 278) / Kimi-K3 B200 聚合式 TP8xPP2 智能体实验——直接 vllm serve（srt-slurm PR 278） #2359

相关实验

三个兄弟 [agentic experiment] PR 以不同的 worker 解析器参数并行竞跑同一配方，先通过者合并、其余关闭：变体 A（本 PR，dynamo --dyn- 命名空间参数）*；变体 B（vLLM 原生写法）#2357；变体 C（不带解析器参数）#2358；变体 D（直接 vllm serve，srt-slurm PR 278 + 多节点补丁，不经 Dynamo）#2359。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30294472010
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30294472010

[claude]

🔴 The recipe deliberately sets health_check.max_attempts: 1440 (4h) for the ~1.4TB Kimi-K3 MXFP4 checkpoint, but runners/launch_b200-dgxc.sh:291 unconditionally runs sed -i 's/^ max_attempts: [0-9]*/ max_attempts: 720/' on the copied config right before srtctl apply, silently clobbering it back to 720 attempts (2h) — the same budget sized for DSR1-FP8 at roughly half this checkpoint's weight size. The recipe file will look correctly configured but the wider window never actually takes effect at runtime; either make the sed a floor (only raise, never lower) or special-case kimik3 like the other model-prefix branches nearby.

Extended reasoning...

The bug: runners/launch_b200-dgxc.sh line 291 runs:

sed -i 's/^  max_attempts: [0-9]*/  max_attempts: 720/' "${CONFIG_FILE%%:*}"

right before srtctl apply -f "$CONFIG_FILE" inside the IS_MULTINODE branch. This is an unconditional hard-set, not a max()/floor operation — whatever numeric value follows max_attempts: (two-space indent) in the config file gets overwritten to exactly 720, no matter what it was before.

The interaction this PR introduces: the new recipe benchmarks/multi_node/srt-slurm-recipes/vllm/kimi-k3/agentic/agg-b200-tp8pp2-agentic.yaml sets:

health_check:
  interval_seconds: 10
  max_attempts: 1440

with a two-space indent that matches the sed's regex exactly. The recipe's own comments make clear this 1440 (= 4h at 10s/attempt) was deliberately sized up from the launcher's usual value, specifically because the native MXFP4 checkpoint is ~1.4TB and has to be pulled over shared Lustre across 2 nodes. But because the sed is unconditional, srtctl apply never sees 1440 — it sees the file after the sed has already rewritten it to 720 (= 2h). The comment directly above the sed ("Bump recipe health-check timeout from 360×10s to 720×10s so large-model loads ... finish in time") shows the sed was written with the mental model of bumping the common 360 default up to 720; it never anticipated a recipe explicitly setting a value above 720, so for this recipe the sed is actually a reduction, not a bump.

Why nothing else catches this: there's no validation step between writing the recipe's health_check block and srtctl apply that would surface the discrepancy — the sed runs silently, will echo the sed invocation but not diff the before/after content of the file, and the recipe file committed to the repo (and reviewed in this PR) legitimately shows 1440. Someone reading the recipe or this diff would have no way to know the value gets overwritten at runtime.

Step-by-step proof:

1. CONFIG_FILE resolves to recipes/vllm/kimi-k3/agentic/agg-b200-tp8pp2-agentic.yaml (per the configs/nvidia-master.yaml entry's additional-settings: CONFIG_FILE=...), a copy of this PR's new recipe overlaid into the srt-slurm-nv clone.
2. That file, as written in this PR, contains the line max_attempts: 1440.
3. At runtime, line ~289 first does sed -i "s/^name:.*/name: \"${RUNNER_NAME}\"/" (unrelated), then line 291 does sed -i 's/^ max_attempts: [0-9]*/ max_attempts: 720/' "${CONFIG_FILE%%:*}".
4. The regex ^ max_attempts: [0-9]* matches max_attempts: 1440 (2-space indent, then digits), so the line becomes max_attempts: 720.
5. srtctl apply -f "$CONFIG_FILE" runs immediately after, reading the now-mutated file — it applies max_attempts: 720, interval_seconds: 10 → a 7200s (2h) health-check window, not the 14400s (4h) the recipe declares.

Impact: if 2h genuinely isn't enough to pull ~1.4TB of MXFP4 weights and bring the engine up (workers loading in parallel off shared Lustre, contended by whatever else is running on the same filesystem), the health check gives up and fails the entire multinode job — the exact scenario this recipe's author was trying to prevent by widening the window, on the largest checkpoint in the fleet and its first-ever bring-up. Because the recipe file looks correct, anyone debugging a health-check failure here would have to know to check the launcher script rather than trusting the recipe as documentation of what will actually run — a debugging detour on a bring-up PR that's likely to already have plenty of other failure modes to sort through.

Fix: make the sed a floor instead of a hard-set, e.g. only replace when the existing value is less than 720 (or compare-and-max in the shell), or skip the sed entirely for kimik3/any recipe that already sets a larger value, similar to how other model prefixes get their own branch in this same script.

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30295407363
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30295407363

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30295861948
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30295861948

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30297837112
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30297837112

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30298269589
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30298269589

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30298672988
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30298672988

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30300683178
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30300683178

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30304440483
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30304440483

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30306482385
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30306482385

[functionstackx]

Closing in favor of the direct-vLLM experiment (Variant D, #2359): serving Kimi-K3 directly with vllm serve removes the entire Dynamo compatibility surface this PR had to fight through (tokenizer 404s, session_control 400s, --dyn-* worker args). The debugging trail here carries over — Variant D already inherits the mamba_hybrid index-dtype container patch, the conv-aware-routing opt-out, gpu-memory-utilization 0.90, and the launcher/model-path plumbing. In-flight sweep cancelled.

中文：关闭本 PR，转向直接 vLLM 实验（变体 D，#2359）：直接以 vllm serve 提供服务，从根本上消除本 PR 需要逐一解决的 Dynamo 兼容性问题（分词器 404、session_control 400、--dyn-* worker 参数）。调试成果均已由变体 D 继承——mamba_hybrid 索引类型容器补丁、会话感知路由关闭、显存利用率 0.90 及启动器/模型路径接线。进行中的扫描已取消。

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30308356213
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=30308356213