[chenfeiz0326]

Dev Engineer Review

The change adds three DeepSeek-V4.1-Flash aggregated performance-sanity cases to the GB300 4-GPU post-merge list: TEP4, DEP4 with attention DP, and TEP4 with embedded DSpark. It adds measured duration entries and sets kv_cache_config.dtype: auto in the DeepSeek configuration.

The supplied change summary also reports GLM-5.3-Flash FP8 and NVFP4 aggregated workloads, a disaggregated GLM-5.3-Flash workload, a multi-node test-list entry, and a model-path mapping. It reports changes to Jenkins shard configuration. This conflicts with the PR objective’s statement that no Jenkins changes are needed, so confirm the actual Jenkins diff before merging. No engine-code changes are reported.

The author reports successful GB300 runs using CI’s wheel and GenPostMergeBuilds 3244. Reported pytest times are 636, 624, and 702 seconds for the three DeepSeek cases. These results were not independently verified here.

QA Engineer Review

The GB300 4-GPU CI list adds the three DeepSeek cases. Other reported test-list additions cover GLM-5.3-Flash cases in the GB300 4-GPU and multi-node lists. The author reports that all three DeepSeek cases passed, but the results were not independently verified. Coverage verdict: needs follow-up until the reported Jenkins changes are reconciled with the stated scope and the CI results are confirmed.

Per-File QA Perspective

• tests/integration/defs/.test_durations: Adds runtime records for DeepSeek and GLM cases. QA should check that the durations match the corresponding test selectors and observed CI runtimes.
• tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml: Adds DeepSeek and GLM performance-sanity entries. Verify that each selector resolves to the intended workload and remains within the expected post-merge runtime.
• tests/scripts/perf-sanity/aggregated/deepseek_v41_flash_grace_blackwell.yaml: Sets the KV-cache dtype to auto in three server configurations. QA should confirm that the harness records the intended dtype.
• jenkins/L0_Test.groovy: The supplied change summary reports changes to the GB300 4-GPU and 8-GPU shard configuration. Confirm this change and its consistency with the stated no-Jenkins-change objective.
• tests/integration/defs/perf/_model_paths.py: Adds a model-path mapping for GLM-5.3-Flash NVFP4. Verify that the path resolves to the expected checkpoint.
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml: Adds a GLM-5.3-Flash multi-node test entry. Verify that its selector maps to the disaggregated workload.
• tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_fp8_grace_blackwell.yaml: Adds FP8 attention-DP and long-prompt workload configurations. QA should verify their client parameters, KV-cache settings, and runtime behavior.
• tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_nvfp4_grace_blackwell.yaml: Adds a GLM-5.3-Flash NVFP4 aggregated workload. QA should verify model resolution and the configured serving and client options.
• tests/scripts/perf-sanity/disaggregated/gb300_glm53-flash-fp8_8k1k_con16_ctx1_tep4_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml: Adds a disaggregated GLM-5.3-Flash workload. QA should verify context/generation worker settings and NIXL KV transfer.

Description

Adds DeepSeek-V4.1-Flash and GLM-5.3-Flash cases to the GB300 post-merge perf-sanity lists. Every case passed on GB300 with CI's own wheel (see Test Coverage). Only perf configs, test lists, the model-path map, .test_durations and Jenkins shard counts change; no library or harness code.

DeepSeek-V4.1-Flash, base checkpoint (aggregated)

tests/scripts/perf-sanity/aggregated/deepseek_v41_flash_grace_blackwell.yaml came in with DeepSeek-V4.1-Flash support (#19727), but no test list ever named it, so CI has never run it. Its three cases join the GB300 4-GPU post-merge list next to the NVFP4 DSpark cases from #20680:

Test id (inside test_e2e[...])	Shape	Load (8k/1k)
aggr_upload-deepseek_v41_flash_grace_blackwell-v41_flash_tep4_8k1k	TEP4	con8, 80 requests
aggr_upload-deepseek_v41_flash_grace_blackwell-v41_flash_dep4_8k1k	DEP4 (attention DP)	con128, 640 requests
aggr_upload-deepseek_v41_flash_grace_blackwell-v41_flash_tep4_dspark_8k1k	TEP4 + embedded DSpark (draft 5, 3 accepted, forced)	con8, 80 requests

The yaml now writes out kv_cache_config.dtype: auto. That is the default, so the engine runs the same; without it, the harness records the case's KV dtype as fp8 in the perf DB.

GLM-5.3-Flash (aggregated and disaggregated)

Four cases on the shapes the GLM-5.3-Flash deployment guide and its perf PRs (#19136, #20251, #20389, #20390, #20454, #20755) measure. They add to the existing glm53_flash_fp8_tp4_mtp3_8k1k case:

Test id (inside test_e2e[...])	Shape	Load	Why
aggr_upload-gb300_glm53_flash_fp8_grace_blackwell-glm53_flash_fp8_dep4_mtp3_fp8kv_8k1k	FP8, attention DP4 / EP4 + MTP3, FP8 KV, batch 64 per rank, 4096 tokens	8k/1k, con256	The high-concurrency frontier shape (#19136, #20755). FP8 KV runs sparse MLA natively under attention DP (#20454); same engine shape as the test_attention_dp_mtp_and_fp8_kv accuracy gate
aggr_upload-gb300_glm53_flash_fp8_grace_blackwell-glm53_flash_fp8_tp4_mtp3_isl13312_osl33	FP8, TP4 / EP4 + MTP3, BF16 KV, 16384 tokens	ISL 13312 / OSL 33, con8 then con1	The long-prompt, short-answer workload of the prefill optimizations (#20251, #20389, #20390, #20454)
aggr_upload-gb300_glm53_flash_nvfp4_grace_blackwell-glm53_flash_nvfp4_tp4_mtp3_8k1k	NVFP4 checkpoint, TP4 / EP4 + MTP3, FP8 KV, TRTLLM MoE	8k/1k, con16 then con1	The NVFP4 checkpoint (#20127, #20547) has no CI perf coverage yet. AUTO resolves it to CUTLASS on main, so TRTLLM is set explicitly, as in #20454
disagg_upload-e2e-gb300_glm53-flash-fp8_8k1k_con16_ctx1_tep4_gen1_tep4_eplb0_mtp3_ccb-NIXL	FP8, 1 ctx + 1 gen worker, each TP4 / EP4, MTP3, Python NIXL transceiver	8k/1k, con16	The deployment guide's disaggregated recipe, at the same load as the aggregated tp4_mtp3 case

• New model-path key glm_5.3_flash_nvfp4 → GLM-5.3-Flash-NVFP4.
• The accepted draft length is pinned to 2.384 of 3, the value of the existing GLM case.
• In the two multi-client cases the larger client runs first. Aggregated clients send no warmup request, and the first few requests a fresh server sees are slow. When con1 ran first, it measured a mean TTFT of 1,122 ms against a 254 ms median (long prompt) and 941 vs 138 ms (NVFP4). Run second, con1 measures 252 / 252 / 258 ms and 138 / 138 / 142 ms (mean / median / P99).

TIMEOUTs, durations and shards

• TIMEOUT follows the README rule (slowest passing run + 25%, at least +15 min, rounded up to 5, floor 30). Every validation run took 8–12 min of pytest, so all seven new entries get 30.
• .test_durations is seeded with the measured pytest times.
• GB300-4_GPUs-PyTorch-PerfSanity-Post-Merge goes from 4 to 5 shards. With the six new aggregated cases it holds 23 tests, and 4 shards would put 7 tests (~120 min) into two of them; with 5 shards the longest is ~96 min and 6 tests.
• GB300-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE1-GPU4-Post-Merge goes from 1 to 2 shards (one test per shard, as submit.py requires).

Cost per post-merge run: about +60 min of pytest on single 4-GPU nodes (~4 GPU-hours) plus one more shard's setup, and about +8.5 min on the 2-node stage (~1.1 GPU-hours).

Test Coverage

Every case passed on GB300 (oci-aga) with CI's own wheel, GenPostMergeBuilds 3244 (= main 7587cc8, this branch's base), and this branch's tree, run through jenkins/scripts/perf/local/submit.py:

Case	Requests	pytest time	Result
DSv4.1 v41_flash_tep4_8k1k	80 / 80	636 s	831 output tok/s, 104.6 tok/s/user
DSv4.1 v41_flash_dep4_8k1k	640 / 640	624 s	5,867 output tok/s, 47.6 tok/s/user
DSv4.1 v41_flash_tep4_dspark_8k1k	80 / 80	702 s	674 output tok/s, 96.2 tok/s/user
GLM glm53_flash_fp8_dep4_mtp3_fp8kv_8k1k	1,280 / 1,280	606 s	7,098 output tok/s (1,775 per GPU), 30.3 tok/s/user
GLM glm53_flash_fp8_tp4_mtp3_isl13312_osl33	160 / 160 + 10 / 10	484 s	con8: median TTFT 581 ms; con1: TTFT 252 ms, E2EL 320 ms
GLM glm53_flash_nvfp4_tp4_mtp3_8k1k	80 / 80 + 10 / 10	541 s	con16: 2,144 output tok/s; con1: 538 tok/s/user, TTFT 138 ms
GLM disagg e2e con16	80 / 80	504 s	3,233 output tok/s, 212.7 tok/s/user, median TTFT 259 ms

• DSv4.1 DSpark is not faster than plain decode at con8 (674 vs 831 output tok/s), even though the forced acceptance is applied: DSpark verifies through SpecWorkerBase.sample_and_accept_draft_tokens, which applies TLLM_SPEC_DECODE_FORCE_NUM_ACCEPTED_TOKENS. The case tracks DSpark as it is.
• Every new engine config validates against tensorrt_llm/schemas/trtllm-serve-config.schema.json (a misspelled control is rejected).
• CPU tests, on this tree and on main's tree as a control: test_disagg_e2e_configs.py (253 passed, one more than main: the new disagg yaml), test_perf_submit.py (97), test_test_to_stage_mapping.py, test_infra_dry_run_pipeline.py and the CBTS tests pass; scripts/check_test_list.py --validate validates 2,037 entries (7 more than main). The remaining local failures are identical on main's tree (missing local modules).
• In CI the new aggregated cases land in GB300-4_GPUs-PyTorch-PerfSanity-Post-Merge-2 to -5, and the disaggregated one in GB300-8_GPUs-2_Nodes-PyTorch-Disagg-PerfSanity-CTX1-NODE1-GPU4-GEN1-NODE1-GPU4-Post-Merge-2.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review.

Recent review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/TensorRT-LLM
• Branch: main
• File: .coderabbit.yaml
• Commit: 6ba2bec
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: a8b9fb6b-1d8e-4771-a257-fdf4baf61df0

Commits

Reviewing files that changed from the base of the PR and between 2c13654 and 58ef3c9.

Files selected for processing: 8

• jenkins/L0_Test.groovy
• tests/integration/defs/.test_durations
• tests/integration/defs/perf/_model_paths.py
• tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml
• tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml
• tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_fp8_grace_blackwell.yaml
• tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_nvfp4_grace_blackwell.yaml
• tests/scripts/perf-sanity/disaggregated/gb300_glm53-flash-fp8_8k1k_con16_ctx1_tep4_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The change adds GB300 aggregated and disaggregated performance sanity cases for DeepSeek V4.1 Flash and GLM-5.3-Flash. It also records timings, maps the NVFP4 checkpoint, and updates Jenkins shard coverage.

Changes

GB300 performance sanity coverage

Layer / File(s)	Summary
Configure and register DeepSeek cases tests/scripts/perf-sanity/aggregated/deepseek_v41_flash_grace_blackwell.yaml, tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml	The TEP4, DEP4, and TEP4 DSpark configurations explicitly set kv_cache_config.dtype to auto. The test list adds the three cases.
Add aggregated GLM cases tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_fp8_grace_blackwell.yaml, tests/scripts/perf-sanity/aggregated/gb300_glm53_flash_nvfp4_grace_blackwell.yaml, tests/integration/test_lists/test-db/l0_gb300_multi_gpus_perf_sanity.yml, tests/integration/defs/perf/_model_paths.py, tests/integration/defs/.test_durations	The change adds GLM-5.3-Flash FP8 attention-DP4 and long-prompt configurations, an NVFP4 configuration, their aggregated test entries, an NVFP4 checkpoint mapping, and timing records.
Add disaggregated GLM case tests/scripts/perf-sanity/disaggregated/gb300_glm53-flash-fp8_8k1k_con16_ctx1_tep4_gen1_tep4_eplb0_mtp3_ccb-NIXL.yaml, tests/integration/test_lists/test-db/l0_gb300_multi_nodes_perf_sanity_ctx1_node1_gpu4_gen1_node1_gpu4.yml, tests/integration/defs/.test_durations	The change adds a disaggregated GLM-5.3-Flash FP8 benchmark configuration and test entry, and records its timing.
Update Jenkins shard coverage jenkins/L0_Test.groovy	The GB300 4-GPU stage increases from four shards to five. The two-node disaggregated stage adds GLM-5.3-Flash and increases from one shard to two.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Suggested reviewers: juney-nvidia

Merge Risk: Minimal · up to 58ef3

The added performance-sanity cases have no identified merge-blocking issue. The long-input GLM case fits its configured sequence limit.

Pre-merge checks: 5

Check name	Status	Explanation
Title check		The title clearly summarizes the addition of DeepSeek-V4.1-Flash and GLM-5.3-Flash performance-sanity cases and follows the required [None][type] format.
Description check		The description includes the required Description, Test Coverage, and PR Checklist sections. It clearly explains the changes, rationale, affected configurations, validation results, and scope.
Docstring Coverage		No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within survi…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

No eligible functions found in the changed files. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 0 functions across 1 files. (7 skipped: 7 unsupported.)

Finishing Touches

🧪 Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}