Skip to content

perf(stats): select exact percentiles for large latency reports - #6

Merged
hatoo merged 1 commit into
masterfrom
codex/perf-latency-percentiles-20261009
Oct 9, 2026
Merged

hatoo merged 1 commit into
masterfrom
codex/perf-latency-percentiles-20261009

Conversation

@hatoo

@hatoo hatoo commented Oct 9, 2026

Copy link
Copy Markdown
Owner

Large benchmark runs retain every latency sample but only report nine percentile ranks. Computing those ranks currently sorts the entire array. Select exact ranks over bounded partitions, retain the existing sort for at most 1,024 samples, and directly read ascending/descending arrays.

All samples and floating-point report values are preserved. Collection, timestamps, request/error counts, and the elapsed-time boundary are unchanged. This improves latency-summary computation after a run; it makes no request-throughput claim.

Measured with immutable baseline e09bfd1 and candidate binaries, full release optimization, CPU affinity, five alternating process pairs, and separate Rust 1.98/seed 73 and Rust 1.91/seed 73073 series:

  • Unordered 1M samples: median summary CPU 18.029 -> 4.025 ms (-77.67%); independent confirmation -78.07%.
  • Unordered 10M samples: 217.896 -> 68.022 ms (-69.25%); confirmation -68.93%. Wall-time results agree.
  • Complete-summary Callgrind instructions at 1M fall 70.61%; three pairs reproduce exactly. Zero summary allocations in both variants.
  • Tradeoff: the 10M-sample, 32-distinct-value workload did not improve (+0.97% primary, +5.74% confirmation). Small ordered timings remain noisy. No universal speed claim.

Validation: 315 unit + 51 integration tests pass in each debug/release profile (two existing ignored unit tests); strict all-target Clippy and formatting pass; standalone MSRV tests pass. New exhaustive/seeded tests compare every output bit and the complete sample multiset. All 36 full text/JSON report comparisons are byte-identical. Fixed-hfast H1/H2/H3 runs complete 196,608 exact status-200 successes with zero errors.

The committed benchmark harnesses and benchmarks/accepted-latency-selection.md contain reproduction instructions, all distribution summaries, uncertainty, and tradeoffs.

@hatoo
hatoo enabled auto-merge (squash) October 9, 2026 14:08
@hatoo
hatoo merged commit 5b8846c into master Oct 9, 2026
4 checks passed
@hatoo
hatoo deleted the codex/perf-latency-percentiles-20261009 branch October 9, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant