Performance: Quicker/Better Hotpath and A/B Testing

[griffinmilsap]

Summary

Add a fast hot-path benchmark suite and an interleaved A/B comparison runner so perf work can happen on short, targeted loops instead of relying only on the long end-to-end ezmsg-perf run suite.

Motivation

ezmsg-perf run is useful as an end-to-end scenario benchmark, but it is expensive and noisy for everyday regression hunting. This branch adds a second layer of perf tooling aimed at:

• fast turnaround during development
• lower-noise hot-path characterization
• more reliable A/B comparisons by interleaving runs across refs instead of running all of A and then all of B

What Changed

New hot-path benchmark suite

Adds ezmsg-perf hotpath, a focused same-process roundtrip benchmark that measures:

• async/local
• async/shm
• async/tcp

with sync/* available as an opt-in API surface.

Notable characteristics:

• prebuilt payloads instead of allocating new messages inside the timed loop
• short, repeatable cases with warmup + timed samples
• per-case summary output in us/msg, msg/s, and coefficient of variation
• optional JSON output for machine-readable comparisons

New interleaved A/B runner

Adds ezmsg-perf ab, which compares two refs by alternating short hot-path runs instead of doing one long monolithic run per side.

Key behaviors:

• provisions comparison refs with temporary git worktrees
• supports comparing CURRENT against another ref like dev
• interleaves round order (A/B and B/A) with a reproducible shuffle seed
• reports per-case paired deltas using median us/msg
• can emit JSON summaries for later inspection

This is meant to reduce machine drift bias during long perf sessions.

Same-process transport selection support

Adds publisher-side controls so same-process hot-path benchmarking can characterize local, shm, and tcp distinctly instead of always collapsing to the local fast path.

Changes include:

• allow_local: None | bool
• force_tcp: None | bool
• process-wide defaults via:
  • EZMSG_ALLOW_LOCAL (defaults to enabled)
  • EZMSG_FORCE_TCP (defaults to disabled)

Behavior:

• allow_local=None inherits EZMSG_ALLOW_LOCAL
• force_tcp=None inherits EZMSG_FORCE_TCP
• explicit per-publisher values override env defaults
• force_tcp=True disables local delivery and logs an INFO message if local would otherwise have been used

This keeps the control general-purpose while still allowing stable characterization of same-process SHM and TCP paths.

Tests

Adds focused coverage for:

• hot-path case generation and smoke execution
• A/B command construction and summary math
• same-process publisher transport selection
• env-default and per-publisher override behavior for allow_local and force_tcp

CLI Examples

uv run ezmsg-perf hotpath
uv run ezmsg-perf hotpath --apis async sync
uv run ezmsg-perf ab --ref-a dev --ref-b CURRENT