Skip to content

perf: replace flume with tokio::sync::mpsc - #1041

Closed
de-sh wants to merge 1 commit into
bytebeamio:mainfrom
de-sh:tokio-sync
Closed

de-sh wants to merge 1 commit into
bytebeamio:mainfrom
de-sh:tokio-sync

Conversation

@de-sh

@de-sh de-sh commented Apr 5, 2026 •

Copy link
Copy Markdown
Contributor

Replace flume channel with tokio::sync::mpsc in the eventloop and client. The sync Client gains a runtime Handle for block_on sends.

Benchmark Results: tokio-sync vs main

refer: bench/throughput

Payload main (median) tokio-sync (median) Δ time Δ throughput
32 B 1.402 ms 1.316 ms −5.4% +5.8%
100 B 1.397 ms 1.383 ms −3.5% +3.6%
1024 B 1.664 ms 1.716 ms +3.3% −3.2%

All results are statistically significant (p < 0.05).


Interpretation

Small messages (32–100 B): tokio-sync wins by 3.5–5.4%. These are channel-bound — the message is tiny relative to the per-message bookkeeping cost, so the reduction in lock contention from dropping flume's mutex pays off directly. The channel send/receive path sees fewer cache line bounces.

Large messages (1024 B): tokio-sync is 3.3% slower. At 1 KB, the payload dominates. The likely cause is that tokio's bounded mpsc uses an intrusive linked-list of Arc-allocated nodes rather than flume's pre-allocated ring buffer, so 1 KB payloads incur an extra allocation + Arc refcount per message where flume's ring buffer would reuse a slot. This is a known design tradeoff in tokio mpsc.

The crossover point is somewhere between 100 B and 1024 B. For the typical MQTT use case (small control messages, moderate publish payloads), the switch is neutral-to-positive overall.

Type of change

Checklist:

  • Formatted with cargo fmt
  • Make an entry to CHANGELOG.md if it's relevant to the users of the library. If it's not relevant mention why.

Replace flume channel with tokio::sync::mpsc in the eventloop and
client. The sync Client gains a runtime Handle for block_on sends;
Connection continues to own the Runtime so the I/O driver stays alive
for the lifetime of the connection.

Results: tokio-sync vs main

  ┌─────────┬───────────────┬─────────────────────┬────────┬──────────────┐
  │ Payload │ main (median) │ tokio-sync (median) │ Δ time │ Δ throughput │
  ├─────────┼───────────────┼─────────────────────┼────────┼──────────────┤
  │ 32 B    │ 1.402 ms      │            1.316 ms │ −5.4%  │ +5.8%        │
  ├─────────┼───────────────┼─────────────────────┼────────┼──────────────┤
  │ 100 B   │ 1.397 ms      │            1.383 ms │ −3.5%  │ +3.6%        │
  ├─────────┼───────────────┼─────────────────────┼────────┼──────────────┤
  │ 1024 B  │ 1.664 ms      │            1.716 ms │ +3.3%  │ −3.2%        │
  └─────────┴───────────────┴─────────────────────┴────────┴──────────────┘

  All results are statistically significant (p < 0.05).

  ---
  Interpretation

  Small messages (32–100 B): tokio-sync wins by 3.5–5.4%. These are channel-bound — the message is tiny relative to the per-message bookkeeping cost, so the reduction in lock
  contention from dropping flume's mutex pays off directly. The channel send/receive path sees fewer cache line bounces.

  Large messages (1024 B): tokio-sync is 3.3% slower. At 1 KB, the payload dominates. The likely cause is that tokio's bounded mpsc uses an intrusive linked-list of Arc-allocated
  nodes rather than flume's pre-allocated ring buffer, so 1 KB payloads incur an extra allocation + Arc refcount per message where flume's ring buffer would reuse a slot. This is a
  known design tradeoff in tokio mpsc.

  The crossover point is somewhere between 100 B and 1024 B. For the typical MQTT use case (small control messages, moderate publish payloads), the switch is neutral-to-positive
  overall.
@de-sh de-sh changed the title perf: replace flume with tokio::sync::mpsc perf: replace flume with tokio::sync::mpsc Apr 5, 2026
@de-sh

de-sh commented Apr 6, 2026 •

Copy link
Copy Markdown
Contributor Author

This PR was for documentation purpose. A note of why both tokio::sync::mpsc and flume are good enough.

@de-sh de-sh closed this Apr 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant