Skip to content

perf(completion): make terminal waits lock-free - #288

Open
ryux1 wants to merge 1 commit into
apache:mainfrom
ryux1:perf/246-completion-atomic-status
Open

perf(completion): make terminal waits lock-free#288
ryux1 wants to merge 1 commit into
apache:mainfrom
ryux1:perf/246-completion-atomic-status

Conversation

@ryux1

@ryux1 ryux1 commented Sep 6, 2026

Copy link
Copy Markdown

Closes #246.

Summary

  • Publish completion status through an atomic so completed and abandoned waits avoid the waiter mutex.
  • Keep pending registration cancellable with one lock and a status recheck, while cloning, dropping, and waking wakers outside the lock.
  • Add a regression test where RawWaker::clone drops the completer, proving clone callbacks cannot deadlock against completion state.

Design notes

Completion and abandonment hold the waiter mutex while detaching the current waiter cohort, then publish the terminal state with a release store. Polls use acquire loads, so observing Completed also observes the initialized OnceLock value and observing either terminal state happens after waiter detachment. A poll that initially sees Pending clones its waker before locking, then rechecks status under the waiter mutex before registering. Wait::drop uses the same load/lock/recheck pattern.

Three cargo x bench runs were collected for upstream main (93b3413) and this branch on an Intel Core i9-10900F (Linux x86_64); the table reports the median of each set:

Completion benchmark main this branch change
ready_wait 16.43 ns 6.848 ns -58.3%
cancel_pending 36.91 ns 40.04 ns +8.5%
complete_then_wait 86.88 ns 78.16 ns -10.0%
notify_pending 121.9 ns 113.9 ns -6.6%
fanout/1 144.1 ns 134.9 ns -6.4%
fanout/2 199.1 ns 183.2 ns -8.0%
fanout/4 308.0 ns 276.5 ns -10.2%
fanout/8 577.1 ns 531.0 ns -8.0%
fanout/32 2.573 us 2.380 us -7.5%

The pending cancellation path pays about 3.1 ns to clone outside the lock; terminal waits and every measured completion/notification path improve.

Validated with:

  • cargo x check
  • cargo x test
  • cargo x lint
  • cargo x miri
  • cargo +nightly miri test --package tests-integration --test completion_test
  • cargo x bench

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore a simpler waiter registration path for Completion

1 participant