Skip to content

Investigate possible missed admission-resumption wakeup with libuv before 1.53 #8436

Description

@cjen1-msft

Concern

Investigate whether inbound admission resumption can miss a wakeup when libuv coalesces notifications. This is a portable memory-ordering concern found during review, not a reproduced failure.

The path exists on upstream 00095f2519a119ac3fd5c384c9064a9302ffc999, before the shared notification locking proposed in #8435.

Relevant ordering

When inbound capacity becomes available, the registered waker does:

recheck_read_interest.store(true, std::memory_order_release);
wake();

The host callback consumes that flag before acquiring out_mutex:

const bool recheck_all =
  recheck_read_interest.exchange(false, std::memory_order_acq_rel);

Unlike pending responses and completions, publication and consumption of this flag do not share a mutex. Its atomic operations avoid a data race, but do not by themselves establish ordering with libuv's separate pending flag.

Potential missed-notification sequence

  1. Libuv clears its pending flag and starts the callback.
  2. The callback consumes recheck_read_interest as false.
  3. A producer sets recheck_read_interest to true and calls wake().
  4. If the older libuv coalescing path observes a stale pending value of one, it returns without scheduling another callback.
  5. The current callback finishes without rechecking admission.

The producer can take and release the lifecycle mutex before the callback's later lifecycle check. That check therefore does not establish the ordering needed before the producer's pending-flag load, nor does it recheck admission.

This sequence needs validation against the complete implementation and memory model. It has not been demonstrated on the deployed compiler, pthread implementation or CPU.

If reachable, paused socket reads could remain paused until another event causes an admission recheck. This would be a liveness issue, not lost queued response data. Its duration and practical reachability are unknown.

libuv contract

Libuv 1.48 documents thread-safe sends and notification coalescing, but not the stronger publication guarantee.

Current libuv documentation states that send/callback sequential consistency was added in 1.53.0 and warns that earlier coalescing cases may require a full sequentially consistent fence. The 1.48 implementation has a relaxed pending-flag fast-path load.

Investigation and acceptance

  • Validate or reject the ordering concern with a reduced model covering the admission flag, both lifecycle-lock acquisitions and libuv's pending flag.
  • Add a regression for admission recovery when unrelated traffic stops, including recovery triggered by a different interface.
  • If confirmed, establish explicit publication/notification ordering for supported libuv versions. Possible approaches include a shared mutex protocol for this flag or the documented fence mitigation; verify the complete protocol before choosing.
  • Keep this separate from Allow concurrent RPC event-loop notifications #8435: the same concern applies to the existing exclusive notifier lock, and does not invalidate the mutex-protected response/completion queue protocol.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions