[jeremydmiller]

Fixes #5254.

Cause

The owner is runtime shutdown, not the EventLog provider or overlapping test hosts.

BufferedReceiver.DrainAsync() only waits for in-flight messages when the receiver is already latched (var waitForCompletion = _latched;). If it isn't latched, it assumes it was called from inside the handler pipeline, where waiting would deadlock.

EndpointCollection.DrainAsync() drains local queues without latching them. DurableLocalQueue.DrainAsync() latches its own receiver first; BufferedLocalQueue, which inherits BufferedReceiver.DrainAsync(), never did. So IHost.StopAsync() returned while a handler on a buffered local queue was still running, and the host then disposed its logging and container under it.

In metrics_source_tag, the MetricFail tracked session completes on the MovedToErrorQueue record. WolverineRuntime.MovedToErrorQueue() writes that record just before it logs the move (WolverineRuntime.Tracking.cs). The test therefore resumes and tears the host down while that log call is still to come, and on Windows the default EventLog provider throws ObjectDisposedException. The recovery path's logger is disposed as well, which produces the follow-on "could not recover from a cascading failure".

Evidence: the new regression test holds a handler at a gate on a buffered local queue and calls StopAsync(). On main, StopAsync() completes while the handler is still held (stopping.IsCompleted is true after 250 ms). With the fix it waits.

Change

EndpointCollection.DrainAsync() latches each BufferedLocalQueue before draining it. The drain then waits for in-flight handlers, bounded by DurabilitySettings.DrainTimeout, as it already does for durable local queues and for listeners.

The in-pipeline drain path (pausing) is untouched. It doesn't go through EndpointCollection.DrainAsync(), so the deadlock guard in BufferedReceiver.DrainAsync() still holds there.

No assertions are weakened, and no ObjectDisposedException is swallowed. metrics_source_tag is unchanged: the intentional handler failure and the metrics assertions stay as they were.

Tests

CoreTests/Runtime/buffered_local_queue_shutdown_waits_for_in_flight_5254.cs. A gated handler on a buffered local queue fails and moves to the error queue, like MetricFail. A logger provider records anything written to it after it has been disposed, standing in for EventLog on macOS and Linux. The test asserts that:

• StopAsync() does not complete while the handler is held;
• the error-queue move is logged before shutdown finishes;
• nothing is logged after the host disposes the provider.

It fails on main and passes with the fix.

Local verification (macOS, net9.0, Release; Windows not run locally):

• dotnet build wolverine.slnx -c Release -f net9.0: clean
• CoreTests full suite: 3928 passed, 8 skipped, 0 failed (includes metrics_source_tag)

For review: a host whose buffered-queue handler never returns now waits up to DrainTimeout (30 s by default) at shutdown, instead of stopping out from under the handler. That is the behaviour durable local queues and listeners already had.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DCggXgxnG3EBXoR9rrq37U