Skip to content

Optimize subscriber slot delivery#104

Open
dallison wants to merge 12 commits into
mainfrom
subspace-latency-queues
Open

Optimize subscriber slot delivery#104
dallison wants to merge 12 commits into
mainfrom
subspace-latency-queues

Conversation

@dallison

@dallison dallison commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Add per-subscriber slot queues and use them as the authoritative unreliable receive path.
  • Add a subscriber option to disable internal dropped-message detection when higher layers already track drops.
  • Avoid reliable-publisher backpressure from unreliable subscribers and skip unnecessary bitset notifications when no reliable subscribers exist.

Test plan

  • bazelisk test //common:common_test //client:client_test --config=macos_arm64 --test_filter=CommonTest.InPlaceSlotQueueEvictsOldestOnOverflow,ClientTest.PublishSingleMessageAndRead:ClientTest.DroppedMessage:ClientTest.DroppedMessageDetectionCanBeDisabled:ClientTest.ReliablePublisherDoesNotBlockOnUnreliableSubscriber --test_output=errors
  • bazelisk build //rust_client:subspace_client_rust --config=macos_arm64

dallison added 12 commits July 3, 2026 15:34
Add per-subscriber slot queues and a drop-detection toggle so unreliable subscribers can avoid scanning slot bitsets on the hot receive path.
Explicitly reset the read message before reusing the publisher buffer in
PublishAndReadWithSubscriberQueue so the slot is released as intended.
Keep bitsets authoritative while making per-subscriber queues bounded, reclaimable, ABI-checked, and consistent across C++, Rust, and shadow recovery.
Make queue delivery recover safely through authoritative bitsets across overflow, rejection, churn, and server failover while preserving C++ and Rust parity.
Decouple publisher arena provisioning in bytes from per-subscriber queue capacity and carry the layout through clients, bridges, and recovery.
Avoid retaining rejected reusable messages and keep the Android-only constructor test aligned with the explicit queue arena API.
Preserve version diagnostics before unmapping, clear partial mapping state, and initialize the synthetic memfd CCB used by Android tests.
Concurrent publishers can reserve queue positions out of ordinal order; recover earlier bits and only discard exact generations already delivered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant