feat: Dynamic Shard Hash Ring Re-Balancing across Active Keeper Nodes

[OG-wura]

Close #907

Dynamic Shard Hash Ring Re-Balancing across Active Keeper Nodes

Summary

Replaces the static modulo-based task sharding fallback with a consistent hash ring that dynamically redistributes task ownership when keeper nodes join or leave the cluster. Integrates with the existing P2P network for automatic node discovery and ring updates.

Changes

keeper/src/sharding.js

• ConsistentHashRing class:
  • SHA-256 based ring with configurable virtual nodes (default 150) for uniform distribution
  • addNode(nodeId) / removeNode(nodeId) — dynamic membership with O(log N) binary search lookup
  • rebuildFromNodeIds(nodeIds) — batch rebuild for P2P sync
  • clear() — reset ring state
  • Deterministic mapping — same task ID always maps to the same node for a given node set
• filterTasksByHashRing(taskIds, ring, selfNodeId) — partitions tasks into owned / skipped using the ring, matching the existing shard return contract (ownedTaskIds, skippedTaskIds, shardCount, shardLabel)
• Existing modulo functions preserved for backward compatibility

keeper/index.js

• Hash ring replaces filterTasksForShard as the non-P2P fallback in selectTaskOwnership()
• P2P disabled: ring populated with synthetic keeper-shard-{i} node IDs from static shardCount
• P2P enabled: ring auto-rebuilds via peer:updated / peer:stale events — nodes joining or leaving the P2P cluster immediately trigger rebalancing
• Configurable via HASH_RING_VNODES env var (default: 150)

keeper/__tests__/sharding.test.js

18 new test cases covering:

• Empty ring, single-node ownership, multi-node distribution
• Deterministic mapping, minimal remapping on node join/leave
• Idempotent add/remove, string/number key consistency
• rebuildFromNodeIds, task partitioning, null/empty edge cases

Key Properties

Property	Modulo (old)	Hash Ring (new)
Node join remap	~all tasks	~1/N tasks
Node leave remap	~all tasks	~1/N tasks
Lookup cost	O(1)	O(log N)
Dynamic membership	No	Yes
Zero coverage gaps	Yes	Yes (wrap-around)

[vercel]

@OG-wura is attempting to deploy a commit to the Ayomide Adeniran's projects Team on Vercel.

A member of the Team first needs to authorize it.

[drips-wave]

@OG-wura Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

[ayomideadeniran]

pr under review, i will get back to you if i find any wrong implementations