Skip to content

Fix open-addressing index generation and wraparound - #836

Open
sleeepyjack wants to merge 2 commits into
NVIDIA:devfrom
sleeepyjack:cuco-issue-834-investigation
Open

Fix open-addressing index generation and wraparound#836
sleeepyjack wants to merge 2 commits into
NVIDIA:devfrom
sleeepyjack:cuco-issue-834-investigation

Conversation

@sleeepyjack

@sleeepyjack sleeepyjack commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Closes #834
Closes #837

  • make hash-to-index reduction defined across the supported hash and extent ranges
  • avoid signed and unsigned overflow when advancing probing iterators
  • reject capacities that cannot be represented after stride or prime rounding
  • add regression coverage for scalar and cooperative-group linear/double probing and signed/unsigned extent boundaries

@sleeepyjack

Copy link
Copy Markdown
Collaborator Author

Benchmark results

Environment:

  • GPU: NVIDIA RTX PRO 6000 Blackwell Max-Q, sm_120
  • CUDA: 13.1.115
  • Host compiler: GCC 14.3
  • Build type: Release
  • Input: 20 million int32_t keys
  • Baseline: upstream/dev at 4b26118c
  • Candidate: e6894e31

The main performance change is the probe iterator advancement. Previously, each collision advanced
the iterator using:

(current + step) % upper_bound

For dynamic extents, this invokes the fast_int division/modulo path on every additional probe.
The new implementation relies on the iterator invariants (current < upper_bound and
step <= upper_bound) and performs at most one wrap using a comparison and add/subtract.

As a result, the improvement scales with probe-chain length:

  • Higher occupancies produce more collisions and therefore larger improvements.
  • Non-matching lookups traverse longer probe sequences than successful lookups and benefit most.
  • Linear probing shows the largest gains at high occupancy because clustering creates long chains.
  • Insert gains are smaller because successful inserts often terminate earlier and include atomic
    insertion costs that are unaffected by this change.
  • Capacity validation runs on the host during construction and does not affect timed kernels.

Across the focused matrix:

Configuration group Cases Faster Unchanged Slower
Default extent, double hashing 15 14 1 0
extent<int32_t>, double hashing 21 21 0 0
extent<int32_t>, linear probing 12 9 3 0
Total 48 44 4 0

Default extent: contains by matching rate

Occupancy is fixed at 0.8.

Key Distribution NumInputs Occupancy MatchingRate Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
I32 UNIQUE 20000000 0.8 0 1.377 ms 0.42% 1.235 ms 0.32% -141.887 us -10.30% FAST
I32 UNIQUE 20000000 0.8 0.5 1.140 ms 0.38% 1.027 ms 0.29% -113.240 us -9.93% FAST
I32 UNIQUE 20000000 0.8 1 826.834 us 0.48% 753.128 us 0.44% -73.706 us -8.91% FAST

Default extent: find by matching rate

Occupancy is fixed at 0.8.

Key Distribution NumInputs Occupancy MatchingRate Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
I32 UNIQUE 20000000 0.8 0 1.442 ms 0.50% 1.275 ms 0.40% -166.867 us -11.57% FAST
I32 UNIQUE 20000000 0.8 0.5 1.261 ms 0.35% 1.137 ms 0.45% -124.080 us -9.84% FAST
I32 UNIQUE 20000000 0.8 1 963.968 us 0.36% 890.785 us 0.34% -73.183 us -7.59% FAST

Default extent: insert by occupancy

Key Distribution NumInputs Occupancy Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
I32 UNIQUE 20000000 0.9 1.577 ms 0.33% 1.486 ms 0.42% -90.445 us -5.74% FAST
I32 UNIQUE 20000000 0.8 1.536 ms 1.48% 1.481 ms 1.12% -54.285 us -3.53% FAST
I32 UNIQUE 20000000 0.5 2.829 ms 0.06% 2.828 ms 0.06% -1.494 us -0.05% SAME

Signed extent, double hashing: contains

NumInputs Occupancy MatchingRate Ref Time Ref Noise Cmp Time Cmp Noise Diff %Diff Status
20000000 0.9 0.5 1.502 ms 0.41% 1.406 ms 0.25% -96.631 us -6.43% FAST
20000000 0.8 0.5 1.063 ms 0.54% 994.997 us 0.29% -67.614 us -6.36% FAST
20000000 0.9 0 1.966 ms 0.40% 1.849 ms 0.19% -117.584 us -5.98% FAST
20000000 0.9 1 858.481 us 0.44% 809.869 us 0.37% -48.612 us -5.66% FAST
20000000 0.8 0 1.269 ms 0.44% 1.199 ms 0.29% -70.282 us -5.54% FAST
20000000 0.8 1 774.082 us 0.50% 732.542 us 0.50% -41.540 us -5.37% FAST
20000000 0.5 0 1.017 ms 2.13% 987.446 us 1.46% -29.280 us -2.88% FAST
20000000 0.5 0.5 954.535 us 0.25% 928.963 us 0.39% -25.571 us -2.68% FAST
20000000 0.5 1 863.941 us 0.19% 851.916 us 0.22% -12.025 us -1.39% FAST

Signed extent, linear probing: representative high-occupancy cases

Operation Occupancy MatchingRate Ref Time Cmp Time %Diff
Contains 0.9 0.5 3.985 ms 3.495 ms -12.28%
Contains 0.9 0 5.084 ms 4.469 ms -12.10%
Contains 0.8 0 1.366 ms 1.211 ms -11.35%
Contains 0.8 0.5 1.119 ms 993.131 us -11.23%
Insert 0.9 - 1.526 ms 1.450 ms -4.97%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant