[BUG] Fix null handling in mixed semi joins

[wjxiz1992]

Description

Closes #23860.

mixed_join_semi builds separate right-side comparators for the equality and conditional tables. The conditional comparator was using the null-state derived from the equality table, so nullable conditional values could be compared as non-null whenever the equality keys contained no nulls. This made right-side representative selection order-dependent and could produce an incorrect left-semi or left-anti gather map.

This change:

• derives the conditional comparator's null-state from right_conditional;
• uses null-equal semantics for conditional-row deduplication, independently of the equality-key compare_nulls setting, so the hash-set comparator remains reflexive and duplicate null rows do not form a long linear-probing cluster;
• expands regression coverage across left-semi and left-anti joins, EQUAL and UNEQUAL, null-first and null-last row order, and predicates that both reject and require null representatives.

The added has_nulls call is a host-side metadata scan over the conditional columns; it does not add a device pass.

Related downstream report: NVIDIA/cudf-spark#14076.

Validation performed locally on an NVIDIA RTX 5880 Ada Generation GPU:

• pre-commit run --files cpp/src/join/mixed_join_semi.cu cpp/tests/join/mixed_join_tests.cu: all applicable hooks passed
• Expanded regression matrix: 1/1 passed
• Expanded regression matrix repeated 100 times: 100/100 passed
• All mixed left-semi/anti tests: 113/113 passed
• Complete JOIN_TEST: 810 passed, 2 skipped
• Focused compute-sanitizer --tool memcheck: 0 errors
• 30,000 duplicate-null right rows sharing one equality key, one left row, 10 iterations: EQUAL 31-33 ms and UNEQUAL 30-31 ms after the fix. Before the follow-up deduplication fix, UNEQUAL took 829-831 ms for the same input.

The same 30-row harness was also run on GH200 against the exact cuDF revision from the downstream failing build (a6bd95e24bebc35c5ea92283410df4f48bf3be89): the original implementation failed all 1,000 null-first iterations, while the patched object failed 0/1,000.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[res-life]

LGTM

[wjxiz1992]

/ok to test 2527072

[Copilot]

Pull request overview

Fixes incorrect null handling in mixed left-semi/left-anti joins by ensuring right-side conditional-row comparison uses the null-state of right_conditional (not the equality table), and by making conditional-row deduplication use null-equal semantics so the hash-set comparator remains reflexive and avoids pathological duplicate-null clustering.

Changes:

• Derive the conditional comparator’s null-state from right_conditional and force null_equality::EQUAL for conditional-row deduplication.
• Add regression coverage for nullable conditional columns with non-nullable equality keys across semi/anti joins, EQUAL/UNEQUAL null-equality, and null-first/null-last ordering.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
cpp/src/join/mixed_join_semi.cu	Fixes conditional comparator null handling and enforces reflexive null-equal semantics for conditional-row deduplication.
cpp/tests/join/mixed_join_tests.cu	Adds a targeted regression test matrix covering the reported null-order-dependent mixed semi/anti join failures.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 19a14cfc-cf8b-4a08-9124-efd5af4f8e8f

📥 Commits

Reviewing files that changed from the base of the PR and between f588e0c and 2527072.

📒 Files selected for processing (2)

• cpp/src/join/mixed_join_semi.cu
• cpp/tests/join/mixed_join_tests.cu

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes

  • Corrected mixed semi and anti joins to consistently match null values in nullable conditional columns.
  • Join results now remain correct regardless of null ordering or null-equality configuration.

• Tests

  • Added coverage for non-nullable equality keys combined with nullable conditional columns.
  • Verified NOT_EQUAL and IS_NULL filtering scenarios across mixed join types.

Walkthrough

The mixed join comparator now derives null handling from right_conditional and treats conditional nulls as equal during deduplication. Tests cover nullable conditional values with non-null equality keys across layouts, null modes, and semi or anti joins.

Changes

Mixed join null handling

Layer / File(s)	Summary
Conditional-row comparator correction cpp/src/join/mixed_join_semi.cu	The right conditional-row comparator uses its own null mask and null_equality::EQUAL during deduplication.
Mixed semi and anti join regression coverage cpp/tests/join/mixed_join_tests.cu	The test covers NOT_EQUAL and IS_NULL predicates, both conditional null layouts, and both null-equality modes with non-null equality keys.

Estimated code review effort: 3 (Moderate) | ~15–30 minutes

Merge Risk: ⚪ Minimal · up to 25270

This localized change corrects nullable conditional-value handling and duplicate-null deduplication in mixed semi joins, with broad regression coverage and sanitizer validation reported; no actionable merge-blocking risk remains.

Suggested reviewers: wence-, misiugodfrey, bdice, pointkernel

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary change: fixing null handling in mixed semi joins.
Description check	✅ Passed	The description directly explains the null-handling bug, the implementation changes, regression coverage, and validation results.
Linked Issues check	✅ Passed	The changes satisfy issue [#23860] by deriving conditional null state from right_conditional, using null-equal deduplication semantics, and testing nullable conditional values with non-null equality k…
Out of Scope Changes check	✅ Passed	The source change and regression tests are directly related to the linked issue and PR objectives. No unrelated code changes are identified.

Full details: Linked Issues check

Explanation

The changes satisfy issue [#23860] by deriving conditional null state from right_conditional, using null-equal deduplication semantics, and testing nullable conditional values with non-null equality keys in both null orderings.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[wjxiz1992]

/merge