Skip to content

[spark] Support AlwaysTrue and AlwaysFalse predicates in SparkV2FilterConverter#7224

Open
kerwin-zk wants to merge 1 commit intoapache:masterfrom
kerwin-zk:always-true-false
Open

[spark] Support AlwaysTrue and AlwaysFalse predicates in SparkV2FilterConverter#7224
kerwin-zk wants to merge 1 commit intoapache:masterfrom
kerwin-zk:always-true-false

Conversation

@kerwin-zk
Copy link
Contributor

Purpose

Support AlwaysTrue and AlwaysFalse predicates in SparkV2FilterConverter

Tests

CI

@kerwin-zk kerwin-zk force-pushed the always-true-false branch 5 times, most recently from 0574ad6 to 101b5e4 Compare February 6, 2026 06:42
@kerwin-zk
Copy link
Contributor Author

cc @JingsongLi

@JingsongLi
Copy link
Contributor

Why do we need to support this? Are there any beneficial queries?

@JingsongLi
Copy link
Contributor

If we really need to introduce this, I prefer introducing TrueTransform and TrueFunction.

@kerwin-zk
Copy link
Contributor Author

kerwin-zk commented Feb 9, 2026

Why do we need to support this? Are there any beneficial queries?

@JingsongLi

DELETE FROM t1 WHERE pt IN (SELECT id FROM t2 WHERE n > 10) OR pt = 3
The subquery (n > 10) returns empty, evalSubquery replaces it with Literal(false). After translateFilterV2, this becomes: V2Or(ALWAYS_FALSE, V2EqualTo(pt, 3)). Without supporting ALWAYS_FALSE in SparkV2FilterConverter, the OR conversion would fail, falling back to the expensive row-level delete instead of partition dropping.

@kerwin-zk kerwin-zk force-pushed the always-true-false branch 2 times, most recently from 9a3e976 to 85ff14d Compare February 11, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants