Skip to content

[spark] Make DeletionVectorTest pass with write.use-v2-write enabled#8614

Merged
Zouxxyy merged 1 commit into
apache:masterfrom
kerwin-zk:fix-deletion-vector-test-v2-write
Jul 14, 2026
Merged

[spark] Make DeletionVectorTest pass with write.use-v2-write enabled#8614
Zouxxyy merged 1 commit into
apache:masterfrom
kerwin-zk:fix-deletion-vector-test-v2-write

Conversation

@kerwin-zk

Copy link
Copy Markdown
Contributor

Purpose

DeletionVectorTest fails when the suite is run with spark.paimon.write.use-v2-write=true.

Its helper runAndCheckSplitScan asserts that a DELETE/UPDATE/MERGE on a deletion-vector append table reads through a PaimonSplitScan (the V1 deletion-vector copy-on-write path). However, since #8539 added V2 delta-based row-level operations, an unaware-bucket deletion-vector append table with V2 write enabled is routed to PaimonSparkDeltaOperation / WriteDelta, whose scan side is a regular pushdown scan (see PaimonSparkDeltaOperation.newScanBuilder) and therefore produces no PaimonSplitScan. The assert(batchScans.nonEmpty) then fails:

- Paimon DeletionVector: delete for append non-partitioned table with bucket = -1 *** FAILED ***
  ArrayBuffer() was empty DELETE FROM T WHERE id = 2 (DeletionVectorTest.scala:85)

With the default (write.use-v2-write=false) the suite still exercises the V1 path, so the failure is only visible when V2 write is enabled. The delta path itself is functionally correct and covered by DeltaRowLevelOpsTest.

Tests

runAndCheckSplitScan now detects the V2 delta plan (WriteDelta, via EXPLAIN EXTENDED) and, in that case, just executes the statement instead of asserting the V1 split scan. Behavior is unchanged for the V1 path (default config and bucketed deletion-vector append tables), and the suite now passes under both write.use-v2-write=false and true.

API and Format

No public API or format change.

Documentation

No documentation change.

@kerwin-zk kerwin-zk force-pushed the fix-deletion-vector-test-v2-write branch from 2ddf56a to 174f9d8 Compare July 14, 2026 06:12

@Zouxxyy Zouxxyy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@Zouxxyy Zouxxyy merged commit e425fb4 into apache:master Jul 14, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants