Skip to content

[core] Improve row id reassign retry reuse#8506

Draft
leaves12138 wants to merge 9 commits into
apache:masterfrom
leaves12138:codex/reassign-row-id-retry-cache
Draft

[core] Improve row id reassign retry reuse#8506
leaves12138 wants to merge 9 commits into
apache:masterfrom
leaves12138:codex/reassign-row-id-retry-cache

Conversation

@leaves12138

Copy link
Copy Markdown
Contributor

Purpose

DataEvolutionRowIdReassigner can lose a commit race when a normal append commits a newer snapshot between planning and replaceManifestList. Previously the next reassignment invocation had to start the manifest read work from scratch.

Changes

  • Retry replaceManifestList conflicts up to three times in the same reassignment call.
  • Reuse already deserialized data manifest and index manifest entries across retry attempts.
  • Recompute assignment from the latest snapshot and latest nextRowId on every retry to avoid row-id range overlap with concurrent appends.
  • Add a deterministic conflict test that appends before the first reassign commit and verifies the retry uses the latest nextRowId.

Tests

  • mvn -pl paimon-core spotless:apply
  • mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -DwildcardSuites=none -Dtest=DataEvolutionRowIdReassignerTest#testReassignRetriesWithLatestNextRowIdAfterConcurrentAppend test
  • mvn -pl paimon-core -am -DfailIfNoTests=false -DwildcardSuites=none -Dtest=DataEvolutionRowIdReassignerTest test
  • git diff --check

@leaves12138 leaves12138 marked this pull request as ready for review July 9, 2026 09:09
@leaves12138 leaves12138 force-pushed the codex/reassign-row-id-retry-cache branch from 9408a9b to a999b5a Compare July 9, 2026 10:15
@leaves12138 leaves12138 marked this pull request as draft July 10, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant