[spark] compact_chain_table should always use dynamic partition overwrite

[juntaozhang]

Purpose

When compact_chain_table is called with overwrite = true on a chain table such as:

CREATE TABLE chain_t (
    `t1` BIGINT COMMENT 't1',
    `t2` BIGINT COMMENT 't2',
    `t3` STRING COMMENT 't3'
  ) PARTITIONED BY (`region` STRING COMMENT 'region', `dt` STRING COMMENT 'dt', `hour` STRING COMMENT 'hour')
TBLPROPERTIES (
     'dynamic-partition-overwrite' = 'true',
     'chain-table.enabled' = 'true',
     'primary-key' = 'region,dt,hour,t1',
     'sequence.field' = 't2',
     'bucket-key' = 't1',
     'bucket' = '1',
     'partition.timestamp-pattern' = '$dt $hour:00:00',
     'partition.timestamp-formatter' = 'yyyyMMdd HH:mm:ss',
     'merge-engine' = 'deduplicate',
     'chain-table.chain-partition-keys' = 'dt,hour'
  )

Users typically compact a chain partition by specifying only the chain dimension:

CALL sys.compact_chain_table('default.chain_t', 'dt=20250810,hr=22', true);

Assume the snapshot branch already contains data for:

  region='FR', dt='20250810', hr='22'
  region='CN', dt='20250810', hr='22'

But the delta branch only contains new data for:

region='CN', dt='20250810', hr='22'

If the table is configured with 'dynamic-partition-overwrite' = 'false', the overwrite commit deletes all full partitions matching dt=20250810, hr=22 in the snapshot branch, including region='FR' which has no corresponding delta data. As a result, the existing snapshot data for region='FR' is lost.

Tests

• Extended CompactChainTableProcedureTest with a group-partition test that sets 'dynamic-partition-overwrite' = 'false' and verifies the snapshot-only group is preserved.

[JingsongLi]

+1