Skip to content

[seekdb][tx] Reduce tx data memtable initial hash buckets - #1206

Open
hnwyllmm wants to merge 2 commits into
masterfrom
task/2026072700117769110
Open

[seekdb][tx] Reduce tx data memtable initial hash buckets#1206
hnwyllmm wants to merge 2 commits into
masterfrom
task/2026072700117769110

Conversation

@hnwyllmm

Copy link
Copy Markdown
Member

Task Description

The tx data memtable hash map starts with 65,536 buckets. Each bucket header is 16 bytes, so an active memtable reserves 1 MiB for the bucket array even when the transaction count is low.

Solution Description

Reduce ObTxDataHashMap::MIN_BUCKETS_CNT from 65,536 to 16,384. This lowers the initial bucket array from 1 MiB to 256 KiB while retaining the existing adaptive expansion and shrink logic.

Passed Regressions

  • Built both the 65,536 baseline and 16,384 patched release binaries successfully.
  • Ran four 300-second ABBA sysbench oltp_update_non_index rounds on a 16-vCPU host, with 8 CPUs assigned to seekdb and 8 CPUs assigned to sysbench.
  • Weighted throughput: 13,410.67 TPS for 65,536 buckets and 13,448.07 TPS for 16,384 buckets (+0.279%, within noise); all rounds completed with zero errors.
  • The adjacent idle PSS comparison saved 718 KiB, close to the expected deterministic saving of 768 KiB per active hash map.
  • git diff --check and git diff --cached --check passed.

Upgrade Compatibility

Compatible. This only changes the initial in-memory bucket allocation and does not modify persisted data, file formats, SQL behavior, or public APIs.

Other Information

This MR contains one commit: 6da09f27dbe reduce tx data memtable initial bucket count. The unrelated untracked file 2 is not included.

Release Note

@hnwyllmm

Copy link
Copy Markdown
Member Author

The mapping Dima issue is about adjusting the memtable_arena parameter.

@hnwyllmm

Copy link
Copy Markdown
Member Author

Local idle-memory A/B result

I re-ran the idle-memory comparison on the current host because the previous cloud host is no longer available.

Method

  • Built the 65,536-bucket baseline and 16,384-bucket patched release binaries from commit 6da09f27dbe; the source difference is only MIN_BUCKETS_CNT and its comment.
  • Reused the same existing base directory, port, and 2 GiB configuration.
  • Waited 180 seconds without client workload and sampled /proc in four ABBA rounds.
  • Because process-level RSS/PSS had multi-MiB background allocation noise, used dump chunk all to isolate the affected MEMTABLE_ARENA allocations.

Allocator dump

  • 65,536 buckets: 1,048,608 + 1,048,608 + 7,936 = 2,105,152 B
  • 16,384 buckets: 1,048,608 + 262,176 + 7,936 = 1,318,720 B
  • Observed idle saving: 786,432 B = 768 KiB = 0.75 MiB

The changed object is the newly created ACTIVE tx-data memtable bucket array. The MINI memtable restored from the same base directory and the 7,936-byte arena object remained unchanged, so the entire 768 KiB delta is directly attributable to this change. This also agrees with the earlier adjacent idle-PSS observation of 718 KiB.

Binary SHA256

  • 65,536: a631c4a472c435cd7bc4e99ad672355cf976f257502a691d681fa8d4aba8a726
  • 16,384: 442dcb42fbf9856697fa96e5f43bae6897ae21f6db193088ac2c77cfd239d4ba

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant