Skip to content

[seekdb][PALF] Remove LogGroupBuffer copy from transaction log flush - #1253

Open
hnwyllmm wants to merge 1 commit into
masterfrom
task/2026073100117914692
Open

[seekdb][PALF] Remove LogGroupBuffer copy from transaction log flush#1253
hnwyllmm wants to merge 1 commit into
masterfrom
task/2026073100117914692

Conversation

@hnwyllmm

@hnwyllmm hnwyllmm commented Aug 4, 2026

Copy link
Copy Markdown
Member

Task Description

Transaction logs are serialized into ObTxLogBlock, copied into the shared 4 MiB LogGroupBuffer, and copied again into LogDIOAlignedBuf before pwrite. The first large copy adds memory-bandwidth cost and couples pending-write ownership, group assembly, flow control, and Hot Cache.

Solution Description

  • Add a transferable PalfLogBuffer with a reserved LogEntryHeader prefix so transaction buffers can move directly into PALF.
  • Store immutable, LSN-ordered segments in LogTask and calculate group checksum from each entry checksum without rescanning payload.
  • Transfer frozen groups to IO tasks, restore ownership when enqueue fails, and release payload only after synchronous pwrite succeeds.
  • Gather fragmented groups and virtual padding into LogDIOAlignedBuf so a continuous same-block batch uses one aligned pwrite.
  • Add a 4 MiB pending-memory limit with ref-counted reservation state, and temporarily disable Hot Cache so reads fall back to disk.
  • Keep the legacy pointer append API compatible by copying once into an owned buffer.

Passed Regressions

  • ob-make test_palf_log_buffer
  • test_palf_log_buffer, test_log_entry_and_group_entry, test_lsn_allocator, and test_fixed_sliding_window: 4/4 passed
  • ob-make seekdb; module-layer DAG check reported no upward edges
  • Isolated runtime smoke: committed a 2.4 MB transaction, restarted seekdb, and recovered all 12 rows and 2.4 MB payload

Upgrade Compatibility

No on-disk log format change. The legacy append API remains compatible. Hot Cache is disabled in this version, which may affect cache-hit performance but does not change read semantics because reads fall back to disk.

Other Information

This MR contains one commit, 5f93727510d, with no unrelated commits or untracked files.

Release Note

@hnwyllmm

hnwyllmm commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

The mapping Dima issue is about optimizing LogGroupBuffer and LogDIOAligned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant