Skip to content

Compress only packets of 512 bytes or more - #1834

Merged
methane merged 1 commit into
go-sql-driver:masterfrom
lohmag:compress-min-512
Oct 27, 2026
Merged

methane merged 1 commit into
go-sql-driver:masterfrom
lohmag:compress-min-512

Conversation

@lohmag

@lohmag lohmag commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Description

The threshold change from #1833, without the klauspost dependency: packets shorter than 512 bytes are sent uncompressed (was 150). It only affects what the driver sends (queries, statement parameters, LOAD DATA), since results are compressed by the server.

The threshold mainly helps Go 1.25 and 1.26. There compress/zlib at level 2 costs at least about 5.6 µs per packet, even for one byte, because the compressor clears its hash tables on the first write after Reset, so every packet of 150 to 511 bytes costs 10 to 14 µs, and one of up to 256 bytes saves at most 68 bytes for it. Go 1.27 rewrote compress/flate (golang/go#75532): the fixed cost is gone, short packets that do not shrink are stored for about 0.2 µs (and the driver then sends them uncompressed), and the rest compresses 2 to 3 times faster. On Go 1.27 the change gives up some savings instead: SQL text below about 350 bytes is stored anyway, but from 384 to 511 bytes it shrinks by about a quarter for 5 to 6 µs, and JSON-like data shrinks well from 150 bytes up (a 384 byte packet to 53% for 4.8 µs).

Per packet, compress/zlib level 2 through a reused writer as in zCompress, AMD Ryzen 9 7950X3D, one pinned core. Compressed size is the zlib output relative to the input, above 100% means the driver sends the packet uncompressed.

SQL text (SELECT ... WHERE uuid = '...', UPDATE ..., multi-row INSERT ... VALUES):

Size Go 1.25.14 compress Go 1.25.14 compressed size Go 1.27.2 compress Go 1.27.2 compressed size
1 B 5.63 µs 0.07 µs
150 B 10.4 µs 96.7% 0.22 µs 108.7%
200 B 11.2 µs 91.0% 0.25 µs 106.5%
256 B 11.6 µs 85.2% 0.29 µs 105.1%
384 B 13.2 µs 73.4% 5.3 µs 77.9%
512 B 14.0 µs 64.6% 6.1 µs 68.9%
1 KiB 17.1 µs 54.1% 8.2 µs 57.0%

JSON-like parameter data:

Size Go 1.25.14 compress Go 1.25.14 compressed size Go 1.27.2 compress Go 1.27.2 compressed size
150 B 9.6 µs 84.7% 3.5 µs 81.3%
200 B 10.2 µs 76.0% 4.1 µs 74.5%
256 B 10.6 µs 70.7% 4.3 µs 69.5%
384 B 11.3 µs 53.4% 4.8 µs 53.4%
512 B 12.1 µs 45.3% 5.1 µs 45.9%
1 KiB 13.4 µs 31.9% 6.2 µs 31.9%

go.mod still allows Go 1.25, so most users of compress=true today pay the Go 1.25 and 1.26 cost. If you would rather tune for Go 1.27, a lower value such as 384 keeps the JSON savings there and still drops the most expensive packets on older Go.

Checklist

  • Code compiles correctly
  • Created tests which fail without the change (if possible): the threshold is a tuning value, so TestCompressThreshold pins the boundary instead: a packet one byte shorter than minCompressLength is sent uncompressed and one of minCompressLength bytes is compressed.
  • All tests passing: go test -race ./..., and the full suite against MySQL 8.0.46 including every compress variant (581 passed, TestConcurrent skipped without MYSQL_TEST_CONCURRENT).
  • Extended the README / documentation, if necessary: nothing in the README mentions the threshold.
  • Added myself / the copyright holder to the AUTHORS file

@coderabbitai

coderabbitai Bot commented Oct 26, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 89d733b9-ac75-4269-8c61-69318e187b64
Commits

Reviewing files that changed from the base of the PR and between 4b12a5b and 2caacfc.

Files selected for processing: 3
  • AUTHORS
  • compress.go
  • compress_test.go

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The minimum packet length for attempting compression increases from 150 to 512 bytes. A test checks packets below and at the threshold. The individual contributors list gains Nikita Massalitin.

Changes

Compression threshold

Layer / File(s) Summary
Threshold behavior
compress.go, compress_test.go
The minimum length for attempting compression increases to 512 bytes. A new test checks packets one byte below and exactly at the threshold. Existing fallback behavior remains unchanged.

Contributor list

Layer / File(s) Summary
Contributor listing
AUTHORS
Nikita Massalitin is added to the individual contributors list.

Priority: Low

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Other

Possibly related PRs

Suggested reviewers: joe-mann

Merge Risk: Minimal · up to 2caac

No actionable merge-blocking risk remains in the reviewed changes.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: increasing compression to apply only to packets of 512 bytes or more.
Description check Passed The description directly explains the threshold change, its performance rationale, affected packet types, testing, and validation results.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 2 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coveralls

Copy link
Copy Markdown

Coverage Status

coverage: 85.771% (+0.07%) from 85.705% — lohmag:compress-min-512 into go-sql-driver:master

@methane
methane merged commit aca61c8 into go-sql-driver:master Oct 27, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants