Skip to content

server, session: fix load data slow log recording#69552

Open
Meowooh wants to merge 6 commits into
pingcap:masterfrom
Meowooh:fix-issue-32034
Open

server, session: fix load data slow log recording#69552
Meowooh wants to merge 6 commits into
pingcap:masterfrom
Meowooh:fix-issue-32034

Conversation

@Meowooh

@Meowooh Meowooh commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #32034

Problem Summary:

RunTestLoadDataForSlowLog was skipped because transactional LOAD DATA LOCAL still kept LoadDataVarKey in the session file-transfer bypass list. That made runStmt skip normal statement finalization, while the connection no longer handled LOAD DATA through handleFileTransInConn, so the slow log and statement summary were not recorded.

What changed and how does it work?

Remove LoadDataVarKey from fileTransInConnKeys so transactional LOAD DATA LOCAL finishes through the normal statement path and records slow log / statement summary data.

Re-enable RunTestLoadDataForSlowLog with an isolated slow log file and update the LoadData plan assertion to match the current runtime fields.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Commands run:

go test -run '^TestLoadDataInitParam$' -count=1 -tags=intest,deadlock ./pkg/executor/test/loaddatatest
go test -run '^TestIssue34358$' -count=1 -tags=intest,deadlock ./pkg/executor/test/writetest
go test -count=1 -tags=intest,deadlock ./pkg/executor/test/loaddatatest ./pkg/executor/test/writetest
./tools/check/failpoint-go-test.sh pkg/server/tests -run '^TestLoadData1$' -count=1
./tools/check/failpoint-go-test.sh pkg/server/tests -count=1
./tools/check/failpoint-go-test.sh pkg/session -count=1
make lint

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

Summary by CodeRabbit

  • New Features

    • Improved support for loading session-related data from local files during statement execution.
  • Bug Fixes

    • Fixed file-based operations so they are handled consistently and complete more reliably.
    • Updated slow-query logging behavior in tests to better reflect real runtime settings and expected output.

@ti-chi-bot ti-chi-bot Bot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. contribution This PR is from a community contributor. labels Jun 30, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

Hi @Meowooh. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@ti-chi-bot ti-chi-bot Bot added the needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. label Jun 30, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jun 30, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign maxshuang for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds a FileTransInConnHandlers registry in the executor package mapping session keys to file-transfer handler functions, refactors handleFileTransInConn in conn.go and hasFileTransInConn in session.go to use this shared registry, and restores the previously skipped slow-query-log load-data test with temporary logging configuration.

Changes

Slow log test re-enablement and file-transfer dispatch cleanup

Layer / File(s) Summary
Centralize file-transfer handlers
pkg/executor/plan_replayer.go, pkg/server/conn.go, pkg/session/session.go
Adds an exported FileTransInConnHandler type and FileTransInConnHandlers registry mapping LoadStatsVarKey/PlanReplayerLoadVarKey to handler functions; refactors handleFileTransInConn to iterate the registry, check capability, invoke handlers, and clear context immediately; removes old inline handler methods; updates hasFileTransInConn to iterate the registry keys instead of a local slice.
Restore slow-log load-data test
pkg/server/internal/testserverclient/server_client.go, pkg/server/internal/testserverclient/BUILD.bazel
Removes the t.Skip in RunTestLoadDataForSlowLog, sets up a temporary slow-query log file with cloned global config and logutil.InitLogger, sets @@tidb_slow_query_file, narrows the expected plan regex, and adds //pkg/config and //pkg/util/logutil Bazel deps.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Session
  participant Conn as ClientConn
  participant Executor as executor.FileTransInConnHandlers
  Session->>Conn: hasFileTransInConn()
  Conn->>Executor: iterate handler keys
  Note over Conn,Executor: handleFileTransInConn()
  Conn->>Executor: for each key, check cc.ctx.Value(key)
  Executor-->>Conn: handler(ctx, value, getDataFromPath)
  Conn->>Conn: clear cc.ctx key, return on error
Loading

Possibly related issues

Suggested labels: approved, lgtm

Suggested reviewers: bb7133, Defined2014

Poem

A rabbit hops through logs anew,
Temp files scribbled, config too,
Handlers gathered in one neat lair,
LoadData's plan trimmed with care,
No more skips — the test runs true! 🐇✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly names the server/session LOAD DATA slow log fix and matches the main change set.
Description check ✅ Passed The description includes the required issue number, problem summary, change summary, tests, and checklist sections.
Linked Issues check ✅ Passed The changes re-enable RunTestLoadDataForSlowLog and fix the LOAD DATA slow-log path, matching issue #32034.
Out of Scope Changes check ✅ Passed The server, session, and executor edits all support the LOAD DATA file-transfer and slow-log fix; no unrelated changes stand out.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/server/internal/testserverclient/server_client.go`:
- Around line 445-449: The deferred cleanup in server_client.go restores only
config, but the process-global logger set by logutil.InitLogger still points at
the temp slow-log sink when os.Remove deletes it. Update the same defer in the
test setup to restore the original logger state before removing
newCfg.Log.SlowQueryFile, using the existing logger/config setup around
logutil.InitLogger and config.StoreGlobalConfig so later tests do not keep
writing to a deleted file.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 083d385e-3005-4566-aae8-719e5b6712fb

📥 Commits

Reviewing files that changed from the base of the PR and between 693e52c and 2ecd410.

📒 Files selected for processing (2)
  • pkg/server/internal/testserverclient/server_client.go
  • pkg/session/session.go
💤 Files with no reviewable changes (1)
  • pkg/session/session.go

Comment on lines +445 to +449
defer func() {
config.StoreGlobalConfig(originCfg)
require.NoError(t, os.Remove(newCfg.Log.SlowQueryFile))
}()
require.NoError(t, logutil.InitLogger(newCfg.Log.ToLogConfig()))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Restore the global logger before deleting the temp slow log.

logutil.InitLogger changes process-global logging state, but the defer only restores config; later tests can keep using the temp slow-log sink after Line 447 removes it. Reinitialize the original logger in the same defer, before removing the file.

Suggested fix
 	defer func() {
 		config.StoreGlobalConfig(originCfg)
+		require.NoError(t, logutil.InitLogger(originCfg.Log.ToLogConfig()))
 		require.NoError(t, os.Remove(newCfg.Log.SlowQueryFile))
 	}()

As per coding guidelines, Go error handling should be actionable and contextual; this keeps the global test process state consistent after the test.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
defer func() {
config.StoreGlobalConfig(originCfg)
require.NoError(t, os.Remove(newCfg.Log.SlowQueryFile))
}()
require.NoError(t, logutil.InitLogger(newCfg.Log.ToLogConfig()))
defer func() {
config.StoreGlobalConfig(originCfg)
require.NoError(t, logutil.InitLogger(originCfg.Log.ToLogConfig()))
require.NoError(t, os.Remove(newCfg.Log.SlowQueryFile))
}()
require.NoError(t, logutil.InitLogger(newCfg.Log.ToLogConfig()))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/server/internal/testserverclient/server_client.go` around lines 445 -
449, The deferred cleanup in server_client.go restores only config, but the
process-global logger set by logutil.InitLogger still points at the temp
slow-log sink when os.Remove deletes it. Update the same defer in the test setup
to restore the original logger state before removing newCfg.Log.SlowQueryFile,
using the existing logger/config setup around logutil.InitLogger and
config.StoreGlobalConfig so later tests do not keep writing to a deleted file.

Source: Coding guidelines

@lance6716

Copy link
Copy Markdown
Contributor

/ok-to-test

@ti-chi-bot ti-chi-bot Bot added ok-to-test Indicates a PR is ready to be tested. and removed needs-ok-to-test Indicates a PR created by contributors and need ORG member send '/ok-to-test' to start testing. labels Jul 1, 2026
@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 47.05882% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.2872%. Comparing base (1174be2) to head (8a2f6df).
⚠️ Report is 20 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #69552        +/-   ##
================================================
- Coverage   76.3270%   74.2872%   -2.0399%     
================================================
  Files          2041       2050         +9     
  Lines        561007     581598     +20591     
================================================
+ Hits         428200     432053      +3853     
- Misses       131906     148634     +16728     
- Partials        901        911        +10     
Flag Coverage Δ
integration 40.8760% <47.0588%> (+1.2480%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 60.4471% <ø> (ø)
parser ∅ <ø> (∅)
br 47.2083% <ø> (-15.5428%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Meowooh

Meowooh commented Jul 1, 2026

Copy link
Copy Markdown
Contributor Author

/retest

Comment thread pkg/session/session.go
@ti-chi-bot ti-chi-bot Bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 3, 2026
Comment thread pkg/executor/file_trans_in_conn.go Outdated
Comment thread pkg/server/internal/testserverclient/server_client.go Outdated
Comment thread pkg/server/conn.go Outdated
Comment thread pkg/server/conn.go
@Meowooh

Meowooh commented Jul 6, 2026

Copy link
Copy Markdown
Contributor Author

/retest

Comment thread pkg/server/conn.go
Comment thread pkg/server/conn.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. ok-to-test Indicates a PR is ready to be tested. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UT get back runTestLoadDataForSlowLog

2 participants