Feature Support large MySQL and MariaDb databases

[pcrosthwaite]

Problem
Large MySQL database backups were failing with Error 2013: Lost connection to server during query mid-dump. This occurs because MySQL's server-side net_read_timeout and net_write_timeout (defaulting to 30s and 60s respectively) kill the connection while mysqldump is still reading rows from a large table.

The root cause was compounded by the Docker image installing mariadb-client, which symlinks mysqldump → mariadb-dump. MariaDB's mysqldump has no mechanism to set these timeouts from the client side — --network-timeout and equivalent options simply don't exist in that binary.

Changes
docker/Dockerfile
Added a mysql:8.4 build stage to extract the real MySQL mysqldump binary and its shared library (libmysqlclient.so.21). The binary is placed at /usr/local/bin/mysqldump, which takes PATH precedence over /usr/bin/mysqldump (MariaDB's symlink). The mariadb-dump binary is unaffected and continues to be used for MariaDB backups.

src/domain/mysql/backup.rs
Added --network-timeout to the mysqldump command. This MySQL 8.0.1+ flag automatically sets net_read_timeout and net_write_timeout to 86400 seconds (24 hours) for the duration of the dump session. Also logs the mysqldump client version as a debug message before each backup to make it easy to verify the correct binary is being used.

src/domain/mariadb/backup.rs
Logs the mariadb-dump client version as a debug message before each backup, consistent with the MySQL change.

src/services/config.rs
Added an optional max_packet_size field to DatabaseConfig (defaults to 512M). This allows per-database tuning of --max-allowed-packet for databases with large row values. Example config:

{
"name": "my-database",
"type": "mysql",
"max_packet_size": "1G"
}

Summary by CodeRabbit

Release Notes

• New Features

  • Added max_packet_size support for database backups (defaults to 512M).
  • Improved PostgreSQL dump tool discovery with an optional PG_BIN_DIR override.

• Bug Fixes

  • Enhanced MySQL/MariaDB backups with debug logging of the detected dump client version.

• Chores

  • Updated Docker images to include mysqldump tools and matching libraries.
  • Added Windows release workflow to build and publish release artifacts automatically.

• Other

  • Reduced PostgreSQL runtime environment logging.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0aa3a5fc-5b65-4e24-976e-aad8edc11af0

📥 Commits

Reviewing files that changed from the base of the PR and between 7328827 and 1fece57.

📒 Files selected for processing (17)

• .github/workflows/windows-release.yml
• .gitignore
• docker-compose.yml
• src/domain/mysql/backup.rs
• src/domain/postgres/connection.rs
• src/domain/postgres/database.rs
• src/domain/postgres/mod.rs
• src/services/config.rs
• src/settings.rs
• src/tests/domain/firebird.rs
• src/tests/domain/mariadb.rs
• src/tests/domain/mongodb.rs
• src/tests/domain/mssql.rs
• src/tests/domain/mysql.rs
• src/tests/domain/postgres.rs
• src/tests/domain/redis.rs
• src/tests/domain/valkey.rs

---

📝 Walkthrough

Walkthrough

The PR extends database configuration with max_packet_size and pg_bin_dir fields, implements cross-platform PostgreSQL binary discovery with environment override support, wires max_packet_size into MySQL and MariaDB backup commands alongside version diagnostics, packages the real mysqldump binary in Docker, updates all database tests for the new config shape, and introduces a Windows CI/CD workflow for release artifact automation.

Changes

Database configuration and tooling

Layer / File(s)	Summary
Settings and DatabaseConfig fields src/settings.rs, src/services/config.rs	Settings gains pg_bin_dir (from PG_BIN_DIR env, default empty). DatabaseConfig and InputDatabaseConfig each gain max_packet_size field; ConfigService::load computes it as "512M" for MySQL/MariaDB and empty string for other types.
PostgreSQL binary discovery and cross-platform resolution src/domain/postgres/connection.rs, src/domain/postgres/mod.rs, src/domain/postgres/database.rs	Replaces hardcoded directory with select_pg_path_with that honors CONFIG.pg_bin_dir override, detects OS-specific install paths by major version, searches PATH as fallback, and retains historical Debian/Ubuntu directory. Removes debug environment logging from build_env(). Makes connection module crate-visible.
Backup command enhancements with version probes and max_packet_size src/domain/mariadb/backup.rs, src/domain/mysql/backup.rs	Both backup runners add best-effort --version probes logged at debug level. Both pass cfg.max_packet_size to --max-allowed-packet; MySQL also adds --set-gtid-purged=OFF, --no-tablespaces, and --network-timeout.
Docker mysqldump binary packaging docker/Dockerfile	New mysql-client-tools stage (from mysql:8.4) exports real mysqldump binary and libmysqlclient.so.21* libraries; both base and production stages copy artifacts and run ldconfig.
Test configuration updates for new max_packet_size field src/tests/domain/firebird.rs, src/tests/domain/mariadb.rs, src/tests/domain/mongodb.rs, src/tests/domain/mssql.rs, src/tests/domain/mysql.rs, src/tests/domain/postgres.rs, src/tests/domain/redis.rs, src/tests/domain/valkey.rs	All test database configs updated to include max_packet_size ("512M" for MySQL/MariaDB, empty string for others). Postgres tests also add select_pg_path_tests module validating override, version detection, platform-specific naming, and PATH fallthrough behavior.
Miscellaneous configuration and cleanup docker-compose.yml, .gitignore	Updates rust-app service EDGE_KEY environment value; adds blank line before /docs in .gitignore.

Windows CI/CD automation

Layer / File(s)	Summary
Windows release workflow definition and implementation .github/workflows/windows-release.yml	New workflow triggered by manual dispatch and semantic version tags on main/master. Runs on windows-latest with Rust MSVC toolchain, sets up OpenSSL via vcpkg, builds release binary, packages deterministically as windows-release-{tag}.zip, uploads as workflow artifact unconditionally, and creates GitHub Release with zip asset when triggered by tag push.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

• Portabase/agent#63: Both PRs rewrite src/domain/postgres/connection.rs to implement cross-platform PostgreSQL binary discovery with environment override support.
• Portabase/agent#49: Both PRs extend the mysqldump invocation path in src/domain/mysql/backup.rs and src/domain/mariadb/backup.rs, modifying dump command construction and subprocess handling.
• Portabase/agent#39: Both PRs modify select_pg_path resolution logic in PostgreSQL connection code, implementing path detection and override mechanisms.

Poem

🐇 Configuration took flight, with pg_bin_dir in sight,
Windows workflows now build through the night!
"512M?" asked the dump, cross-platform and bright,
Postgres finds its home, Docker shines so tight.
Hop hop! The tooling is right! 🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main feature being added: support for large MySQL and MariaDB databases by addressing timeout and configuration issues.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docker/Dockerfile`:
- Around line 157-159: The runtime image is not configured to run as a non-root
user, which is a security risk. After the COPY commands that add mysqldump and
its libraries (lines 157-158), adjust the ownership and permissions of the
copied directories and binaries so they are readable by a non-root user, then
create a dedicated non-root user account using the USER instruction and set it
as the default user for the container. Ensure the non-root user has appropriate
read and execute permissions on /usr/local/bin/mysqldump and /usr/local/lib/
directories.

In `@src/services/config.rs`:
- Line 57: The `DatabaseConfig` struct now has a required `max_packet_size`
field that breaks all existing struct literal constructions that don't provide
it (such as in src/tests/domain/mysql.rs). Either update all `DatabaseConfig {
... }` struct literal instantiations throughout the codebase to include the
`max_packet_size` field with an appropriate value, or implement a constructor
method or builder pattern for `DatabaseConfig` that provides sensible defaults
and is used consistently instead of direct struct instantiation.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3b328185-e9b5-47dd-b6c1-aa3ecf88d105

📥 Commits

Reviewing files that changed from the base of the PR and between 23678bf and 7328827.

📒 Files selected for processing (4)

• docker/Dockerfile
• src/domain/mariadb/backup.rs
• src/domain/mysql/backup.rs
• src/services/config.rs

[RambokDev]

Hi, thanks for that PR, I will review it before the end of the week end.

Let me know once good in your side for review

[pcrosthwaite]

Hi @RambokDev - good to go.

[RambokDev]

Hi @pcrosthwaite,

Merged, thanks for you contribution to Portabase,

Regards,