Repository navigation
Support PostgreSQL 14.25, 15.20 and 16.16 in Citus 12.1 - #8974
Open
ibrahim halatci (ihalatci) wants to merge 19 commits into
Open
ibrahim halatci (ihalatci) wants to merge 19 commits into
ibrahim halatci (ihalatci) wants to merge 19 commits into
Conversation
Test with paired the-process dev images from 62955d16510b401bf1eeaa41bc5c4ebfb19c484e. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Decorate injected Vars with their outer joins' nulling relids on PostgreSQL 16 and later so join removal does not invalidate catalog filters. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep recursive planning consistent with PostgreSQL join removal without bypassing managed-local routing. Preserve dropped attribute positions and cover right joins, join aliases and materialized CTEs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Backport the justified PostgreSQL minor output adaptations, retaining release-12.1 alternatives for older minor versions. Existing function propagation and isolation schedules pass on PostgreSQL 16.16. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Carry outer-join nullability through nested joins before replacing eliminated relation RTEs. Cover the nested distributed LEFT JOIN on the nullable side of a managed-local RIGHT JOIN. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pin the Brotli-compatible legacy mitmproxy fork and narrow compatible dependency updates without upgrading unrelated packages. Retain Python 3.9 and legacy urwid/Werkzeug limits; unsupported newer security targets remain unresolved. Regenerated with Pipenv 2025.0.4 on Python 3.9.25. Export base requirements with pipenv requirements and development requirements with pipenv requirements --dev from src/test/regress. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reconcile the lock with requirements shipped by the-process 62955d16510b401bf1eeaa41bc5c4ebfb19c484e instead of regressing to the older Citus lock. Preserve all unrelated base/development pins while retaining the Python 3.9-compatible updates. Keep Construct 2.10.70 from the existing SSOT/stylechecker consistently in base exports; existing COPY failure coverage passes. Apply the three blank lines required by Black 25.11. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pair Citus source 0d87983 with the-process a1a81efff85b0e1ca42e8490891f91ba18482544. All 12 development image jobs passed in run 43416274880; stable publication was skipped. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Backport validated correction 3ccb168. Preserve all matching transformed copies and nullable traversal; add a deterministic wrong-result regression compatible with Citus 12. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the count assertion while allowing older minors to retain the distributed join and newer minors to eliminate it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the complete wrong-result fixture and assertion in local_dist_join_load so regular-user and random-worker configurations do not execute privileged DDL. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep coordinator-only fixture setup in the load schedule and the query assertion in local_dist_join, including regular-user and MX execution. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
function_propagation: PostgreSQL 14.25/15.20/16.16 (CVE-2026-6460) reject ALTER TABLE table_1_for_circ_dep ADD COLUMN ... because the default of table_2_for_circ_dep uses table_1_for_circ_dep's row type, so PostgreSQL errors before Citus detects the cycle. Create the column up front and set its default with ALTER COLUMN ... SET DEFAULT instead. That creates the same function/table cycle without changing the row type, so Citus raises its circular-dependency error on old and new minors alike. isolation_citus_schema_distribute_undistribute: PostgreSQL 14.25/15.20/16.16 (CVE-2026-6467) lock the schema during ALTER SCHEMA ... RENAME, so the rename waits for s1 on new minors but not on old ones. Make s1 lock the schema's pg_namespace row before s2 renames it, so the rename waits for s1 on every minor. Remove function_propagation_0.out and isolation_citus_schema_distribute_undistribute_0.out.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## release-12.1 #8974 +/- ##
================================================
+ Coverage 89.65% 89.68% +0.02%
================================================
Files 276 276
Lines 59790 59903 +113
Branches 7471 7497 +26
================================================
+ Hits 53605 53722 +117
- Misses 4041 4109 +68
+ Partials 2144 2072 -72 🚀 New features to boost your workflow:
|
Adding nulling relids to the shard-hiding and Citus-object-hiding filters moved them above the outer join. As a result, a LEFT JOIN row that matched only a hidden row was removed instead of being null-extended. When the meta table is on the nullable side of an outer join, add the filter to the relation's security quals so it runs while scanning the relation. Elsewhere keep the old WHERE clause filter.
The query returns true whether or not the planner removes the outer join, and also returns true when the filter is placed above the join, so it does not test what its comment claims.
Restore the release branch dependency manifest and lock, and remove formatting introduced only by the newer dependency tools. Preserve PostgreSQL updates and compatibility fixes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Pair verified the-process f81aced images after removing release dependency upgrades. Retain stylechecker tools 0.8.18 and the updated PostgreSQL minors. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Backport 48be61f for PostgreSQL 14/15/16. Normalize after initial planning and recursive replanning, retain modifying CTE effects and surviving distributed subqueries, and remap dropped-column aliases and references. Cover local-execution-disabled joins and limit the dropped-column alias collision assertion to majors using normalization; older majors retain the same unaliased query coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DESCRIPTION: Support PostgreSQL 14.25, 15.20 and 16.16 in Citus 12.1
Readiness: exact-head stable-image CI and packaging passed; ready for review, not approved for Citus merge. Head
3111ddd580e73558f271683f0aeb20fa8447acd1passed Build & Test 45005156142, attempt 2 and packaging 45005156077. Refreshed overall checks: 120 successful, 3 skipped, none failed or pending. This includes all supported PG14/15/16 jobs, 18 arbitrary configurations, all three query generators, upgrades and style.The latest commit changes only
.github/workflows/build_and_test.ymlimage suffix from-dev-f81acedto verified stable-vbe937ba. PostgreSQL 14.25/15.20/16.16, tools 0.8.18 and release-baseline dependencies are unchanged; dependency upgrades remain main-only.Paired image PR: citusdata/the-process#269, merged as
be937ba0f5c5263d4c69a22b0e12b987d002fa13. Stable publication 45002696412 succeeded. The image owner verified all 12 GHCR manifests against workflow push receipts and six runtime checks for exact PostgreSQL/libpq packages, original Python pins, pip check, mitmdump and style tooling. Required images: extbuilder/exttester/failtester for 14.25/15.20/16.16; citusupgradetester14.25; pgupgradetester14.25-15.20-16.16; stylechecker0.8.18, all using-vbe937ba. Merged image content matches the previously verified development baseline.First-failure disclosure: stable run attempt 1 failed only PG14
failure_distributed_results. The injected disconnect produced ERRORserver closed the connection unexpectedlyinstead of expected WARNINGconnection not open, aborting later transaction assertions. Exact artifact 14747093976 is retained. On the actual stable failtester14.25-vbe937ba image,make -s -C src/test/regress check-failurepassed all 37 tests with immutable current product binaries (same C-source as suffix-only head) and separately all 37 with pre-normalization fd7fd93 binaries. Attempt 2 passed at unchanged head without code, expectation or CI changes. The original failure remains unreproduced with root cause unproven; it is not classified as baseline or a flake.Includes the focused backport of main
48be61fc2eabb8fe6bd01d5b227ca9fa958902ba, adapted to PG14/15/16 APIs. Normalization runs after initial PostgreSQL planning and subquery/CTE replanning, not the fast path; relation identities are assigned first. It handles surviving distributed subqueries, modifying CTEs, inheritance clearing, dropped-column/alias/parent-join/whole-row/correlated remapping. PG16-only normalization remains compiled out on older majors. Direct tests retain numeric assertions, CTE side effects and disabled-local-execution joins. Exact dummy-1/dummy-5 aliases are PG16-only because older planners do not remove those joins and their preexisting placeholder deparser cannot handle these aliases.Earlier focused validation on each supported major:
./configure --enable-debug CFLAGS="-O0 -g -Werror",make -s -j4,make -s install: passed on PG14.25, PG15.20 and PG16.16. Configure warns that --enable-debug is unrecognized; debug settings come from CFLAGS.make -s -C src/test/regress check-custom-schedule SCHEDULE=normalization_port_schedule WORKERCOUNT=2: 9/9 passed on each major, covering helpers, local_dist_join_mixed/load/query, recurring outer joins and citus_depended_object.make -s -C src/test/regress check-custom-schedule SCHEDULE=normalization_single_enterprise_schedule WORKERCOUNT=2: passed 8/8 on each baseline/current PG14 snapshot.make -s -C src/test/regress check-failure-custom-schedule SCHEDULE=failure_base_schedule EXTRA_TESTS=failure_distributed_results WORKERCOUNT=2: passed 8/8 on each baseline/current PG14 snapshot.Historical development-image head
1b55794b8fde5eed72949a8d359a22f8c8c8ed67passed CI 44578164168 attempt 2 and packaging 44578163705. Its attempt 1 had the same PG14 disconnect outcome plus two extra transaction NOTICEs in single_node_enterprise (count 53 unchanged). Both tests passed eight times on each baseline/current snapshot (32 controls total) and then unchanged-head retry; those original failures likewise remain root-unproven.No dependency upgrades, expectation weakening or unrelated runtime fixes were introduced by stable-suffix promotion. Citus merge is not authorized.