Skip to content

[14.0] Support PostgreSQL 16.16 / 17.12 / 18.7 by restoring removed outer-join relations - #8978

Draft
eaydingol wants to merge 3 commits into
release-14.0from
ebruaydingol-microsoft-review-pr-8973-pg-minor-bump
Draft

eaydingol wants to merge 3 commits into
release-14.0from
ebruaydingol-microsoft-review-pr-8973-pg-minor-bump

Conversation

@eaydingol

Copy link
Copy Markdown
Contributor

DESCRIPTION: Fixes outer join planning on PostgreSQL 16.16, 17.12 and 18.7

Alternative to #8975 (and #8973 on main). It uses a different approach to the same breakage.

Problem

PostgreSQL 16.16, 17.12 and 18.7 include upstream commit 9f25197bf27c. With it, useless outer joins are removed by editing parse->jointree, which Citus plans after standard_planner(). Distributed relations then disappear from the query Citus plans. The result is wrong recursive-planning decisions (recursive complex joins are only supported when all distributed tables are co-located and joined on their distribution columns), or an assertion failure in cassert builds.

The same minors also change two behaviors that tests relied on (CVE-2026-6460, CVE-2026-6467).

Approach

  1. Restore the jointree (distributed_planner.c). When standard_planner() leaves fewer relations in the jointree than the original query had, restore the original jointree.
    • This is the code main already uses for PG19 (Fix PostgreSQL 19 beta4 compatibility #8870), applied to all supported versions. release-14.0 and main therefore share one mechanism instead of a new planner heuristic.
    • There is no compile-time minor guard, because a package built against an older minor can run on a newer one. On older minors the jointree is never edited, so nothing is restored.
  2. Catalog filters (worker_shard_visibility.c, citus_depended_object.c). When pg_class or another meta table is on the nullable side of an outer join, put the hiding filter in the relation's securityQuals. Elsewhere it stays in the WHERE clause.
  3. Tests
  4. CI moves to 16.16 / 17.12 / 18.7 using the -dev-bf6c2ae images. There are no Python dependency or devcontainer changes.

Known limitation, shared with main's PG19 code: the restore compares top-level relation counts. If subquery pull-up adds a relation while join removal drops one, the counts can match and nothing is restored.

Credits: the filter helper and tests are by Onur Tirtir (@onurctirtir) and ibrahim halatci (@ihalatci) (#8975); the restore is from #8870.

Validation

Run locally in WSL against PostgreSQL built from source (--enable-cassert), with Citus built against each version:

Command 18.7 18.6 16.16
make -C src/test/regress check-custom-schedule WORKERCOUNT=2 SCHEDULE=<minimal_schedule + function_propagation, local_dist_join_load, local_dist_join, citus_depended_object> passed passed passed
make -C src/test/regress check-custom-schedule WORKERCOUNT=2 SCHEDULE=<minimal_schedule + local_dist_join_mixed> passed passed passed
make -C src/test/regress check-custom-schedule WORKERCOUNT=2 SCHEDULE=<multi_mx_schedule through multi_mx_hide_shard_names> passed passed passed
make -C src/test/regress check-isolation-custom-schedule WORKERCOUNT=2 SCHEDULE=<base_isolation_schedule + isolation_citus_schema_distribute_undistribute> passed passed passed
Ad-hoc probe: 10 outer-join shapes, INSERT/UPDATE, prepared statements, catalog hiding same results same results same results

Additional checks:

  • PG 17.12: compile only (make), no C warnings.
  • Baseline: unpatched release-14.0 with the new tests on 18.7 fails local_dist_join, local_dist_join_mixed and the probe with the co-location error above.

Not run locally: full regression and isolation suites, PG17 runtime tests, upgrade/N-1 tests, and check-style (local uncrustify is 0.68, not the pinned 0.82.0). CI covers these.

eaydingol and others added 3 commits November 17, 2026 11:05
DESCRIPTION: Fixes outer join planning on PostgreSQL 16.16, 17.12 and 18.7

PostgreSQL 16.16, 17.12 and 18.7 (commit 9f25197bf27c upstream) remove
useless outer joins by editing parse->jointree, which Citus plans after
standard_planner(). Distributed relations then disappear from the query
that Citus plans, which produces wrong recursive-planning decisions and
errors such as "recursive complex joins are only supported when all
distributed tables are co-located", or an assertion failure in
cassert builds.

Port the jointree restore that main already uses for PostgreSQL 19
(#8870) to all supported versions: when standard_planner() leaves fewer
relations in the jointree than the original query had, restore the
original jointree. There is no compile-time minor version guard,
because a package built against an older minor can run on a newer one.
On older minors the jointree is never edited, so nothing is restored.

The filters that hide shards and Citus-dependent objects from pg_class
and the other meta tables are added to the WHERE clause without
nullingrels, so on the nullable side of an outer join PostgreSQL
evaluates them when scanning the meta table and null-extends the outer
row. The new join removal can then remove the meta table while the
WHERE clause still references it, which fails an assertion in cassert
builds. When the meta table is on the nullable side of an outer join,
put the filter into its securityQuals instead: it is still evaluated
at scan level, and it is removed together with the relation.
Elsewhere, keep adding it to the WHERE clause.

Co-authored-by: Ibrahim Halatci <ihalatci@gmail.com>
Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DESCRIPTION: Makes tests stable across PostgreSQL security minor releases

PostgreSQL 16.16, 17.12 and 18.7 changed two behaviors that tests
relied on:

- CVE-2026-6460: ALTER TABLE ADD COLUMN with a default whose type
  depends on the table's row type is rejected before Citus detects the
  circular dependency. function_propagation now sets the default on an
  existing column.
- CVE-2026-6467: ALTER SCHEMA RENAME also locks the schema. The
  isolation test now locks the pg_namespace row explicitly, so the
  rename waits on every minor version.

Co-authored-by: Onur Tirtir <onurcantirtir@gmail.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DESCRIPTION: Bumps CI to PostgreSQL 16.16, 17.12 and 18.7

Points CI at the -dev-bf6c2ae images, which carry the new PostgreSQL
minors.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.05882% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 88.57%. Comparing base (cefd6ce) to head (adc77c1).

Additional details and impacted files
@@               Coverage Diff                @@
##           release-14.0    #8978      +/-   ##
================================================
- Coverage         88.58%   88.57%   -0.01%     
================================================
  Files               292      292              
  Lines             65369    65401      +32     
  Branches           8237     8241       +4     
================================================
+ Hits              57905    57932      +27     
- Misses             5088     5093       +5     
  Partials           2376     2376              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant