CBO: data-movement-aware greedy join reordering

[deniskuzZ]

What changes were proposed in this pull request?

The HEP planner's cumulative row-count cost is data-movement-blind
(cpu=io=0). The extended cost model (hive.cbo.costmodel.extended) does
model CPU and IO, but it only rescores plans after the fact: the greedy
ordering in HiveLoptOptimizeJoinRule is driven by factor weights with a
max-NDV tie-break and never consults it, so the shuffle-heavy ordering
is generated either way (runtime-validated on TPC-DS query24, where
enabling the extended model brought no improvement). The rule can thus
emit orderings that shuffle a large fact table raw before the selective
dimension joins that would have reduced it, inflating the bytes crossing
the network by orders of magnitude on queries like query24, query59 and
query72.

This adds a shuffle-cost model to the join reordering, enabled by
hive.cbo.join.reorder.shuffle.cost (default true):

• Generation: on a factor-weight tie in getBestNextFactor, prefer a
  factor whose build side fits the map-join size budget
  (hive.auto.convert.join.noconditionaltask.size), producing a
  fact-reducing ordering the legacy max-NDV tie-break never yields.
• Selection: from the pool of N legacy orderings plus N
  broadcast-preferring variants, emit only the plan with the lowest
  estimated shuffle cost. A broadcast-eligible join (INNER, hash-joinable
  key, build side under the threshold) is charged its build side;
  anything else is charged both inputs. Outer joins are deliberately
  never modelled as broadcasts: letting the model move them on the
  strength of estimates alone was runtime-validated as harmful (an
  estimate-better outer-join placement in query72 ran 14x slower).
• Fail-safe: unknown statistics make a candidate uncostable (NaN) and
  any estimation failure logs a warning and falls back to emitting the
  legacy N orderings unchanged.

The reordering only applies when map-join conversion is enabled, since
the cost model ranks orderings against broadcast joins the physical
planner must be able to materialize.

Why are the changes needed?

Performance regression in TPC-DS 1Tb after adding constraints

Does this PR introduce any user-facing change?

No

How was this patch tested?

Cluster

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.9% Duplication on New Code

See analysis details on SonarQube Cloud