# [fix](topn lazy materialization) Disable topn lazy materialization on non-light-schema-change tables#65415
Conversation
… non-light-schema-change tables TopN lazy materialization appends a synthetic global row-id column (__DORIS_GLOBAL_ROWID_COL__<table>) to the OLAP scan. The BE only rebuilds the tablet schema from FE's columns_desc when columns_desc[0].col_unique_id >= 0, which only holds for light_schema_change tables. On a non-light-schema-change table every column's uniqueId is -1, so the synthetic row-id column never enters the BE tablet schema and the scan fails with: field name is invalid. field=__DORIS_GLOBAL_ROWID_COL__<table> (forcing the column into the scan schema instead makes the second-phase fetch silently return NULL for the lazily-materialized columns, which is worse). Fix: disable topn lazy materialization for non-light-schema-change OLAP tables in MaterializeProbeVisitor and fall back to normal topn (the working two-phase read path). A new helper supportOlapTopnLazyMaterialize() consolidates the existing AGG_KEYS exclusion with the new light_schema_change requirement, applied at visitPhysicalOlapScan, visitPhysicalCatalogRelation and visitPhysicalFilter. Add regression test topn_lazy_light_schema_change verifying: - light_schema_change=false: no lazy materialization in the plan, correct results. - light_schema_change=true: lazy materialization still applies, correct results. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
|
/review |
There was a problem hiding this comment.
Reviewed PR 65415 for TopN lazy materialization gating on non-light-schema-change OLAP tables.
Critical checkpoint conclusions:
- Goal and coverage: The PR disables OLAP TopN lazy materialization before source selection when the table is AGG_KEYS or has light_schema_change=false, and adds a regression covering explicit false and true table properties.
- Scope: The change is small and focused on MaterializeProbeVisitor support checks plus one targeted regression suite.
- Concurrency/lifecycle: No new concurrency, locks, async lifecycle, static initialization, persistence, or transaction paths are introduced.
- Config/session compatibility: No new config is added. Existing topn_lazy_materialization_threshold and topn_lazy_materialization_using_index paths remain unchanged; the new table-property gate is applied in the normal scan, catalog-relation, and using-index filter probe paths.
- Parallel paths: The relevant OLAP probe entry points are covered; non-OLAP lazy materialization paths are unchanged.
- Test coverage: The regression checks both plan shape and query result for light_schema_change=false fallback and light_schema_change=true lazy behavior. I did not run the regression locally in this shallow Actions checkout because thirdparty/installed and thirdparty/installed/bin/protoc are missing; GitHub CI currently has style/license checks passing, with compile/FE UT/performance still pending.
- User focus: No additional user-provided focus points were supplied.
Subagent conclusions: optimizer-rewrite reported NO_NEW_VALUABLE_FINDINGS; tests-session-config reported NO_NEW_VALUABLE_FINDINGS. No subagent candidate became an inline comment, and none were dismissed or merged as duplicates. Final convergence round 1 ended with both live subagents reporting NO_NEW_VALUABLE_FINDINGS for the same empty inline-comment set and this comment-only review body.
FE Regression Coverage ReportIncrement line coverage |
A top-N query that emits a non-order-by column fails on a non-light-schema-change OLAP table (a table created/upgraded with
light_schema_change = false, where every column'suniqueIdis-1):Fix: disable topn lazy materialization for non-light-schema-change OLAP tables in MaterializeProbeVisitor and fall back to normal topn (the working two-phase read path). A new helper supportOlapTopnLazyMaterialize() consolidates the existing AGG_KEYS exclusion with the new light_schema_change requirement, applied at visitPhysicalOlapScan, visitPhysicalCatalogRelation and visitPhysicalFilter.
Add regression test topn_lazy_light_schema_change verifying:
Behavior after the fix
light_schema_change = falsePhysicalOlapScan(no lazy) → safe two-phase readlight_schema_change = truePhysicalLazyMaterialize/PhysicalLazyMaterializeOlapScan(unchanged)What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)