fix: late-materialize blob columns read as binary#7593
Open
wkalt wants to merge 1 commit into
Open
Conversation
is_early_field force-classified every blob field as early materialization, so
a selective filter that projected a blob read the whole blob column instead of
taking matched rows. That is only correct for the default blobs_descriptions
handling, where a blob is a tiny {offset, size} description; all_binary (and
the SomeBinary variants) materialize the full value, so reading it for the
whole table defeats late materialization. A TODO at the call site already
flagged this.
Force early materialization only when the blob is returned as a description;
otherwise fall through to the width-based heuristic, which late-materializes a
wide binary leaf. The decision is per leaf, so a blob nested in a struct is
handled like a top-level column. Default and explicit AllEarly/AllLate are
unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wkalt
force-pushed
the
ticket/oss-1382/late-materialize-all-binary-blobs
branch
from
582c1f1 to
5faa90b
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
brendanclement
approved these changes
Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
is_early_field force-classified every blob field as early materialization, so
a selective filter that projected a blob read the whole blob column instead of
taking matched rows. That is only correct for the default blobs_descriptions
handling, where a blob is a tiny {offset, size} description; all_binary (and
the SomeBinary variants) materialize the full value, so reading it for the
whole table defeats late materialization. A TODO at the call site already
flagged this.
Force early materialization only when the blob is returned as a description;
otherwise fall through to the width-based heuristic, which late-materializes a
wide binary leaf. The decision is per leaf, so a blob nested in a struct is
handled like a top-level column. Default and explicit AllEarly/AllLate are
unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com