support specify field ids for table read context #59
+84
−19
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
lxy-9602
reviewed
Jan 18, 2026
| /// @return Reference to this builder for method chaining. | ||
| /// @note Currently supports top-level field selection. Future versions may support | ||
| /// nested field selection using ArrowSchema for more granular projection | ||
| ReadContextBuilder& SetReadFieldIds(const std::vector<int32_t>& read_field_ids); |
Collaborator
There was a problem hiding this comment.
Consider add comments to clarify the behavior when both SetReadFieldIds and SetReadSchema are called. E.g., SetReadFieldIds and SetReadSchema are mutually exclusive. Calling both may result in undefined behavior. Only one should be used.
Collaborator
There was a problem hiding this comment.
Or fieldId take priority? Or should this be treated as an error?
Author
There was a problem hiding this comment.
thanks, will update according to your suggestion.
lszskye
reviewed
Jan 20, 2026
| read_data_fields.reserve(context->GetReadFieldIds().size()); | ||
| for (const auto& field_id : context->GetReadFieldIds()) { | ||
| // if enable row tracking or data evolution, check special fields | ||
| if (core_options.RowTrackingEnabled() && field_id == SpecialFields::RowId().Id()) { |
Collaborator
There was a problem hiding this comment.
If users need to read certain special fields, they will need to know the corresponding field IDs. In that case, we should consider exposing the SpecialFields to users in /include directory.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
paimon-cpp currently only support select columns using column name, the patch is used to support select columns using field id as well.
Linked issue: #34
Tests
add UT internal_read_context_test.cpp#TestReadWithSpecifiedFieldId
update UT read_context_test.cpp
API and Format
add SetReadFieldIds(const std::vector<int32_t>& read_field_ids) in class ReadContextBuilder
Documentation