refactor: improve type safety and const-correctness#192
Open
lxy-9602 wants to merge 8 commits intoalibaba:mainfrom
Open
refactor: improve type safety and const-correctness#192lxy-9602 wants to merge 8 commits intoalibaba:mainfrom
lxy-9602 wants to merge 8 commits intoalibaba:mainfrom
Conversation
Contributor
|
I think it would be good to have some coding style guideline for paimon-cpp contribution docs. 😸 |
Collaborator
Author
Great idea! We'll add that in later. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: #93
LookupLevelsto support key fields at any position in schema.For 1.
KeyValueDataFileRecordReaderpreviously usedkey_arity(an integer) to extract key fields by positional index, assuming they always appear right after the two special fields (_SEQUENCE_NUMBER,_VALUE_KIND).For the normal read path this was not an issue, because
MergeFileSplitReadreorders the schema to place key fields before value fields. However,LookupLevelsdoes not perform this reordering, so during compaction the lookup process would fail to correctly extract key fields when they appear after value fields.This fix replaces
int32_t key_aritywithstd::shared_ptr<arrow::Schema> key_schemaand usesGetFieldByNameto locate key columns by name regardless of their physical position.Tests
LookupLevelsTest, TestLevelsWithValueFieldAppearBeforeKey
KeyValueDataFileRecordReaderTest, TestKeyFieldAfterValueField
Generative AI tooling
Partially Generated-by: Claude-4.6-Opus