Skip to content

feat: migrate Paimon C++ 0.3 development history - #161

Merged
SteNicholas merged 138 commits into
apache:mainfrom
lxy-9602:migrate-0.3
Aug 1, 2026
Merged

feat: migrate Paimon C++ 0.3 development history#161
SteNicholas merged 138 commits into
apache:mainfrom
lxy-9602:migrate-0.3

Conversation

@lxy-9602

@lxy-9602 lxy-9602 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Purpose

Migrate the Paimon C++ development history after release-0.2 from the original Alibaba repository into the Apache repository.

This PR:

  • migrates 133 non-merge commits in their original order;
  • preserves the original commit authors, author dates, and commit boundaries;
  • migrates the source code, public APIs, tests, benchmarks, documentation, CMake configuration, and test data required for the 0.3 development line;
  • resolves cherry-pick conflicts against the Apache repository structure and existing Apache-specific build and release infrastructure;
  • converts Alibaba-owned source headers to ASF license headers;
  • preserves applicable third-party and borrowed-file license headers and declarations;
  • removes target-only migration leftovers that are not present in the source repository.

The resulting branch contains 134 commits in total: 133 migrated source commits and one migration cleanup commit.

Tests

API and Format

Documentation

Generative AI tooling

Migrate-by: Codex

suxiaogang223 and others added 30 commits July 31, 2026 17:25
@suxiaogang223 @lxy-9602
feat: Add metadata system tables
7138229
@lszskye @lxy-9602
fix: add break for CHAR_END_SUBTYPE
3ae0723
@lxy-9602
fix: Fix MemorySegment::Compare to use big-endian byte-order comparis…
d5d94d8 
…on semantics
@zjw1111 @lxy-9602
chore(license): refine build support notices
5e25655
@lxy-9602
fix: Fix undefined behavior in BinarySection bitmask constants caused…
184cda7 
… by signed left shift
@lxy-9602
refactor: ColumnarArray: change constructor parameter to raw pointer …
2b5426a 
…to clarify non-owning semantics
@suxiaogang223 @lxy-9602
feat: Support macOS builds with AppleClang and Homebrew Clang
566586b
@lxy-9602
fix: fix undefined behavior in bitwise left-shift operations across c…
e98acc9 
…odebase
@liangjie3138 @lxy-9602
feat: optimize parquet reads with page-level filtering
002daa4
@zjw1111 @lxy-9602
fix: add ScopeGuard to wait async tasks before early return in Orphan…
3f02435 
…FilesCleanerImpl::Clean()
@lxy-9602
feat(lumina): support null values in vector column during index building
ed17877
@lszskye @lxy-9602
chore: modify comment in RecordBatch & add check in GenericRow
8372606
@zjw1111 @lxy-9602
fix: fix clang-tidy issues
98c87aa
@lszskye @lxy-9602
chore: add check scale in BinaryRowWriter
48d55dd
@lszskye @lxy-9602
fix: fix ub in Blob
2a4aa2b
@lxy-9602
refact: Refactor Literal operator== to delegate to CompareTo for cons…
6f90168 
…istent equality semantics
@zhf999 @lxy-9602
refactor: Reuse RowGroupPageIndexReader across columns to improve pag…
4cb047c 
…e-level predicate pushdown performance
@lxy-9602
feat(blob): Support blob-descriptor-field for inline blob descriptor …
3fc019a 
…storage
@lxy-9602
fix: Like::TestString to align with Java LIKE semantics
93db3e3
@zjw1111 @lxy-9602
chore: update LICENSE with additional PyTorch copyright information
5ea204d
@dalingmeng @lxy-9602
fix: date validation reject invalid dates instead of silent normaliza…
a15fc72 
…tion
@lucasfang @lxy-9602
fix: fix thread-safe problem for file store path factory
c394b0c
@lxy-9602
feat: Improve PK MOR read performance with batch queue and cached col…
b412c2d 
…umn metadata
@lxy-9602
fix: avoid signed integer overflow UB in BloomFilter and DeltaVarintC…
17b7c2a 
…ompressor
@lucasfang @dalingmeng @lxy-9602
feat: optimize count(*) for pk/append table
996be78 
Co-authored-by: dalingmeng <menglingda.mld@alibaba-inc.com>
@lszskye @lxy-9602
feat(blob): support blob-view-field for cross-table blob reference re…
e88a8d4
@lucasfang @lxy-9602
fix: fix unstable prefetch reader case
08c0cc0
@lszskye @lxy-9602
feat: remove support for FLOAT/DOUBLE type partition field
0f099cc
@liujiayi771 @lxy-9602
fix: skip object store check when UseRESTCatalogCommit is enabled
beaae3c
@fourier307 @lxy-9602
feat: add Bucket() virtual interface to DataSplit base class
28e2b82
lxy-9602 and others added 23 commits July 31, 2026 17:25
@lxy-9602
feat(lumina): support Lumina tag filtering
598c8bc
@lxy-9602
fix(reader): ensure batch readers return zero-offset Arrow arrays
90c3bb7
@suxiaogang223 @lxy-9602
feat: add global system tables framework under sys database
9ed061e
@zjw1111 @lxy-9602
refactor(format): remove Lance support
60011b4
@zjw1111 @lxy-9602
refactor(tantivy): move FFI crate under crates
bd4466d
@zjw1111 @lxy-9602
chore: clean up attribution and test data whitespace
1eb50bb
@lxy-9602
fix(blob): support blob-only writes and reject nested blob fields
1577b0c
@XiaoHongbo-Hope @lxy-9602
fix: nested list schema evolution
1df1baa
@zhangweilst @lxy-9602
fix(abi): dynamic_cast error on Predicate across shared libraries
0cd7ef1
@mrdrivingduck @lxy-9602
feat(fs): add object store and S3 file systems
0d8929d 
Add a reusable read-only object store layer with a curl-based HTTP
transport.

Use the AWS C authentication components for credential resolution and
request signing while keeping S3 data access independent of the full AWS
SDK.
@zhf999 @lxy-9602
feat(parquet): support page-level (page index) filtering for nested c…
83a9c41 
…olumn types (patch arrow)
@lucasfang @lxy-9602
test: add test for test coverage
04bc024
@lszskye @lxy-9602
test: add ut for release-0.3
c4f9f45
@zjw1111 @lxy-9602
chore: remove Git LFS and refresh documentation
f15596b
@lxy-9602
fix(blob): allow blob files across schema IDs
c6f2b91
@SteNicholas @lxy-9602
fix(core): support fallback keys for ignore-delete option
27f6d89
@lxy-9602
feat(shredding): support adaptive schemas across rolling files
662bccd
@lszskye @lxy-9602
fix(lucene): normalize prefix and wildcard queries
77b9042
@lszskye @lxy-9602
chore: update version to 0.3.0
b58d238
@SteNicholas @lxy-9602
feat(blob): support placeholder fallback for partial updates
e543f76
@zhf999 @lxy-9602
feat(parquet): support configuring Arrow pre-buffer hole-size-limit f…
2b74264 
…or range coalescing
@lxy-9602
fix: align floating-point semantics with Java
eab6a6c
@lxy-9602
chore: remove target-only migration leftover
eb172b8
SteNicholas
SteNicholas approved these changes Aug 1, 2026
View reviewed changes

@SteNicholas SteNicholas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@SteNicholas
SteNicholas merged commit a899262 into apache:main Aug 1, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SteNicholas SteNicholas SteNicholas approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.