feat(inspect): implement SnapshotsTable scanning#801
Open
WZhuo wants to merge 1 commit into
Open
Conversation
WZhuo
marked this pull request as ready for review
July 2, 2026 08:46
WZhuo
force-pushed
the
inspect
branch
3 times, most recently
from
July 6, 2026 03:37
28c4513 to
a53f8ce
Compare
WZhuo
changed the title
WZhuo
force-pushed
the
inspect
branch
3 times, most recently
from
July 7, 2026 02:04
9bc23e1 to
04b657f
Compare
- Add Scan() virtual method and Scan() convenience overload to MetadataTable - Add SnapshotSelection struct for time-travel snapshot resolution - Add supports_time_travel() concrete method driven by kind() - Implement SnapshotsTable::Scan() to materialize snapshot rows via ArrowRowBuilder
There was a problem hiding this comment.
Pull request overview
Implements initial metadata-table scanning support by adding a Scan() API to MetadataTable (with snapshot-selection parameters for future time-travel) and providing a concrete SnapshotsTable::Scan() implementation that materializes snapshot rows into Arrow arrays. The PR also restructures/extends the metadata-table test suite to validate schemas and snapshot scanning behavior.
Changes:
- Added
SnapshotSelectionand a virtualMetadataTable::Scan()API (with a convenience overload) plussupports_time_travel(). - Implemented
SnapshotsTable::Scan()to emit snapshot rows (6 columns) viaArrowRowBuilder. - Added/expanded tests and wired new test sources into the metadata-table test target.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/iceberg/inspect/metadata_table.h | Adds SnapshotSelection, Scan() API, and supports_time_travel() declaration/docs. |
| src/iceberg/inspect/metadata_table.cc | Implements default Scan() + supports_time_travel() and wires factory for kinds. |
| src/iceberg/inspect/snapshots_table.h | Declares SnapshotsTable::Scan() override. |
| src/iceberg/inspect/snapshots_table.cc | Implements snapshot scanning into Arrow via ArrowRowBuilder. |
| src/iceberg/test/metadata_table_test.cc | Simplifies base setup and adds SupportsTimeTravel test. |
| src/iceberg/test/metadata_table_test_base.h | New shared fixture/helpers for metadata table tests. |
| src/iceberg/test/snapshots_table_test.cc | New tests validating snapshots table construction/schema/scan output. |
| src/iceberg/test/history_table_test.cc | New schema test for history table. |
| src/iceberg/test/CMakeLists.txt | Adds new test sources to the metadata-table test target. |
Comment on lines
+65
to
+68
| /// Time travel is supported for tables that read from a single snapshot's | ||
| /// manifests (e.g., Entries, Files, Manifests, Partitions). Tables that | ||
| /// scan all snapshots (All*) or return in-memory history (Snapshots, | ||
| /// History, Refs) do not support time travel. |
Comment on lines
+38
to
+48
| bool MetadataTable::supports_time_travel() const noexcept { | ||
| // Time travel is supported for tables that read from a single snapshot's | ||
| // manifests. Tables that scan all snapshots or return in-memory history do | ||
| // not. | ||
| switch (kind()) { | ||
| case Kind::kSnapshots: | ||
| case Kind::kHistory: | ||
| return false; | ||
| } | ||
| return false; | ||
| } |
Comment on lines
+81
to
+83
| ArrowSchema c_schema; | ||
| EXPECT_THAT(ToArrowSchema(schema, &c_schema), IsOk()); | ||
| auto arrow_schema = ::arrow::ImportSchema(&c_schema).ValueOrDie(); |
| /// | ||
| /// The default implementation returns NotSupported. Subclasses override this | ||
| /// to materialize their data. | ||
| virtual Result<ArrowArray> Scan(std::optional<SnapshotSelection> snapshot_selection); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implement
Scan()forSnapshotsTable, adding the core scanning infrastructure toMetadataTableand materializing snapshot rows into Arrow arrays viaArrowRowBuilder.Changes
metadata_table.h/.ccSnapshotSelectionstruct withsnapshot_id,as_of_timestamp,ref_nameoptional fields for time-travel queriesScan()virtual method returningResult<ArrowArray>with a convenience zero-arg overloadsupports_time_travel()concrete method driven bykind(), matching Java'sTIME_TRAVEL_TABLE_TYPESstatic set (currently bothkSnapshotsandkHistoryreturnfalse)Scan()returnsNotSupportedsnapshots_table.h/.ccSnapshotsTable::Scan()— iteratessource_table()->snapshots()and writes 6 columns (committed_at,snapshot_id,parent_id,operation,manifest_list,summary) viaArrowRowBuildersnap.timestampMillis() \* 1000Tests
metadata_table_test_base.h— sharedMetadataTableTestBasewithMockFileIO+MockCatalogfixture,FinishAndImport(),MakeTestSnapshots(),MakeTableWithSnapshots()helpersmetadata_table_test.cc— 2 framework tests (FactoryRejectsNullSourceTable,SupportsTimeTravel)snapshots_table_test.cc— 11 tests (Construct,SchemaMatchesIcebergSchema, 9 scan data tests matching JavaTestDataTaskParserfixture)history_table_test.cc— 1 schema test