From e5e3b85b011a08b38d88c8e641ac18121157d87c Mon Sep 17 00:00:00 2001 From: kanthi subramanian Date: Thu, 16 Apr 2026 13:48:56 -0500 Subject: [PATCH 1/3] Added docker compose to start etcd cluster and instructions to point ice-rest-catalog to etcd cluster --- docs/etcd-backend-schema.md | 109 ++++++++++++++++ docs/etcd-cluster-setup.md | 107 ++++++++++++++++ docs/sqlite-backend-schema.md | 98 +++++++++++++++ .../docker-compose/docker-compose-etcd.yaml | 116 ++++++++++++++++++ 4 files changed, 430 insertions(+) create mode 100644 docs/etcd-backend-schema.md create mode 100644 docs/etcd-cluster-setup.md create mode 100644 docs/sqlite-backend-schema.md create mode 100644 examples/docker-compose/docker-compose-etcd.yaml diff --git a/docs/etcd-backend-schema.md b/docs/etcd-backend-schema.md new file mode 100644 index 00000000..03624c8a --- /dev/null +++ b/docs/etcd-backend-schema.md @@ -0,0 +1,109 @@ +# Iceberg REST Catalog — etcd Backend Schema + +This document describes the etcd key/value schema used by the Iceberg REST Catalog to track namespaces and tables. + +## Key Prefixes + +For the `default` catalog, keys use bare prefixes: + +| Prefix | Purpose | +|--------|---------| +| `n/` | Namespace entries | +| `t/` | Table entries | + +For non-default catalogs, the catalog name is prepended: `/n/` and `/t/`. + +Defined in `EtcdCatalog.java`: + +```java +private static final String NAMESPACE_PREFIX = "n/"; +private static final String TABLE_PREFIX = "t/"; +``` + +## Schema + +### Namespaces (`n/`) + +**Key format:** `n/` + +**Value:** JSON map of namespace properties (may be empty `{}`). + +| Field | Description | +|-------|-------------| +| key | `n/` + namespace name (levels joined by `/` for nested namespaces) | +| value | JSON object with namespace properties | + +**Example:** + +``` +key: n/flowers +value: {} +``` + +``` +key: n/nyc +value: {} +``` + +### Tables (`t/`) + +**Key format:** `t//` + +**Value:** JSON object with table metadata pointers. + +| Field | Description | +|-------|-------------| +| `table_type` | Object type (e.g. `ICEBERG`) | +| `metadata_location` | S3/file path to the current metadata JSON file | +| `previous_metadata_location` | S3/file path to the previous metadata JSON file (empty if first version) | + +**Example:** + +``` +key: t/flowers/iris2 +value: { + "table_type": "ICEBERG", + "metadata_location": "s3://bucket1/flowers/iris2/metadata/00001-5d6604f9-d6f1-4ced-a036-8f20d6c0def2.metadata.json", + "previous_metadata_location": "s3://bucket1/flowers/iris2/metadata/00000-3f6b12e4-0a85-42bf-8ed0-d7cb443d5830.metadata.json" +} +``` + +## Mapping to SQLite Backend + +The SQLite backend stores the same information in two relational tables. Here is how the etcd keys correspond: + +``` +etcd key SQLite table +─────────────────────────── ────────────────────────────────── +n/ -> iceberg_namespace_properties +t// -> iceberg_tables +``` + +### `n/` -> `iceberg_namespace_properties` + +| etcd | SQLite column | +|------|---------------| +| key prefix after `n/` | `namespace` | +| (implicit) | `catalog_name` = catalog name (e.g. `default`) | +| each entry in JSON value | one row per property: `property_key` + `property_value` | + +In SQLite, each namespace property is stored as a separate row. In etcd, all properties for a namespace are stored as a single JSON blob. + +### `t/` -> `iceberg_tables` + +| etcd | SQLite column | +|------|---------------| +| namespace segment of key | `table_namespace` | +| table segment of key | `table_name` | +| (implicit) | `catalog_name` = catalog name (e.g. `default`) | +| `table_type` in value | `iceberg_type` | +| `metadata_location` in value | `metadata_location` | +| `previous_metadata_location` in value | `previous_metadata_location` | + +## Notes + +- The composite primary key equivalent in etcd is the key itself (`t//
` is unique per table). +- Tables with an empty `previous_metadata_location` are at their initial version (version 0). +- The metadata version can be inferred from the filename prefix (e.g. `00003-...` = version 3). +- All metadata files follow the pattern: `s3:////
/metadata/-.metadata.json`. +- Namespace levels are joined by `/` in the key (e.g. `n/parent/child` for nested namespace `parent.child`). diff --git a/docs/etcd-cluster-setup.md b/docs/etcd-cluster-setup.md new file mode 100644 index 00000000..2266ac6d --- /dev/null +++ b/docs/etcd-cluster-setup.md @@ -0,0 +1,107 @@ +# ice-rest-catalog with etcd Cluster + +This guide walks through setting up ice-rest-catalog backed by a 3-node etcd cluster, inserting data with the ice CLI, verifying replication, and querying from ClickHouse. + +## Prerequisites + +- Docker Compose +- ice CLI (`ice`) +- `etcdctl` (v3) + +## 1. Start the etcd Cluster + +Use the provided docker-compose file to bring up etcd, MinIO, ice-rest-catalog, and ClickHouse: + +```bash +cd examples/docker-compose +docker compose -f docker-compose-etcd.yaml up -d +``` + +This starts a single-node etcd by default. For a 3-node cluster, replace the `etcd` service definition with three separate nodes (etcd1, etcd2, etcd3) each with their own ports mapped to the host: + +| Node | Client Port | +|-------|-------------| +| etcd1 | 12379 | +| etcd2 | 12479 | +| etcd3 | 12579 | + +## 2. Configure ice-rest-catalog + +Update the `uri` in your `ice-rest-catalog.yaml` to point at all three etcd endpoints: + +```yaml +uri: etcd:http://127.0.0.1:12379,http://127.0.0.1:12479,http://127.0.0.1:12579 +warehouse: s3://bucket1 + +s3: + endpoint: http://localhost:9000 + pathStyleAccess: true + accessKeyID: miniouser + secretAccessKey: miniopassword + region: minio + +bearerTokens: + - value: foo +``` + +Start (or restart) ice-rest-catalog so it picks up the new config. + +## 3. Insert Data + +Use the ice CLI to create a table and insert a Parquet file: + +```bash +ice insert flowers.iris file://iris.parquet +``` + +## 4. Verify Replication with etcdctl + +Query the table key across all three etcd endpoints to confirm the data was replicated: + +```bash +ETCDCTL_API=3 etcdctl \ + --endpoints=http://127.0.0.1:12379,http://127.0.0.1:12479,http://127.0.0.1:12579 \ + get t/flowers/iris +``` + +Expected output: + +``` +t/flowers/iris +{"table_type":"ICEBERG","metadata_location":"s3://bucket1/flowers/iris/metadata/00002-b2cd8da0-74a7-460d-ac3c-12f85d65225b.metadata.json","previous_metadata_location":"s3://bucket1/flowers/iris/metadata/00001-659c1907-5ac7-4ae1-b8c2-573d2f61b45b.metadata.json"} +``` + +This confirms the key was replicated across all etcd instances. + +## 5. Query from ClickHouse + +Connect to ClickHouse and create the Iceberg catalog database: + +```sql +CREATE DATABASE ice + ENGINE = DataLakeCatalog('http://host.docker.internal:5000') + SETTINGS + catalog_type = 'rest', + auth_header = 'Authorization: Bearer foo', + storage_endpoint = 'http://host.docker.internal:9000', + warehouse = 's3://bucket1', + aws_access_key_id = 'miniouser', + aws_secret_access_key = 'miniopassword'; +``` + +Query the table: + +```sql +SELECT * FROM ice.`flowers.iris`; +``` + +``` +┌─sepal.length─┬─sepal.width─┬─petal.length─┬─petal.width─┬─variety────┐ +│ 5.1 │ 3.5 │ 1.4 │ 0.2 │ Setosa │ +│ 4.9 │ 3 │ 1.4 │ 0.2 │ Setosa │ +│ 4.7 │ 3.2 │ 1.3 │ 0.2 │ Setosa │ +│ 4.6 │ 3.1 │ 1.5 │ 0.2 │ Setosa │ +│ 5 │ 3.6 │ 1.4 │ 0.2 │ Setosa │ +│ 5.4 │ 3.9 │ 1.7 │ 0.4 │ Setosa │ +└──────────────┴─────────────┴──────────────┴─────────────┴────────────┘ +``` diff --git a/docs/sqlite-backend-schema.md b/docs/sqlite-backend-schema.md new file mode 100644 index 00000000..0dd8839d --- /dev/null +++ b/docs/sqlite-backend-schema.md @@ -0,0 +1,98 @@ +# Iceberg REST Catalog — SQLite Backend Schema + +This document describes the SQLite database schema used by the Iceberg REST Catalog to track namespaces and tables. + +--- + +## ERD Diagram + +```mermaid +erDiagram + iceberg_namespace_properties { + string catalog_name PK + string namespace PK + string property_key PK + string property_value + } + + iceberg_tables { + string catalog_name PK + string table_namespace PK + string table_name PK + string metadata_location + string previous_metadata_location + string iceberg_type + } + + iceberg_namespace_properties ||--o{ iceberg_tables : "catalog_name, namespace → table_namespace" +``` + +> **Relationship:** `iceberg_namespace_properties.catalog_name` + `namespace` logically maps to `iceberg_tables.catalog_name` + `table_namespace`. A namespace can contain zero or more tables. + +--- + +## Schema + +### `iceberg_tables` + +Stores metadata references for every registered Iceberg table. + +| Column | Key | Description | +|---|---|---| +| `catalog_name` | PK | Catalog identifier (e.g. `default`) | +| `table_namespace` | PK | Dot-separated namespace the table belongs to | +| `table_name` | PK | Name of the table | +| `metadata_location` | | S3 path to the **current** metadata JSON file | +| `previous_metadata_location` | | S3 path to the **previous** metadata JSON file (empty if first version) | +| `iceberg_type` | | Object type — always `TABLE` | + +### `iceberg_namespace_properties` + +Stores key/value properties for each namespace. + +| Column | Key | Description | +|---|---|---| +| `catalog_name` | PK | Catalog identifier | +| `namespace` | PK | Namespace name | +| `property_key` | PK | Property key | +| `property_value` | | Property value | + +--- + +## Sample Data + +### `iceberg_tables` + +| catalog_name | table_namespace | table_name | metadata_location | previous_metadata_location | iceberg_type | +|---|---|---|---|---|---| +| default | test | ll2 | `s3://bucket1/test/ll2/metadata/00003-bb602d70-...json` | `s3://bucket1/test/ll2/metadata/00002-1debf62b-...json` | TABLE | +| default | test | ll2_p | `s3://bucket1/test/ll2_p/metadata/00001-690b44b8-...json` | `s3://bucket1/test/ll2_p/metadata/00000-d87bf260-...json` | TABLE | +| default | test | ll3_ | `s3://bucket1/test/ll3_/metadata/00001-005c7284-...json` | `s3://bucket1/test/ll3_/metadata/00000-08751e01-...json` | TABLE | +| default | test | type_test | `s3://bucket1/test/type_test/metadata/00003-688d3504-...json` | `s3://bucket1/test/type_test/metadata/00002-2aab973f-...json` | TABLE | +| default | test | type_test2 | `s3://bucket1/test/type_test2/metadata/00000-8573b865-...json` | *(none)* | TABLE | +| default | test | type_test_no_copy | `s3://bucket1/test/type_test_no_copy/metadata/00001-e2ecf4d0-...json` | `s3://bucket1/test/type_test_no_copy/metadata/00000-2e38b5ba-...json` | TABLE | +| default | test | type_test_no_copy_2 | `s3://bucket1/test/type_test_no_copy_2/metadata/00000-90ca965f-...json` | *(none)* | TABLE | +| default | test | type_test_no_copy_3 | `s3://bucket1/test/type_test_no_copy_3/metadata/00001-05050286-...json` | `s3://bucket1/test/type_test_no_copy_3/metadata/00000-a9a4f859-...json` | TABLE | +| default | test | type_test_no_copy_44 | `s3://bucket1/test/type_test_no_copy_44/metadata/00001-5ae8dcb9-...json` | `s3://bucket1/test/type_test_no_copy_44/metadata/00000-01ad9f2d-...json` | TABLE | +| default | test | type_test_no_copy_0 | `s3://bucket1/test/type_test_no_copy_0/metadata/00001-c3aa9443-...json` | `s3://bucket1/test/type_test_no_copy_0/metadata/00000-b64bfb12-...json` | TABLE | +| default | test | type_test_no_copy_02 | `s3://bucket1/test/type_test_no_copy_02/metadata/00000-6c2832c0-...json` | *(none)* | TABLE | +| default | nyc | taxis_p_by_day | `s3://bucket1/nyc/taxis_p_by_day/metadata/00001-dd00d7f3-...json` | `s3://bucket1/nyc/taxis_p_by_day/metadata/00000-afe27a5c-...json` | TABLE | +| default | nyc | simple_table | `s3://bucket1/nyc/simple_table/metadata/00000-0fc867be-...json` | *(none)* | TABLE | +| default | test | simple_table | `s3://bucket1/test/simple_table/metadata/00000-fe34018d-...json` | *(none)* | TABLE | + +### `iceberg_namespace_properties` + +| catalog_name | namespace | property_key | property_value | +|---|---|---|---| +| default | test.ll2 | exists | true | +| default | nyc | exists | true | + +--- + +## Notes + +- The composite primary key for `iceberg_tables` is (`catalog_name`, `table_namespace`, `table_name`). +- The composite primary key for `iceberg_namespace_properties` is (`catalog_name`, `namespace`, `property_key`). +- Tables with an empty `previous_metadata_location` are at their initial version (version 0). +- The metadata version can be inferred from the filename prefix (e.g. `00003-...` = version 3, meaning 3 commits since creation). +- All metadata files are stored in S3 under the pattern: `s3:////
/metadata/-.metadata.json`. diff --git a/examples/docker-compose/docker-compose-etcd.yaml b/examples/docker-compose/docker-compose-etcd.yaml new file mode 100644 index 00000000..25cd1867 --- /dev/null +++ b/examples/docker-compose/docker-compose-etcd.yaml @@ -0,0 +1,116 @@ +# Iceberg REST catalog with etcd as the metastore (instead of SQLite). +# Usage: docker compose -f docker-compose-etcd.yaml up +services: + etcd: + image: quay.io/coreos/etcd:v3.5.12 + restart: unless-stopped + command: + - etcd + - --name=etcd + - --data-dir=/etcd-data + - --listen-client-urls=http://0.0.0.0:2379 + - --advertise-client-urls=http://etcd:2379 + - --listen-peer-urls=http://0.0.0.0:2380 + - --initial-advertise-peer-urls=http://etcd:2380 + - --initial-cluster=etcd=http://etcd:2380 + - --initial-cluster-token=ice-rest-catalog-etcd + - --initial-cluster-state=new + volumes: + - etcd-data:/etcd-data + healthcheck: + test: ["CMD", "etcdctl", "--endpoints=http://127.0.0.1:2379", "endpoint", "health"] + interval: 5s + timeout: 3s + retries: 10 + # Uncomment to run etcdctl from the host against this cluster: + # ports: + # - "2379:2379" + + minio: + image: minio/minio:RELEASE.2025-03-12T18-04-18Z + restart: unless-stopped + command: [ 'server', '/data', '--address', ':8999', '--console-address', ':9001' ] + environment: + MINIO_ROOT_USER: miniouser + MINIO_ROOT_PASSWORD: miniopassword + ports: + - '8999:8999' # 9000 is taken by clickhouse + - '9001:9001' # web console + volumes: + - minio:/data + minio-init: + image: minio/mc:RELEASE.2025-03-12T17-29-24Z + restart: on-failure # run once and exit + entrypoint: > + /bin/sh -c " + sleep 1; until /usr/bin/mc alias set local http://minio:8999 $$MINIO_ROOT_USER $$MINIO_ROOT_PASSWORD; do echo waiting for minio to start...; sleep 1; done; + /usr/bin/mc mb --ignore-existing local/bucket1; + exit 0; + " + environment: + MINIO_ROOT_USER: miniouser + MINIO_ROOT_PASSWORD: miniopassword + depends_on: + - minio + ice-rest-catalog: + image: altinity/ice-rest-catalog:${ICE_REST_CATALOG_TAG:-debug-with-ice-latest-master@sha256:9f5308309b98d2e0b76346cd29c22557076bfcb0070eb8f32b69b87932b9e37c} + pull_policy: ${ICE_REST_CATALOG_PULL_POLICY:-always} + restart: unless-stopped + ports: + - '5001:5000' # iceberg/http + configs: + - source: ice-rest-catalog-yaml + target: /etc/ice/ice-rest-catalog.yaml + depends_on: + etcd: + condition: service_healthy + minio-init: + condition: service_completed_successfully + clickhouse: + image: altinity/clickhouse-server:25.8.9.20496.altinityantalya-alpine + restart: unless-stopped + environment: + CLICKHOUSE_SKIP_USER_SETUP: "1" # insecure + ports: + - "8123:8123" # clickhouse/http + - "9000:9000" # clickhouse/native + configs: + - source: clickhouse-init + target: /docker-entrypoint-initdb.d/init-db.sh + volumes: + # for access to clickhouse-logs + - ./data/docker-compose/clickhouse/var/log/clickhouse-server:/var/log/clickhouse-server + # - ./config.xml:/etc/clickhouse-server/conf.d/config.xml + depends_on: + - ice-rest-catalog +configs: + clickhouse-init: + content: | + #!/bin/bash + exec clickhouse client --query $" + SET allow_experimental_database_iceberg = 1; + + DROP DATABASE IF EXISTS ice; + + CREATE DATABASE ice + ENGINE = DataLakeCatalog('http://ice-rest-catalog:5000') + SETTINGS catalog_type = 'rest', + auth_header = 'Authorization: Bearer foo', + storage_endpoint = 'http://minio:8999', + warehouse = 's3://bucket1'; + " + ice-rest-catalog-yaml: + content: | + uri: etcd:http://etcd:2379 + warehouse: s3://bucket1 + s3: + endpoint: http://minio:8999 + pathStyleAccess: true + accessKeyID: miniouser + secretAccessKey: miniopassword + region: minio + bearerTokens: + - value: foo +volumes: + minio: + etcd-data: From 805f1b2f54aa36e0e8e841cc9abb8eb2afd6a7ee Mon Sep 17 00:00:00 2001 From: kanthi subramanian Date: Thu, 16 Apr 2026 15:16:04 -0500 Subject: [PATCH 2/3] Added links to documents in docs --- ice-rest-catalog/README.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/ice-rest-catalog/README.md b/ice-rest-catalog/README.md index de2a9d72..e0ee2d05 100644 --- a/ice-rest-catalog/README.md +++ b/ice-rest-catalog/README.md @@ -10,3 +10,12 @@ and then execute `ice-rest-catalog`. That's it. Examples of `.ice-rest-catalog.yaml` (as well as Kubernetes deployment manifests) can be found [here](../examples/). + +## Documentation + +- [Architecture](../docs/architecture.md) -- components, design principles, HA, backup/recovery +- [Kubernetes Setup](../docs/k8s_setup.md) -- k8s deployment with etcd StatefulSet and replicas +- [etcd Cluster Setup](../docs/etcd-cluster-setup.md) -- docker-compose setup with 3-node etcd, data insertion, replication verification, ClickHouse queries +- [GCS Setup](../docs/ice-rest-catalog-gcs.md) -- configuring ice-rest-catalog with Google Cloud Storage +- [etcd Backend Schema](../docs/etcd-backend-schema.md) -- etcd key/value schema (`n/`, `t/` prefixes) and mapping to SQLite +- [SQLite Backend Schema](../docs/sqlite-backend-schema.md) -- SQLite tables (`iceberg_tables`, `iceberg_namespace_properties`) From f1227eec9874759c887db541199b507c1f80abe1 Mon Sep 17 00:00:00 2001 From: kanthi subramanian Date: Fri, 17 Apr 2026 10:36:42 -0500 Subject: [PATCH 3/3] Updated ice cli usage examples --- ice/README.md | 161 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 161 insertions(+) diff --git a/ice/README.md b/ice/README.md index e12bc126..21ccc07e 100644 --- a/ice/README.md +++ b/ice/README.md @@ -2,6 +2,22 @@ A CLI for loading data into Iceberg REST catalogs. +## Table of Contents + +- [Usage](#usage) +- [Examples](#examples) + - [Partitioned Insert](#partitioned-insert) + - [Sorted Insert](#sorted-insert) + - [Compression](#compression) + - [Schema Evolution](#schema-evolution) + - [Delete Partition](#delete-partition) + - [Insert Without Copy](#insert-without-copy) + - [Multiple Files](#multiple-files) + - [Namespace Management](#namespace-management) + - [Inspect](#inspect) + - [S3 with Public Data](#s3-with-public-data) + - [Describe Metadata](#describe-metadata) + ## Usage ```shell @@ -34,3 +50,148 @@ ice delete-table flowers.iris ``` See `ice --help` for more. + +## Examples + +For a full walkthrough (including MinIO and ClickHouse setup), see [examples/scratch](../examples/scratch/README.md). + +### Partitioned Insert + +```shell +# partition by day +ice insert nyc.taxis_p_by_day -p \ + https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet \ + --partition='[{"column":"tpep_pickup_datetime","transform":"day"}]' + +# partition by identity +ice insert flowers.iris_partitioned -p file://iris.parquet \ + --partition='[{"column":"sepal.length","transform":"identity"}]' + +# partition by bucket with custom name +ice insert flowers.iris_bucketed -p file://iris.parquet \ + --partition='[{"column":"variety","transform":"bucket[3]","name":"var_bucket"}]' +``` + +Supported transforms: `identity`, `day`, `month`, `year`, `hour`, `bucket[N]`. + +### Sorted Insert + +```shell +ice insert nyc.taxis_s_by_day -p \ + https://d37ci6vzurychx.cloudfront.net/trip-data/yellow_tripdata_2025-01.parquet \ + --sort='[{"column":"tpep_pickup_datetime"}]' +``` + +### Compression + +```shell +ice insert flowers.iris -p file://iris.parquet --compression=zstd + +ice insert flowers.iris file://iris2.parquet --compression=snappy +``` + +Supported codecs: `zstd`, `snappy`, `gzip`, `lz4`. + +### Schema Evolution + +```shell +# add a column to an existing table +ice alter-table flowers.iris '[{"op":"add_column","name":"extra","type":"string"}]' + +# verify the schema change +ice describe -s flowers.iris +``` + +### Delete Partition + +```shell +# dry run first (default) +ice delete nyc.taxis_p_by_day \ + --partition '[{"name": "tpep_pickup_datetime_day", "values": ["2024-12-31"]}]' + +# actually delete +ice delete nyc.taxis_p_by_day \ + --partition '[{"name": "tpep_pickup_datetime_day", "values": ["2024-12-31"]}]' \ + --dry-run=false +``` + +### Insert Without Copy + +Insert an S3 file by reference (no data copy) when it's already in the warehouse: + +```shell +ice create-table flowers.iris_no_copy --schema-from-parquet=file://iris.parquet +mc cp iris.parquet local/bucket1/flowers/iris_no_copy/ +ice insert flowers.iris_no_copy --no-copy s3://bucket1/flowers/iris_no_copy/iris.parquet +``` + +### Multiple Files + +Pipe a list of files to insert as a single atomic transaction: + +```shell +cat filelist | ice insert flowers.iris -p - +``` + +where `filelist` contains one file path per line. If any file fails, the entire transaction is rolled back. + +### Namespace Management + +```shell +ice create-namespace flowers +ice list-namespaces +ice delete-namespace flowers +``` + +### Inspect + +```shell +# show all tables +ice describe + +# show table with schema, partition spec, and sort order +ice describe -s flowers.iris + +# show everything (schema, properties, metrics) +ice describe -a flowers.iris + +# scan table data +ice scan flowers.iris --limit 10 + +# list data files in current snapshot +ice files flowers.iris + +# list partitions +ice list-partitions nyc.taxis_p_by_day + +# describe a parquet file directly +ice describe-parquet file://iris.parquet +``` + +### S3 with Public Data + +Access public S3 datasets without credentials: + +```shell +ice insert btc.transactions -p \ + --s3-region=us-east-2 --s3-no-sign-request \ + s3://aws-public-blockchain/v1.0/btc/transactions/date=2025-01-01/*.parquet +``` + +### Describe Metadata + +Inspect Iceberg metadata files directly (without a running catalog): + +```shell +# summary (default) +ice describe-metadata /path/to/v3.metadata.json + +# full schema +ice describe-metadata -S /path/to/v3.metadata.json + +# all snapshots +ice describe-metadata --snapshots s3://bucket/metadata/v3.metadata.json + +# everything as JSON +ice describe-metadata -a --json /path/to/v3.metadata.json +```