Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/styles/config/vocabularies/docs/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Opendata
APIs
(?i)datapoint
(?i)datapoints
(?i)queryable
dataclass
subtask
subtasks
Expand Down
7 changes: 7 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ Common page flow patterns in this repo:
3. Step-by-step sections for procedures.
4. `Next steps` links/cards at the end.

When documenting behavior shared by Tilebox clients and the CLI:

1. State the behavior once as a Tilebox capability. Treat support across Python, Go, and the CLI as the default instead of enumerating each interface in prose.
2. Let code examples show interface-specific syntax. Use `CodeGroup` when every alternative is a single code snippet. Use `Tabs` only when alternatives contain other elements such as text, tables, or multiple code blocks.
3. Name a specific client only when documenting a real difference, limitation, version requirement, or unavailable feature. Put syntax-only guidance inside the relevant example tab when possible.
4. Keep introductions to examples brief. State the feature facts and let the code demonstrate the mechanics instead of describing each snippet line by line.

For command examples in user guides, optimize for reader copy/paste rather than fully scripted automation. Prefer direct `tilebox` commands and simple shell setup commands such as `cd`, `mkdir`, and `export`. Avoid Bash command substitution, uppercase helper variables such as `RELEASE_ID` or `JOB_ID`, `jq` pipelines, heredocs, and `--json` unless the page is explicitly about structured output or automation. When a later command needs a value returned by an earlier command, show the earlier command output and use a placeholder such as `<JOB_ID>` in follow-up commands.

## Terminology, Capitalization, And Naming
Expand Down
16 changes: 14 additions & 2 deletions api-reference/go/datasets/Create.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,9 @@ func (datasetClient) Create(

Create a dataset with the given code name, display name, schema kind, and custom fields.

Fields created with `field.String`, `field.Bool`, `field.Int32`, `field.Int64`, `field.Uint64`, or `field.Float64` can be
marked queryable. Queryable fields cannot be repeated, and a dataset can contain at most two queryable string fields.

## Parameters

<ParamField path="kind" type="datasets.DatasetKind" required>
Expand Down Expand Up @@ -44,6 +47,12 @@ Create a dataset with the given code name, display name, schema kind, and custom
Set the dataset's markdown description.
</ParamField>

## Field options

<ParamField path="Queryable()">
Make the field available for server-side custom field filters. Choose queryable fields before ingesting datapoints.
</ParamField>

## Dataset kinds

<ParamField path="datasets.KindTemporal" type="datasets.DatasetKind">
Expand All @@ -64,10 +73,13 @@ dataset, err := client.Datasets.Create(ctx,
"my_catalog",
"My catalog",
[]datasets.Field{
field.String("source").Description("Source system"),
field.Float64("cloud_cover"),
field.String("source").Description("Source system").Queryable(),
field.Float64("cloud_cover").Queryable(),
},
datasets.WithSummary("Scenes prepared for analysis"),
)
```
</RequestExample>

See [Queryable fields](/datasets/concepts/datasets#queryable-fields) for schema constraints and
[Filter by custom fields](/datasets/query/filter-by-fields) for query syntax.
31 changes: 28 additions & 3 deletions api-reference/go/datasets/CreateOrUpdate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ func (datasetClient) CreateOrUpdate(
Create a dataset or update an existing dataset if a dataset with the given `codeName` already exists.

If the dataset already exists, Tilebox applies the same schema update rules as a direct update. New fields can be added to non-empty datasets. Breaking schema changes are only allowed for empty datasets.
New queryable fields and changes to an existing field's queryable annotation are also only allowed for empty datasets.

## Parameters

Expand Down Expand Up @@ -66,6 +67,9 @@ If the dataset already exists, Tilebox applies the same schema update rules as a
<ParamField path="field.Bool(name string)" type="Field">
A boolean field
</ParamField>
<ParamField path="field.Int32(name string)" type="Field">
A 32-bit signed integer field
</ParamField>
<ParamField path="field.Int64(name string)" type="Field">
A 64-bit signed integer field
</ParamField>
Expand Down Expand Up @@ -99,6 +103,26 @@ If the dataset already exists, Tilebox applies the same schema update rules as a
<ParamField path="ExampleValue(exampleValue string)">
Set the example value of the field for documentation purposes
</ParamField>
<ParamField path="SourceJSONPointer(sourceJSONPointer string)">
Optional. Set the RFC 6901 path to this field in the source JSON, such as `/properties/eo:cloud_cover`. This is useful
when transforming datapoints to JSON because Tilebox can reconstruct nested source objects from flattened dataset
fields.
</ParamField>
<ParamField path="Queryable()">
Make a non-repeated `String`, `Bool`, `Int32`, `Int64`, `Uint64`, or `Float64` field available for server-side custom
field filters. A dataset can contain at most two queryable string fields.
</ParamField>
<ParamField path="JSONSchemaRef(jsonSchemaRef string)">
Optional. Set a JSON Schema reference URI or URI fragment for the field. Use this when the field follows a well-known
schema, such as a STAC extension. Tilebox emits the reference as `$ref` when advertising the field in STAC queryables.
</ParamField>
<ParamField path="Roles(roles ...FieldRole)">
Set semantic display roles for the field. The only currently supported role is `field.RolePrimaryTitle`.
</ParamField>

Queryable string values can contain at most 1,024 Unicode code points. See
[Queryable fields](/datasets/concepts/datasets#queryable-fields) for schema constraints and
[Filter by custom fields](/datasets/query/filter-by-fields) for query syntax.

## Returns

Expand All @@ -111,9 +135,10 @@ dataset, err := client.Datasets.CreateOrUpdate(ctx,
"my_catalog",
"My catalog",
[]datasets.Field{
field.String("field1"),
field.Int64("field2").Repeated(),
field.Geometry("field3").Description("Field 3").ExampleValue("Value 3"),
field.String("platform").Queryable(),
field.Float64("cloud_cover").Queryable(),
field.Int64("shape").Repeated(),
field.Geometry("footprint").Description("Source product footprint"),
},
datasets.WithSummary("Scenes prepared for analysis"),
)
Expand Down
6 changes: 6 additions & 0 deletions api-reference/go/datasets/Datapoints.Query.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ The output sequence can be transformed into a typed `proto.Message` using [Colle
<ParamField path="WithCollectionIDs(collectionIDs ...uuid.UUID)">
Restrict the query to specific dataset collections by collection ID.
</ParamField>
<ParamField path="WithFilters(filters ...query.Expression)">
Filter by fields marked queryable in the dataset schema. Multiple expressions are combined with each other and with
temporal, spatial, and collection filters using logical AND.
</ParamField>
<ParamField path="WithSkipData()" default="false">
Skip the data when querying datapoints.
If set, only the required and auto-generated fields will be returned.
Expand Down Expand Up @@ -82,3 +86,5 @@ datapoints, err := datasets.CollectAs[*v1.Sentinel1Sar](
)
```
</RequestExample>

See [Filter by custom fields](/datasets/query/filter-by-fields) for comparisons, boolean expressions, and null checks.
6 changes: 6 additions & 0 deletions api-reference/go/datasets/Datapoints.QueryInto.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ QueryInto is a convenience function for [Query](/api-reference/go/datasets/Datap
<ParamField path="WithCollectionIDs(collectionIDs ...uuid.UUID)">
Restrict the query to specific dataset collections by collection ID.
</ParamField>
<ParamField path="WithFilters(filters ...query.Expression)">
Filter by fields marked queryable in the dataset schema. Multiple expressions are combined with each other and with
temporal, spatial, and collection filters using logical AND.
</ParamField>
<ParamField path="WithSkipData()" default="false">
Skip the data when querying datapoints.
If set, only the required and auto-generated fields will be returned.
Expand Down Expand Up @@ -84,3 +88,5 @@ err := client.Datapoints.QueryInto(ctx,
)
```
</RequestExample>

See [Filter by custom fields](/datasets/query/filter-by-fields) for comparisons, boolean expressions, and null checks.
8 changes: 8 additions & 0 deletions api-reference/go/datasets/Datapoints.QueryPage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ Use `QueryPage` when you need manual pagination. Use [`Datapoints.Query`](/api-r
<ParamField path="datasets.WithCollectionIDs(collectionIDs ...uuid.UUID)">
Restrict the query to specific dataset collections by collection ID.
</ParamField>
<ParamField path="datasets.WithFilters(filters ...query.Expression)">
Filter by fields marked queryable in the dataset schema. Multiple expressions are combined with each other and with
temporal, spatial, and collection filters using logical AND.
</ParamField>
<ParamField path="datasets.WithSkipData()" default="false">
Skip datapoint data and return only required and generated fields.
</ParamField>
Expand All @@ -62,6 +66,7 @@ page, err := client.Datapoints.QueryPage(ctx,
dataset.ID,
datasets.WithTemporalExtent(queryInterval),
datasets.WithCollectionIDs(collection.ID),
datasets.WithFilters(query.Field("quality").GreaterThanOrEqual(80)),
datasets.WithLimit(100),
)
if err != nil {
Expand All @@ -73,6 +78,7 @@ if page.NextCursor != nil {
dataset.ID,
datasets.WithTemporalExtent(queryInterval),
datasets.WithCollectionIDs(collection.ID),
datasets.WithFilters(query.Field("quality").GreaterThanOrEqual(80)),
datasets.WithCursor(page.NextCursor),
datasets.WithLimit(100),
)
Expand All @@ -81,3 +87,5 @@ if page.NextCursor != nil {
}
```
</RequestExample>

See [Filter by custom fields](/datasets/query/filter-by-fields) for comparisons, boolean expressions, and null checks.
17 changes: 15 additions & 2 deletions api-reference/go/datasets/Update.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,9 @@ func (datasetClient) Update(

Update an existing dataset by ID with the given code name, display name, schema kind, custom fields, and metadata.

New non-queryable fields can be added to a non-empty dataset. Changing whether an existing field is queryable or adding
a new queryable field is only supported while the dataset is empty.

## Parameters

<ParamField path="id" type="uuid.UUID" required>
Expand Down Expand Up @@ -48,6 +51,13 @@ Update an existing dataset by ID with the given code name, display name, schema
Set the dataset's markdown description.
</ParamField>

## Field options

<ParamField path="Queryable()">
Make a non-repeated `String`, `Bool`, `Int32`, `Int64`, `Uint64`, or `Float64` field available for server-side custom
field filters. A dataset can contain at most two queryable string fields.
</ParamField>

## Returns

The updated dataset object.
Expand All @@ -60,11 +70,14 @@ dataset, err := client.Datasets.Update(ctx,
"my_catalog",
"My catalog",
[]datasets.Field{
field.String("source").Description("Source system"),
field.Float64("cloud_cover"),
field.String("source").Description("Source system").Queryable(),
field.Float64("cloud_cover").Queryable(),
field.Timestamp("processed_at"),
},
datasets.WithDescription("Catalog of scenes prepared for analysis."),
)
```
</RequestExample>

See [Queryable fields](/datasets/concepts/datasets#queryable-fields) for schema constraints and
[Filter by custom fields](/datasets/query/filter-by-fields) for query syntax.
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ def Client.create_or_update_dataset(

Create a dataset, or update the existing dataset with the same code name.

New non-queryable fields can be added to a non-empty dataset. Changing whether an existing field is queryable or adding
a new queryable field is only supported while the dataset is empty.

## Parameters

<ParamField path="kind" type="DatasetKind">
Expand Down Expand Up @@ -89,6 +92,26 @@ Note that the type can also be a list of one of the types, indicating that the f
<ParamField path="example_value" type="str">
Set the example value of the field for documentation purposes
</ParamField>
<ParamField path="source_json_pointer" type="str | None">
Optional. Set the RFC 6901 path to this field in the source JSON, such as `/properties/eo:cloud_cover`. This is useful
when transforming datapoints to JSON because Tilebox can reconstruct nested source objects from flattened dataset
fields.
</ParamField>
<ParamField path="queryable" type="bool">
Make the field available for server-side custom field filters. Queryable fields must be non-repeated `str`, `bool`,
`int`, `np.uint64`, or `float` fields. A dataset can contain at most two queryable string fields.
</ParamField>
<ParamField path="json_schema_ref" type="str | None">
Optional. Set a JSON Schema reference URI or URI fragment for the field. Use this when the field follows a well-known
schema, such as a STAC extension. Tilebox emits the reference as `$ref` when advertising the field in STAC queryables.
</ParamField>
<ParamField path="roles" type='list[FieldRole] | list[Literal["primary_title"]]'>
Set semantic display roles for the field. The only currently supported role is `primary_title`.
</ParamField>

Queryable string values can contain at most 1,024 Unicode code points. See
[Queryable fields](/datasets/concepts/datasets#queryable-fields) for schema constraints and
[Filter by custom fields](/datasets/query/filter-by-fields) for query syntax.

## Returns

Expand All @@ -107,18 +130,24 @@ dataset = client.create_or_update_dataset(
code_name="my_catalog",
fields=[
{
"name": "field1",
"name": "platform",
"type": str,
"queryable": True,
},
{
"name": "cloud_cover",
"type": float,
"queryable": True,
},
{
"name": "field2",
"name": "shape",
"type": list[int],
},
{
"name": "field3",
"name": "footprint",
"type": Geometry,
"description": "Field 3",
"example_value": "Value 3",
"description": "Source product footprint",
"example_value": "POLYGON ((11 46, 12 46, 12 47, 11 47, 11 46))",
},
],
name="My personal catalog",
Expand Down
8 changes: 8 additions & 0 deletions api-reference/python/tilebox.datasets/Collection.query.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ icon: layer-group
def Collection.query(
*,
temporal_extent: TimeIntervalLike,
filter: Expression | None = None,
spatial_extent: SpatialFilterLike | None = None,
skip_data: bool = False,
show_progress: bool | Callable[[float], None] = False,
Expand All @@ -33,6 +34,11 @@ If no data exists for the requested time or interval, an empty `xarray.Dataset`

</ParamField>

<ParamField path="filter" type="Expression | None">
Optional expression over fields marked queryable in the dataset schema. Build expressions with `field()` and combine
them with `&`, `|`, and `~`. See [Filter by custom fields](/datasets/query/filter-by-fields).
</ParamField>

<ParamField path="spatial_extent" type="SpatialFilterLike | None">
Optional spatial filter. Use this for spatial queries in spatio-temporal datasets.
</ParamField>
Expand All @@ -52,6 +58,7 @@ An [`xarray.Dataset`](/sdks/python/xarray) containing the requested data points.
<RequestExample>
```python Python
from datetime import datetime
from tilebox.datasets import field
from tilebox.datasets.query import TimeInterval

# querying a specific time
Expand All @@ -72,6 +79,7 @@ data = collection.query(
data = collection.query(
temporal_extent=interval,
spatial_extent=geometry,
filter=(field("cloud_cover") < 10) & (field("platform") == "sentinel-2c"),
)

# querying a time interval with TimeInterval
Expand Down
14 changes: 14 additions & 0 deletions api-reference/python/tilebox.datasets/Dataset.query.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ def Dataset.query(
*,
collections: list[str] | list[UUID] | list[Collection] | list[CollectionInfo] | list[CollectionClient] | dict[str, CollectionClient] | None = None,
temporal_extent: TimeIntervalLike,
filter: Expression | None = None,
spatial_extent: SpatialFilterLike | None = None,
skip_data: bool = False,
show_progress: bool | Callable[[float], None] = False,
Expand Down Expand Up @@ -37,6 +38,11 @@ If no data matches the filters, an empty `xarray.Dataset` is returned.
The time or time interval to query. This can be a single time scalar, a tuple of two time scalars, or a `TimeInterval` object.
</ParamField>

<ParamField path="filter" type="Expression | None">
Optional expression over fields marked queryable in the dataset schema. Build expressions with `field()` and combine
them with `&`, `|`, and `~`. See [Filter by custom fields](/datasets/query/filter-by-fields).
</ParamField>

<ParamField path="spatial_extent" type="SpatialFilterLike | None">
Optional spatial filter. Use this for spatial queries in spatio-temporal datasets.
</ParamField>
Expand Down Expand Up @@ -69,11 +75,19 @@ An [`xarray.Dataset`](/sdks/python/xarray) containing matching datapoints.

<RequestExample>
```python Python
from tilebox.datasets import field

# query all collections in the dataset
data = dataset.query(
temporal_extent=("2025-04-01", "2025-05-01"),
)

# query using custom fields marked queryable in the dataset schema
data = dataset.query(
temporal_extent=("2026-07-20", "2026-07-28"),
filter=(field("cloud_cover") < 1) & (field("platform") == "sentinel-2c"),
)

# query selected collections by name
data = dataset.query(
collections=["S2A_S2MSI2A", "S2B_S2MSI2A"],
Expand Down
Loading