Skip to content

GEOMESA-3582 Removes BBOX predicates from datastore SQL, adds as feature flag to the connector#3589

Open
cwdobbins wants to merge 3 commits into
locationtech:mainfrom
cwdobbins:feature/sql-refactor
Open

GEOMESA-3582 Removes BBOX predicates from datastore SQL, adds as feature flag to the connector#3589
cwdobbins wants to merge 3 commits into
locationtech:mainfrom
cwdobbins:feature/sql-refactor

Conversation

@cwdobbins

@cwdobbins cwdobbins commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

GEOMESA-3582: consolidate spatial pushdown in the connector

Splits spatial-filter handling cleanly between the two geomesa-trino modules and makes query plans deterministic regardless of how a query arrives.

Background

Previously the datastore rewrote CQL spatial predicates into composite SQL — __X_bbox__ overlap prefilters plus CASE WHEN row-level shortcuts — and the spatial_iceberg connector independently derived the same pruning from either ST_* calls or the recognized bbox-comparison pattern. The same logical query could therefore produce different plans depending on whether it came through the datastore or as hand-written Trino SQL, and near-identical logic was maintained in two places.

1. Datastore emits pure ST_* (commit 1)

TrinoFilterToSQL now emits only row-level ST_* function calls (plus temporal / attribute predicates). Swapped operand forms are normalized column-first via the OGC complement (within(a,b) ⇔ contains(b,a)) so the connector's ST_*(ST_GeomFromBinary(col), literal) recognizer always matches. The connector is now the single place bbox/Z2/XZ2 pushdown is derived, so datastore queries and hand-written Trino SQL get identical treatment and deterministic plans.

2. Connector-side page-source bbox reject (commit 2, configuration-enabled via feature flag)

Emitting pure ST_* removes the datastore's inline CASE WHEN row-level shortcut, which had let a row whose bbox was inside the query rectangle skip the WKB decode. This commit optionally recovers a sound version of that saving entirely inside the connector, with no datastore coupling.

The connector already injects the query envelope as REAL domains on a geometry's __X_bbox__ sub-fields for stat pruning; those domains ride to the worker in the table handle's unenforcedPredicate. SpatialPageSourceProvider wraps the Iceberg page source, reads only those cheap bbox columns, and drops any row whose bbox cannot overlap the
envelope before the geometry column is materialized (via SourcePage.selectPositions), so the engine's exact ST_Intersects/ST_Within residual never decodes those rows' WKB.

  • Sound / non-destructive: only rows failing a necessary condition are dropped, so no matching row is ever removed; the engine's exact predicate still runs on every survivor (this is a pre-filter). Null bbox values are kept; OR-of-envelope degrades to the bounding span. Validated by FilterParityIT and by cross-checking counts against the stock iceberg catalog (no pushdown) across rectangle, triangle, ST_Within, and non-spatial filters.
  • Zero-overhead when inapplicable: non-spatial queries (no bbox domains) delegate unchanged.
  • Opt-in, off by default (geomesa.spatial.bbox-page-filter): its value is inversely proportional to spatial partition quality — on a well-Z2/XZ2-partitioned table, file/row-group pruning already removes most non-matching rows, so the extra bbox-column read can outweigh the saved decodes. Enable it for coarsely-partitioned tables, where row-level rejection catches what file pruning misses.

…ng datastore SQL as limited to ST_* predicates.
…uiting logic to the plugin connector, which shortcuts the more expensive WKB decode during row-filtering, provides additional performance improvement for coarsely-pruned datasets.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant