Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
- **Remote GFQL sends the resolved strictness level (#1916)**: `gfql_remote()` previously hardcoded its client-side preflight to `strict=False` and sent the server nothing, so the same query was strict locally and loose remotely. The preflight now honors the resolved level, and the request body carries a new `strictness` field (`"strict"` / `"warn"` / `"quiet"`) alongside the existing `engine` field. **Server-side honoring is a server change and is not in this repository**: a server that does not read `strictness` applies its own default, so a non-default level requested remotely warns once, in the same shape as the existing Let/DAG compatibility warning (#1955). A client holding only a `dataset_id` can now also preflight names when `bind(schema=...)` supplied them, since a declared schema is names without data.

### Fixed
- **All-null Boolean `sum()` on `engine='polars-gpu'` now returns integer zero instead of null (#1997)**: cudf-polars 26.02 reports null for an all-null Boolean reduction, while GFQL's documented Boolean aggregate extension follows the Cypher `sum()` empty-input identity and returns `0`. The result normalization now fills only Boolean `sum` before the shared Int64 cast; Boolean `count` and non-Boolean `sum` keep their existing null behavior at this helper boundary. Direct boundary tests pin the positive cell and both negative controls.
- **A whole-entity endpoint projection (`RETURN b`) answered a deduplicated node set instead of the openCypher bag (#1994)**: on nodes 1-5 with edges (1,2) (1,3) (2,3) (3,4), `MATCH (a)-->(b) RETURN b` returned 3 rows where openCypher returns 4 — node 3 is bound twice, once from node 1 and once from node 2 — and `MATCH (a)-->(b) RETURN a` returned `[1, 2, 3]` for the 4-row bag `[1, 1, 2, 3]`. Parallel edges made it starker: two 1->2 edges are two matches, but the answer could not represent them at all. It was **silent**, and the engine disagreed with itself: every *property* spelling of the same projection (`RETURN b.id`, and even `RETURN a, b`) was already bag-correct, so only the single whole-entity spelling was wrong. Two independent vetoes sent it to the per-alias node table, which *is* a set: the multiplicity predicate bailed on any bare-alias projected item, and the projection lowering vetoed binding rows whenever the plan had a whole-row output. Fixing the lane alone was not enough — the polars projector could not render a whole entity off a binding-row frame at all, which is why `MATCH (a)-->(b) RETURN a, b` raised `NotImplementedError` on polars while pandas and cuDF answered it. That projector now resolves each alias's `{alias}.{field}` columns through a per-alias view (the polars twin of the pandas `_projection_alias_rows`), so single-entity and multi-entity binding rows render alike and the polars decline is gone. Four scopes are deliberately unchanged: `RETURN DISTINCT b` keeps the node-set lane (DISTINCT asks for exactly that dedup, and the binding-row frame carries sibling-alias columns a lone whole-row output does not functionally determine), a whole-row `WITH` carry into a trailing `MATCH` keeps it too (re-entry cannot yet separate matched from unmatched rows on a duplicated prefix, so #1935 item 1 stays open rather than turning into a decline), a variable-length arm keeps it (its bag is the relationship-unique walk expansion, not the edge bag this lane counts — that shape's own whole-entity/property disagreement is left open rather than swapped for a second unvalidated answer), and a pattern with no relationship has no multiplicity to keep. The seeded fast path recognizes the whole-entity bag lowering and re-expands one destination row per matched edge, so the LDBC IS5 entity shape (200k nodes / 1M edges, pandas, median of 20) stays on the fast lane at 7.8ms against 7.9ms before, rather than the 65ms the general lane costs; it defers on a zero-row bag so the empty-frame dtype contract stays single-sourced in the full path. An unseeded whole-entity scan necessarily gets slower in proportion to the rows it stopped dropping (94ms/199k rows before, 863ms/1.0M rows after).
- **Every string predicate over a CATEGORICAL column answered an empty/null result on cuDF where pandas answered rows**: `MATCH (n) WHERE searchAny(n, 'x', {columns: ['cat']}) RETURN n.id` over a categorical `cat` returned 15 rows on pandas and **0 rows on cuDF** — silently, with no warning and no error. A categorical-of-strings is string-VALUED on every engine, but only pandas lends it a `.str` accessor; cuDF raises `AttributeError` on `.str` for a categorical. The predicates' accessor probe read that raise as "this column is not string-valued" and returned the non-string result for the whole column — null, or `False` under the `na=False` that `searchAny` passes, so every row was dropped. `Contains`/`Startswith`/`Endswith`/`Match`/`Fullmatch` now decode a categorical whose CATEGORIES are strings back to its string values before the accessor, which is exact and null-preserving on both engines, and the unguarded `isalpha()`-family predicates take the same path instead of surfacing the raw `AttributeError` cuDF-only. A categorical with NUMERIC or temporal categories is unchanged and still refuses to stringify — `searchAny` keeps declining it on cuDF with a typed `NotImplementedError`, because that rendering diverges pandas↔cuDF. pandas answers are unchanged; polars already declined `searchAny` with explicit `columns=` and is unaffected.
- **NULL edge endpoints now follow one identity-resolution contract on all three engines (#1995)**: production answered this both ways -- eight sites implemented "a null never links" while the polars hop's `_keep_edges_with_both_endpoints_resolvable` (#1888 round 6) resolved a NULL endpoint to a NULL node id, so `MATCH (a)-[x]-(b) RETURN count(*)` over a graph with one NULL endpoint answered polars 4, pandas 6, cuDF 6. The contract is now stated in `docs/source/gfql/spec/language.md`: **a NULL id is not a graph identity**, so an edge with a NULL endpoint matches no pattern edge on any surface, from either direction, with a bound or synthesized node table. Input validity is a separate policy: this change preserves permissive DataFrame ingestion and current node-only row scans without declaring NULL-id source rows valid graph nodes, while `OPTIONAL MATCH` NULL bindings remain valid result values. Two defects that were wrong under either endpoint policy are fixed: the polars hop kept a NULL-endpoint edge whose NULL endpoint got no node row, and pandas/cuDF answered the same undirected chain with 2 edges unnamed and 3 edges named. Three kernels enforce endpoint resolution (shared pandas/cuDF `hop`, polars `hop_eager`, polars chain fast path). The seven strict-xfail cells from #1888 rounds 6-7 are removed and replaced by 13 green contract pins plus one non-strict compatibility probe (136 engine-parametrized cells, 39 red at the merge base) over both-sided-NULL, NULL-free, and string-id fixtures.
Expand Down
4 changes: 4 additions & 0 deletions graphistry/compute/gfql/agg_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,13 @@ def polars_agg_result_cast(func: str, input_dtype: "Optional[pl.DataType]") -> "
def polars_conform_agg_dtype(expr: "pl.Expr", func: str, input_dtype: "Optional[pl.DataType]",
alias: str) -> "pl.Expr":
"""Land a polars aggregate on its CONTRACT dtype rather than on its kernel dtype."""
import polars as pl

target = polars_agg_result_cast(func, input_dtype)
if target is None:
return expr.alias(alias)
if func == "sum" and input_dtype == pl.Boolean:
expr = expr.fill_null(0)
Comment thread
lmeyerov marked this conversation as resolved.
return expr.cast(target).alias(alias) # hygiene-ok: explicit-cast -- polars dtype conversion


Expand Down
19 changes: 19 additions & 0 deletions graphistry/tests/compute/gfql/test_aggregate_type_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -853,6 +853,25 @@ def test_polars_agg_result_cast_fires_only_where_polars_misses_the_contract():
assert cast_to("collect", pl.Boolean) is None


def test_polars_boolean_sum_null_fill_is_scoped_to_exact_boundary():
"""Only Boolean ``sum`` repairs a null aggregate result to Cypher's zero."""
from graphistry.compute.gfql.agg_types import polars_conform_agg_dtype as conform

source = pl.DataFrame({"one": [1]})

boolean_sum = source.select(conform(pl.lit(None), "sum", pl.Boolean, "out"))
assert boolean_sum.schema["out"] == pl.Int64
assert boolean_sum["out"][0] == 0

boolean_count = source.select(conform(pl.lit(None), "count", pl.Boolean, "out"))
assert boolean_count.schema["out"] == pl.Int64
assert boolean_count["out"][0] is None

integer_sum = source.select(conform(pl.lit(None), "sum", pl.Int64, "out"))
assert integer_sum.schema["out"] == pl.Null
assert integer_sum["out"][0] is None


def test_polars_all_null_literal_is_a_typed_integer_zero():
"""A bare ``pl.lit(0)`` is ``Int32`` -- a width neither pandas nor cuDF ever produces, so the
all-null substitution would reintroduce the very dtype divergence the cast above removes."""
Expand Down
Loading