diff --git a/CHANGELOG.md b/CHANGELOG.md index dc3e07ff91..4c49b54071 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm - **Remote GFQL sends the resolved strictness level (#1916)**: `gfql_remote()` previously hardcoded its client-side preflight to `strict=False` and sent the server nothing, so the same query was strict locally and loose remotely. The preflight now honors the resolved level, and the request body carries a new `strictness` field (`"strict"` / `"warn"` / `"quiet"`) alongside the existing `engine` field. **Server-side honoring is a server change and is not in this repository**: a server that does not read `strictness` applies its own default, so a non-default level requested remotely warns once, in the same shape as the existing Let/DAG compatibility warning (#1955). A client holding only a `dataset_id` can now also preflight names when `bind(schema=...)` supplied them, since a declared schema is names without data. ### Fixed +- **All-null Boolean `sum()` on `engine='polars-gpu'` now returns integer zero instead of null (#1997)**: cudf-polars 26.02 reports null for an all-null Boolean reduction, while GFQL's documented Boolean aggregate extension follows the Cypher `sum()` empty-input identity and returns `0`. The result normalization now fills only Boolean `sum` before the shared Int64 cast; Boolean `count` and non-Boolean `sum` keep their existing null behavior at this helper boundary. Direct boundary tests pin the positive cell and both negative controls. - **A whole-entity endpoint projection (`RETURN b`) answered a deduplicated node set instead of the openCypher bag (#1994)**: on nodes 1-5 with edges (1,2) (1,3) (2,3) (3,4), `MATCH (a)-->(b) RETURN b` returned 3 rows where openCypher returns 4 — node 3 is bound twice, once from node 1 and once from node 2 — and `MATCH (a)-->(b) RETURN a` returned `[1, 2, 3]` for the 4-row bag `[1, 1, 2, 3]`. Parallel edges made it starker: two 1->2 edges are two matches, but the answer could not represent them at all. It was **silent**, and the engine disagreed with itself: every *property* spelling of the same projection (`RETURN b.id`, and even `RETURN a, b`) was already bag-correct, so only the single whole-entity spelling was wrong. Two independent vetoes sent it to the per-alias node table, which *is* a set: the multiplicity predicate bailed on any bare-alias projected item, and the projection lowering vetoed binding rows whenever the plan had a whole-row output. Fixing the lane alone was not enough — the polars projector could not render a whole entity off a binding-row frame at all, which is why `MATCH (a)-->(b) RETURN a, b` raised `NotImplementedError` on polars while pandas and cuDF answered it. That projector now resolves each alias's `{alias}.{field}` columns through a per-alias view (the polars twin of the pandas `_projection_alias_rows`), so single-entity and multi-entity binding rows render alike and the polars decline is gone. Four scopes are deliberately unchanged: `RETURN DISTINCT b` keeps the node-set lane (DISTINCT asks for exactly that dedup, and the binding-row frame carries sibling-alias columns a lone whole-row output does not functionally determine), a whole-row `WITH` carry into a trailing `MATCH` keeps it too (re-entry cannot yet separate matched from unmatched rows on a duplicated prefix, so #1935 item 1 stays open rather than turning into a decline), a variable-length arm keeps it (its bag is the relationship-unique walk expansion, not the edge bag this lane counts — that shape's own whole-entity/property disagreement is left open rather than swapped for a second unvalidated answer), and a pattern with no relationship has no multiplicity to keep. The seeded fast path recognizes the whole-entity bag lowering and re-expands one destination row per matched edge, so the LDBC IS5 entity shape (200k nodes / 1M edges, pandas, median of 20) stays on the fast lane at 7.8ms against 7.9ms before, rather than the 65ms the general lane costs; it defers on a zero-row bag so the empty-frame dtype contract stays single-sourced in the full path. An unseeded whole-entity scan necessarily gets slower in proportion to the rows it stopped dropping (94ms/199k rows before, 863ms/1.0M rows after). - **Every string predicate over a CATEGORICAL column answered an empty/null result on cuDF where pandas answered rows**: `MATCH (n) WHERE searchAny(n, 'x', {columns: ['cat']}) RETURN n.id` over a categorical `cat` returned 15 rows on pandas and **0 rows on cuDF** — silently, with no warning and no error. A categorical-of-strings is string-VALUED on every engine, but only pandas lends it a `.str` accessor; cuDF raises `AttributeError` on `.str` for a categorical. The predicates' accessor probe read that raise as "this column is not string-valued" and returned the non-string result for the whole column — null, or `False` under the `na=False` that `searchAny` passes, so every row was dropped. `Contains`/`Startswith`/`Endswith`/`Match`/`Fullmatch` now decode a categorical whose CATEGORIES are strings back to its string values before the accessor, which is exact and null-preserving on both engines, and the unguarded `isalpha()`-family predicates take the same path instead of surfacing the raw `AttributeError` cuDF-only. A categorical with NUMERIC or temporal categories is unchanged and still refuses to stringify — `searchAny` keeps declining it on cuDF with a typed `NotImplementedError`, because that rendering diverges pandas↔cuDF. pandas answers are unchanged; polars already declined `searchAny` with explicit `columns=` and is unaffected. - **NULL edge endpoints now follow one identity-resolution contract on all three engines (#1995)**: production answered this both ways -- eight sites implemented "a null never links" while the polars hop's `_keep_edges_with_both_endpoints_resolvable` (#1888 round 6) resolved a NULL endpoint to a NULL node id, so `MATCH (a)-[x]-(b) RETURN count(*)` over a graph with one NULL endpoint answered polars 4, pandas 6, cuDF 6. The contract is now stated in `docs/source/gfql/spec/language.md`: **a NULL id is not a graph identity**, so an edge with a NULL endpoint matches no pattern edge on any surface, from either direction, with a bound or synthesized node table. Input validity is a separate policy: this change preserves permissive DataFrame ingestion and current node-only row scans without declaring NULL-id source rows valid graph nodes, while `OPTIONAL MATCH` NULL bindings remain valid result values. Two defects that were wrong under either endpoint policy are fixed: the polars hop kept a NULL-endpoint edge whose NULL endpoint got no node row, and pandas/cuDF answered the same undirected chain with 2 edges unnamed and 3 edges named. Three kernels enforce endpoint resolution (shared pandas/cuDF `hop`, polars `hop_eager`, polars chain fast path). The seven strict-xfail cells from #1888 rounds 6-7 are removed and replaced by 13 green contract pins plus one non-strict compatibility probe (136 engine-parametrized cells, 39 red at the merge base) over both-sided-NULL, NULL-free, and string-id fixtures. diff --git a/graphistry/compute/gfql/agg_types.py b/graphistry/compute/gfql/agg_types.py index 7fb10f77a5..f58bd8e48f 100644 --- a/graphistry/compute/gfql/agg_types.py +++ b/graphistry/compute/gfql/agg_types.py @@ -129,9 +129,13 @@ def polars_agg_result_cast(func: str, input_dtype: "Optional[pl.DataType]") -> " def polars_conform_agg_dtype(expr: "pl.Expr", func: str, input_dtype: "Optional[pl.DataType]", alias: str) -> "pl.Expr": """Land a polars aggregate on its CONTRACT dtype rather than on its kernel dtype.""" + import polars as pl + target = polars_agg_result_cast(func, input_dtype) if target is None: return expr.alias(alias) + if func == "sum" and input_dtype == pl.Boolean: + expr = expr.fill_null(0) return expr.cast(target).alias(alias) # hygiene-ok: explicit-cast -- polars dtype conversion diff --git a/graphistry/tests/compute/gfql/test_aggregate_type_contract.py b/graphistry/tests/compute/gfql/test_aggregate_type_contract.py index 1e52b1e90b..67dd3c8269 100644 --- a/graphistry/tests/compute/gfql/test_aggregate_type_contract.py +++ b/graphistry/tests/compute/gfql/test_aggregate_type_contract.py @@ -853,6 +853,25 @@ def test_polars_agg_result_cast_fires_only_where_polars_misses_the_contract(): assert cast_to("collect", pl.Boolean) is None +def test_polars_boolean_sum_null_fill_is_scoped_to_exact_boundary(): + """Only Boolean ``sum`` repairs a null aggregate result to Cypher's zero.""" + from graphistry.compute.gfql.agg_types import polars_conform_agg_dtype as conform + + source = pl.DataFrame({"one": [1]}) + + boolean_sum = source.select(conform(pl.lit(None), "sum", pl.Boolean, "out")) + assert boolean_sum.schema["out"] == pl.Int64 + assert boolean_sum["out"][0] == 0 + + boolean_count = source.select(conform(pl.lit(None), "count", pl.Boolean, "out")) + assert boolean_count.schema["out"] == pl.Int64 + assert boolean_count["out"][0] is None + + integer_sum = source.select(conform(pl.lit(None), "sum", pl.Int64, "out")) + assert integer_sum.schema["out"] == pl.Null + assert integer_sum["out"][0] is None + + def test_polars_all_null_literal_is_a_typed_integer_zero(): """A bare ``pl.lit(0)`` is ``Int32`` -- a width neither pandas nor cuDF ever produces, so the all-null substitution would reintroduce the very dtype divergence the cast above removes."""