fix(gfql): preserve bag multiplicity for whole-entity projections - #2000
Conversation
|
This probably needs to get restacked on 1999 because likely conflicts |
| from graphistry.Engine import is_polars_df | ||
| from graphistry.compute.typing import DataFrameT | ||
|
|
||
| EndpointFrameT = typing.TypeVar("EndpointFrameT") |
There was a problem hiding this comment.
Strange, clarify why we track else drop...
There was a problem hiding this comment.
Kept and clarified in 696327559. The TypeVar preserves the caller's exact pandas/cuDF/Polars frame flavor through the runtime dispatcher; DataFrameT intentionally models pandas during type checking, so replacing the signature with DataFrameT -> DataFrameT would erase Polars return precision. The declaration now carries a single guard-compliant constraint comment. Focused CPU, lint, and mypy are green.
|
I can go either way here. What does gql do vs cypher ? |
…projection `MATCH (a)-->(b) RETURN b` answered the deduplicated node SET instead of the openCypher bag: 3 rows where the 4-edge match bag binds node 3 twice. Silent, and self-inconsistent -- every property spelling of the same projection (`RETURN b.id`, and `RETURN a, b`) was already row-correct. Two vetoes sent it to the per-alias node table: the multiplicity predicate bailed on any bare-alias projected item, and the projection lowering vetoed binding rows whenever the plan carried a whole-row output. Both now admit a whole-entity node output over a relationship pattern. The lane switch alone was not enough. polars could not render a whole entity off a binding-row frame at all -- `MATCH (a)-->(b) RETURN a, b` raised NotImplementedError there while pandas and cuDF answered it. The polars projector now resolves each alias through a per-alias view (the twin of the pandas _projection_alias_rows), so single-entity and multi-entity binding rows render alike, and alias-relative property sources resolve through the same map. Deliberately unchanged: DISTINCT (asks for exactly the node-set dedup), a whole-row WITH carry into re-entry (the trailing MATCH cannot yet separate matched from unmatched rows on a duplicated prefix), and relationship-free patterns. The seeded fast path recognizes the whole-entity bag lowering so the seeded shape stays on the fast lane, and defers on a zero-row bag so the empty-frame dtype contract stays single-sourced in the full path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AjbKuKheqDu78oapRT5AYm
Variable-length arms keep the node-set lane: their openCypher bag is the
relationship-unique walk expansion, not the edge bag this lane counts, and the
polars text fallback cannot render that frame. Excluding them matches the
multiplicity predicate's documented scope and leaves the varlen shape byte-for-
byte as before.
Pins updated, each because the lowering legitimately moved:
- test_alias_scoping_semantics: `RETURN a AS b` pinned the 3-name node set while
its own sibling property spelling pinned the 4-row bag -- it encoded the
defect. Now the bag.
- test_hop_semantics_pins: the polars whole-entity NIE pin carried an AUDIT NOTE
asking for a row-level parity assertion once the gate was narrowed. Flipped to
exactly that.
- test_engine_polars_row_pipeline: `MATCH (n)-[e]->(m) RETURN n, m` moved from
DEFERRED to NATIVE_LOWERED (parity + polars-typed both hold).
- cypher/test_lowering: two bracketless-relationship tests asserted the raw
node-table column names of the node-set lowering; binding rows spell them
"{alias}.{field}". The third pinned a compile-time refusal of a query whose
`root` is out of scope after an aggregating WITH; it is still refused, now at
execution.
- test_row_multiplicity_semantics: the strict-xfail residual is now a real
assertion (2 xfails removed).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AjbKuKheqDu78oapRT5AYm
9de0b68 to
6963275
Compare
|
Recommendation: keep the non-DISTINCT bag behavior. Current Cypher documents plain |
Summary
Correctness oracle
For nodes 1-5 and edges
(1,2) (1,3) (2,3) (3,4),MATCH (a)-->(b) RETURN bnow returnsb.id = [2,3,3,4], including node 3 once per relationship match. Parallel-edge and two-hop fixtures discriminate a bag from a node set. Cross-engine pins cover pandas, Polars, and cuDF; the final DGX canary covers explicit Polars-GPU.Validation
696327559297d8ddcfc65e3955ba2a6e4d8e0404libnvrtc.so.12./bin/lint.sh: clean (Ruff, type-hygiene, comment-density, relative-import gates)./bin/typecheck.sh: 336 source files cleanReview resolution
EndpointFrameTstays: it preserves the caller's pandas/cuDF/Polars frame flavor across the runtime dispatcher;DataFrameTintentionally models pandas during type checking. Commit696327559adds the guard-compliant one-line explanation.RETURNand GQL-conformantRETURN ALLas equivalent and duplicate-preserving;DISTINCTperforms deduplication.Reviewer handoff
Exact state: rebased linear head
696327559297d8ddcfc65e3955ba2a6e4d8e0404onmaster@190b802ba867081f253f21e5c22679de59f7e8f4.Reviewer decisions:
_AliasViewfield mapping for prefixed multi-entity Polars rows, including OPTIONAL-null entity markers and structured versus text rendering.Safe local reproduction:
python -m pytest -q -k 'not cudf' graphistry/tests/compute/gfql/test_whole_entity_projection_bag_1994.py graphistry/tests/compute/gfql/cypher/test_lowering.py graphistry/tests/compute/gfql/test_alias_scoping_semantics.py graphistry/tests/compute/gfql/test_engine_polars_row_pipeline.py graphistry/tests/compute/gfql/test_hop_semantics_pins.py graphistry/tests/compute/gfql/test_row_multiplicity_semantics.py ./bin/lint.sh ./bin/typecheck.shStack / landing
Position: #1999 is landed; #2000 is rebased directly on master. After #2000 lands, rebase successor #2001 onto fresh master before its own checks and merge.
Landing rule: merge commit only, exact-head guard, every check settled green, then ancestry verification. Do not delete this branch until dependent successor rebases are complete.