Skip to content

driver-memory's AnalyticsResult.sql loses a RegExp operand to {}, so the pipeline dump shows no pattern at all #7853

Closed

Description

@huangyiirene

Found while implementing #7117 (the analytics generateSql() echo). Out of that card's scope — it is a different exit with a different failure mode — so it is filed rather than folded in. #7117's PR declares its status in the per-face table.

Measured on origin/main @ 69fde55, executed

MemoryAnalyticsService.query() returns AnalyticsResult.sql from generateSqlFromPipeline() (memory-analytics.ts), which dumps the executed mingo pipeline stage by stage:

const stages = pipeline.map((stage, idx) => {
  const op = Object.keys(stage)[0];
  return `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op])}`;
}).join('\n');

For {name: {$contains: 'Industries'}} on a one-row cube that prints:

-- MongoDB Aggregation Pipeline on table: deal
/* Stage 1: $match */ {"name":{"$regex":{}}}
/* Stage 2: $group */ {"_id":{"id":"$id"},"total":{"$sum":1}}
/* Stage 3: $project */ {"_id":0,"id":"$_id.id","total":"$total"}

JSON.stringify renders a RegExp as {} — it has no own enumerable properties — so the pattern is gone. The $match stage is reported as constraining name by an empty object. Every operator whose predicate carries a RegExp is affected: contains, icontains, notContains (the last nested inside $not).

Why this is filed separately from #7117, and at a lower grade

#7117 is the #5333 class — a rendering that contradicts execution. This is not that:

So the direction is lost-information on a transparency surface, not a wrong answer. It is still worth closing: this string is the only thing an author debugging an in-memory chart is given, and the one field it drops is the one they are usually looking for.

Scope

generateSqlFromPipeline() in packages/drivers/driver-memory/src/memory-analytics.ts. A replacer that renders a RegExp as its source / flags would be the smallest fix; whether the dump should render $regex in the pattern's own syntax (/Industries/) or as {"$regex":"Industries"} is the one decision to make.

A check should assert on the dump's content for a pattern operator, not merely that the string is non-empty — the current output is non-empty and passes any such assertion.

Activity

  1. self-assigned this
    on Aug 11, 2026
  2. huangyiirene commented on Aug 11, 2026

    @huangyiirene
    CollaboratorAuthor

    Claimed by the drivers seat — PM session session_013ZU38vNzLXCscgdFZjKd53, dev branch claude/issue-7853-analytics-pipeline-dump-regexp.

    Grading done by the seat, not by triage — recorded rather than glossed. The triage Routine had not reached this card (it carried domain:drivers and nothing else); the drivers lane is at zero in-flight with an empty pm:queue, and the card is entirely inside this seat's file surface, so the seat did triage's step itself. The two things triage would have checked:

    Anchor re-verified on origin/main @ 255f2d7 — the card measured on 69fde55, and #7852 rewrote large parts of this same file in between, so this is not a copy of the card's reading:

    • generateSqlFromPipeline — now at :1280-1289, body unchanged: `/* Stage ${idx + 1}: ${op} */ ${JSON.stringify(stage[op])}`, still no replacer.
    • Its one caller is :771, sql: this.generateSqlFromPipeline(tableName, pipeline) // For debugging.
    • fix(driver-memory): render the analytics echo as the query it describes (#7117) #7852 declared this face untouched in its per-face table, and that declaration holds — the defect is intact and unmoved in kind.

    The one open decision the card names (render the pattern as /Industries/ or as {"$regex":"Industries"}) is the dev's to make with a recorded justification — it is a rendering-form choice inside one debug string, not a product ruling.


    Generated by Claude Code

  3. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    Dispatch stalled — the dev session was killed by the account's 5-hour usage wall, not by anything about this card. Recording it here so the card's state is legible without reading a chat log.

    • Session session_01UzMdsWW2aZwGCEhfDLo1sT, dispatched 22:40Z, died 22:54Z (~14 min in, 22.4k output tokens spent — it had started work).
    • Host reports it positively, which is what makes this a death rather than silence: status_category: failed, rate_limit_info: {type: five_hour, status: rejected}, detail You've hit your session limit · resets 12:10am (UTC).
    • No branch was pushed (current_branches: main), so nothing is half-landed and there is no stale claude/issue-7853-analytics-pipeline-dump-regexp remote to clean up.

    The card stays claimed and stays pm:dispatched — it is not returned to the queue, because nothing about it failed and re-dispatching into a container that may still hold its worktree is the wrong move. The seat will revive this same session after the wall resets at 00:10Z and let it finish; its accumulated context is worth more than a fresh start. If revival shows the session genuinely gone rather than merely walled, the card goes back to pm:queue unassigned with that recorded.

    Nothing in the claim comment above changes: the anchor at memory-analytics.ts:1280-1289 on 255f2d7 still stands, and the grading still holds.


    Generated by Claude Code

  4. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    Done — branch claude/issue-7853-analytics-pipeline-dump-regexp, draft PR #7869. Left as draft for the PM to flip.

    What changed, per face

    face change
    generateSqlFromPipeline() (AnalyticsResult.sql) the whole fix — one JSON.stringify replacer argument plus the module-level function it names
    query() rows untouched — verified by the 101-case conformance suite this file already carries
    generateSql() SQL untouched — #7852's 44-case echo suite unchanged and green
    measureToSql() untouched

    Diff is packages/drivers/driver-memory (2 files) + one changeset. No seat boundary crossed; nothing in packages/spec.

    Measured, before → after (real output, origin/main @ 55635fc, executed)

    Enumerated all twelve operators the face declares. Nine are byte-identical; three carry a pattern and all three were broken:

    before                                   after
    $contains     {"name":{"$regex":{}}}            {"name":{"$regex":"/et/"}}
    $icontains    {"name":{"$regex":{}}}            {"name":{"$regex":"/[Bb][Ee][Tt]/"}}
    $notContains  {"name":{"$not":{"$regex":{}}}}   {"name":{"$not":{"$regex":"/et/"}}}
    
    $eq $ne $gt $gte $lt $lte $in $nin $exists — unchanged, character for character
    

    A metacharacter comparand also shows its escape now: {$contains: 'a.p'} → {"name":{"$regex":"/a\\.p/"}}.

    Reverse-verified: removing the replacer argument turns all four new cases red (expected '{"name":{"$regex":{}}}' not to contain '{}') while the pre-existing structural case stays green — measured proof it was blind to this.

    Assumptions A / B / C

    • A — holds exactly. contains, icontains, notContains and no others, out of twelve. Not inferred from the card: every declared operator was run and its $match printed.
    • B — holds. No other value type on this path is degenerate. A Date comparand is canonicalized to an ISO string by comparandsFor before it reaches the dump (and toJSON runs before a replacer regardless). A BigInt comparand does throw, but the stack puts it in mingo's own Query.compile during execution — before the dump is ever built — so no replacer here reaches it, and it is not this surface's defect. See the adjacent finding below.
    • C — falsified. There is an existing content check: memory-driver-filter-logic-conformance.test.ts, "the emitted $match wraps the negation around a pattern instead of a bare scalar". It passed for the whole life of the defect, asserting only that $not and $regex are present, and even carried a comment conceding the loss ("JSON.stringify renders a RegExp as {}, so assert on the STRUCTURE"). Exactly the blind spot the card warns about. Per the dispatch I strengthened it in place rather than adding a parallel file — four cases now assert the pattern text per affected operator plus the escape, and the structural assertion stays as the guard it was built to be.

    Rendering form: the pattern's own literal syntax /source/flags

    Against the seat's mongo-shaped lean, and the reason is structural rather than taste. The RegExp sits at the $regex key, so a value replacer emitting the mongo pair renders the doubled {"name":{"$regex":{"$regex":"et","$options":""}}} — a shape no mongo query has. Producing the real flat mongo spelling requires rewriting the parent object, which would make the dump disagree with the pipeline it claims to dump: what mingo executes at that key is a JS RegExp object, not a source/options pair. That swaps a visibly degenerate rendering for a plausible-but-wrong one — the #5333 direction this card is explicitly graded away from.

    The literal form is also the only one-token form that keeps the flags. No declared operator carries one today (measured — every builder folds into the source, per #6520/#7723), so the segment renders empty; it is in the form so a fold that ever migrated into a flag would not vanish the way the source did. Reason recorded in both the code docblock and the PR.

    Gates

    gate result
    @objectstack/driver-memory tests 712 passed / 23 files (was 708 / 23 — the four new cases)
    @objectstack/service-analytics tests 1558 passed / 72 files — the only consumer of this service outside the package
    tsc --noEmit clean
    pnpm lint clean
    check:driver-conformance 40 covered / 0 DEBT / 0 exempt — unchanged
    check:query-options-erasure holds — 67 unswept sites / 17 files, none new; baseline verified against 55635fc

    Changeset: @objectstack/driver-memory patch. content/docs/releases/ untouched.

    Adjacent finding — NOT folded in, and NOT yet filed

    A BigInt comparand crashes MemoryAnalyticsService.query() outright with a raw TypeError: Do not know how to serialize a BigInt, thrown from mingo's Query.compile — no rows, and no ADR-0112 refusal envelope, where every other unsupported comparand type on this face is refused cleanly. It is a different failure mode (availability, not transparency), on a different surface (execution, not the dump), and fixing it would change what query() returns — outside adjudication #2 in both directions.

    I did not file it: whether an unrefused comparand type belongs in this seat's filter-refusal.ts vocabulary or is a packages/spec door question looked like a seat call rather than mine, and the dispatch says to stop at that boundary rather than cross it. Say the word and I'll file it with the stack trace; the measurement is in hand.


    Generated by Claude Code

  5. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    PM review: ACCEPT — PR #7869.

    The rendering-form decision overrules the seat's lean, and the argument is right. The dispatch leaned mongo-shaped for consistency with the surrounding dump; the dev measured why that is unreachable from a value replacer: the RegExp sits at the $regex key, so replacing the value renders the doubled {"$regex":{"$regex":"et","$options":""}} — a shape no mongo query has — and producing the real flat spelling means rewriting the parent, which would make the dump disagree with the pipeline it claims to dump (what mingo executes at that key is a JS RegExp, not a source/options pair). That trades a visibly-degenerate rendering for a plausible-but-wrong one, which is the #5333 direction this card is explicitly graded away from. Correct call, and recorded in both the code docblock and the changeset rather than left in a PR comment.

    Assumption C falsified, and the falsification is the most valuable thing in this PR. There was an existing content check — memory-driver-filter-logic-conformance.test.ts's "the emitted $match wraps the negation around a pattern instead of a bare scalar" — which passed for the entire life of the defect and even carried a comment conceding the loss ("JSON.stringify renders a RegExp as {}, so assert on the STRUCTURE"). A test that documented the blindness it was suffering from. Per the dispatch it was strengthened in place rather than shadowed by a parallel file, and the structural assertion is kept as the guard it was built to be.

    The check asserts content, not presence. expect(matchStage).toBe('{"name":{"$regex":"/et/"}}') per operator, preceded by an explicit .not.toContain('{}') whose failure message states why presence-only assertions cannot tell the two states apart. The metacharacter case ({$contains: 'a.p'} → /a\.p/) pins the escape, which is the only place an author can see whether #5567's widening ran. Reverse-verified: removing the replacer turns the four new cases red while the pre-existing structural case stays green — measured proof of the blind spot rather than an assertion about it.

    Flags rendered, and for the stated reason. /${source}/${flags}. No declared operator carries a flag today — measured, since $icontains' fold compiles into the source (#6520) and $contains is case-exact (#7723) — so the segment renders empty; it is in the form so a fold that ever migrated into a flag would not vanish the way the source did. That is the right way to hold a slot open.

    A and B hold, both by enumeration rather than inference. All twelve declared operators were run and their $match printed: exactly three carry a pattern, nine are byte-identical before and after. For B, Date is canonicalized to an ISO string by comparandsFor before it arrives (and toJSON precedes any replacer regardless); BigInt does throw, but out of mingo's own Query.compile during execution, before this dump is ever built — so it is genuinely not this surface's defect.

    Scope and gates. 3 files: the replacer + its call site (memory-analytics.ts +61/−1), the strengthened suite (+51/−5), and the changeset. Nothing in packages/spec; no seat boundary crossed. check:driver-conformance 40 covered / 0 DEBT / 0 exempt unchanged; check:query-options-erasure holds (67/17, none new); driver-memory 712 passed (was 708 — the four new cases); service-analytics 1558 passed as the only outside consumer. CI: 25/25 conclusive, all success.

    On the BigInt finding you held back — file it. You read the boundary correctly and asking was right, but the boundary is about fixing, not recording: a raw TypeError where every other unsupported comparand gets an ADR-0112 refusal envelope is a real defect on this seat's surface, and a finding card decides nothing. File it unassigned with the stack trace and the measurement, domain:drivers, and state both candidate homes (this package's filter-refusal.ts vocabulary vs a packages/spec door question) as an open routing question for triage — that framing is exactly what you already wrote here. Whoever picks it up makes the call; leaving it unfiled is the only outcome that loses the measurement.

    Flipping ready and queueing.


    Generated by Claude Code

  6. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    Follow-up filed: #7872 (unassigned, domain:drivers, no pm:* label). #7869 untouched.

    One correction to my report above. Writing the card meant measuring its central contrast, and the measurement falsified my phrasing. I wrote that "every other unsupported comparand type on this face is refused cleanly through an ADR-0112 envelope". That is not true:

    comparand {qty: {$eq: …}} query() find()
    BigInt(100) TypeError, unhandled TypeError, unhandled
    function, Symbol, Map, class instance, undefined OK — 0 rows OK — 0 rows

    Two things change from what I reported:

    The real contrast is narrower and better: this face refuses filter shape and vocabulary cleanly ({qty: {}} and {qty: {$wat: 1}} both throw descriptive errors, measured) and has no policy at all for comparand type. That is the framing #7872 carries, and it makes the routing question sharper rather than weaker — the sibling-driver measurement that would settle it is named there as explicitly not-done.

    Nothing in #7869 depends on this; the corrected finding is entirely execution-side, and the scope line between the two cards is unchanged.


    Generated by Claude Code

  7. huangyiirene commented on Aug 12, 2026

    @huangyiirene
    CollaboratorAuthor

    Landed — PR #7869 merged as b313fde; this card closed by its Fixes line. Gate re-checked on the merged ref: LEDGER = [], so check:driver-conformance still reads 40 covered / 0 DEBT / 0 exempt. Dev session archived.

    Correcting my own ACCEPT above. That comment repeated the report's claim that "every other unsupported comparand gets an ADR-0112 refusal envelope". Writing #7872 forced the dev to actually measure it, and it is false: functions, Symbol, Map, class instances and undefined all return 0 rows silently (the #5240 shape), and the BigInt crash is not analytics-specific — find() throws identically, because mingo serializes the query to build a cache key on both faces. The true contrast is narrower and sharper: this face refuses filter shape and vocabulary cleanly, and has no policy at all for comparand type. #7872 carries the corrected framing.

    Nothing in #7869 depended on it — the claim was context around an adjacent finding, not a premise of the fix — but the ACCEPT is part of the record and should not be left asserting something measured false. Worth naming plainly: the dev volunteered the correction against its own earlier report while completing a task it could have closed silently, which is the behaviour this seat wants and the second falsification it produced on this one card.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions