Skip to content

Extend SQL coverage: triggers, window frames, roles/RLS, GRANT, position(), deferrable FKs, exact numeric literals#174

Open
sanketsahu wants to merge 24 commits into
oguimbal:masterfrom
sanketsahu:master
Open

Extend SQL coverage: triggers, window frames, roles/RLS, GRANT, position(), deferrable FKs, exact numeric literals#174
sanketsahu wants to merge 24 commits into
oguimbal:masterfrom
sanketsahu:master

Conversation

@sanketsahu

@sanketsahu sanketsahu commented Jul 8, 2026

Copy link
Copy Markdown

Intent

This extends pgsql-ast-parser to parse a range of common PostgreSQL syntax that wasn't
yet supported. It was developed while evolving pg-mem toward fuller Postgres
conformance — see the companion PR oguimbal/pg-mem#476, which consumes these AST additions.

Every addition follows the repo's conventions: grammar rule → AST node → toSql
round-trip → spec.

What's added

  • Window frame clausesROWS | RANGE | GROUPS with BETWEEN … AND … and the
    UNBOUNDED PRECEDING/FOLLOWING, CURRENT ROW, <n> PRECEDING/FOLLOWING bounds
    (CallOver.frame, CallOverFrame, FrameBound).
  • CREATE TRIGGER / DROP TRIGGERCREATE [CONSTRAINT] TRIGGER, BEFORE | AFTER | INSTEAD OF, INSERT | UPDATE [OF cols] | DELETE | TRUNCATE (OR-ed), ON table,
    FOR EACH ROW | STATEMENT, WHEN (...), EXECUTE FUNCTION | PROCEDURE f(...)
    (CreateTriggerStatement, TriggerEvent); and DROP TRIGGER [IF EXISTS] name ON table [CASCADE | RESTRICT] (DropTriggerStatement, with onTable).
  • EXECUTE <name>(args) to run a PREPAREd statement (ExecuteStatement). PREPARE
    and DEALLOCATE were already parsed; EXECUTE was the missing piece.
  • TABLESPACE clause on CREATE TABLE (CREATE INDEX already supported it). Physical
    storage is meaningless to most AST consumers, but it appears in pg_dump output.
  • CREATE DOMAIN — a base type with NOT NULL / NULL / CHECK constraints and an
    optional DEFAULT (CreateDomainStatement, DomainConstraint).
  • RETURNS SETOF <type> on CREATE FUNCTION (a setof flag on the statement).
  • #> JSON path operator (complements the existing #>>).
  • Composite-type field access(expr).field, the standard syntax for reading a
    field off a parenthesized composite/row value (e.g. (center).x, (row(1,2)).f1).
    ExprMember.op is widened to include '.'. No grammar ambiguity introduced.
  • MERGE (PG 15+) — MERGE INTO target USING source ON cond WHEN [NOT] MATCHED [AND cond] THEN { UPDATE SET … | DELETE | INSERT [(cols)] VALUES (…) | INSERT DEFAULT VALUES | DO NOTHING }. New MergeStatement / MergeAction AST, merge.ne grammar,
    to-sql + ast-mapper support.
  • Declarative partitioningPARTITION BY {RANGE|LIST|HASH} (cols) on CREATE TABLE,
    and CREATE TABLE child PARTITION OF parent FOR VALUES { FROM (..) TO (..) | IN (..) | WITH (MODULUS m, REMAINDER r) } | DEFAULT. New PartitionBy / PartitionOf /
    PartitionBound* AST; bare MINVALUE/MAXVALUE in a range bound resolved without
    grammar ambiguity.
  • INTERSECT / EXCEPT — the UNION grammar/AST is widened to the full set-operation
    family (SetOperationType: union/intersect/except, each with optional ALL).
  • IS DISTINCT FROM / IS NOT DISTINCT FROM null-safe comparison operators (new
    expr_is alternative; EqualityOperator widened).
  • week / decade / century / millennium units in interval literals (normalized to
    days/years).
  • GROUP BY GROUPING SETS (...) (emitted as a grouping sets call alongside the
    existing rollup/cube calls).
  • position(substring IN string) special syntax (new ExprPosition node). The
    generic position(...) call form is rejected to avoid an ambiguity with IN.
  • RolesCREATE ROLE/CREATE USER, DROP ROLE, SET [SESSION|LOCAL] ROLE,
    RESET.
  • Row-level security DDLCREATE/DROP POLICY, and ALTER TABLE … {ENABLE | DISABLE | FORCE | NO FORCE} ROW LEVEL SECURITY.
  • GRANT / REVOKE — table/sequence privileges, plus ON SCHEMA | DATABASE | FUNCTION | PROCEDURE | ROUTINE, ON ALL {TABLES|SEQUENCES|FUNCTIONS|ROUTINES} IN SCHEMA, and ALTER ROLE / ALTER DEFAULT PRIVILEGES.
  • NOTIFY / LISTEN / UNLISTEN (NotifyStatement, ListenStatement,
    UnlistenStatement).
  • COMMENT ON POLICY / COMMENT ON FUNCTION.
  • ALTER COLUMN ... TYPE ... USING <expr> — the conversion expression is captured
    on AlterColumnSetType.using.
  • CREATE INDEX ... INCLUDE (cols) — covering-index payload columns
    (CreateIndexStatement.include).
  • CREATE FUNCTION ... SECURITY DEFINER | INVOKER (CreateFunctionStatement.security).
  • Comment-only / whitespace-only input now parses to [] rather than throwing,
    matching Postgres accepting an empty command string.
  • Dollar-quoted strings with named tags$tag$…$tag$ (not just $$…$$) now lex as
    string literals, both as expression values (e.g. INSERT … VALUES ($body$…$body$)) and
    as DO/function bodies. The matching close tag is resolved in the lexer.next wrapper
    (a moo rule can't backreference the tag).
  • Array slicingarr[lo:hi], arr[:hi], arr[lo:] (ExprArraySlice with optional
    from/to), complementing the existing arr[i] indexing.
  • DEFERRABLE [INITIALLY DEFERRED | IMMEDIATE] on foreign keys.
  • Optional column list on WITH RECURSIVE (WITH RECURSIVE t AS …).
  • Exact numeric-literal precisionExprInteger/ExprNumeric now carry an optional
    valueText holding the exact source digits. JS numbers lose precision above 2^53 and
    for many decimals, so 9007199254740993 and 1.005 were being corrupted at parse
    time. The field is backward-compatible: it is absent unless the literal actually
    needs it (large ints, fractional/exponent numerics), so existing consumers and ASTs are
    unchanged.

Tests / coverage

  • All new syntax has round-tripping specs (parse → toSql → re-parse is stable, per the
    existing checkTree* harness). 836 tests passing.
  • Each feature was additionally validated against PostgreSQL 16 through pg-mem's
    differential conformance harness (same SQL run on pg-mem and a real Postgres, results
    diffed).

Bundle-size impact

Compiled grammar only grows a little, and there are no new dependencies:

bundle (gzipped) before after delta
pgsql-ast-parser 48.9 KB 55.5 KB +6.6 KB (~+13%)

(measured on the compiled lib/index.js, gzipped, this branch vs upstream master.)

Independence

This PR is self-contained and can be merged and released on its own. The companion pg-mem PR (oguimbal/pg-mem#476) depends on these AST additions (and on this being published), not the other way
around.

sanketsahu and others added 8 commits July 8, 2026 13:11
Adds the frame clause to OVER():
  [ROWS | RANGE | GROUPS] (<bound> | BETWEEN <bound> AND <bound>)
with UNBOUNDED PRECEDING/FOLLOWING, CURRENT ROW and <offset>
PRECEDING/FOLLOWING bounds. Offsets are restricted to integer
literals, parameters or parenthesized expressions (a general expr
would be ambiguous against the keyword bounds).

New AST nodes: CallOver.frame, CallOverFrame, FrameBound - emitted
by toSql and covered by roundtripping specs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- WITH RECURSIVE column list is now optional (columnNames?: Name[] | nil),
  inferred from the seed query by consumers when absent
- add the '#>' json path operator (lexer token + ComparisonOperator)
- SAVEPOINT / RELEASE [SAVEPOINT] / ROLLBACK TO [SAVEPOINT] statements
  with new SavepointStatement / ReleaseSavepointStatement AST nodes and
  RollbackStatement.to; emitted by toSql, covered by roundtripping specs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New ExprPosition AST node (mirrors substring/overlay special-call
handling), emitted by toSql as POSITION(x IN y). The generic call
form 'position(...)' is rejected in the grammar (postgres has no such
form - only the IN syntax), which also removes the parse ambiguity
with 'x IN y' as a boolean argument.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for row-level security:
- CREATE ROLE / CREATE USER (SUPERUSER/LOGIN/BYPASSRLS attributes and
  their NO- forms; PASSWORD and a few common options parsed & ignored;
  USER implies LOGIN) -> CreateRoleStatement
- DROP ROLE / DROP USER (added to drop_what) -> 'drop role'
- SET [SESSION|LOCAL] ROLE name | NONE -> SetRoleStatement
- RESET role/param -> ResetStatement
Unquoted 'none' after SET ROLE disambiguated in the postprocessor.
New AST nodes emitted by toSql, covered by specs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ECURITY

- CREATE POLICY name ON table [AS PERMISSIVE|RESTRICTIVE]
  [FOR ALL|SELECT|INSERT|UPDATE|DELETE] [TO roles] [USING (expr)]
  [WITH CHECK (expr)] -> CreatePolicyStatement
- DROP POLICY [IF EXISTS] name ON table -> DropPolicyStatement
- ALTER TABLE ... {ENABLE|DISABLE|FORCE|NO FORCE} ROW LEVEL SECURITY ->
  TableAlterationRowLevelSecurity
New AST nodes emitted by toSql, covered by specs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Table/sequence GRANT and REVOKE (privileges | ALL, TO/FROM roles,
WITH GRANT OPTION). New GrantStatement / RevokeStatement AST nodes
emitted by toSql, covered by specs. Enables loading dumps and RLS
setup scripts (pg-mem treats them as no-ops - no privilege system).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds DEFERRABLE / NOT DEFERRABLE and INITIALLY DEFERRED / IMMEDIATE to
the FK reference grammar (column- and table-level), surfaced on
TableReference as deferrable / initiallyDeferred and emitted by toSql.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JS numbers lose precision for large integers (>2^53) and can't exactly
represent many decimals, corrupting literals at parse time. Integer and
numeric AST nodes now carry an optional valueText with the exact source
digits - integers only when the value isn't a safe JS integer, numerics
for any fractional/exponent literal (canonicalized so equivalent forms
like .5 and 0.5 match). toSql emits valueText when present. This is the
groundwork for arbitrary-precision bigint/numeric in consumers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sanketsahu and others added 3 commits July 9, 2026 00:54
CREATE [CONSTRAINT] TRIGGER name {BEFORE|AFTER|INSTEAD OF}
{INSERT|UPDATE [OF cols]|DELETE|TRUNCATE [OR ...]} ON table
[FOR EACH {ROW|STATEMENT}] [WHEN (cond)]
EXECUTE {FUNCTION|PROCEDURE} f(args).
New CreateTriggerStatement / TriggerEvent AST, emitted by toSql, specs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DROP TRIGGER in postgres requires the owning table:
  DROP TRIGGER [IF EXISTS] name ON table [CASCADE|RESTRICT]

The generic drop rule modelled it like DROP TABLE (a bare name list),
which rejected the ON clause. Add a dedicated DropTriggerStatement
(name + onTable) with its own grammar rule, mapper and to-sql visitor,
and drop 'drop trigger' from the generic DropStatement union.

Companion to the CREATE TRIGGER support; needed by pg-mem's trigger
execution engine (oguimbal/pg-mem).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sanketsahu sanketsahu changed the title Extend SQL coverage: window frames, roles/RLS policies, GRANT, position(), deferrable FKs, exact numeric literals Extend SQL coverage: triggers, window frames, roles/RLS, GRANT, position(), deferrable FKs, exact numeric literals Jul 8, 2026
sanketsahu and others added 13 commits July 9, 2026 11:58
- EXECUTE <name>[(args)] to run a PREPAREd statement (new
  ExecuteStatement node, grammar, mapper, to-sql). PREPARE and
  DEALLOCATE already parsed; EXECUTE was the missing piece.
- optional TABLESPACE clause on CREATE TABLE (parsed into the existing
  `tablespace` shape; CREATE INDEX already supported it).

Consumed by pg-mem's SQL-level PREPARE/EXECUTE and its tablespace
no-op handling (oguimbal/pg-mem).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CREATE DOMAIN name [AS] type [COLLATE c] [DEFAULT expr]
    [ [CONSTRAINT n] { NOT NULL | NULL | CHECK (expr) } ]*

New CreateDomainStatement / DomainConstraint AST, grammar, mapper and
to-sql. Consumed by pg-mem's domain type (oguimbal/pg-mem).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CREATE FUNCTION ... RETURNS SETOF <type> now parses (a set-returning
function). Adds a `setof` flag on CreateFunctionStatement + to-sql; the
element type is carried in `returns`. Consumed by pg-mem's plpgsql
set-returning functions (oguimbal/pg-mem).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a `.` member operator so a parenthesized expression can have a field
read off it, e.g. `(center).x` or `(row(1,2)).f1`. This is the standard
PostgreSQL syntax for accessing a field of a composite/row value.

- expr.ne: new `expr_paren %dot word` production in expr_member
- ast.ts: ExprMember.op widened to include '.'
- to-sql.ts: member visitor emits `(operand).field` for op '.'

799 parser tests pass; no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses PostgreSQL MERGE (PG 15+):

  MERGE INTO target [AS] alias
  USING source [AS] alias
  ON join_condition
  WHEN [NOT] MATCHED [AND cond] THEN
    { UPDATE SET ... | DELETE | INSERT [(cols)] VALUES (...) | INSERT DEFAULT VALUES | DO NOTHING }
  [...]

- new MergeStatement / MergeAction / MergeAction{Update,Delete,Insert,DoNothing} AST
- merge.ne grammar (reuses update_set_list, insert_expr_list_raw, select_from_subject)
- kw_merge / kw_matched keywords
- to-sql visitor + ast-mapper default recursion (merge)

803 tests pass (+4 merge round-trip specs); no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses PostgreSQL declarative partitioning:

  CREATE TABLE t (...) PARTITION BY {RANGE|LIST|HASH} (cols)
  CREATE TABLE child PARTITION OF parent
    FOR VALUES { FROM (..) TO (..) | IN (..) | WITH (MODULUS m, REMAINDER r) } | DEFAULT

- new PartitionBy / PartitionOf / PartitionBound* AST on CreateTableStatement
- create-table.ne: PARTITION BY clause + a second createtable_statement alternative
  for PARTITION OF; bare MINVALUE/MAXVALUE in a range bound are recognized in
  post-processing (avoids grammar ambiguity with column refs)
- kw_list / kw_hash / kw_modulus / kw_remainder keywords
- to-sql + ast-mapper support (mapper no longer drops a column-less partition child)

810 tests pass (+7 partition round-trip specs); no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the UNION grammar/AST to the full set-operation family:
INTERSECT / INTERSECT ALL / EXCEPT / EXCEPT ALL.

- SelectFromUnion.type widened to a SetOperationType union
- union.ne accepts %kw_intersect / %kw_except alongside %kw_union
- to-sql + ast-mapper handle the new types (same code path as UNION)

814 tests pass (+4 round-trip specs); no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the null-safe comparison operators to the grammar (a new alternative in `expr_is`
producing a binary expr with op 'IS DISTINCT FROM' / 'IS NOT DISTINCT FROM'; the right
operand also accepts a parenthesized expression). EqualityOperator widened accordingly;
to-sql renders the op verbatim.

816 tests pass (+2 specs); no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The interval-literal lexer only recognized year..millisecond units. Adds week, decade,
century, and millennium; they are normalized in buildInterval (week -> 7 days,
decade/century/millennium -> 10/100/1000 years), matching Postgres. Mixed forms like
'2 weeks 3 days' work (normalization happens after the unit-uniqueness check).

816 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses `GROUP BY GROUPING SETS ( element, ... )` where an element is a bare expression,
a parenthesized column group, or an empty group `()` (grand total). Emitted as a
`grouping sets` call in the groupBy list (alongside the existing rollup/cube calls). The
grammar stays unambiguous by leaning on the fact that `(a)` / `(a,b)` already parse as
expressions; only `()` needs a dedicated rule.

- kw_grouping / kw_sets keywords
- select_groupby now takes a group_by_list of grouping_sets_call | expr

816 tests pass; no grammar ambiguity introduced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… INCLUDE, SECURITY, comment tolerance

Parser surface needed to run a full Supabase-style bootstrap + real migrations:

- GRANT/REVOKE on schema/database/function/sequence + ALL ... IN SCHEMA,
  ALTER ROLE, ALTER DEFAULT PRIVILEGES (parsed; engine treats as no-ops)
- NOTIFY / LISTEN / UNLISTEN statements
- COMMENT ON POLICY / FUNCTION
- ALTER COLUMN ... TYPE ... USING <expr>
- CREATE INDEX ... INCLUDE (covering-index payload columns)
- CREATE FUNCTION ... SECURITY DEFINER | INVOKER
- Comment-only / whitespace-only input now parses to [] instead of throwing
  (matches Postgres accepting an empty command string)

Round-trip (to-sql) + ast-mapper handlers added for every new node; specs added.
827 -> 829 parser tests passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dollar-quoted string literals with named tags ($tag$…$tag$, not just $$…$$) now
  lex correctly, both as expression values (e.g. INSERT VALUES) and in DO/function
  bodies. The matching close tag is resolved in the lexer.next wrapper (a moo regex
  can't backreference the tag).
- Array slice syntax: arr[lo:hi], arr[:hi], arr[lo:] (ExprArraySlice) — grammar, AST,
  to-sql round-trip, ast-mapper.

836 parser tests passing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sanketsahu added a commit to sanketsahu/pgsql-ast-parser that referenced this pull request Jul 10, 2026
…#174)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant