Extend SQL coverage: triggers, window frames, roles/RLS, GRANT, position(), deferrable FKs, exact numeric literals#174
Open
sanketsahu wants to merge 24 commits into
Open
Extend SQL coverage: triggers, window frames, roles/RLS, GRANT, position(), deferrable FKs, exact numeric literals#174sanketsahu wants to merge 24 commits into
sanketsahu wants to merge 24 commits into
Conversation
Adds the frame clause to OVER(): [ROWS | RANGE | GROUPS] (<bound> | BETWEEN <bound> AND <bound>) with UNBOUNDED PRECEDING/FOLLOWING, CURRENT ROW and <offset> PRECEDING/FOLLOWING bounds. Offsets are restricted to integer literals, parameters or parenthesized expressions (a general expr would be ambiguous against the keyword bounds). New AST nodes: CallOver.frame, CallOverFrame, FrameBound - emitted by toSql and covered by roundtripping specs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- WITH RECURSIVE column list is now optional (columnNames?: Name[] | nil), inferred from the seed query by consumers when absent - add the '#>' json path operator (lexer token + ComparisonOperator) - SAVEPOINT / RELEASE [SAVEPOINT] / ROLLBACK TO [SAVEPOINT] statements with new SavepointStatement / ReleaseSavepointStatement AST nodes and RollbackStatement.to; emitted by toSql, covered by roundtripping specs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New ExprPosition AST node (mirrors substring/overlay special-call handling), emitted by toSql as POSITION(x IN y). The generic call form 'position(...)' is rejected in the grammar (postgres has no such form - only the IN syntax), which also removes the parse ambiguity with 'x IN y' as a boolean argument. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Foundation for row-level security: - CREATE ROLE / CREATE USER (SUPERUSER/LOGIN/BYPASSRLS attributes and their NO- forms; PASSWORD and a few common options parsed & ignored; USER implies LOGIN) -> CreateRoleStatement - DROP ROLE / DROP USER (added to drop_what) -> 'drop role' - SET [SESSION|LOCAL] ROLE name | NONE -> SetRoleStatement - RESET role/param -> ResetStatement Unquoted 'none' after SET ROLE disambiguated in the postprocessor. New AST nodes emitted by toSql, covered by specs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ECURITY
- CREATE POLICY name ON table [AS PERMISSIVE|RESTRICTIVE]
[FOR ALL|SELECT|INSERT|UPDATE|DELETE] [TO roles] [USING (expr)]
[WITH CHECK (expr)] -> CreatePolicyStatement
- DROP POLICY [IF EXISTS] name ON table -> DropPolicyStatement
- ALTER TABLE ... {ENABLE|DISABLE|FORCE|NO FORCE} ROW LEVEL SECURITY ->
TableAlterationRowLevelSecurity
New AST nodes emitted by toSql, covered by specs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Table/sequence GRANT and REVOKE (privileges | ALL, TO/FROM roles, WITH GRANT OPTION). New GrantStatement / RevokeStatement AST nodes emitted by toSql, covered by specs. Enables loading dumps and RLS setup scripts (pg-mem treats them as no-ops - no privilege system). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds DEFERRABLE / NOT DEFERRABLE and INITIALLY DEFERRED / IMMEDIATE to the FK reference grammar (column- and table-level), surfaced on TableReference as deferrable / initiallyDeferred and emitted by toSql. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
JS numbers lose precision for large integers (>2^53) and can't exactly represent many decimals, corrupting literals at parse time. Integer and numeric AST nodes now carry an optional valueText with the exact source digits - integers only when the value isn't a safe JS integer, numerics for any fractional/exponent literal (canonicalized so equivalent forms like .5 and 0.5 match). toSql emits valueText when present. This is the groundwork for arbitrary-precision bigint/numeric in consumers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
CREATE [CONSTRAINT] TRIGGER name {BEFORE|AFTER|INSTEAD OF}
{INSERT|UPDATE [OF cols]|DELETE|TRUNCATE [OR ...]} ON table
[FOR EACH {ROW|STATEMENT}] [WHEN (cond)]
EXECUTE {FUNCTION|PROCEDURE} f(args).
New CreateTriggerStatement / TriggerEvent AST, emitted by toSql, specs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DROP TRIGGER in postgres requires the owning table: DROP TRIGGER [IF EXISTS] name ON table [CASCADE|RESTRICT] The generic drop rule modelled it like DROP TABLE (a bare name list), which rejected the ON clause. Add a dedicated DropTriggerStatement (name + onTable) with its own grammar rule, mapper and to-sql visitor, and drop 'drop trigger' from the generic DropStatement union. Companion to the CREATE TRIGGER support; needed by pg-mem's trigger execution engine (oguimbal/pg-mem). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- EXECUTE <name>[(args)] to run a PREPAREd statement (new ExecuteStatement node, grammar, mapper, to-sql). PREPARE and DEALLOCATE already parsed; EXECUTE was the missing piece. - optional TABLESPACE clause on CREATE TABLE (parsed into the existing `tablespace` shape; CREATE INDEX already supported it). Consumed by pg-mem's SQL-level PREPARE/EXECUTE and its tablespace no-op handling (oguimbal/pg-mem). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CREATE DOMAIN name [AS] type [COLLATE c] [DEFAULT expr]
[ [CONSTRAINT n] { NOT NULL | NULL | CHECK (expr) } ]*
New CreateDomainStatement / DomainConstraint AST, grammar, mapper and
to-sql. Consumed by pg-mem's domain type (oguimbal/pg-mem).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CREATE FUNCTION ... RETURNS SETOF <type> now parses (a set-returning function). Adds a `setof` flag on CreateFunctionStatement + to-sql; the element type is carried in `returns`. Consumed by pg-mem's plpgsql set-returning functions (oguimbal/pg-mem). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a `.` member operator so a parenthesized expression can have a field read off it, e.g. `(center).x` or `(row(1,2)).f1`. This is the standard PostgreSQL syntax for accessing a field of a composite/row value. - expr.ne: new `expr_paren %dot word` production in expr_member - ast.ts: ExprMember.op widened to include '.' - to-sql.ts: member visitor emits `(operand).field` for op '.' 799 parser tests pass; no grammar ambiguity introduced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses PostgreSQL MERGE (PG 15+):
MERGE INTO target [AS] alias
USING source [AS] alias
ON join_condition
WHEN [NOT] MATCHED [AND cond] THEN
{ UPDATE SET ... | DELETE | INSERT [(cols)] VALUES (...) | INSERT DEFAULT VALUES | DO NOTHING }
[...]
- new MergeStatement / MergeAction / MergeAction{Update,Delete,Insert,DoNothing} AST
- merge.ne grammar (reuses update_set_list, insert_expr_list_raw, select_from_subject)
- kw_merge / kw_matched keywords
- to-sql visitor + ast-mapper default recursion (merge)
803 tests pass (+4 merge round-trip specs); no grammar ambiguity introduced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses PostgreSQL declarative partitioning:
CREATE TABLE t (...) PARTITION BY {RANGE|LIST|HASH} (cols)
CREATE TABLE child PARTITION OF parent
FOR VALUES { FROM (..) TO (..) | IN (..) | WITH (MODULUS m, REMAINDER r) } | DEFAULT
- new PartitionBy / PartitionOf / PartitionBound* AST on CreateTableStatement
- create-table.ne: PARTITION BY clause + a second createtable_statement alternative
for PARTITION OF; bare MINVALUE/MAXVALUE in a range bound are recognized in
post-processing (avoids grammar ambiguity with column refs)
- kw_list / kw_hash / kw_modulus / kw_remainder keywords
- to-sql + ast-mapper support (mapper no longer drops a column-less partition child)
810 tests pass (+7 partition round-trip specs); no grammar ambiguity introduced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extends the UNION grammar/AST to the full set-operation family: INTERSECT / INTERSECT ALL / EXCEPT / EXCEPT ALL. - SelectFromUnion.type widened to a SetOperationType union - union.ne accepts %kw_intersect / %kw_except alongside %kw_union - to-sql + ast-mapper handle the new types (same code path as UNION) 814 tests pass (+4 round-trip specs); no grammar ambiguity introduced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the null-safe comparison operators to the grammar (a new alternative in `expr_is` producing a binary expr with op 'IS DISTINCT FROM' / 'IS NOT DISTINCT FROM'; the right operand also accepts a parenthesized expression). EqualityOperator widened accordingly; to-sql renders the op verbatim. 816 tests pass (+2 specs); no grammar ambiguity introduced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The interval-literal lexer only recognized year..millisecond units. Adds week, decade, century, and millennium; they are normalized in buildInterval (week -> 7 days, decade/century/millennium -> 10/100/1000 years), matching Postgres. Mixed forms like '2 weeks 3 days' work (normalization happens after the unit-uniqueness check). 816 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parses `GROUP BY GROUPING SETS ( element, ... )` where an element is a bare expression, a parenthesized column group, or an empty group `()` (grand total). Emitted as a `grouping sets` call in the groupBy list (alongside the existing rollup/cube calls). The grammar stays unambiguous by leaning on the fact that `(a)` / `(a,b)` already parse as expressions; only `()` needs a dedicated rule. - kw_grouping / kw_sets keywords - select_groupby now takes a group_by_list of grouping_sets_call | expr 816 tests pass; no grammar ambiguity introduced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… INCLUDE, SECURITY, comment tolerance Parser surface needed to run a full Supabase-style bootstrap + real migrations: - GRANT/REVOKE on schema/database/function/sequence + ALL ... IN SCHEMA, ALTER ROLE, ALTER DEFAULT PRIVILEGES (parsed; engine treats as no-ops) - NOTIFY / LISTEN / UNLISTEN statements - COMMENT ON POLICY / FUNCTION - ALTER COLUMN ... TYPE ... USING <expr> - CREATE INDEX ... INCLUDE (covering-index payload columns) - CREATE FUNCTION ... SECURITY DEFINER | INVOKER - Comment-only / whitespace-only input now parses to [] instead of throwing (matches Postgres accepting an empty command string) Round-trip (to-sql) + ast-mapper handlers added for every new node; specs added. 827 -> 829 parser tests passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Dollar-quoted string literals with named tags ($tag$…$tag$, not just $$…$$) now lex correctly, both as expression values (e.g. INSERT VALUES) and in DO/function bodies. The matching close tag is resolved in the lexer.next wrapper (a moo regex can't backreference the tag). - Array slice syntax: arr[lo:hi], arr[:hi], arr[lo:] (ExprArraySlice) — grammar, AST, to-sql round-trip, ast-mapper. 836 parser tests passing. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sanketsahu
added a commit
to sanketsahu/pgsql-ast-parser
that referenced
this pull request
Jul 10, 2026
…#174) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
This extends
pgsql-ast-parserto parse a range of common PostgreSQL syntax that wasn'tyet supported. It was developed while evolving pg-mem toward fuller Postgres
conformance — see the companion PR oguimbal/pg-mem#476, which consumes these AST additions.
Every addition follows the repo's conventions: grammar rule → AST node →
toSqlround-trip → spec.
What's added
ROWS | RANGE | GROUPSwithBETWEEN … AND …and theUNBOUNDED PRECEDING/FOLLOWING,CURRENT ROW,<n> PRECEDING/FOLLOWINGbounds(
CallOver.frame,CallOverFrame,FrameBound).CREATE TRIGGER/DROP TRIGGER—CREATE [CONSTRAINT] TRIGGER,BEFORE | AFTER | INSTEAD OF,INSERT | UPDATE [OF cols] | DELETE | TRUNCATE(OR-ed),ON table,FOR EACH ROW | STATEMENT,WHEN (...),EXECUTE FUNCTION | PROCEDURE f(...)(
CreateTriggerStatement,TriggerEvent); andDROP TRIGGER [IF EXISTS] name ON table [CASCADE | RESTRICT](DropTriggerStatement, withonTable).EXECUTE <name>(args)to run a PREPAREd statement (ExecuteStatement). PREPAREand DEALLOCATE were already parsed; EXECUTE was the missing piece.
TABLESPACEclause onCREATE TABLE(CREATE INDEXalready supported it). Physicalstorage is meaningless to most AST consumers, but it appears in
pg_dumpoutput.CREATE DOMAIN— a base type withNOT NULL/NULL/CHECKconstraints and anoptional
DEFAULT(CreateDomainStatement,DomainConstraint).RETURNS SETOF <type>onCREATE FUNCTION(asetofflag on the statement).#>JSON path operator (complements the existing#>>).(expr).field, the standard syntax for reading afield off a parenthesized composite/row value (e.g.
(center).x,(row(1,2)).f1).ExprMember.opis widened to include'.'. No grammar ambiguity introduced.MERGE(PG 15+) —MERGE INTO target USING source ON cond WHEN [NOT] MATCHED [AND cond] THEN { UPDATE SET … | DELETE | INSERT [(cols)] VALUES (…) | INSERT DEFAULT VALUES | DO NOTHING }. NewMergeStatement/MergeActionAST,merge.negrammar,to-sql+ast-mappersupport.PARTITION BY {RANGE|LIST|HASH} (cols)onCREATE TABLE,and
CREATE TABLE child PARTITION OF parent FOR VALUES { FROM (..) TO (..) | IN (..) | WITH (MODULUS m, REMAINDER r) } | DEFAULT. NewPartitionBy/PartitionOf/PartitionBound*AST; bareMINVALUE/MAXVALUEin a range bound resolved withoutgrammar ambiguity.
INTERSECT/EXCEPT— the UNION grammar/AST is widened to the full set-operationfamily (
SetOperationType: union/intersect/except, each with optionalALL).IS DISTINCT FROM/IS NOT DISTINCT FROMnull-safe comparison operators (newexpr_isalternative;EqualityOperatorwidened).days/years).
GROUP BY GROUPING SETS (...)(emitted as agrouping setscall alongside theexisting
rollup/cubecalls).position(substring IN string)special syntax (newExprPositionnode). Thegeneric
position(...)call form is rejected to avoid an ambiguity withIN.CREATE ROLE/CREATE USER,DROP ROLE,SET [SESSION|LOCAL] ROLE,RESET.CREATE/DROP POLICY, andALTER TABLE … {ENABLE | DISABLE | FORCE | NO FORCE} ROW LEVEL SECURITY.GRANT/REVOKE— table/sequence privileges, plusON SCHEMA | DATABASE | FUNCTION | PROCEDURE | ROUTINE,ON ALL {TABLES|SEQUENCES|FUNCTIONS|ROUTINES} IN SCHEMA, andALTER ROLE/ALTER DEFAULT PRIVILEGES.NOTIFY/LISTEN/UNLISTEN(NotifyStatement,ListenStatement,UnlistenStatement).COMMENT ON POLICY/COMMENT ON FUNCTION.ALTER COLUMN ... TYPE ... USING <expr>— the conversion expression is capturedon
AlterColumnSetType.using.CREATE INDEX ... INCLUDE (cols)— covering-index payload columns(
CreateIndexStatement.include).CREATE FUNCTION ... SECURITY DEFINER | INVOKER(CreateFunctionStatement.security).[]rather than throwing,matching Postgres accepting an empty command string.
$tag$…$tag$(not just$$…$$) now lex asstring literals, both as expression values (e.g.
INSERT … VALUES ($body$…$body$)) andas
DO/function bodies. The matching close tag is resolved in thelexer.nextwrapper(a
moorule can't backreference the tag).arr[lo:hi],arr[:hi],arr[lo:](ExprArraySlicewith optionalfrom/to), complementing the existingarr[i]indexing.DEFERRABLE [INITIALLY DEFERRED | IMMEDIATE]on foreign keys.WITH RECURSIVE(WITH RECURSIVE t AS …).ExprInteger/ExprNumericnow carry an optionalvalueTextholding the exact source digits. JS numbers lose precision above 2^53 andfor many decimals, so
9007199254740993and1.005were being corrupted at parsetime. The field is backward-compatible: it is absent unless the literal actually
needs it (large ints, fractional/exponent numerics), so existing consumers and ASTs are
unchanged.
Tests / coverage
toSql→ re-parse is stable, per theexisting
checkTree*harness). 836 tests passing.differential conformance harness (same SQL run on pg-mem and a real Postgres, results
diffed).
Bundle-size impact
Compiled grammar only grows a little, and there are no new dependencies:
pgsql-ast-parser(measured on the compiled
lib/index.js, gzipped, this branch vs upstreammaster.)Independence
This PR is self-contained and can be merged and released on its own. The companion pg-mem PR (oguimbal/pg-mem#476) depends on these AST additions (and on this being published), not the other way
around.