Parse returns the tree without the protobuf round trip

[kyleconroy]

pg_query_go's Parse decodes protobuf because the parse tree is built in C and that is how it crosses the cgo boundary. oliphant inherited the shape: the Go parser built a tree, ParseToProtobuf marshalled it, and Parse unmarshalled it straight back.

That deep copy is roughly three quarters of Parse's running time, and sqlc pays it on every schema and query file of every run:

input	before	after
one query (150 B)	87 µs, 341 allocs	24 µs, 179 allocs	3.7×
a six-statement query file	214 µs, 845 allocs	62 µs, 428 allocs	3.5×
1.1 MB stress INSERT	278 ms, 67 MB	90 ms, 40 MB	3.1×

Parse now calls parser.ParseToTree. That is oliphant's one deliberate addition to pg_query_go's surface, so PLAN.md § 1 records it as the exception to "the API does not change" and CLAUDE.md's hard rule points at it. ParseToProtobuf is untouched — it is upstream API, and consumers who want the bytes still get them.

The UTF-8 fallback

The round trip survives for the one input class where it is load-bearing rather than redundant. proto3 string fields must be valid UTF-8, so a tree carrying a raw invalid byte fails to marshal and upstream surfaces that as an error from Parse:

SELECT '\xff'   →  string field contains invalid UTF-8

Such a byte can only come from the input verbatim. The scanner already rejects escapes that would synthesize one (E'\xff' raises invalid byte sequence for encoding "UTF8", matching PostgreSQL), and truncateIdentifier is pg_mbcliplen-equivalent, so it never splits a character. utf8.ValidString(input) is therefore a sufficient guard, and it costs about 1 ms on that 1.1 MB input against the 188 ms it saves.

Subtree sharing

One difference cannot be guarded away, so it is made explicit rather than left to be discovered: a direct tree can share subtrees where the round trip deep-copied them. A multi-column assignment points every ResTarget at one MultiAssignRef source, exactly as the C tree does, and serialization has no pointers to preserve:

UPDATE t SET (a, b, c) = (1, 2, 3)

526 sites across the corpus, all of this shape. Callers that only read the tree — which is all pg_query_go's API admits, and all sqlc's convert.go does — cannot tell. proto.Equal cannot either, since it compares by value.

Testing

parser/tree_test.go pins all three properties:

• TestParseToTreeMatchesProtobuf — across the parse corpus, all 46,756 cases that parse, the direct tree is proto.Equal to the round-tripped one and the two paths agree on every error.
• TestParseToTreeInvalidUTF8 — an invalid byte still fails with the protobuf error; valid-UTF-8 inputs that stress the escape and truncation paths still take the fast path and parse.
• TestParseToTreeSharesMultiAssignSource — the sharing above, asserted against the round trip's deep copy, with the trees still equal by value.

go test ./... is green. Checked against the real consumer too: with a replace pointing here, sqlc builds and its whole TestReplay/base corpus passes, so every PostgreSQL golden regenerates identically.

Follow-ups, not in this PR

Scan, Deparse and Summary have the same marshal/unmarshal shape and could take the same treatment. This is deliberately just Parse, which is the one on sqlc's hot path.

---

Generated by Claude Code