CREATE TABLE AS into a columnar table fails when the source plan is parallel, which is the default at scale

[ChronicallyJD]

CREATE TABLE ... USING pgcolumnar AS SELECT ... fails whenever the planner picks a
parallel plan for the source query:

ERROR:  cannot update SecondarySnapshot during a parallel operation

I hit this doing something ordinary, building a clustered copy of the TSBS table on the
bench to measure the question docs/benchmarks.md leaves open.

It fails by default, and only at scale

Nothing is set here. Stock cluster, stock session:

=# EXPLAIN (COSTS OFF) CREATE TABLE zz USING pgcolumnar AS SELECT * FROM cpu_pgc ORDER BY hostname, time;
 Gather Merge
   Workers Planned: 6
   ->  Sort
         ->  Parallel Custom Scan (ColumnarScan) on cpu_pgc

Small sources get a serial plan and work. Large ones get a parallel plan and fail. So
the failure arrives exactly when someone bulk loads a real table, which is the main way
a columnar table gets created from existing data.

Minimal repro

Zeroing the parallel costs makes a small table go parallel, so this runs in seconds:

CREATE TABLE src AS SELECT g AS id, 'h'||(g%97) AS host FROM generate_series(1,200000) g;

SET parallel_setup_cost=0; SET parallel_tuple_cost=0;
SET min_parallel_table_scan_size=0; SET max_parallel_workers_per_gather=2;

CREATE TABLE t USING pgcolumnar AS SELECT * FROM src ORDER BY host;
ERROR:  cannot update SecondarySnapshot during a parallel operation

What is and is not affected

I checked the plan of each case rather than trusting the result, because a green run
proves nothing if the plan was never parallel.

case	plan	result
CTAS into heap, same forced costs	Gather Merge, 2 workers	ok
CTAS into pgcolumnar	Gather Merge, 2 workers	ERROR
CTAS into pgcolumnar, max_parallel_workers_per_gather=0	serial	ok
CTAS into pgcolumnar, WITH NO DATA, then INSERT ... SELECT	serial insert	ok
plain parallel SELECT from a columnar table	Gather	ok
parallel CTAS reading a columnar table into a heap	Gather Merge	ok

The first two rows are the finding. Same plan, same costs, heap works and columnar does
not, so it is the target and not the query.

One row I want to be careful about. INSERT INTO <columnar> SELECT ... ORDER BY
also succeeds, but when I checked its plan it had no Gather at all:

 Insert on tgt_p
   ->  Sort
         ->  Seq Scan on src_p

So INSERT ... SELECT is untested here, not proven safe. I could not get PG18 to
build a parallel plan under it for a columnar target.

Where it comes from

backtrace_functions='GetLatestSnapshot', frames resolved with addr2line:

columnar_tuple_insert                src/columnar_tableam.c:638
ColumnarWriteRow                     src/columnar_write_state.c:794
columnar_flush_row_group             src/columnar_write_state.c:1369
ColumnarInsertNativeStorageRow       src/columnar_metadata.c:1543

Line 1543 is the re-check after taking the storage-row creation advisory lock:

	PushActiveSnapshot(GetLatestSnapshot());
	snapshot = ColumnarCatalogSnapshot(GetActiveSnapshot());

GetLatestSnapshot() raises this exact error under IsInParallelMode(). The comment
above it explains why the fresh snapshot is needed, and that reasoning looks right to
me: the loser of a cross-transaction first-write race has to see the winner's committed
row or it fails on storage_pkey. The problem is only that the call is unconditional,
and CTAS runs the whole executor inside parallel mode even though the leader does the
writing.

Thoughts on a fix, not a proposal

I have not written one, and I would rather not guess at the concurrency reasoning in
that function.

Worth noting though that in the CTAS case the relation is created by the same statement,
so no other session can see it, and the race the fresh snapshot defends against cannot
happen. There is also already precedent for making this conditional: the
columnar_bulk_parallel_writer fast path directly above does exactly that for
pgcolumnar.parallel_copy.

Happy to take it if you want, but you know that lock's history better than I do.

Environment

PostgreSQL 18, pgcolumnar 1.0-dev at author/main, on the bench. I have not checked
whether 15 through 17 behave the same.

Workaround

SET max_parallel_workers_per_gather = 0 for the load, or create the table empty and
INSERT ... SELECT into it.

[jdatcmd]

Plan, with two of your open questions answered first

It is not PG18-only. All majors.

You said you had not checked 15 through 17. Reproduced with your forced-cost repro,
plan captured each time so a green run cannot be a serial one:

major	plan	columnar CTAS	heap CTAS (control)	columnar CTAS serial (control)
PG18	Gather Merge, 2 workers	ERROR cannot update SecondarySnapshot	ok, 200,000	ok, 200,000
PG15	Gather Merge, 2 workers	ERROR, identical	ok, 200,000	ok, 200,000

So this is not a recent regression and there is no version gate to hide behind.

INSERT ... SELECT confirmed unreachable, for a reason

Your caution was right and the reason is structural. With the same forced costs the
plan has no Gather at all:

 Insert on t4
   ->  Sort
         Sort Key: src.host

PostgreSQL will not put a Gather under a writing ModifyTable here, so that path cannot
enter parallel mode and cannot hit this. CTAS is different because the leader consumes
Gather output and does the writing itself. CTAS is the whole exposure, which makes
the fix narrower than the title suggests.

The fix I would write

Your instinct is right and I want to state the condition more precisely, because
"skip it in parallel mode" is not safe and is the tempting version.

IsInParallelMode() is the wrong discriminator. It is true for CTAS, where the
relation is new, and it would also be true for any future path that writes to an
existing table under a parallel plan. There the first-writer race is real: two
sessions can both be first to write a committed-but-empty columnar table, and skipping
the fresh-snapshot re-check would reintroduce exactly the storage_pkey failure the
concurrent differential suite found.

The safe condition is "no other session can see this relation", which for CTAS is
true by construction because the relation is created by the same statement. Postgres
records that: rd_createSubid != InvalidSubTransactionId on the Relation, the standard
test for created-in-this-transaction.

So:

• thread the relation (or a boolean derived from it) into
  PgColumnarInsertNativeStorageRow, which today takes only NativeStorageMetadata;
• when the relation is new in this transaction, skip both the advisory lock and the
  GetLatestSnapshot() re-check, and insert directly. No race exists to defend against,
  and no fresh snapshot is needed;
• otherwise the current path is untouched.

This leaves a non-new relation written under parallel mode still erroring. That is
deliberate: it is unreachable today, and if some future change makes it reachable I
would rather it error loudly than silently weaken the lock. The precedent you spotted,
pgcolumnar_bulk_parallel_writer, is the same shape but a different justification (the
coordinator pre-committed the row); this one rests on invisibility, not pre-creation.

Verification, in the order I would do it

1. Repro as a test. test/native_ctas.sh (new suite, registered), forced parallel
   costs, asserting the plan is parallel before asserting the CTAS succeeds. Without
   the plan assertion the test passes on a serial plan and proves nothing, which is the
   trap the issue's own table avoids.
2. Removal proof. Revert the condition, confirm the new check fails. Nothing counts
   until it fails with the fix removed.
3. The race must still be serialized. The guard exists because concurrent first
   writes broke storage_pkey. Re-run concurrent_diff and unique_conc in native
   mode, and add a direct two-session first-write race on a committed empty columnar
   table, which is the case the new condition must not touch.
4. Content, not just exit status. CTAS output compared row-for-row against the heap
   equivalent, including a clustered ORDER BY, since building a clustered copy is what
   you were doing when you hit it.
5. Five-major matrix, since the bug is on all five.

Scope note

docs/limitations.md should carry the workaround until this lands, because "set
max_parallel_workers_per_gather=0 for the CTAS" is not discoverable from the error.

Happy for you to take it. If you would rather I did, say so and I will start from the
repro above.

[ChronicallyJD]

Suggesting you keep this one

You reproduced it on all majors and answered the question I had left open, so you are
ahead of me on it already.

More to the point, the fix touches the storage-row creation advisory lock, and that lock
has a history: it exists for the concurrent first-write race, and #300 pivoted away from
2PC because holding it to transaction end deadlocks. Deciding what may be skipped in
parallel mode needs that context. I said on the issue that you know it better than I do,
and I still think so.

I am staying off it. Shout if you would rather hand it back.

[jdatcmd]

Taking this, per your offer. Fix is written and the five-major matrix is running; PR shortly.

The condition I landed on is creation, not parallel mode, for the reason in my plan above: parallel mode says nothing about whether a second writer exists, so keying on it would restore the storage_pkey race on a committed empty table. rd_createSubid is the invisibility test, and I deliberately did not use rd_newRelfilelocatorSubid, since a rewritten relation is still visible to other sessions in its previous form.

I will ask for your review on the PR. No benchmark needed for this one.