docs: object storage, parquet_schema field_id, and today's user-facing changes

[jdatcmd]

Documentation backfill for the work that merged today, audited against the SQL surface on main.

What it documents

• A new "Object storage" section in sql-reference.md — the day's headline, and previously undocumented. Covers the three URL schemes (s3://, http://, https://), the separate pgcolumnar_objstore module loaded on first remote use, the full credential model (ambient AWS_* for the function API; server endpoint/region + user-mapping secrets for the FDW; credentials_required and the ambient-is-a-privilege rule), the objstore_allowed_endpoints allow-list with its unconditional link-local refusal, and export to s3:// with the multipart nothing-visible-before-complete property. The export/read/FDW entries and both section intros now point at it.
• parquet_schema.field_id (Review what it would take to support Apache Iceberg #388): signature updated, column described (NULL vs a real 0).
• configuration.md: the allow-list GUC under a new Object storage subsection — it is security-relevant and superuser-only, so it belongs in the config reference.
• CHANGELOG: object-storage read/write, credentials, allow-list, field_id (Added); export atomicity and the batch-fold EXPLAIN report (Fixed).

What it deliberately does not cover

• The pgcolumnar_autovacuum daemon (feat: pgcolumnar_autovacuum — the maintenance daemon autovacuum cannot reach (#415) #624) ships its own docs/administration.md in that PR; it is still open, so documenting it here would duplicate and race.
• recluster self-gate and sort_status owner fix were documented by docs: reference maintenance_due, recluster self-gate, and the sort_status owner fix (#415, #608) #625 (merged).
• Internal/test-only changes (test: red any suite assigning to a bash special, and shellcheck in CI #618 harness guard, feat: fold-path payload deferral with an adaptive gate (#405 step 2) #617 fold deferral behind a default-off GUC, fix: EXPLAIN ANALYZE reports the batch fold that ran, not the prediction (#602) #603 EXPLAIN plumbing) carry no user-facing surface.

Gate

Docs only. No em/en-dash anywhere in the additions; every added sentence is within the 25-word STE limit (five were split to satisfy it); docs_style.sh passes 9/9.

🤖 Generated with Claude Code

[ChronicallyJD]

Reviewed as docs against the SQL and C surface on main, not taken on trust. Every factual claim I could check holds. This is accurate and well-scoped — a comment, with one trivial nit and one heads-up that is mine to handle, not a block.

Verified against main

• parquet_schema signature — RETURNS TABLE(..., field_id integer) matches the doc's 4th column.
• field_id NULL-vs-0 — the reader sets field_id = -1 when absent and emits the column only for field_id >= 0, so 0 really is a distinct real id. The parquet_schema.c comment says as much verbatim. The doc's "a NULL and a 0 mean different things" is correct.
• objstore_allowed_endpoints — PGC_SUSET, default "". Doc's "string, default empty, superuser-only, empty refuses every remote endpoint" matches.
• Link-local — os_addr_is_linklocal refuses 169.254.0.0/16 and fe80::/10 after DNS resolution, unconditionally, with the metadata address called out in the source comment. Doc's "including 169.254.169.254, refused whether or not they are listed" matches, and it is even stronger than stated (a hostname resolving there is caught too).
• Credentials model — server takes endpoint/region only, user-mapping takes the secrets + credentials_required only, any other placement errors at CREATE/ALTER (columnar_parquet_reader.c:4613), and only a superuser may set credentials_required 'false' (:4623). Ambient env is used only for a superuser or a credentials_required 'false' mapping (the function-path caller is the separately-documented function API). All matches.
• AWS_DEFAULT_REGION — actually read as the fallback to AWS_REGION (columnar_objstore_module.c:1336), so the sql-reference "AWS_REGION or AWS_DEFAULT_REGION" is right.
• Export to s3:// — ABI is v3 on main with sink_create and the remote PgColumnarSinkOpen dispatch (#622 merged 00:33Z), so the multipart / nothing-visible-before-complete / failed-export-removes-the-upload text describes real, shipped behavior. The doc correctly does not claim transactionality.
• docs_style.sh — I ran it on the branch: 9/9 PASS. Anchors and relative link paths (#object-storage, sql-reference.md#object-storage, docs/sql-reference.md#object-storage from the root CHANGELOG) all resolve.

One nit (optional)

• The CHANGELOG credentials bullet lists AWS_REGION but not AWS_DEFAULT_REGION, while the sql-reference table lists both. Not wrong — the primary is there — just less complete than the reference entry two files over. Add AWS_DEFAULT_REGION for parity if you touch it again.

Heads-up, mine to fix, not a defect here

• This PR and my still-open #624 both insert at the top of CHANGELOG ### Added. Whichever merges second takes a one-line rebase there. Since #624 is mine, I'll rebase it onto whatever lands first — no action for you.

Scope note: I verified the signatures, GUC contexts, env vars, link-local guard, and credential validation directly in the merged source; the #602 batch-fold-prediction wording I took as consistent with the merged fix and my own prior work on that marker rather than re-running an EXPLAIN.

— ChronicallyJD

[ChronicallyJD]

Re-reviewed at 4005a56. The delta since my last pass is one commit, "keep the configuration intro's superuser-setting list accurate," and it is a correct, complete fix.

• The intro previously said "There are two exceptions." Adding objstore_allowed_endpoints (superuser-only) made that count wrong. The rewrite drops the count and names all the special settings.
• I checked completeness rather than trusting it: across all 32 pgcolumnar.* GUCs on main, exactly three are neither PGC_USERSET nor PGC_SIGHUP — enable_end_truncation (SUSET), objstore_allowed_endpoints (SUSET), and unique_lock_buckets (POSTMASTER). The updated intro names all three and omits none.
• docs_style.sh still passes 9/9 at this head; the new intro sentences are within the STE limit and carry no dash.

Everything from my first review stands (the sql-reference and CHANGELOG content is untouched by this commit). The only open item is the optional AWS_DEFAULT_REGION parity nit in the CHANGELOG, which is fine to leave.

Looks good to me.

— ChronicallyJD