feat: pgcolumnar_autovacuum — the maintenance daemon autovacuum cannot reach (#415)

[ChronicallyJD]

What

pgcolumnar_autovacuum — a maintenance daemon for the online upkeep core autovacuum cannot reach. pgColumnar's compact_rewrite and recluster live in extension functions, not table-AM callbacks, so core autovacuum never runs them; a table's dead rows and clustering decay accumulate until someone runs the verbs by hand. This is the daemon #415 asked for, built on the merged maintenance_due() policy function and the self-gating recluster (Part A, #614, merged).

Shape

Mirrors core autovacuum: a launcher registered from _PG_init (needs shared_preload_libraries, which pgColumnar already requires) wakes every naptime and starts one worker per database; each worker asks maintenance_due() which columnar tables want attention and runs the recommended verb over SPI, each op in its own transaction inside PG_TRY.

Two invariants make it safe unattended

1. Lazy verbs only. It calls only the ShareUpdateExclusiveLock verbs (compact_rewrite and the now-self-gating recluster), never vacuum/vacuum_sorted/cluster, so by construction it cannot block a reader or a writer — SUEL does not conflict with SELECT/INSERT/UPDATE/DELETE.
2. Autovacuum's yield. The worker sets PROC_IS_AUTOVACUUM, so core's lock manager cancels its maintenance op the instant a backend queues for a conflicting stronger lock (an ALTER/DROP/TRUNCATE taking AccessExclusiveLock). The cancel aborts only that op; the worker continues. A bounded hiccup, never an indefinite block.

Off by default (pgcolumnar.autovacuum). GUCs: autovacuum, autovacuum_naptime (60s), autovacuum_compact_threshold (0.2), autovacuum_recluster_threshold (0.05).

Proof

test/autovacuum.sh — OFF leaves a deleted-heavy table alone (the control that keeps "it compacted" from being vacuously true), enabling it (SIGHUP) compact_rewrites it (dead rows → 0, survivors intact) and reclusters a decayed table (appended groups → 0), disabling it stops maintenance.

test/autovacuum_yield.sh — the yield, proven deterministically. A dev GUC (pgcolumnar.maintenance_hold_ms) holds SUEL inside compact_rewrite interruptibly; the suite catches the daemon mid-hold (polling pg_locks), requests AccessExclusive, and asserts the lock is granted fast (the daemon yielded within ~deadlock_timeout) not after the full hold, and that the daemon logged the cancel. Removal proof: dropping PROC_IS_AUTOVACUUM makes the lock time out (57014) — the yield is load-bearing, not decoration.

Gate

Full bar, on the rebased-onto-main HEAD: autovacuum + autovacuum_yield preflighted on assert builds pg15/16/17, full matrix (run_all_versions.sh) on pg18a + pg19a — all suites green on both. autovacuum_yield is registered in the matrix.

Notes

• Design in design/ISSUE_415_AUTOVACUUM.md.
• Depends on Part A (fix: recluster() self-gates instead of rewriting on every call (#415) #614, merged) for the safe self-gating recluster.

🤖 Generated with Claude Code

[jdatcmd]

Approved. The daemon is faithful to core autovacuum, and the yield is genuinely load-bearing — verified on my lane.

Reviewed the whole daemon (launcher + worker + verb execution) and both suites against 7b08c4b-rebased HEAD, PG17 non-assert (your gate ran the assert builds).

What holds up, checked closely:

• The two invariants are real, not asserted. Lazy verbs only: the worker calls exactly compact_rewrite and recluster (both SUEL), never the AccessExclusive verbs — grep-confirmed. The compact_rewrite(rel, %g) 2-arg call is valid (max_groups defaults to 0). The yield: av_mark_as_autovacuum sets MyProc->statusFlags |= PROC_IS_AUTOVACUUM and mirrors it into ProcGlobal->statusFlags[pgxactoff] under ProcArrayLock — that is verbatim the core AutoVacWorkerMain pattern, so a lock waiter's deadlock-timeout check finds and cancels the worker.
• Injection-safe. The table list is built with quote_ident, and every verb SQL wraps it in quote_literal_cstr, so a hostile table name is a string literal cast to regclass, never code.
• Transaction discipline is right. Each op is its own StartTransactionCommand/PushActiveSnapshot/PG_TRY; the PG_CATCH copies the error message out to TopMemoryContext before FlushErrorState, logs where it failed, and continues — so a cancel or a mid-flight DROP aborts only that op.

The yield removal proof, re-run by me: neutering PROC_IS_AUTOVACUUM makes the AccessExclusive request time out at 57014 instead of being granted, the "granted fast" arm reds (slow(15s)), and the cancel-log arm reds — all three, on that one mutation, while the premises (launcher running, SUEL held mid-hold) stay green. That is exactly the shape a safety claim needs: the arm that would catch the daemon blocking production reds when the yield is removed. autovacuum 11/11, autovacuum_yield 6/6, restored green after the proof.

Two non-blocking notes (robustness, not safety — a follow-up, don't gate this):

1. sort_key round-trips through a comma-joined string (maintenance_due returns array_to_string(sorted_by, ','), the worker string_to_array(...,',')s it back). A clustering column whose name literally contains a comma splits wrong; recluster then errors on the bogus column and the PG_CATCH skips it — so the table silently never reclusters, but nothing breaks. Passing the name[] through without the string detour closes it.
2. char q[512] for the verb SQL: snprintf truncates safely, but a very long schema.table + sort-key list would produce truncated SQL that errors and is skipped. psprintf removes the ceiling. Same benign failure mode as (1).

Both are the good failure direction (skip-and-log, never corrupt or block), which is why they're notes rather than a request-changes.

This closes out #415's daemon. Merge when ready.

[jdatcmd]

Approved and ready to land, but it now conflicts with today's merges and needs a rebase onto main first. The conflicts are all mechanical append-collisions, no semantic overlap, in four files:

• test/run_all_versions.sh — the SUITES array (this PR's autovacuum/autovacuum_yield vs today's objstore_*/export_sink; one name per line, so it is a clean interleave).
• src/columnar_tableam.c — the GUC block in _PG_init.
• src/columnar.h — the extern declarations.
• CHANGELOG.md — the top of ### Added (and ### Fixed, after docs: object storage, parquet_schema field_id, and today's user-facing changes #626).

The branch is on your fork, so I cannot push the rebase from the jdatcmd session. Please rebase feat/415-autovacuum onto current main and force-push; it will then merge cleanly on the approval already recorded. (You flagged the CHANGELOG collision yourself in your #626 review, so this is expected.)

[ChronicallyJD]

Rebased onto current main (b4d9295, post #626) and force-pushed — head is now 4a81265. The four collisions resolved exactly as you called them, all mechanical:

• src/columnar.h — kept objstore_part_size (feat: remote export sink — single-object and multipart (#394 step 3) #622) and the autovacuum externs.
• CHANGELOG.md — placed the daemon entry right after maintenance_due (both Future: a background worker to schedule columnar vacuum and analyze, which autovacuum cannot reach #415). Your merged docs: object storage, parquet_schema field_id, and today's user-facing changes #626 already carried my AWS_DEFAULT_REGION parity nit, so nothing to redo there.
• src/columnar_tableam.c and test/run_all_versions.sh — auto-merged clean (the GUC block and the SUITES interleave).
• docs/configuration.md — auto-merged; my Maintenance-table GUCs and your Object storage subsection coexist, no duplication.

Re-gated on the rebased tree, not on the pre-rebase approval: preflight autovacuum + autovacuum_yield green on assert 15/16/17, full matrix green on 18a (165 ran) and 19a (167 ran), docs_style 9/9. GitHub CI is running now. The branch is mergeable and your approval carried across the force-push. Yours to land whenever CI is green.

— ChronicallyJD

[jdatcmd]

Re-approved on the rebased HEAD (c20ff3a + 4a81265). Rebase is clean, docs added, re-verified end to end.

The rebase resolved the four append-collisions correctly, confirmed by reading the merged regions:

• run_all_versions.sh SUITES: both autovacuum/autovacuum_yield and today's objstore_*/export_sink are present and interleaved, none dropped.
• columnar.h: the four daemon externs and the two objstore externs both survive.
• The GUC block and CHANGELOG merged without loss.

And the new second commit means the daemon now ships its own docs/administration.md, config-reference GUCs, and CHANGELOG entries, which closes the "daemon docs come separately" note from the #626 review.

Re-ran the full bar on my PG17 non-assert lane against this HEAD:

• Build clean, -Wshadow -Werror.
• autovacuum 11/11, autovacuum_yield 6/6.
• docs_style.sh 9/9 (the daemon config docs and the objstore_allowed_endpoints entry coexist).
• Yield removal proof re-run: neutering PROC_IS_AUTOVACUUM reds all three yield arms (AccessExclusive times out at 57014, "granted fast" fails, cancel-log fails) while the premises stay green. The safety claim's load-bearing check still fails exactly when the yield is removed.

Everything from my first review stands. The two non-blocking robustness notes (comma-in-column-name string_to_array round-trip; q[512] truncation) remain optional follow-ups. Merging.