feat: pgcolumnar_autovacuum — the maintenance daemon autovacuum cannot reach (#415)

CHANGELOG.md

@@ -55,6 +55,26 @@ which was true until that script existed.
   directly. It is `SECURITY DEFINER` and checks that the caller may `SELECT` the
   table, so the owner can run it without superuser rights.
 
+- `pgcolumnar.autovacuum`, a maintenance daemon for the online upkeep that core
+  autovacuum cannot reach (#415). pgColumnar's `compact_rewrite` and `recluster`
+  live in extension functions, not table access method callbacks, so core
+  autovacuum never runs them. A table's dead rows and clustering decay then
+  accumulate until someone runs the verbs by hand. This daemon runs them for you.
+
+  It is off by default. When on, a launcher wakes every
+  `pgcolumnar.autovacuum_naptime` seconds (default 60) and starts one worker per
+  database. Each worker asks `pgcolumnar.maintenance_due()` which tables crossed a
+  threshold, then runs the recommended verb over SPI in its own transaction.
+
+  Two properties make it safe unattended. It calls only the
+  `ShareUpdateExclusiveLock` verbs, never `vacuum`, `vacuum_sorted`, or `cluster`,
+  so it cannot block a reader or a writer. And it yields the way autovacuum does:
+  the worker sets `PROC_IS_AUTOVACUUM`, so the lock manager cancels its
+  maintenance the moment a backend queues for a stronger lock. New settings:
+  `pgcolumnar.autovacuum`, `autovacuum_naptime`, `autovacuum_compact_threshold`
+  (0.2), and `autovacuum_recluster_threshold` (0.05). See the administration
+  guide for the operator's view.
+
 - `pgcolumnar.parallel_flush` dispatches a stripe flush across background workers
   (#445). Default off. When on, a flush of two or more columns fans the per-column
   encode and compress work out to a worker pool. Any column a worker does not

Makefile

@@ -29,7 +29,8 @@ OBJS = \
 	src/columnar_parallel_copy.o \
 	src/columnar_parallel_export.o \
 	src/columnar_objstore.o \
-	src/columnar_sink.o
+	src/columnar_sink.o \
+	src/columnar_autovacuum.o
 
 EXTENSION = pgcolumnar
 DATA = pgcolumnar--1.0-alpha.sql pgcolumnar--1.0-dev--1.0-alpha.sql

docs/administration.md

@@ -156,6 +156,36 @@ others work in place and run beside your queries.
 Schedule the exclusive three in a maintenance window. Anything that runs
 unattended, such as a cron entry, should call the online ones.
 
+### The maintenance daemon (pgcolumnar.autovacuum)
+
+pgColumnar's online maintenance verbs, `compact_rewrite` and `recluster`, live
+in extension functions. PostgreSQL's autovacuum never calls them. Without a
+schedule, a table's dead rows and clustering decay accumulate unattended. The
+`pgcolumnar.autovacuum` daemon runs those verbs for you.
+
+It is off by default. When it is on, a launcher wakes every
+`pgcolumnar.autovacuum_naptime` seconds (default 60) and starts one worker per
+database. Each worker asks `pgcolumnar.maintenance_due()` which columnar tables
+have crossed a threshold, then runs the verb it recommends.
+
+The daemon calls only the online `ShareUpdateExclusiveLock` verbs. It never
+calls `vacuum`, `vacuum_sorted`, or `cluster`. So it does not block readers or
+writers. It also yields the way autovacuum does: it cancels its own maintenance
+the moment a statement needs a stronger lock on the table.
+
+```
+-- turn it on (SIGHUP, no restart)
+ALTER SYSTEM SET pgcolumnar.autovacuum = on;
+SELECT pg_reload_conf();
+```
+
+The thresholds are reloadable. `pgcolumnar.autovacuum_compact_threshold` is the
+deleted fraction (default 0.2). `pgcolumnar.autovacuum_recluster_threshold` is
+the appended fraction (default 0.05). A table is reclustered only when it has a
+recorded clustering key, from a prior `recluster` or from
+`set_options(..., sort_by => ...)`. The launcher needs `pgcolumnar` in
+`shared_preload_libraries`, which the extension already requires.
+
 ### Online maintenance and disk reclaim
 
 The online maintenance functions run under ShareUpdateExclusiveLock, so reads and

docs/configuration.md

@@ -79,6 +79,10 @@ disk. It never changes the values that a table returns.
 | --- | --- | --- | --- |
 | `pgcolumnar.reclaim_coalesce` | boolean | `on` | During online compaction, split an oversized freed range on reuse and coalesce adjacent freed ranges, so space is reclaimed under fragmentation. Off reverts to whole-range reuse. |
 | `pgcolumnar.enable_end_truncation` | boolean | `off` | Allow `pgcolumnar.truncate()` to return trailing reclaimed blocks to the operating system. Off makes `pgcolumnar.truncate()` a no-op. Requires superuser to set. |
+| `pgcolumnar.autovacuum` | boolean | `off` | Run the maintenance daemon. When on, it runs `compact_rewrite` and `recluster` on columnar tables that cross a threshold. It uses only `ShareUpdateExclusiveLock` and yields to any stronger lock. It never blocks a reader or a writer. See the [administration guide](administration.md#the-maintenance-daemon-pgcolumnarautovacuum). Reloadable, not a per-session setting. |
+| `pgcolumnar.autovacuum_naptime` | integer | `60` | Seconds between daemon sweeps. Each sweep starts one worker per database. Range 1 to 86400. Reloadable. |
+| `pgcolumnar.autovacuum_compact_threshold` | float | `0.2` | Deleted fraction at which the daemon rewrites a table with `compact_rewrite`. Range 0.0 to 1.0. Reloadable. |
+| `pgcolumnar.autovacuum_recluster_threshold` | float | `0.05` | Appended fraction at which the daemon reclusters a table that has a recorded clustering key. Range 0.0 to 1.0. Reloadable. |
 
 ### Object storage
 
@@ -105,6 +109,7 @@ appears in `pg_settings` and a reader who finds it there deserves an answer.
 | Setting | Type | Default | Description |
 | --- | --- | --- | --- |
 | `pgcolumnar.bulk_parallel_writer` | boolean | `off` | Internal. Set by `pgcolumnar.parallel_copy` loader workers so they skip the storage-row creation lock when the row already exists committed, which is what lets several atomic writers load one table at once. Marked `GUC_NOT_IN_SAMPLE`; leave it alone. Setting it by hand is safe but pointless: the skip only fires when the storage row is already committed, which is exactly when the lock guards nothing. |
+| `pgcolumnar.maintenance_hold_ms` | integer | `0` | Internal, for tests. A maintenance verb holds `ShareUpdateExclusiveLock` this many milliseconds, interruptibly, so a test can observe the daemon yield to a stronger lock. `0` disables it. Range 0 to 600000. Leave it at `0`. |
 
 ## Per-table storage options
 

src/columnar.h

@@ -193,6 +193,13 @@ extern bool pgcolumnar_enable_bloom_filter;	/* bloom equality skipping (I7) */
 extern bool pgcolumnar_objstore_buffered;	/* chunk-granular remote reads (#393) */
 extern char *pgcolumnar_objstore_allowed_endpoints; /* #393 allow-list, SUSET */
 extern int	pgcolumnar_objstore_part_size;	/* #394 remote multipart part size */
+/* #415 autovacuum daemon */
+extern bool pgcolumnar_autovacuum;
+extern int	pgcolumnar_autovacuum_naptime;
+extern double pgcolumnar_autovacuum_compact_threshold;
+extern double pgcolumnar_autovacuum_recluster_threshold;
+extern void PgColumnarAutovacuumRegister(void);
+extern int	pgcolumnar_maintenance_hold_ms;	/* dev/test: hold SUEL this long in a maintenance verb */
 
 /* Phase 6 GUCs (spec 8.3) */
 extern bool pgcolumnar_enable_vectorization;	/* vectorized aggregate path */