diff --git a/CHANGELOG.md b/CHANGELOG.md index 8dd23d64..fb0dbb57 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,15 @@ which was true until that script existed. ### Added +- `pgcolumnar.maintenance_due(rel, compact_due_fraction, recluster_due_fraction)` + reports whether an online maintenance verb is worth running on a table, from its + statistics alone (#415). It takes no lock and rewrites nothing. It returns the + deleted and appended fractions, whether `compact_rewrite` or `recluster` has + crossed its threshold, and a `recommendation`. This is the policy the + `pgcolumnar.autovacuum` daemon consults, and a monitoring role can call it + directly. It is `SECURITY DEFINER` and checks that the caller may `SELECT` the + table, so the owner can run it without superuser rights. + - `pgcolumnar.parallel_flush` dispatches a stripe flush across background workers (#445). Default off. When on, a flush of two or more columns fans the per-column encode and compress work out to a worker pool. Any column a worker does not @@ -85,6 +94,13 @@ which was true until that script existed. ### Fixed +- `pgcolumnar.sort_status` works for a non-superuser who owns the table (#608). + The function reads pgColumnar's internal catalogs, which carry no `GRANT`. As an + invoker-rights function it therefore failed for any caller who was not a + superuser. It is now `SECURITY DEFINER` and checks that the caller may `SELECT` + the table. The owner can read the sort status of their own table, and no caller + gains access to a table they could not already read. + - `pgcolumnar.analyze()` counts `null_frac` over live rows (#485). It came from the zone maps, which record what was written, so a deleted row kept counting toward the denominator until the table was rewritten. `VACUUM` did not correct @@ -180,6 +196,15 @@ which was true until that script existed. ### Changed +- `pgcolumnar.recluster` no longer rewrites a table that is already clustered by + the requested key (#415). The function records the clustering key and kind it + establishes. A later call with the same key returns 0 and touches nothing when + the existing sorted run still covers every row group. Before this, it re-sorted + on every call, so a scheduled recluster rewrote the whole table each time, which + is why the maintenance daemon could not have run it safely. `pgcolumnar.sort_status` + now reports this recorded key as `sort_key`, and falls back to the declared + `sort_by` when there is no recorded key. + - A columnar scan whose filter cannot be pushed down now skips decoding the projected columns of a 1024-row vector that holds no matching row (#452). The scan decodes the filter columns first, rules out the vectors with no match, and diff --git a/docs/sql-reference.md b/docs/sql-reference.md index a6ed57a4..41ccd44c 100644 --- a/docs/sql-reference.md +++ b/docs/sql-reference.md @@ -129,6 +129,12 @@ reclustered. SELECT pgcolumnar.recluster('events', 'customer_id', 'ts'); ``` +Re-running `recluster` with the same key on a table whose clustering is intact is +a fast no-op. The function records the key it last established. It returns 0 +without rewriting anything when the recorded key matches, the kind is Z-order, +and the existing sorted run already covers every row group. This is what lets the +maintenance daemon call it on a schedule without churning storage. + ### pgcolumnar.compact(tablename regclass) returns bigint Retires row groups that are fully deleted, dropping their metadata so scans skip @@ -263,7 +269,7 @@ Reports how much of a table's sorted order is still in place. Returns one row: | Column | Type | Meaning | | --- | --- | --- | -| `sort_key` | name[] | The `sort_by` key declared by `set_options`, or NULL. | +| `sort_key` | name[] | The clustering key in effect. It is the key the last `recluster` recorded, or the `sort_by` declared by `set_options`, or NULL. | | `total_groups` | bigint | Row groups in the table. | | `sorted_groups` | bigint | Row groups written by the last ordering rewrite. | | `appended_groups` | bigint | Row groups written after it. | @@ -304,6 +310,34 @@ which counts as appended. The record is internal storage metadata and `pg_dump` does not carry it, so a restored table reports no sorted groups until you sort it again. +### pgcolumnar.maintenance_due(rel regclass, compact_due_fraction float8 DEFAULT 0.2, recluster_due_fraction float8 DEFAULT 0.05) + +Reports whether an online maintenance verb is worth running on a table, from its +statistics alone. It takes no lock and rewrites nothing. This is the policy the +`pgcolumnar.autovacuum` daemon consults on each sweep. You can also call it from a +monitoring query. Returns one row: + +| Column | Type | Meaning | +| --- | --- | --- | +| `total_rows` | bigint | Stored rows, including deleted rows not yet reclaimed. | +| `deleted_rows` | bigint | Rows deleted but still stored. | +| `deleted_fraction` | float8 | `deleted_rows` over `total_rows`. | +| `sort_key` | name[] | The recorded clustering key, or NULL. | +| `appended_groups` | bigint | Row groups written after the last ordering. | +| `appended_rows` | bigint | Rows in those appended groups. | +| `appended_fraction` | float8 | Appended rows over the sorted plus appended rows. | +| `compact_rewrite_due` | boolean | True when `deleted_fraction` reaches `compact_due_fraction`. | +| `recluster_due` | boolean | True when a sorted run exists and `appended_fraction` reaches `recluster_due_fraction`. | +| `recommendation` | text | The verbs to run, comma-separated, or NULL when nothing is due. | + +The two thresholds default to the values the daemon uses. The function is +`SECURITY DEFINER` and checks that the caller may `SELECT` the table. A monitoring +role that owns the table can therefore call it without superuser rights. + +```sql +SELECT recommendation FROM pgcolumnar.maintenance_due('events'); +``` + ## Projections A projection is a named subset of a table's columns stored a second time,