diff --git a/CHANGELOG.md b/CHANGELOG.md index b3be1ec..11158bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,10 +6,40 @@ pre-release; the version marker is `1.0-alpha`, recorded in `VERSION`. New table are written in the native on-disk format, PGCN v1. For the forward-looking plan see [design/ROADMAP.md](design/ROADMAP.md); for full history see the git log. -The extension's own `default_version` is still `1.0-dev`. That is deliberate: it -governs `ALTER EXTENSION UPDATE`, and moving it needs an upgrade script that does -not exist yet. `SELECT extversion FROM pg_extension` therefore reports `1.0-dev` -on a 1.0-alpha build. +The extension's `default_version` is `1.0-alpha`, and an upgrade script ships with +it. Older notes in this file describe `default_version` as pinned at `1.0-dev`, +which was true until that script existed. + +## [Unreleased] + +### Changed + +- The extension's exported C symbols are namespaced under `pgcolumnar` (#382). + Two extensions that both call themselves `columnar` could define the same + symbol. `columnar_handler` and `columnar_relation_storageid` collided with + Citus columnar. Four settings variables such as `columnar_stripe_row_limit` + also shared names with the same settings there. That case binds one library's + setting to the other's storage. +- `default_version` moves from `1.0-dev` to `1.0-alpha`, so + `SELECT extversion FROM pg_extension` now agrees with `VERSION`. + +### Upgrading + +**Run `ALTER EXTENSION pgcolumnar UPDATE;` in every database that has the +extension, after installing this build.** + +The rename moves the C symbol names that each installed function recorded when it +was created. Replace the shared library without this step and those records +point at symbols the new library does not export. The extension then stops +working until the catalog is updated. Reading an existing columnar table fails +with `could not find function "columnar_handler"`. + +Nothing happens to your data, and no conversion runs. The upgrade replaces +catalog entries only, and keeps each function's identity, so the access method +binding and every dependency survive. The SQL you write does not change. + +See [Upgrade](docs/installation.md#upgrade) for the commands, including how to +list the databases that need it. ## [1.0-alpha] - 2026-08-04 diff --git a/Makefile b/Makefile index a1c9e0d..f206f2b 100644 --- a/Makefile +++ b/Makefile @@ -30,7 +30,7 @@ OBJS = \ src/columnar_parallel_export.o EXTENSION = pgcolumnar -DATA = pgcolumnar--1.0-dev.sql +DATA = pgcolumnar--1.0-alpha.sql pgcolumnar--1.0-dev--1.0-alpha.sql PGFILEDESC = "pgColumnar - column-oriented table access method" # make installcheck. Not the project's gate -- that is test/run_all_versions.sh, diff --git a/docs/installation.md b/docs/installation.md index dbd0277..7986117 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -110,6 +110,49 @@ To install a new build of the extension: 1. Run `make install` with the same `PG_CONFIG`. 2. Start the server again, so that it loads the new library. +3. Run `ALTER EXTENSION pgcolumnar UPDATE;` **in every database that has the + extension**. + +Step 3 is not optional, and it is easy to miss because nothing prompts for it. +The first two steps replace the shared library. The third updates the catalog to +match it. + +```sql +-- in each database that ran CREATE EXTENSION pgcolumnar +ALTER EXTENSION pgcolumnar UPDATE; +SELECT extversion FROM pg_extension WHERE extname = 'pgcolumnar'; +``` + +To find the databases that need it: + +```sql +SELECT datname FROM pg_database WHERE datallowconn + AND EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'pgcolumnar'); +``` + +### If you skipped step 3 + +Every function the extension installs records the name of a C symbol. When those +names change between builds, the recorded names no longer resolve, and the +extension stops working until the catalog is updated. Reading an existing +columnar table then fails: + +``` +ERROR: could not find function "columnar_handler" in file ".../pgcolumnar.so" +``` + +The fix is step 3. Run `ALTER EXTENSION pgcolumnar UPDATE;` in that database and +the error goes away. **Your data is not affected.** The tables are intact and no +conversion happens. Only the catalog entry is stale. + +Do not run `DROP EXTENSION`. It removes your columnar tables with it. + +### Upgrading from 1.0-dev to 1.0-alpha + +This release renames the extension's C symbols into the `pgcolumnar` namespace, +so that two extensions named `columnar` can be loaded without colliding. That is +the change step 3 applies. The SQL you write does not change. Function names, +settings and table syntax are all the same. The source records the on-disk format version. The specification also records it, in [../design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md](https://github.com/commandprompt/pgcolumnar/blob/main/design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md). diff --git a/docs/limitations.md b/docs/limitations.md index 2983ab4..9f31fb7 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -89,16 +89,28 @@ A physical copy does not replace the source across a version change. The same posture covers the extension's own catalog, not only the on-disk data format. The install script of a build defines the `pgcolumnar` catalog tables for a fresh -`CREATE EXTENSION`. The pre-release ships no `ALTER EXTENSION UPDATE` scripts, so -there is no in-place catalog migration either. +`CREATE EXTENSION`. An `ALTER EXTENSION UPDATE` script ships when a build needs one. +The 1.0-dev to 1.0-alpha script is the first. -You can replace the shared library and the SQL script, and then restart, without -a new `CREATE EXTENSION`. This can leave a catalog table without a column that a -newer build needs. For example, `sort_by` was added to `pgcolumnar.options`. A function that +**Replacing the shared library is not sufficient on its own.** After installing a new +build, run `ALTER EXTENSION pgcolumnar UPDATE;` in every database that has the extension. +The library and the catalog have to agree, and only that command updates the catalog. + +Skipping it can leave the catalog describing the previous build. Each installed function +records the name of a C symbol. 1.0-alpha moved those names, so an un-updated catalog +names symbols the new library does not export. Reading an existing columnar table then +fails with `could not find function "columnar_handler"`. The data is untouched, and the +command above fixes it. See [Upgrade](installation.md#upgrade). + +A catalog can also lack a column that a newer build needs, where no upgrade script covers +the gap. For example, `sort_by` was added to `pgcolumnar.options`. A function that uses that column fails against an `options` table that an older build created. -Across an incompatible build, recreate the extension with `DROP EXTENSION` and -`CREATE EXTENSION`, and load the data again. Do not replace the files in place. A +Across an incompatible build, meaning one where no upgrade script covers the change, +recreate the extension with `DROP EXTENSION` and `CREATE EXTENSION`, and load the data +again. This is not the remedy for the un-updated catalog described above, where +`ALTER EXTENSION pgcolumnar UPDATE` is enough. `DROP EXTENSION` removes your columnar +tables with it. Do not replace the files in place. A dump that exists still restores into a newer build. `pg_dump` writes an explicit column list for the configuration tables of the extension, and a new column takes its default of NULL. diff --git a/pgcolumnar--1.0-dev.sql b/pgcolumnar--1.0-alpha.sql similarity index 96% rename from pgcolumnar--1.0-dev.sql rename to pgcolumnar--1.0-alpha.sql index 39da81c..decad2a 100644 --- a/pgcolumnar--1.0-dev.sql +++ b/pgcolumnar--1.0-alpha.sql @@ -277,7 +277,7 @@ CREATE INDEX free_space_fit CREATE FUNCTION pgcolumnar.columnar_handler(internal) RETURNS table_am_handler LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_handler'; + AS 'MODULE_PATHNAME', 'pgcolumnar_handler'; CREATE ACCESS METHOD pgcolumnar TYPE TABLE @@ -492,7 +492,7 @@ COMMENT ON FUNCTION pgcolumnar.reset_options(regclass, bool, bool, bool, bool, b CREATE FUNCTION pgcolumnar.get_storage_id(rel regclass) RETURNS bigint LANGUAGE C STABLE STRICT - AS 'MODULE_PATHNAME', 'columnar_relation_storageid'; + AS 'MODULE_PATHNAME', 'pgcolumnar_relation_storageid'; COMMENT ON FUNCTION pgcolumnar.get_storage_id(regclass) IS 'storage id linking a columnar table to its metadata rows'; @@ -504,7 +504,7 @@ CREATE FUNCTION pgcolumnar.add_projection( sort_key text[] DEFAULT '{}') RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_add_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_add_projection'; COMMENT ON FUNCTION pgcolumnar.add_projection(regclass, text, text[], text[]) IS 'declare a physical projection: a named column subset sorted on sort_key (gap 26)'; @@ -512,7 +512,7 @@ COMMENT ON FUNCTION pgcolumnar.add_projection(regclass, text, text[], text[]) CREATE FUNCTION pgcolumnar.drop_projection(rel regclass, name text) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_drop_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_drop_projection'; COMMENT ON FUNCTION pgcolumnar.drop_projection(regclass, text) IS 'drop a declared projection and free its storage (gap 26)'; @@ -572,7 +572,7 @@ COMMENT ON FUNCTION pgcolumnar.rebuild_projections(regclass) CREATE FUNCTION pgcolumnar.read_projection(rel regclass, name text) RETURNS SETOF text LANGUAGE C STABLE - AS 'MODULE_PATHNAME', 'columnar_read_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_read_projection'; COMMENT ON FUNCTION pgcolumnar.read_projection(regclass, text) IS 'read a projection''s stored columns (live rows), joined by | -- verification/debug (gap 26)'; @@ -580,7 +580,7 @@ COMMENT ON FUNCTION pgcolumnar.read_projection(regclass, text) CREATE FUNCTION pgcolumnar.reconstruct_via_projection(rel regclass, name text) RETURNS SETOF text LANGUAGE C STABLE - AS 'MODULE_PATHNAME', 'columnar_reconstruct_via_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_reconstruct_via_projection'; COMMENT ON FUNCTION pgcolumnar.reconstruct_via_projection(regclass, text) IS 'read all live rows via a projection, reconstructing non-covered columns from the base by row number (gap 26)'; @@ -707,7 +707,7 @@ COMMENT ON FUNCTION pgcolumnar.sort_status(regclass) CREATE FUNCTION pgcolumnar.vacuum(tablename regclass, stripe_count int DEFAULT 0) RETURNS void LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_vacuum'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum'; COMMENT ON FUNCTION pgcolumnar.vacuum(regclass, int) IS 'compact a columnar table by combining stripes and reclaiming deleted rows'; @@ -717,7 +717,7 @@ CREATE FUNCTION pgcolumnar.vacuum_sorted( VARIADIC sort_columns name[]) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vacuum_sorted'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass, name[]) IS 'compact a columnar table, storing rows sorted ascending (NULLS LAST) on the given columns. With no columns, applies the table''s declared sort_by key from set_options (#288), like a bare CLUSTER re-applying a remembered index; errors if none is declared. Supports any btree-orderable column including text (unlike the numeric-only Z-order cluster()). One-shot: not auto-maintained.'; @@ -732,7 +732,7 @@ COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass, name[]) CREATE FUNCTION pgcolumnar.vacuum_sorted(tablename regclass) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vacuum_sorted'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass) IS 'apply the table''s declared sort_by key from set_options (#288); errors if none is declared. Equivalent to a bare CLUSTER re-applying a remembered index.'; @@ -742,7 +742,7 @@ CREATE FUNCTION pgcolumnar.cluster( VARIADIC columns name[]) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_cluster'; + AS 'MODULE_PATHNAME', 'pgcolumnar_cluster'; COMMENT ON FUNCTION pgcolumnar.cluster(regclass, name[]) IS 'eager reorg: rewrite a columnar table with rows ordered by the Z-order space-filling curve over the given columns. Holds AccessExclusiveLock like CLUSTER/VACUUM FULL; the online incremental path is Phase F3'; @@ -750,7 +750,7 @@ COMMENT ON FUNCTION pgcolumnar.cluster(regclass, name[]) CREATE FUNCTION pgcolumnar.compact(tablename regclass) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_compact'; + AS 'MODULE_PATHNAME', 'pgcolumnar_compact'; COMMENT ON FUNCTION pgcolumnar.compact(regclass) IS 'lazy online compaction: retire row groups that are fully deleted, dropping their metadata so scans skip them. Holds only ShareUpdateExclusiveLock (concurrent reads and writes). Returns the number of groups retired (Phase F3a)'; @@ -758,7 +758,7 @@ COMMENT ON FUNCTION pgcolumnar.compact(regclass) CREATE FUNCTION pgcolumnar.truncate(tablename regclass) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_truncate'; + AS 'MODULE_PATHNAME', 'pgcolumnar_truncate'; COMMENT ON FUNCTION pgcolumnar.truncate(regclass) IS 'physical end-truncation: return trailing reclaimed blocks to the OS. Best-effort -- takes AccessExclusiveLock conditionally for the brief physical step and returns 0 without waiting if the table is busy. Only removes space freed before the oldest-xmin horizon. Gated by pgcolumnar.enable_end_truncation. Returns the number of blocks truncated (Phase F)'; @@ -769,7 +769,7 @@ CREATE FUNCTION pgcolumnar.compact_rewrite( max_groups int DEFAULT 0) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_compact_rewrite'; + AS 'MODULE_PATHNAME', 'pgcolumnar_compact_rewrite'; COMMENT ON FUNCTION pgcolumnar.compact_rewrite(regclass, float8, int) IS 'lazy online space reclaim: rewrite partially-deleted row groups (deleted fraction >= min_deleted_fraction) to drop their dead rows, under ShareUpdateExclusiveLock (concurrent reads and writes). Returns the number of groups rewritten (Phase F3b)'; @@ -779,7 +779,7 @@ CREATE FUNCTION pgcolumnar.recluster( VARIADIC columns name[]) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_recluster'; + AS 'MODULE_PATHNAME', 'pgcolumnar_recluster'; COMMENT ON FUNCTION pgcolumnar.recluster(regclass, name[]) IS 'lazy online reclustering: re-establish global Z-order clustering over the given columns under ShareUpdateExclusiveLock (concurrent reads and writes), unlike the eager cluster() which holds AccessExclusiveLock. Returns the number of groups reclustered (Phase F3c)'; @@ -787,7 +787,7 @@ COMMENT ON FUNCTION pgcolumnar.recluster(regclass, name[]) CREATE FUNCTION pgcolumnar.export_arrow(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_export_arrow'; + AS 'MODULE_PATHNAME', 'pgcolumnar_export_arrow'; COMMENT ON FUNCTION pgcolumnar.export_arrow(regclass, text) IS 'export a columnar table to an Arrow IPC stream file; returns rows written'; @@ -795,7 +795,7 @@ COMMENT ON FUNCTION pgcolumnar.export_arrow(regclass, text) CREATE FUNCTION pgcolumnar.export_parquet(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_export_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_export_parquet'; COMMENT ON FUNCTION pgcolumnar.export_parquet(regclass, text) IS 'export a columnar table to a Parquet file; returns rows written'; @@ -804,7 +804,7 @@ CREATE FUNCTION pgcolumnar.parallel_export_parquet(target regclass, path text, workers int DEFAULT NULL) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_parallel_export_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_export_parquet'; COMMENT ON FUNCTION pgcolumnar.parallel_export_parquet(regclass, text, int) IS 'parallel Parquet export using read-only background workers into a directory readable by pgcolumnar.read_parquet: a single columnar table split by row-group ranges, or a partitioned columnar table one file per partition; returns rows written (#300)'; @@ -812,7 +812,7 @@ COMMENT ON FUNCTION pgcolumnar.parallel_export_parquet(regclass, text, int) CREATE FUNCTION pgcolumnar.import_arrow(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_import_arrow'; + AS 'MODULE_PATHNAME', 'pgcolumnar_import_arrow'; COMMENT ON FUNCTION pgcolumnar.import_arrow(regclass, text) IS 'insert rows from an Arrow IPC stream file into a columnar table; returns rows inserted'; @@ -820,7 +820,7 @@ COMMENT ON FUNCTION pgcolumnar.import_arrow(regclass, text) CREATE FUNCTION pgcolumnar.import_parquet(rel regclass, path text) RETURNS bigint LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_import_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_import_parquet'; COMMENT ON FUNCTION pgcolumnar.import_parquet(regclass, text) IS 'insert rows from a Parquet file, directory, or glob into a table; returns rows inserted (gap 27)'; @@ -828,7 +828,7 @@ COMMENT ON FUNCTION pgcolumnar.import_parquet(regclass, text) CREATE FUNCTION pgcolumnar.parquet_schema(path text) RETURNS TABLE(column_name text, data_type text, nullable boolean) LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_parquet_schema'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_schema'; COMMENT ON FUNCTION pgcolumnar.parquet_schema(text) IS 'report the leaf columns of a Parquet file and the PostgreSQL type each maps to; for a directory or glob, of its first file (Phase G scan core)'; @@ -836,7 +836,7 @@ COMMENT ON FUNCTION pgcolumnar.parquet_schema(text) CREATE FUNCTION pgcolumnar.read_parquet(path text) RETURNS SETOF record LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_read_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_read_parquet'; COMMENT ON FUNCTION pgcolumnar.read_parquet(text) IS 'read a Parquet file, directory, or glob in place as a set of rows; requires a column definition list covering every leaf column, e.g. SELECT * FROM pgcolumnar.read_parquet(path) AS t(id int, name text) (Phase G)'; @@ -872,7 +872,7 @@ COMMENT ON FOREIGN DATA WRAPPER pgcolumnar_parquet CREATE FUNCTION pgcolumnar.vm_selftest(rel regclass, blk int) RETURNS boolean LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vm_selftest'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_selftest'; COMMENT ON FUNCTION pgcolumnar.vm_selftest(regclass, int) IS 'gap 28 phase-1 self-test: set a VM-fork all-visible bit and read it back'; @@ -880,7 +880,7 @@ COMMENT ON FUNCTION pgcolumnar.vm_selftest(regclass, int) CREATE FUNCTION pgcolumnar.vm_is_visible(rel regclass, blk int) RETURNS boolean LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vm_is_visible'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_is_visible'; COMMENT ON FUNCTION pgcolumnar.vm_is_visible(regclass, int) IS 'gap 28: is the synthetic block marked all-visible in the VM fork?'; @@ -927,7 +927,7 @@ COMMENT ON FUNCTION pgcolumnar.vacuum_full(name, real, int) CREATE FUNCTION pgcolumnar.file_split_offsets(path text, workers int) RETURNS bigint[] LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_file_split_offsets'; + AS 'MODULE_PATHNAME', 'pgcolumnar_file_split_offsets'; COMMENT ON FUNCTION pgcolumnar.file_split_offsets(text, int) IS 'byte offsets that split a COPY text-format file into N record-aligned ranges (#300)'; @@ -956,7 +956,7 @@ CREATE FUNCTION pgcolumnar.parallel_copy(target regclass, filename text, workers int DEFAULT NULL) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_parallel_copy'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_copy'; COMMENT ON FUNCTION pgcolumnar.parallel_copy(regclass, text, int) IS 'atomic parallel bulk load of a COPY text file into a columnar table using background workers: a single columnar table (any row order), or a RANGE-partitioned columnar table sorted by the partition key with one distinct partition set per worker (#300)'; diff --git a/pgcolumnar--1.0-dev--1.0-alpha.sql b/pgcolumnar--1.0-dev--1.0-alpha.sql new file mode 100644 index 0000000..4cedaa2 --- /dev/null +++ b/pgcolumnar--1.0-dev--1.0-alpha.sql @@ -0,0 +1,192 @@ +/* + * pgcolumnar 1.0-dev -> 1.0-alpha + * + * The C link names moved into the pgcolumnar namespace (#382), so a build from this + * version no longer exports the symbols an existing pg_proc recorded. Without this + * script, replacing the shared library leaves the extension inert: reading an existing + * columnar table fails with "could not find function columnar_handler", and so does + * creating a new one. + * + * CREATE OR REPLACE keeps each function's OID, so the CREATE ACCESS METHOD binding and + * every dependency survive. Only prosrc changes. No signature, name, or permission + * changes here, and no catalog or on-disk format change. + */ + +\echo Use "ALTER EXTENSION pgcolumnar UPDATE" to load this file. \quit + +CREATE OR REPLACE FUNCTION pgcolumnar.columnar_handler(internal) + RETURNS table_am_handler + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_handler'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.get_storage_id(rel regclass) + RETURNS bigint + LANGUAGE C STABLE STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_relation_storageid'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.add_projection( + rel regclass, + name text, + columns text[], + sort_key text[] DEFAULT '{}') + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_add_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.drop_projection(rel regclass, name text) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_drop_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.read_projection(rel regclass, name text) + RETURNS SETOF text + LANGUAGE C STABLE + AS 'MODULE_PATHNAME', 'pgcolumnar_read_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.reconstruct_via_projection(rel regclass, name text) + RETURNS SETOF text + LANGUAGE C STABLE + AS 'MODULE_PATHNAME', 'pgcolumnar_reconstruct_via_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum(tablename regclass, stripe_count int DEFAULT 0) + RETURNS void + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum_sorted( + tablename regclass, + VARIADIC sort_columns name[]) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum_sorted(tablename regclass) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.cluster( + tablename regclass, + VARIADIC columns name[]) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_cluster'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.compact(tablename regclass) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_compact'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.truncate(tablename regclass) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_truncate'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.compact_rewrite( + tablename regclass, + min_deleted_fraction float8 DEFAULT 0.2, + max_groups int DEFAULT 0) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_compact_rewrite'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.recluster( + tablename regclass, + VARIADIC columns name[]) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_recluster'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.export_arrow(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_export_arrow'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.export_parquet(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_export_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parallel_export_parquet(target regclass, path text, + workers int DEFAULT NULL) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_export_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.import_arrow(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_import_arrow'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.import_parquet(rel regclass, path text) + RETURNS bigint + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_import_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_schema(path text) + RETURNS TABLE(column_name text, data_type text, nullable boolean) + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_schema'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.read_parquet(path text) + RETURNS SETOF record + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_read_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_fdw_handler() + RETURNS fdw_handler + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_fdw_handler'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_fdw_validator(text[], oid) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_fdw_validator'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vm_selftest(rel regclass, blk int) + RETURNS boolean + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_selftest'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vm_is_visible(rel regclass, blk int) + RETURNS boolean + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_is_visible'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.file_split_offsets(path text, workers int) + RETURNS bigint[] + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_file_split_offsets'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parallel_copy(target regclass, filename text, + workers int DEFAULT NULL) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_copy'; + diff --git a/pgcolumnar.control b/pgcolumnar.control index 4420323..66e34a6 100644 --- a/pgcolumnar.control +++ b/pgcolumnar.control @@ -4,7 +4,7 @@ # independent MIT implementation; the re-origination line builds from # design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md. comment = 'Columnar storage for PostgreSQL (pgColumnar)' -default_version = '1.0-dev' +default_version = '1.0-alpha' module_pathname = '$libdir/pgcolumnar' relocatable = false schema = pgcolumnar diff --git a/src/columnar.h b/src/columnar.h index 7c52fb2..6741e6c 100644 --- a/src/columnar.h +++ b/src/columnar.h @@ -84,7 +84,7 @@ /* * Round a logical byte length up to a whole number of pages. Every reservation - * (ColumnarReserveOffset) starts on a page boundary and the next one starts on + * (PgColumnarReserveOffset) starts on a page boundary and the next one starts on * the next page boundary, so a group's on-disk footprint is its data length * rounded up to a page. Physical reclaim keeps free ranges page-aligned in both * offset and length by working in these footprints. @@ -114,7 +114,7 @@ /* * Value-stream encoding codes (I1, format 2.1). An encoding is a reversible * transform of the raw value-stream bytes, applied before block compression on - * write and reversed after decompression on read (columnar_encoding.c). + * write and reversed after decompression on read (pgcolumnar_encoding.c). */ #define COLUMNAR_ENCODING_NONE 0 #define COLUMNAR_ENCODING_RLE 1 /* run-length of a fixed-width value */ @@ -146,7 +146,7 @@ #define COLUMNAR_ENCODE_EFFORT_FULL 0 #define COLUMNAR_ENCODE_EFFORT_FAST 1 -typedef struct ColumnarOptions +typedef struct PgColumnarOptions { bool chunkGroupRowLimitSet; int chunkGroupRowLimit; @@ -158,47 +158,47 @@ typedef struct ColumnarOptions int compressionLevel; bool encodeEffortSet; int encodeEffort; /* one of COLUMNAR_ENCODE_EFFORT_* */ -} ColumnarOptions; +} PgColumnarOptions; /* GUC-backed instance defaults (spec 8.3) */ -extern int columnar_stripe_row_limit; -extern int columnar_chunk_group_row_limit; -extern int columnar_encoding_sample_rows; -extern int columnar_compression; /* one of COLUMNAR_COMPRESSION_* */ -extern int columnar_compression_level; /* zstd level */ -extern int columnar_fsst_min_gain_percent; /* min compressed FSST win to keep it (#155) */ -extern bool columnar_enable_qual_pushdown; -extern bool columnar_enable_column_projection; -extern bool columnar_enable_custom_scan; -extern bool columnar_enable_bloom_filter; /* bloom equality skipping (I7) */ +extern int pgcolumnar_stripe_row_limit; +extern int pgcolumnar_chunk_group_row_limit; +extern int pgcolumnar_encoding_sample_rows; +extern int pgcolumnar_compression; /* one of COLUMNAR_COMPRESSION_* */ +extern int pgcolumnar_compression_level; /* zstd level */ +extern int pgcolumnar_fsst_min_gain_percent; /* min compressed FSST win to keep it (#155) */ +extern bool pgcolumnar_enable_qual_pushdown; +extern bool pgcolumnar_enable_column_projection; +extern bool pgcolumnar_enable_custom_scan; +extern bool pgcolumnar_enable_bloom_filter; /* bloom equality skipping (I7) */ /* Phase 6 GUCs (spec 8.3) */ -extern bool columnar_enable_vectorization; /* vectorized aggregate path */ -extern bool columnar_enable_group_vectorization; /* GROUP BY vectorized agg (#289) */ -extern bool columnar_enable_ungrouped_vector_agg; /* filtered/extended ungrouped agg (#289) */ -extern bool columnar_enable_parallel_vector_agg; /* parallel-aware ungrouped batch fold (#289 phase 5/6) */ -extern int columnar_groupagg_max_groups; /* plan-time group-count cap (#289) */ -extern bool columnar_enable_read_stream; /* stream/prefetch block reads (PG17+) */ -extern bool columnar_enable_index_only_scan; /* allow index-only scans (gap 28) */ -extern bool columnar_bulk_parallel_writer; /* internal: parallel_copy loader skips the storage-row creation lock (#300) */ -extern bool columnar_enable_projection_scan; /* scan a covering projection (gap 26) */ -extern bool columnar_enable_index_fetch_penalty; /* price a columnar index scan's per-row fetch (#355) */ +extern bool pgcolumnar_enable_vectorization; /* vectorized aggregate path */ +extern bool pgcolumnar_enable_group_vectorization; /* GROUP BY vectorized agg (#289) */ +extern bool pgcolumnar_enable_ungrouped_vector_agg; /* filtered/extended ungrouped agg (#289) */ +extern bool pgcolumnar_enable_parallel_vector_agg; /* parallel-aware ungrouped batch fold (#289 phase 5/6) */ +extern int pgcolumnar_groupagg_max_groups; /* plan-time group-count cap (#289) */ +extern bool pgcolumnar_enable_read_stream; /* stream/prefetch block reads (PG17+) */ +extern bool pgcolumnar_enable_index_only_scan; /* allow index-only scans (gap 28) */ +extern bool pgcolumnar_bulk_parallel_writer; /* internal: parallel_copy loader skips the storage-row creation lock (#300) */ +extern bool pgcolumnar_enable_projection_scan; /* scan a covering projection (gap 26) */ +extern bool pgcolumnar_enable_index_fetch_penalty; /* price a columnar index scan's per-row fetch (#355) */ /* - * Statement-scoped by-row-number fetch cache cap (columnar_reader.c). Named here + * Statement-scoped by-row-number fetch cache cap (pgcolumnar_reader.c). Named here * so the index-fetch cost model (#355) can tell when a stripe is too wide to be * retained across fetches and must be treated as re-decoded per row. */ #define COLUMNAR_FETCH_CACHE_MAX_BYTES (32 * 1024 * 1024) /* issue #5: concurrent unique-key insert serialization */ -extern bool columnar_enable_unique_lock; /* serialize same-key inserters */ -extern int columnar_unique_lock_buckets; /* advisory-lock buckets per index */ +extern bool pgcolumnar_enable_unique_lock; /* serialize same-key inserters */ +extern int pgcolumnar_unique_lock_buckets; /* advisory-lock buckets per index */ /* ------------------------------------------------------------------------- * Metapage (spec 3) * ------------------------------------------------------------------------- */ -typedef struct ColumnarMetapage +typedef struct PgColumnarMetapage { uint32 versionMajor; uint32 versionMinor; @@ -207,7 +207,7 @@ typedef struct ColumnarMetapage uint64 reservedRowNumber; uint64 reservedOffset; bool unloggedReset; -} ColumnarMetapage; +} PgColumnarMetapage; @@ -251,7 +251,7 @@ typedef struct NativeColumnChunkMetadata * One pgcolumnar.zone_map row (native spec 7.1, Phase D5): a Small Materialized * Aggregate for one vector of a column chunk (vectorIndex 0-based) or for the * whole column chunk (vectorIndex -1). minimum and maximum are the column's - * value serialized with ColumnarEncodeValue (NULL when the type has no btree + * value serialized with PgColumnarEncodeValue (NULL when the type has no btree * ordering); sum is a numeric Datum (D5a leaves it unset, hasSum false; the * zone-map-only aggregate that consumes it lands in D5b). value_count and * null_count are always present. @@ -263,7 +263,7 @@ typedef struct NativeZoneMapMetadata int columnIndex; int vectorIndex; /* 0-based vector; -1 for the whole chunk */ bool hasMinMax; - const char *minimum; /* ColumnarEncodeValue bytes, when hasMinMax */ + const char *minimum; /* PgColumnarEncodeValue bytes, when hasMinMax */ uint32 minimumLen; const char *maximum; uint32 maximumLen; @@ -276,7 +276,7 @@ typedef struct NativeZoneMapMetadata /* * One pgcolumnar.bloom row (native spec 7.2, Phase D5b): a per-column-chunk bloom * filter over the chunk's hashable values, for equality skipping on unsorted - * columns. filter is the ColumnarBloomBuild byte image. + * columns. filter is the PgColumnarBloomBuild byte image. */ typedef struct NativeBloomMetadata { @@ -309,7 +309,7 @@ typedef struct DeleteVectorMetadata * projectionId 0 is the implicit base projection. attnums are 1-based; sortKey * attnums are a subset of columns. */ -typedef struct ColumnarProjection +typedef struct PgColumnarProjection { uint64 storageId; /* the table's base storage id */ int projectionId; /* 0 = base, 1..N additional */ @@ -319,51 +319,51 @@ typedef struct ColumnarProjection int sortKeyLen; int16 *columns; /* stored attnums, columnsLen entries */ int columnsLen; -} ColumnarProjection; +} PgColumnarProjection; /* ------------------------------------------------------------------------- - * storage layer (columnar_storage.c) + * storage layer (pgcolumnar_storage.c) * ------------------------------------------------------------------------- */ struct SMgrRelationData; -extern void ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, +extern void PgColumnarWriteNewMetapage(const RelFileLocator *newrlocator, struct SMgrRelationData *srel, char persistence, uint64 storageId); -extern void ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta); -extern uint64 ColumnarStorageId(Relation rel); -extern void ColumnarEnsureStorageRow(Relation rel); /* pre-create storage row (#300 parallel_copy) */ -extern void ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, +extern void PgColumnarReadMetapage(Relation rel, PgColumnarMetapage *meta); +extern uint64 PgColumnarStorageId(Relation rel); +extern void PgColumnarEnsureStorageRow(Relation rel); /* pre-create storage row (#300 parallel_copy) */ +extern void PgColumnarReserveRowNumbers(Relation rel, uint64 rowCount, uint64 *stripeId, uint64 *firstRowNumber); -extern void ColumnarReserveOffset(Relation rel, uint64 dataLength, +extern void PgColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset); -extern void ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes); -extern void ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, +extern void PgColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes); +extern void PgColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor); -extern void ColumnarSetReservedOffset(Relation rel, uint64 newOffset); -extern void ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks); -extern void ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, +extern void PgColumnarSetReservedOffset(Relation rel, uint64 newOffset); +extern void PgColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks); +extern void PgColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, char *data, uint64 length); -extern void ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, +extern void PgColumnarReadLogicalData(Relation rel, uint64 logicalOffset, char *dest, uint64 length); -extern void ColumnarResetMetapage(Relation rel); +extern void PgColumnarResetMetapage(Relation rel); /* row number <-> item pointer (spec 6) */ -extern void ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid); -extern uint64 ColumnarItemPointerToRowNumber(ItemPointer tid); +extern void PgColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid); +extern uint64 PgColumnarItemPointerToRowNumber(ItemPointer tid); /* ------------------------------------------------------------------------- - * visibility map for index-only scans (columnar_visibilitymap.c, gap 28) + * visibility map for index-only scans (pgcolumnar_visibilitymap.c, gap 28) * ------------------------------------------------------------------------- */ -extern void ColumnarVMSetVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMClearVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMClearForRow(Relation rel, uint64 rowNumber); -extern bool ColumnarVMIsVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMSetVisibleForRelation(Relation rel); -extern void ColumnarDiscardFetchCache(void); +extern void PgColumnarVMSetVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMClearVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMClearForRow(Relation rel, uint64 rowNumber); +extern bool PgColumnarVMIsVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMSetVisibleForRelation(Relation rel); +extern void PgColumnarDiscardFetchCache(void); /* index maintenance for callers that insert rows without an executor (#153) */ -typedef struct ColumnarIndexInsertState +typedef struct PgColumnarIndexInsertState { EState *estate; TupleTableSlot *slot; @@ -384,55 +384,55 @@ typedef struct ColumnarIndexInsertState Relation *rels; IndexInfo **infos; ExprState **predicates; /* partial-index predicate, or NULL */ -} ColumnarIndexInsertState; +} PgColumnarIndexInsertState; /* * enforceConstraints is fixed for the life of the state rather than passed per * row, because it selects which of the two routes above is built: an importer * enforces for every row it inserts and a rewrite enforces for none. */ -extern ColumnarIndexInsertState *ColumnarIndexInsertBegin(Relation rel, +extern PgColumnarIndexInsertState *PgColumnarIndexInsertBegin(Relation rel, bool enforceConstraints); -extern void ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, +extern void PgColumnarIndexInsertRow(PgColumnarIndexInsertState *st, Relation rel, Datum *values, bool *isnull, uint64 rowNumber); -extern void ColumnarIndexInsertEnd(ColumnarIndexInsertState *st); -extern bool ColumnarRelationHasIndexes(Relation rel); +extern void PgColumnarIndexInsertEnd(PgColumnarIndexInsertState *st); +extern bool PgColumnarRelationHasIndexes(Relation rel); /* a contiguous run of all-visible row numbers (gap 28 phase 3) */ -typedef struct ColumnarRowRange +typedef struct PgColumnarRowRange { uint64 firstRowNumber; uint64 rowCount; -} ColumnarRowRange; +} PgColumnarRowRange; /* row groups every one of whose rows is deleted as-of oldestXmin. Returns a * List of palloc'd uint64 group numbers. */ -extern List *ColumnarComputeFullyDeletedGroups(uint64 storageId, +extern List *PgColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin); -extern void ColumnarRetireGroup(uint64 storageId, uint64 groupNumber); -extern int64 ColumnarRetireFullyDeletedGroups(Relation rel); -extern void ColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber); -extern bool ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, +extern void PgColumnarRetireGroup(uint64 storageId, uint64 groupNumber); +extern int64 PgColumnarRetireFullyDeletedGroups(Relation rel); +extern void PgColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber); +extern bool PgColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, TransactionId oldestXmin, uint64 *fileOffset); -extern bool ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, +extern bool PgColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, TransactionId oldestXmin); -extern void ColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd); -extern void ColumnarReconcileFreeList(Relation dataRel); +extern void PgColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd); +extern void PgColumnarReconcileFreeList(Relation dataRel); /* all-visible chunk-group row ranges: stripe committed past the horizon and no - * deletes (committed or in-progress). Returns a List of ColumnarRowRange *. */ -extern List *ColumnarComputeAllVisibleGroups(uint64 storageId, + * deletes (committed or in-progress). Returns a List of PgColumnarRowRange *. */ +extern List *PgColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin); /* physical reclaim: split freed ranges on allocate and coalesce on free (GUC) */ -extern bool columnar_reclaim_coalesce; +extern bool pgcolumnar_reclaim_coalesce; /* physical end-truncation opt-in (GUC) */ -extern bool columnar_enable_end_truncation; +extern bool pgcolumnar_enable_end_truncation; /* error unless the current user owns the relation (maintenance/DDL gate) */ -extern void ColumnarRequireTableOwner(Relation rel); +extern void PgColumnarRequireTableOwner(Relation rel); /* * Assert-only invariant: a storage's live row-group footprints and its @@ -440,64 +440,64 @@ extern void ColumnarRequireTableOwner(Relation rel); * called only in assert builds (the version matrix builds with asserts). */ #ifdef USE_ASSERT_CHECKING -extern void ColumnarCheckFreeSpaceNoOverlap(uint64 storageId); -#define COLUMNAR_ASSERT_NO_OVERLAP(sid) ColumnarCheckFreeSpaceNoOverlap(sid) +extern void PgColumnarCheckFreeSpaceNoOverlap(uint64 storageId); +#define COLUMNAR_ASSERT_NO_OVERLAP(sid) PgColumnarCheckFreeSpaceNoOverlap(sid) #else #define COLUMNAR_ASSERT_NO_OVERLAP(sid) ((void) 0) #endif /* ------------------------------------------------------------------------- - * metadata layer (columnar_metadata.c) + * metadata layer (pgcolumnar_metadata.c) * ------------------------------------------------------------------------- */ -extern uint64 ColumnarNextStorageId(void); -extern void ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s); -extern void ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, +extern uint64 PgColumnarNextStorageId(void); +extern void PgColumnarInsertNativeStorageRow(const NativeStorageMetadata *s); +extern void PgColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup); -extern void ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName); -extern void ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg); -extern void ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc); -extern void ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z); -extern void ColumnarInsertBloomRow(const NativeBloomMetadata *b); -extern List *ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot); -extern List *ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, +extern void PgColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName); +extern void PgColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg); +extern void PgColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc); +extern void PgColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z); +extern void PgColumnarInsertBloomRow(const NativeBloomMetadata *b); +extern List *PgColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot); +extern List *PgColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern NativeBloomMetadata *ColumnarReadBloomForColumn(uint64 storageId, +extern NativeBloomMetadata *PgColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, int columnIndex, Snapshot snapshot); -extern void ColumnarDeleteMetadata(uint64 storageId); +extern void PgColumnarDeleteMetadata(uint64 storageId); /* per-table options catalog (spec 7.4) */ -extern bool ColumnarReadOptions(Oid relid, ColumnarOptions *opts); -extern void ColumnarDeleteOptions(Oid relid); +extern bool PgColumnarReadOptions(Oid relid, PgColumnarOptions *opts); +extern void PgColumnarDeleteOptions(Oid relid); /* declared physical sort key (#288); List of pstrdup'd column names, NIL if * none is declared. Names (not attnums) so the value survives dump/restore. */ -extern List *ColumnarReadSortBy(Oid relid); +extern List *PgColumnarReadSortBy(Oid relid); -/* projection catalog (gap 26, format 2.2). List entries are ColumnarProjection* +/* projection catalog (gap 26, format 2.2). List entries are PgColumnarProjection* * palloc'd in the current context, ordered by projection_id. */ -extern List *ColumnarListProjections(uint64 storageId); -extern void ColumnarInsertProjectionRow(const ColumnarProjection *proj); +extern List *PgColumnarListProjections(uint64 storageId); +extern void PgColumnarInsertProjectionRow(const PgColumnarProjection *proj); /* The dumpable declaration behind a projection, keyed by regclass and stored as * column names so a dump and restore can carry it (#266). */ -extern void ColumnarRecordProjectionDeclaration(Oid relid, const char *name, +extern void PgColumnarRecordProjectionDeclaration(Oid relid, const char *name, ArrayType *columns, ArrayType *sortKey); -extern void ColumnarDeleteProjectionDeclaration(Oid relid, const char *name); +extern void PgColumnarDeleteProjectionDeclaration(Oid relid, const char *name); /* Every declaration for a relation, for the drop hook: a dropped table must not * leave rows behind whose regclass no longer resolves (#304). */ -extern void ColumnarDeleteProjectionDeclarationsForRel(Oid relid); -extern void ColumnarDeleteProjectionRow(uint64 storageId, int projectionId); +extern void PgColumnarDeleteProjectionDeclarationsForRel(Oid relid); +extern void PgColumnarDeleteProjectionRow(uint64 storageId, int projectionId); /* whether a relation uses the columnar table access method */ -extern bool ColumnarIsColumnarRelation(Oid relid); +extern bool PgColumnarIsColumnarRelation(Oid relid); /* * A snapshot suitable for reading the columnar metadata catalog during a scan @@ -509,113 +509,113 @@ extern bool ColumnarIsColumnarRelation(Oid relid); * MVCC snapshot. The result is palloc'd in the current context and shares the * base snapshot's arrays, so the base must outlive it. */ -extern Snapshot ColumnarCatalogSnapshot(Snapshot base); +extern Snapshot PgColumnarCatalogSnapshot(Snapshot base); /* delete_vector catalog access (spec 7.5) */ -extern List *ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, +extern List *PgColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot); -extern bool ColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot); -extern void ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm); +extern bool PgColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot); +extern void PgColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm); /* ------------------------------------------------------------------------- - * writer (columnar_write_state.c) + * writer (pgcolumnar_write_state.c) * ------------------------------------------------------------------------- */ -typedef struct ColumnarWriteState ColumnarWriteState; +typedef struct PgColumnarWriteState PgColumnarWriteState; -extern ColumnarWriteState *ColumnarGetWriteState(Relation rel); -extern int ColumnarWriteStateStripeCount(ColumnarWriteState *ws); -extern uint64 *ColumnarWriteStateStripeIds(ColumnarWriteState *ws, int *n); -extern uint64 *ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n); -extern uint64 ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, +extern PgColumnarWriteState *PgColumnarGetWriteState(Relation rel); +extern int PgColumnarWriteStateStripeCount(PgColumnarWriteState *ws); +extern uint64 *PgColumnarWriteStateStripeIds(PgColumnarWriteState *ws, int *n); +extern uint64 *PgColumnarWriteStateProjStripeIds(PgColumnarWriteState *ws, int *n); +extern uint64 PgColumnarWriteRow(PgColumnarWriteState *writeState, Relation rel, Datum *values, bool *nulls); -extern void ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, +extern void PgColumnarProjectionFanoutRow(Relation rel, PgColumnarWriteState *baseWs, uint64 rowNumber, Datum *values, bool *nulls); -extern void ColumnarBackfillProjection(Relation rel, - const ColumnarProjection *proj); -extern bool ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, +extern void PgColumnarBackfillProjection(Relation rel, + const PgColumnarProjection *proj); +extern bool PgColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, Datum *values, bool *nulls); -extern void ColumnarFlushWriteStateForRelation(Oid relid); -extern void ColumnarForgetWriteStateForRelation(Oid relid); -extern void ColumnarFlushAllPendingWrites(void); -extern void ColumnarDiscardAllPendingWrites(void); -extern void ColumnarWriteStateDiscardSubXact(SubTransactionId subid); -extern void ColumnarWriteStatePromoteSubXact(SubTransactionId subid, +extern void PgColumnarFlushWriteStateForRelation(Oid relid); +extern void PgColumnarForgetWriteStateForRelation(Oid relid); +extern void PgColumnarFlushAllPendingWrites(void); +extern void PgColumnarDiscardAllPendingWrites(void); +extern void PgColumnarWriteStateDiscardSubXact(SubTransactionId subid); +extern void PgColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent); /* ------------------------------------------------------------------------- - * delete vector / delete tracking (columnar_delete_vector.c, spec 7.5, 9) + * delete vector / delete tracking (pgcolumnar_delete_vector.c, spec 7.5, 9) * ------------------------------------------------------------------------- */ -extern void ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber); -extern bool ColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber); -extern void ColumnarFlushDeleteVectorForRelation(Relation rel); -extern void ColumnarFlushAllDeleteVectors(void); -extern void ColumnarDiscardAllDeleteVectors(void); -extern void ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid); -extern void ColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, +extern void PgColumnarMarkRowDeleted(Relation rel, uint64 rowNumber); +extern bool PgColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber); +extern void PgColumnarFlushDeleteVectorForRelation(Relation rel); +extern void PgColumnarFlushAllDeleteVectors(void); +extern void PgColumnarDiscardAllDeleteVectors(void); +extern void PgColumnarDeleteVectorDiscardSubXact(SubTransactionId subid); +extern void PgColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent); /* ------------------------------------------------------------------------- - * reader (columnar_reader.c) + * reader (pgcolumnar_reader.c) * ------------------------------------------------------------------------- */ -typedef struct ColumnarReadState ColumnarReadState; +typedef struct PgColumnarReadState PgColumnarReadState; -extern ColumnarReadState *ColumnarBeginRead(Relation rel, Snapshot snapshot, +extern PgColumnarReadState *PgColumnarBeginRead(Relation rel, Snapshot snapshot, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys); -/* like ColumnarBeginRead but reads an explicit storage id with an explicit +/* like PgColumnarBeginRead but reads an explicit storage id with an explicit * tuple descriptor -- used to read a projection's storage (gap 26) */ -extern ColumnarReadState *ColumnarBeginReadWithStorage(Relation rel, +extern PgColumnarReadState *PgColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, uint64 storageId, TupleDesc tupdesc, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys); -extern bool ColumnarReadNextRow(ColumnarReadState *readState, +extern bool PgColumnarReadNextRow(PgColumnarReadState *readState, Datum *values, bool *nulls, uint64 *rowNumber); -extern void ColumnarRescanRead(ColumnarReadState *readState); -extern void ColumnarEndRead(ColumnarReadState *readState); +extern void PgColumnarRescanRead(PgColumnarReadState *readState); +extern void PgColumnarEndRead(PgColumnarReadState *readState); /* * Batch-fold accessors (#289): expose the current loaded group's decoded buffer * so an ungrouped aggregate can fold it column-at-a-time instead of one Datum - * tuple per row. See the block comment in columnar_reader.c for the contract. + * tuple per row. See the block comment in pgcolumnar_reader.c for the contract. */ -extern bool ColumnarReadFoldNextGroup(ColumnarReadState *readState); -extern void ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, +extern bool PgColumnarReadFoldNextGroup(PgColumnarReadState *readState); +extern void PgColumnarReadFoldGroupInfo(PgColumnarReadState *readState, uint64 *nrows, const char **deleteMask, uint32 *deleteMaskLen, const bool **skipVec, const uint32 **vecStart, int *vectorCount); -extern bool ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, +extern bool PgColumnarReadFoldColumn(PgColumnarReadState *readState, int attidx, const char **validity, const char **packed, int16 *attlen, const uint32 **vecRawLen); /* * Restrict a scan to a set of row groups (issue #149). Groups outside the set * are skipped without their bytes being read. Must be called before the first - * ColumnarReadNextRow; ngroups == 0 makes the scan return no rows. + * PgColumnarReadNextRow; ngroups == 0 makes the scan return no rows. */ -extern void ColumnarReadRestrictToGroups(ColumnarReadState *readState, +extern void PgColumnarReadRestrictToGroups(PgColumnarReadState *readState, const uint64 *groupNumbers, int ngroups); /* Parquet export helpers, shared by the serial and parallel exporters - * (src/columnar_parquet.c). */ -extern int64 ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, + * (src/pgcolumnar_parquet.c). */ +extern int64 PgColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, const uint64 *restrictGroups, int nRestrictGroups); -extern void ColumnarParquetCheckExportable(Relation rel); +extern void PgColumnarParquetCheckExportable(Relation rel); /* * Parallel scan (gap 23): point the read state at a shared atomic that hands out * stripe indices, so several workers scanning the same relation each claim * distinct stripes. Set by the custom scan's DSM init callbacks. */ -extern void ColumnarReadSetParallelCounter(ColumnarReadState *readState, +extern void PgColumnarReadSetParallelCounter(PgColumnarReadState *readState, pg_atomic_uint32 *counter); /* @@ -623,26 +623,26 @@ extern void ColumnarReadSetParallelCounter(ColumnarReadState *readState, * scan's EXPLAIN output to show how many chunk groups the min/max skip lists * removed. total = read + skipped over the groups the scan has reached. */ -extern void ColumnarReadStats(ColumnarReadState *readState, +extern void PgColumnarReadStats(PgColumnarReadState *readState, uint64 *groupsRead, uint64 *groupsSkipped, uint64 *groupsTotal); -extern uint64 ColumnarVectorsSkipped(ColumnarReadState *readState); +extern uint64 PgColumnarVectorsSkipped(PgColumnarReadState *readState); /* cached base-liveness for a projection scan (gap 26): build once per scan, * probe per row with a binary search instead of a per-row catalog scan */ -typedef struct ColumnarLivenessCache ColumnarLivenessCache; -extern ColumnarLivenessCache *ColumnarBuildLivenessCache(Relation rel, +typedef struct PgColumnarLivenessCache PgColumnarLivenessCache; +extern PgColumnarLivenessCache *PgColumnarBuildLivenessCache(Relation rel, Snapshot snapshot); -extern bool ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, +extern bool PgColumnarLivenessCacheIsLive(PgColumnarLivenessCache *cache, uint64 rowNumber); -extern void ColumnarFreeLivenessCache(ColumnarLivenessCache *cache); +extern void PgColumnarFreeLivenessCache(PgColumnarLivenessCache *cache); /* * Fetch a single row by its 1-based row number (spec 6), for the table AM's * fetch-by-tid callback used by UPDATE. Fills values/nulls (by-reference values * are allocated in the current memory context) and returns true when the row * exists and is not marked deleted in the delete vector. */ -extern bool ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, +extern bool PgColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls); /* @@ -651,93 +651,93 @@ extern bool ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, * this deliberately has no "NULL means all" convention, because a Bitmapset * cannot distinguish empty from NULL, so a caller whose computed set came out * empty would silently get the opposite of what it asked for. For every column - * call ColumnarReadRowByNumber, which takes no set. + * call PgColumnarReadRowByNumber, which takes no set. * * Decoding every column regardless makes a wide table exceed the fetch cache's * size cap, so the entry is dropped after every fetch and the group is decoded * again for the next row (issue #157). */ -extern bool ColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, +extern bool PgColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, Bitmapset *needed); /* Is the row visible? Decodes nothing. */ -extern bool ColumnarRowIsLive(Relation rel, Snapshot snapshot, +extern bool PgColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber); /* ------------------------------------------------------------------------- - * Decoded chunk group (columnar_vector.c aggregate path) + * Decoded chunk group (pgcolumnar_vector.c aggregate path) * - * A ColumnarVector is one decoded chunk group: for each projected column, the + * A PgColumnarVector is one decoded chunk group: for each projected column, the * whole group's values and null flags as flat arrays, plus the per-row deleted * flag resolved from the delete vector. The vectorized aggregate builds a selection * vector over it; the scan itself is the scalar per-row - * reader (ColumnarReadNextRow). + * reader (PgColumnarReadNextRow). * ------------------------------------------------------------------------- */ -typedef struct ColumnarVector +typedef struct PgColumnarVector { uint64 nrows; /* rows in this chunk group */ uint64 firstRowNumber; /* row number of local row 0 */ Datum **values; /* [natts]; values[c] is Datum[nrows] or NULL */ bool **isnull; /* [natts]; isnull[c] is bool[nrows] or NULL */ bool *deleted; /* [nrows]; true when row-mask-deleted */ -} ColumnarVector; +} PgColumnarVector; /* value stream encode/decode shared by writer and reader */ -extern void ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, +extern void PgColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value); -extern Datum ColumnarDecodeValue(Form_pg_attribute att, char **cursor, +extern Datum PgColumnarDecodeValue(Form_pg_attribute att, char **cursor, MemoryContext targetContext); /* ------------------------------------------------------------------------- - * lightweight value-stream encodings (columnar_encoding.c, I1) + * lightweight value-stream encodings (pgcolumnar_encoding.c, I1) * ------------------------------------------------------------------------- */ -extern int ColumnarEncodeChunk(const char *raw, uint32 rawLen, +extern int PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen); -extern char *ColumnarDecodeChunk(const char *enc, uint32 encLen, +extern char *PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx); -extern const char *ColumnarEncodingName(int encodingType); +extern const char *PgColumnarEncodingName(int encodingType); /* * Build one FSST symbol table for a whole column chunk from a sample of its * concatenated varlena value streams (E3b). Returns true and sets *tableOut / * *tableLenOut (palloc'd, serialized as [uint8 nSym][ nSym x (uint8 len, bytes)]) * when a table was built; false for non-varlena columns or when no useful table - * exists. The table is passed back into ColumnarEncodeChunk / ColumnarDecodeChunk + * exists. The table is passed back into PgColumnarEncodeChunk / PgColumnarDecodeChunk * as fsstTable so the per-vector build cost is paid once per chunk. */ -extern bool ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut); /* Cheap distinct-count pre-check: true when dictionary encoding wins outright, so * the costly FSST table build can be skipped with byte-identical output (#155). */ -extern bool ColumnarFsstDictWins(const char *corpus, uint32 corpusLen); +extern bool PgColumnarFsstDictWins(const char *corpus, uint32 corpusLen); /* * True when encoding the chunk with the table just built is still a win after * the block compressor runs over the result, judged on the same sample the - * table was trained on. The per-vector test inside ColumnarEncodeChunk compares + * table was trained on. The per-vector test inside PgColumnarEncodeChunk compares * uncompressed lengths, which is the wrong objective when a codec is configured: * FSST codes are smaller than repetitive text but much less compressible, so * FSST can win every vector and still enlarge the chunk. Callers pass NULL for * fsstTable when this returns false. */ -extern bool ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, const char *table, uint32 tableLen, int compressionType, int compressionLevel); /* ------------------------------------------------------------------------- - * per-chunk bloom filters (columnar_bloom.c, I7) + * per-chunk bloom filters (pgcolumnar_bloom.c, I7) * ------------------------------------------------------------------------- */ -extern bool ColumnarBloomBuild(const uint32 *hashes, uint32 n, +extern bool PgColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen); -extern bool ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); +extern bool PgColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); /* * True when a column of the given collation can carry a bloom filter (I7/gap 25): @@ -745,10 +745,10 @@ extern bool ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); * values are byte-identical and hash consistently between build and probe. * Nondeterministic collations return false and are left unbloomed. */ -extern bool ColumnarCollationIsDeterministic(Oid collid); +extern bool PgColumnarCollationIsDeterministic(Oid collid); /* ------------------------------------------------------------------------- - * compression-block run iterator (columnar_encoding.c, I2) + * compression-block run iterator (pgcolumnar_encoding.c, I2) * * Exposes a column chunk's (non-null) values as a sequence of (value, run * length) pairs so operators run once per run instead of once per row (I3 @@ -756,15 +756,15 @@ extern bool ColumnarCollationIsDeterministic(Oid collid); * adjacent equal fixed-width values, so a repetitive or run-length-encoded * column yields long runs. Fixed-width columns only. * ------------------------------------------------------------------------- */ -typedef struct ColumnarBlockReader +typedef struct PgColumnarBlockReader { const char *raw; /* raw value stream (packed fixed-width values) */ uint64 valueCount; /* number of values in the stream */ int width; /* bytes per value (attlen) */ uint64 pos; /* next value index */ -} ColumnarBlockReader; +} PgColumnarBlockReader; -extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +extern void PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width); /* @@ -772,65 +772,65 @@ extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, * while the underlying stream is), *runLen is how many consecutive values equal * it. Returns false at end of stream. */ -extern bool ColumnarBlockNextRun(ColumnarBlockReader *br, +extern bool PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen); /* ------------------------------------------------------------------------- - * compression (columnar_compression.c, spec 5) + * compression (pgcolumnar_compression.c, spec 5) * ------------------------------------------------------------------------- */ -extern bool ColumnarCodecAvailable(int compressionType); -extern void ColumnarCompressValueStream(const char *raw, uint32 rawLen, +extern bool PgColumnarCodecAvailable(int compressionType); +extern void PgColumnarCompressValueStream(const char *raw, uint32 rawLen, int requestedType, int level, char **outData, uint32 *outLen, int *usedType, int *usedLevel); -extern char *ColumnarDecompressValueStream(const char *comp, uint32 compLen, +extern char *PgColumnarDecompressValueStream(const char *comp, uint32 compLen, int compressionType, uint32 rawLen, MemoryContext targetContext); /* ------------------------------------------------------------------------- - * concurrent unique-key insert serialization (columnar_unique.c, issue #5) + * concurrent unique-key insert serialization (pgcolumnar_unique.c, issue #5) * * Before an inserted row is handed to the executor's index maintenance, the - * table AM insert paths call ColumnarLockUniqueKeys to take a transaction- + * table AM insert paths call PgColumnarLockUniqueKeys to take a transaction- * scoped advisory lock per applicable unique index key, so a concurrent * inserter of an equal key serializes behind this transaction until it commits * (and has therefore flushed its row), at which point the ordinary btree - * uniqueness check catches the duplicate. See columnar_unique.c. + * uniqueness check catches the duplicate. See pgcolumnar_unique.c. * ------------------------------------------------------------------------- */ -extern void ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot); -extern void ColumnarUniqueInit(void); +extern void PgColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot); +extern void PgColumnarUniqueInit(void); /* ------------------------------------------------------------------------- - * planner integration (columnar_customscan.c, spec 8.3, 9) + * planner integration (pgcolumnar_customscan.c, spec 8.3, 9) * ------------------------------------------------------------------------- */ -extern void ColumnarCustomScanInit(void); +extern void PgColumnarCustomScanInit(void); /* * The single registered CustomScanMethods, shared by the base custom scan and * the vectorized aggregate. The create-state callback dispatches on scanrelid: * a scanrelid==0 upper node is the vectorized aggregate. */ -extern const CustomScanMethods columnar_scan_methods; -extern Node *ColumnarCreateAggScanState(CustomScan *cscan); -extern Node *ColumnarCreateGroupAggScanState(CustomScan *cscan); +extern const CustomScanMethods pgcolumnar_scan_methods; +extern Node *PgColumnarCreateAggScanState(CustomScan *cscan); +extern Node *PgColumnarCreateGroupAggScanState(CustomScan *cscan); /* * Build the chunk-group skip scan keys from a plan's restriction clauses. * Shared by the base custom scan and the vectorized aggregate (spec 9). Clauses * that are not simple "column op const" comparisons are ignored. */ -extern ScanKey ColumnarBuildScanKeys(List *qual, Index scanrelid, +extern ScanKey PgColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, int *nkeys); /* ------------------------------------------------------------------------- - * vectorized aggregation and filtering (columnar_vector.c, spec 9) + * vectorized aggregation and filtering (pgcolumnar_vector.c, spec 9) * ------------------------------------------------------------------------- */ -extern void ColumnarVectorInit(void); +extern void PgColumnarVectorInit(void); /* * The vectorized predicate machinery that used to be declared here is private to - * columnar_vector.c. ColumnarVecRowPasses and ColumnarVecSelect were exported + * pgcolumnar_vector.c. PgColumnarVecRowPasses and PgColumnarVecSelect were exported * with no call site anywhere in the tree and are gone; see issue #200. What * remains of it is a convertibility probe the planner uses, and it has no * business outside that file. @@ -869,7 +869,7 @@ extern void ColumnarVectorInit(void); * duplicating. */ static inline uint32 -ColumnarVarSizeAnyUnaligned(const char *p) +PgColumnarVarSizeAnyUnaligned(const char *p) { uint32 hdr; diff --git a/src/columnar_arrow.c b/src/columnar_arrow.c index 81ebcdb..4ff8bcd 100644 --- a/src/columnar_arrow.c +++ b/src/columnar_arrow.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_arrow.c + * pgcolumnar_arrow.c * Arrow IPC stream export for pgColumnar (gap 27, piece 1). * * pgcolumnar.export_arrow(rel regclass, path text) writes a columnar table @@ -52,8 +52,8 @@ #include "utils/typcache.h" #include "utils/uuid.h" -PG_FUNCTION_INFO_V1(columnar_export_arrow); -PG_FUNCTION_INFO_V1(columnar_import_arrow); +PG_FUNCTION_INFO_V1(pgcolumnar_export_arrow); +PG_FUNCTION_INFO_V1(pgcolumnar_import_arrow); /* one RecordBatch per this many rows */ #define ARROW_BATCH_ROWS 16384 @@ -158,7 +158,7 @@ numeric_to_int128(Datum numd, int scale, __int128 *out) /* ---- Arrow Type table for one column; returns tag via *typetag ---- */ static uint32 -fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) +pgc_fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) { switch (kind) { @@ -168,80 +168,80 @@ fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) { int32 bits = (kind == A_INT16) ? 16 : (kind == A_INT32) ? 32 : 64; - fb_start(b, 2); - fb_add_i32(b, 0, bits, 0); /* bitWidth */ - fb_add_bool(b, 1, true, false); /* is_signed */ + pgc_fb_start(b, 2); + pgc_fb_add_i32(b, 0, bits, 0); /* bitWidth */ + pgc_fb_add_bool(b, 1, true, false); /* is_signed */ *typetag = ARROW_TYPE_Int; - return fb_end(b); + return pgc_fb_end(b); } case A_FLOAT32: case A_FLOAT64: - fb_start(b, 1); - fb_add_i16(b, 0, (kind == A_FLOAT32) ? 1 : 2, 0); /* SINGLE/DOUBLE */ + pgc_fb_start(b, 1); + pgc_fb_add_i16(b, 0, (kind == A_FLOAT32) ? 1 : 2, 0); /* SINGLE/DOUBLE */ *typetag = ARROW_TYPE_FloatingPoint; - return fb_end(b); + return pgc_fb_end(b); case A_BOOL: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Bool; - return fb_end(b); + return pgc_fb_end(b); case A_UTF8: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Utf8; - return fb_end(b); + return pgc_fb_end(b); case A_BINARY: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Binary; - return fb_end(b); + return pgc_fb_end(b); case A_DATE32: /* Date { unit: DateUnit = MILLISECOND (1) }; want DAY (0) */ - fb_start(b, 1); - fb_add_i16(b, 0, 0, 1); + pgc_fb_start(b, 1); + pgc_fb_add_i16(b, 0, 0, 1); *typetag = ARROW_TYPE_Date; - return fb_end(b); + return pgc_fb_end(b); case A_TIME64: /* Time { unit: TimeUnit = MILLISECOND (1); bitWidth: int = 32 } */ - fb_start(b, 2); - fb_add_i16(b, 0, 2, 1); /* MICROSECOND */ - fb_add_i32(b, 1, 64, 32); + pgc_fb_start(b, 2); + pgc_fb_add_i16(b, 0, 2, 1); /* MICROSECOND */ + pgc_fb_add_i32(b, 1, 64, 32); *typetag = ARROW_TYPE_Time; - return fb_end(b); + return pgc_fb_end(b); case A_TIMESTAMP: case A_TIMESTAMPTZ: { uint32 tzOff = 0; if (kind == A_TIMESTAMPTZ) - tzOff = fb_create_string(b, "UTC"); + tzOff = pgc_fb_create_string(b, "UTC"); /* Timestamp { unit: TimeUnit = SECOND (0); timezone: string } */ - fb_start(b, 2); - fb_add_i16(b, 0, 2, 0); /* MICROSECOND */ - fb_add_offset(b, 1, tzOff); + pgc_fb_start(b, 2); + pgc_fb_add_i16(b, 0, 2, 0); /* MICROSECOND */ + pgc_fb_add_offset(b, 1, tzOff); *typetag = ARROW_TYPE_Timestamp; - return fb_end(b); + return pgc_fb_end(b); } case A_UUID: /* FixedSizeBinary { byteWidth: int } */ - fb_start(b, 1); - fb_add_i32(b, 0, 16, 0); + pgc_fb_start(b, 1); + pgc_fb_add_i32(b, 0, 16, 0); *typetag = ARROW_TYPE_FixedSizeBinary; - return fb_end(b); + return pgc_fb_end(b); case A_DECIMAL128: /* Decimal { precision: int; scale: int; bitWidth: int = 128 } */ - fb_start(b, 3); - fb_add_i32(b, 0, precision, 0); - fb_add_i32(b, 1, scale, 0); + pgc_fb_start(b, 3); + pgc_fb_add_i32(b, 0, precision, 0); + pgc_fb_add_i32(b, 1, scale, 0); *typetag = ARROW_TYPE_Decimal; - return fb_end(b); + return pgc_fb_end(b); case A_LIST: /* List {} -- the element type is carried in the Field's children */ - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_List; - return fb_end(b); + return pgc_fb_end(b); case A_STRUCT: /* Struct_ {} -- the field types are the Field's children */ - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Struct; - return fb_end(b); + return pgc_fb_end(b); } *typetag = 0; return 0; /* unreachable */ @@ -809,7 +809,7 @@ arrow_emit_buffers(StringInfo body, ArrowCol *c, static uint32 arrow_build_field(FBB *b, ArrowCol *c) { - uint32 nameOff = fb_create_string(b, c->name ? c->name : ""); + uint32 nameOff = pgc_fb_create_string(b, c->name ? c->name : ""); uint8 typetag; uint32 typeOff; uint32 childrenVec = 0; @@ -821,22 +821,22 @@ arrow_build_field(FBB *b, ArrowCol *c) childOff = palloc(sizeof(uint32) * c->nchildren); for (i = 0; i < c->nchildren; i++) childOff[i] = arrow_build_field(b, &c->children[i]); - fb_start_vector(b, 4, c->nchildren, 4); + pgc_fb_start_vector(b, 4, c->nchildren, 4); for (i = c->nchildren - 1; i >= 0; i--) - fb_push_uoffset(b, childOff[i]); - childrenVec = fb_end_vector(b, c->nchildren); + pgc_fb_push_uoffset(b, childOff[i]); + childrenVec = pgc_fb_end_vector(b, c->nchildren); } - typeOff = fb_arrow_type(b, c->kind, c->precision, c->scale, &typetag); + typeOff = pgc_fb_arrow_type(b, c->kind, c->precision, c->scale, &typetag); - fb_start(b, 7); - fb_add_offset(b, 0, nameOff); /* name */ - fb_add_bool(b, 1, true, false); /* nullable */ - fb_add_u8(b, 2, typetag, 0); /* type_type */ - fb_add_offset(b, 3, typeOff); /* type */ + pgc_fb_start(b, 7); + pgc_fb_add_offset(b, 0, nameOff); /* name */ + pgc_fb_add_bool(b, 1, true, false); /* nullable */ + pgc_fb_add_u8(b, 2, typetag, 0); /* type_type */ + pgc_fb_add_offset(b, 3, typeOff); /* type */ if (c->nchildren > 0) - fb_add_offset(b, 5, childrenVec); /* children (Field slot 5) */ - return fb_end(b); + pgc_fb_add_offset(b, 5, childrenVec); /* children (Field slot 5) */ + return pgc_fb_end(b); } /* build one RecordBatch (metadata + body) and write it */ @@ -883,41 +883,41 @@ write_record_batch(FILE *f, ArrowCol *cols, int ncols, int64 nrows) } /* ---- RecordBatch metadata flatbuffer ---- */ - fb_init(&b); + pgc_fb_init(&b); /* nodes vector: [FieldNode{length,null_count}] structs, 16B/8-align */ - fb_start_vector(&b, 16, nnodes, 8); + pgc_fb_start_vector(&b, 16, nnodes, 8); for (i = nnodes - 1; i >= 0; i--) { - fb_prep(&b, 8, 0); - fb_place(&b, &nodeNull[i], 8); /* null_count (higher) */ - fb_place(&b, &nodeLen[i], 8); /* length (lower) */ + pgc_fb_prep(&b, 8, 0); + pgc_fb_place(&b, &nodeNull[i], 8); /* null_count (higher) */ + pgc_fb_place(&b, &nodeLen[i], 8); /* length (lower) */ } - nodesVec = fb_end_vector(&b, nnodes); + nodesVec = pgc_fb_end_vector(&b, nnodes); /* buffers vector: [Buffer{offset,length}] structs */ - fb_start_vector(&b, 16, nbuf, 8); + pgc_fb_start_vector(&b, 16, nbuf, 8); for (i = nbuf - 1; i >= 0; i--) { - fb_prep(&b, 8, 0); - fb_place(&b, &bufLen[i], 8); /* length (higher) */ - fb_place(&b, &bufOff[i], 8); /* offset (lower) */ + pgc_fb_prep(&b, 8, 0); + pgc_fb_place(&b, &bufLen[i], 8); /* length (higher) */ + pgc_fb_place(&b, &bufOff[i], 8); /* offset (lower) */ } - bufsVec = fb_end_vector(&b, nbuf); - - fb_start(&b, 4); - fb_add_i64(&b, 0, nrows, 0); - fb_add_offset(&b, 1, nodesVec); - fb_add_offset(&b, 2, bufsVec); - rbOff = fb_end(&b); - - fb_start(&b, 5); - fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); - fb_add_u8(&b, 1, ARROW_MSG_RecordBatch, 0); - fb_add_offset(&b, 2, rbOff); - fb_add_i64(&b, 3, body.len, 0); /* bodyLength */ - msgOff = fb_end(&b); - fb_finish(&b, msgOff); + bufsVec = pgc_fb_end_vector(&b, nbuf); + + pgc_fb_start(&b, 4); + pgc_fb_add_i64(&b, 0, nrows, 0); + pgc_fb_add_offset(&b, 1, nodesVec); + pgc_fb_add_offset(&b, 2, bufsVec); + rbOff = pgc_fb_end(&b); + + pgc_fb_start(&b, 5); + pgc_fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); + pgc_fb_add_u8(&b, 1, ARROW_MSG_RecordBatch, 0); + pgc_fb_add_offset(&b, 2, rbOff); + pgc_fb_add_i64(&b, 3, body.len, 0); /* bodyLength */ + msgOff = pgc_fb_end(&b); + pgc_fb_finish(&b, msgOff); /* ---- write encapsulated message ---- */ metaLen = b.tail; @@ -940,13 +940,13 @@ write_record_batch(FILE *f, ArrowCol *cols, int ncols, int64 nrows) } /* - * columnar_export_arrow + * pgcolumnar_export_arrow * SQL: pgcolumnar.export_arrow(rel regclass, path text) -> bigint. * Write a columnar table to an Arrow IPC stream file; returns the number * of rows written. */ Datum -columnar_export_arrow(PG_FUNCTION_ARGS) +pgcolumnar_export_arrow(PG_FUNCTION_ARGS) { Oid relid; text *pathText; @@ -956,7 +956,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) int ncols; ArrowCol *cols; Snapshot snapshot; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *values; bool *nulls; uint64 rowNumber; @@ -987,7 +987,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) path = text_to_cstring(pathText); rel = table_open(relid, AccessShareLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessShareLock); ereport(ERROR, @@ -1045,25 +1045,25 @@ columnar_export_arrow(PG_FUNCTION_ARGS) /* ---- Schema message ---- */ fieldOff = palloc(sizeof(uint32) * ncols); - fb_init(&b); + pgc_fb_init(&b); for (i = 0; i < ncols; i++) fieldOff[i] = arrow_build_field(&b, &cols[i]); - fb_start_vector(&b, 4, ncols, 4); + pgc_fb_start_vector(&b, 4, ncols, 4); for (i = ncols - 1; i >= 0; i--) - fb_push_uoffset(&b, fieldOff[i]); - vec = fb_end_vector(&b, ncols); + pgc_fb_push_uoffset(&b, fieldOff[i]); + vec = pgc_fb_end_vector(&b, ncols); - fb_start(&b, 4); + pgc_fb_start(&b, 4); /* endianness Little=0 is the default, so omit slot 0 */ - fb_add_offset(&b, 1, vec); /* fields */ - schemaOff = fb_end(&b); + pgc_fb_add_offset(&b, 1, vec); /* fields */ + schemaOff = pgc_fb_end(&b); - fb_start(&b, 5); - fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); - fb_add_u8(&b, 1, ARROW_MSG_Schema, 0); - fb_add_offset(&b, 2, schemaOff); - msgOff = fb_end(&b); - fb_finish(&b, msgOff); + pgc_fb_start(&b, 5); + pgc_fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); + pgc_fb_add_u8(&b, 1, ARROW_MSG_Schema, 0); + pgc_fb_add_offset(&b, 2, schemaOff); + msgOff = pgc_fb_end(&b); + pgc_fb_finish(&b, msgOff); { uint32 cont = 0xFFFFFFFF; @@ -1089,9 +1089,9 @@ columnar_export_arrow(PG_FUNCTION_ARGS) nulls = palloc(sizeof(bool) * ncols); snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(readState, values, nulls, &rowNumber)) { CHECK_FOR_INTERRUPTS(); for (i = 0; i < ncols; i++) @@ -1110,7 +1110,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) batchRows = 0; } } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); if (batchRows > 0) { @@ -1205,7 +1205,7 @@ fbr_i64(const uint8 *b, uint32 len, uint32 pos) /* absolute position of field `i` of the table at `tab`, or 0 if absent */ static uint32 -fb_field(const uint8 *b, uint32 len, uint32 tab, int i) +pgc_fb_field(const uint8 *b, uint32 len, uint32 tab, int i) { int32 soff = fbr_i32(b, len, tab); int64 vt = (int64) tab - soff; @@ -1226,7 +1226,7 @@ fb_field(const uint8 *b, uint32 len, uint32 tab, int i) /* follow the uoffset stored at `pos` to the object it points at */ static uint32 -fb_indirect(const uint8 *b, uint32 len, uint32 pos) +pgc_fb_indirect(const uint8 *b, uint32 len, uint32 pos) { return pos + fbr_u32(b, len, pos); } @@ -1726,13 +1726,13 @@ imp_check_bounds(ImpNode *n, const uint8 *body, const int64 *bufOff, } /* - * columnar_import_arrow + * pgcolumnar_import_arrow * SQL: pgcolumnar.import_arrow(rel regclass, path text) -> bigint. * Insert the rows of an Arrow IPC stream file into a columnar table; * returns the number of rows inserted. */ Datum -columnar_import_arrow(PG_FUNCTION_ARGS) +pgcolumnar_import_arrow(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1743,7 +1743,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) int totalBuffers = 0; FILE *f; TupleTableSlot *slot; - ColumnarIndexInsertState *indexes; + PgColumnarIndexInsertState *indexes; CommandId cid; MemoryContext rowCtx; int64 total = 0; @@ -1763,7 +1763,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) path = text_to_cstring(PG_GETARG_TEXT_PP(1)); rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1812,8 +1812,8 @@ columnar_import_arrow(PG_FUNCTION_ARGS) slot = table_slot_create(rel, NULL); cid = GetCurrentCommandId(true); - indexes = ColumnarRelationHasIndexes(rel) - ? ColumnarIndexInsertBegin(rel, true) : NULL; + indexes = PgColumnarRelationHasIndexes(rel) + ? PgColumnarIndexInsertBegin(rel, true) : NULL; /* * Per-row scratch context. Reconstructing a nested value (array/composite) @@ -1855,12 +1855,12 @@ columnar_import_arrow(PG_FUNCTION_ARGS) if (fread(meta, 1, metaLen, f) != metaLen) IMPORT_CORRUPT("truncated metadata"); - msg = fb_indirect(meta, metaLen, 0); - pos = fb_field(meta, metaLen, msg, 1); /* header_type (u8) */ + msg = pgc_fb_indirect(meta, metaLen, 0); + pos = pgc_fb_field(meta, metaLen, msg, 1); /* header_type (u8) */ headerType = pos ? fbr_u8(meta, metaLen, pos) : 0; - pos = fb_field(meta, metaLen, msg, 2); /* header (offset) */ - hdr = pos ? fb_indirect(meta, metaLen, pos) : 0; - pos = fb_field(meta, metaLen, msg, 3); /* bodyLength (i64) */ + pos = pgc_fb_field(meta, metaLen, msg, 2); /* header (offset) */ + hdr = pos ? pgc_fb_indirect(meta, metaLen, pos) : 0; + pos = pgc_fb_field(meta, metaLen, msg, 3); /* bodyLength (i64) */ bodyLength = pos ? fbr_i64(meta, metaLen, pos) : 0; if (bodyLength < 0) IMPORT_CORRUPT("negative body length"); @@ -1874,9 +1874,9 @@ columnar_import_arrow(PG_FUNCTION_ARGS) if (headerType == ARROW_MSG_Schema) { - uint32 fieldsVecPos = hdr ? fb_field(meta, metaLen, hdr, 1) : 0; + uint32 fieldsVecPos = hdr ? pgc_fb_field(meta, metaLen, hdr, 1) : 0; uint32 fieldsVec = fieldsVecPos ? - fb_indirect(meta, metaLen, fieldsVecPos) : 0; + pgc_fb_indirect(meta, metaLen, fieldsVecPos) : 0; uint32 nfields = fieldsVec ? fbr_u32(meta, metaLen, fieldsVec) : 0; if ((int) nfields != ncols) @@ -1898,16 +1898,16 @@ columnar_import_arrow(PG_FUNCTION_ARGS) IMPORT_CORRUPT("RecordBatch before Schema"); if (!hdr) IMPORT_CORRUPT("missing RecordBatch header"); - if (fb_field(meta, metaLen, hdr, 3) != 0) + if (pgc_fb_field(meta, metaLen, hdr, 3) != 0) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("columnar.import_arrow does not support compressed Arrow bodies"))); - pos = fb_field(meta, metaLen, hdr, 0); /* length */ + pos = pgc_fb_field(meta, metaLen, hdr, 0); /* length */ nrows = pos ? fbr_i64(meta, metaLen, pos) : 0; - nodesVecPos = fb_field(meta, metaLen, hdr, 1); - buffersVecPos = fb_field(meta, metaLen, hdr, 2); - buffersVec = buffersVecPos ? fb_indirect(meta, metaLen, buffersVecPos) : 0; + nodesVecPos = pgc_fb_field(meta, metaLen, hdr, 1); + buffersVecPos = pgc_fb_field(meta, metaLen, hdr, 2); + buffersVec = buffersVecPos ? pgc_fb_indirect(meta, metaLen, buffersVecPos) : 0; nbuffers = buffersVec ? fbr_u32(meta, metaLen, buffersVec) : 0; (void) nodesVecPos; @@ -1968,9 +1968,9 @@ columnar_import_arrow(PG_FUNCTION_ARGS) * (issue #153). tts_tid carries the assigned row number. */ if (indexes != NULL) - ColumnarIndexInsertRow(indexes, rel, slot->tts_values, + PgColumnarIndexInsertRow(indexes, rel, slot->tts_values, slot->tts_isnull, - ColumnarItemPointerToRowNumber(&slot->tts_tid)); + PgColumnarItemPointerToRowNumber(&slot->tts_tid)); MemoryContextSwitchTo(oldCtx); MemoryContextReset(rowCtx); total++; @@ -1993,7 +1993,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) FreeFile(f); MemoryContextDelete(rowCtx); if (indexes != NULL) - ColumnarIndexInsertEnd(indexes); + PgColumnarIndexInsertEnd(indexes); ExecDropSingleTupleTableSlot(slot); /* diff --git a/src/columnar_bloom.c b/src/columnar_bloom.c index 462e4c3..fd78b02 100644 --- a/src/columnar_bloom.c +++ b/src/columnar_bloom.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_bloom.c + * pgcolumnar_bloom.c * Per-chunk bloom filters for equality chunk-group skipping (I7). * * min/max skip lists (spec 7.2) prune equality predicates only when the probed @@ -28,14 +28,14 @@ #include "utils/syscache.h" /* - * ColumnarCollationIsDeterministic + * PgColumnarCollationIsDeterministic * Whether a bloom filter is safe for this collation: InvalidOid (a * non-collatable type) and deterministic collations qualify; a * nondeterministic collation does not, since equal values need not be * byte-identical and would hash inconsistently. */ bool -ColumnarCollationIsDeterministic(Oid collid) +PgColumnarCollationIsDeterministic(Oid collid) { HeapTuple tp; bool result = true; @@ -77,12 +77,12 @@ bloom_hashes(uint32 h, uint32 *h1, uint32 *h2) } /* - * ColumnarBloomBuild + * PgColumnarBloomBuild * Build a filter over n precomputed value hashes. Returns false (no filter) * when n is too small for a filter to be worthwhile. */ bool -ColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) +PgColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) { uint32 nbits; uint32 nbytes; @@ -142,13 +142,13 @@ ColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) } /* - * ColumnarBloomProbe + * PgColumnarBloomProbe * Return true when the hash may be present (all k bits set), false when it * is definitely absent. A malformed/empty filter conservatively returns * true (never skips wrongly). */ bool -ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash) +PgColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash) { uint32 nbits; uint8 k; diff --git a/src/columnar_compat.h b/src/columnar_compat.h index 3db88a6..e7c871c 100644 --- a/src/columnar_compat.h +++ b/src/columnar_compat.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_compat.h + * pgcolumnar_compat.h * PostgreSQL major-version compatibility shims for pgColumnar. * * pgColumnar keeps a single source tree that builds on PostgreSQL 13 through @@ -72,10 +72,10 @@ #include "utils/lsyscache.h" #if PG_VERSION_NUM >= 180000 -#define ColumnarOpInterpretation OpIndexInterpretation -#define ColumnarGetOpInterpretation(opno) get_op_index_interpretation(opno) +#define PgColumnarOpInterpretation OpIndexInterpretation +#define PgColumnarGetOpInterpretation(opno) get_op_index_interpretation(opno) static inline int -ColumnarOpInterpStrategy(const OpIndexInterpretation *o) +PgColumnarOpInterpStrategy(const OpIndexInterpretation *o) { switch (o->cmptype) { @@ -94,10 +94,10 @@ ColumnarOpInterpStrategy(const OpIndexInterpretation *o) } } #else -#define ColumnarOpInterpretation OpBtreeInterpretation -#define ColumnarGetOpInterpretation(opno) get_op_btree_interpretation(opno) +#define PgColumnarOpInterpretation OpBtreeInterpretation +#define PgColumnarGetOpInterpretation(opno) get_op_btree_interpretation(opno) static inline int -ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) +PgColumnarOpInterpStrategy(const OpBtreeInterpretation *o) { return o->strategy; } @@ -122,10 +122,10 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * only (locator, persistence). Heap passes true, and so do we. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 150000 -#define ColumnarRelationCreateStorage(loc, persistence) \ +#define PgColumnarRelationCreateStorage(loc, persistence) \ RelationCreateStorage((loc), (persistence)) #else -#define ColumnarRelationCreateStorage(loc, persistence) \ +#define PgColumnarRelationCreateStorage(loc, persistence) \ RelationCreateStorage((loc), (persistence), true) #endif @@ -134,7 +134,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * index_delete_tuples(Relation, TM_IndexDeleteOp *) in PG14+. In PG13 the slot * was compute_xid_horizon_for_tuples(Relation, ItemPointerData *, int). The two * take different arguments, so the callback itself is compiled per major (see - * columnar_tableam.c); this macro only selects the struct field name. + * pgcolumnar_tableam.c); this macro only selects the struct field name. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 140000 #define COLUMNAR_AM_INDEX_DELETE_FIELD compute_xid_horizon_for_tuples @@ -156,15 +156,15 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) /* ------------------------------------------------------------------------- * tuple_update() reports which indexes to maintain through a pointer whose * target changed from bool (PG13-15) to the TU_UpdateIndexes enum (PG16+). We - * want "maintain all indexes", which is true / TU_All. ColumnarUpdateIndexes is + * want "maintain all indexes", which is true / TU_All. PgColumnarUpdateIndexes is * a macro (not a typedef) so the enum name is not referenced until the .c file * has already included access/tableam.h, which defines it. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 160000 -#define ColumnarUpdateIndexes bool +#define PgColumnarUpdateIndexes bool #define COLUMNAR_TU_ALL true #else -#define ColumnarUpdateIndexes TU_UpdateIndexes +#define PgColumnarUpdateIndexes TU_UpdateIndexes #define COLUMNAR_TU_ALL TU_All #endif @@ -198,13 +198,13 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) Relation rel, ItemPointer otid, TupleTableSlot *slot, CommandId cid, \ uint32 options, Snapshot snapshot, Snapshot crosscheck, bool wait, \ TM_FailureData *tmfd, LockTupleMode *lockmode, \ - ColumnarUpdateIndexes *update_indexes + PgColumnarUpdateIndexes *update_indexes #else #define COLUMNAR_TUPLE_UPDATE_ARGS \ Relation rel, ItemPointer otid, TupleTableSlot *slot, CommandId cid, \ Snapshot snapshot, Snapshot crosscheck, bool wait, \ TM_FailureData *tmfd, LockTupleMode *lockmode, \ - ColumnarUpdateIndexes *update_indexes + PgColumnarUpdateIndexes *update_indexes #endif /* ------------------------------------------------------------------------- @@ -252,7 +252,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * scan_analyze_next_block() took (BlockNumber, BufferAccessStrategy) through * PG16 and (ReadStream *) from PG17 (the read-stream ANALYZE rework). Pre-17 * has no ReadStream type. The callback is compiled per major in - * columnar_tableam.c; this macro supplies its parameter list. + * pgcolumnar_tableam.c; this macro supplies its parameter list. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 170000 #define COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS \ @@ -267,7 +267,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * argument in PG18. An access method supplying its own slot operations has to * match the struct exactly, so the wrapper is declared and forwarded through * these two macros. Keep the boundary here in step with the callback in - * columnar_tableam.c; when the analyze block callback's guard drifted from the + * pgcolumnar_tableam.c; when the analyze block callback's guard drifted from the * macro that supplies its parameters, PG17 stopped compiling. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM >= 180000 @@ -294,9 +294,9 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * snapshot. Callers include "storage/procarray.h". * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM >= 140000 -#define ColumnarOldestXmin(rel) GetOldestNonRemovableTransactionId(rel) +#define PgColumnarOldestXmin(rel) GetOldestNonRemovableTransactionId(rel) #else -#define ColumnarOldestXmin(rel) GetOldestXmin((rel), PROCARRAY_FLAGS_VACUUM) +#define PgColumnarOldestXmin(rel) GetOldestXmin((rel), PROCARRAY_FLAGS_VACUUM) #endif /* ------------------------------------------------------------------------- @@ -306,9 +306,9 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * smgrextend require on those majors. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 160000 -#define ColumnarAllocPage() ((Page) palloc0(BLCKSZ)) +#define PgColumnarAllocPage() ((Page) palloc0(BLCKSZ)) #else -#define ColumnarAllocPage() \ +#define PgColumnarAllocPage() \ ((Page) palloc_aligned(BLCKSZ, PG_IO_ALIGN_SIZE, MCXT_ALLOC_ZERO)) #endif @@ -380,7 +380,7 @@ extern void _PG_init(void); #include "catalog/index.h" static inline void -ColumnarReindexRelation(Oid relid, int flags) +PgColumnarReindexRelation(Oid relid, int flags) { #if PG_VERSION_NUM < 140000 reindex_relation(relid, flags, 0); diff --git a/src/columnar_compression.c b/src/columnar_compression.c index 084ab29..004adfb 100644 --- a/src/columnar_compression.c +++ b/src/columnar_compression.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_compression.c + * pgcolumnar_compression.c * Value-stream compression codecs for pgColumnar (spec 5): none, pglz, * lz4, and zstd. Each chunk's value stream is compressed independently. * If a codec does not shrink the data, or the codec is not built into @@ -29,12 +29,12 @@ #endif /* - * ColumnarCodecAvailable + * PgColumnarCodecAvailable * Whether a compression type can be produced by this binary. none and * pglz are always available; lz4 and zstd depend on the build. */ bool -ColumnarCodecAvailable(int compressionType) +PgColumnarCodecAvailable(int compressionType) { switch (compressionType) { @@ -114,7 +114,7 @@ try_zstd(const char *raw, uint32 rawLen, char *dest, size_t destCap, int level) #endif /* - * ColumnarCompressValueStream + * PgColumnarCompressValueStream * Compress rawLen bytes at raw using the requested codec at the given * level. On success at shrinking the data, returns a palloc'd buffer in * *outData with its length in *outLen and the codec actually used in @@ -125,7 +125,7 @@ try_zstd(const char *raw, uint32 rawLen, char *dest, size_t destCap, int level) * The output is always allocated in the current memory context. */ void -ColumnarCompressValueStream(const char *raw, uint32 rawLen, +PgColumnarCompressValueStream(const char *raw, uint32 rawLen, int requestedType, int level, char **outData, uint32 *outLen, int *usedType, int *usedLevel) @@ -227,14 +227,14 @@ ColumnarCompressValueStream(const char *raw, uint32 rawLen, } /* - * ColumnarDecompressValueStream + * PgColumnarDecompressValueStream * Decompress compLen bytes at comp of the given codec into a fresh buffer * of rawLen bytes allocated in targetContext, and return it. For type * none the bytes are copied. Errors out if the codec is not built in or * the decoded length does not match rawLen. */ char * -ColumnarDecompressValueStream(const char *comp, uint32 compLen, +PgColumnarDecompressValueStream(const char *comp, uint32 compLen, int compressionType, uint32 rawLen, MemoryContext targetContext) { diff --git a/src/columnar_customscan.c b/src/columnar_customscan.c index 4414fb2..6572757 100644 --- a/src/columnar_customscan.c +++ b/src/columnar_customscan.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_customscan.c + * pgcolumnar_customscan.c * Planner and executor integration for pgColumnar (spec 8.3, 9). * * A set_rel_pathlist_hook replaces the sequential-scan path of a columnar @@ -60,19 +60,19 @@ #include "utils/typcache.h" /* GUC: use the columnar custom scan path (spec 8.3) */ -bool columnar_enable_custom_scan = true; +bool pgcolumnar_enable_custom_scan = true; /* GUC: let the planner scan a covering projection instead of the base (gap 26) */ -bool columnar_enable_projection_scan = true; +bool pgcolumnar_enable_projection_scan = true; /* GUC: price a columnar index scan's per-row heap fetch (#355) */ -bool columnar_enable_index_fetch_penalty = true; +bool pgcolumnar_enable_index_fetch_penalty = true; static set_rel_pathlist_hook_type prev_set_rel_pathlist_hook = NULL; /* our executor-time scan state embeds CustomScanState as its first field */ -typedef struct ColumnarCustomScanState +typedef struct PgColumnarCustomScanState { CustomScanState css; - ColumnarReadState *readState; + PgColumnarReadState *readState; Bitmapset *projectedColumns; /* 0-based; NULL means all columns */ ScanKey scanKeys; int nScanKeys; @@ -97,63 +97,63 @@ typedef struct ColumnarCustomScanState int *projColMap; /* base attno-1 -> index into projValues, or -1 */ Datum *projValues; /* scratch, length K+1 (index 0 = rownumber) */ bool *projNulls; - ColumnarLivenessCache *livenessCache; /* cached base liveness for the scan */ -} ColumnarCustomScanState; + PgColumnarLivenessCache *livenessCache; /* cached base liveness for the scan */ +} PgColumnarCustomScanState; /* path -> plan */ -static Plan *ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, +static Plan *PgColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans); /* plan -> scan state (dispatches base vs vectorized-aggregate) */ -static Node *ColumnarCreateScanState(CustomScan *cscan); -static Node *ColumnarCreateBaseScanState(CustomScan *cscan); +static Node *PgColumnarCreateScanState(CustomScan *cscan); +static Node *PgColumnarCreateBaseScanState(CustomScan *cscan); /* executor callbacks */ -static void ColumnarBeginCustomScan(CustomScanState *node, EState *estate, +static void PgColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags); -static TupleTableSlot *ColumnarExecCustomScan(CustomScanState *node); -static void ColumnarEndCustomScan(CustomScanState *node); -static void ColumnarReScanCustomScan(CustomScanState *node); -static Size ColumnarEstimateDSMCustomScan(CustomScanState *node, +static TupleTableSlot *PgColumnarExecCustomScan(CustomScanState *node); +static void PgColumnarEndCustomScan(CustomScanState *node); +static void PgColumnarReScanCustomScan(CustomScanState *node); +static Size PgColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt); -static void ColumnarInitializeDSMCustomScan(CustomScanState *node, +static void PgColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate); -static void ColumnarReInitializeDSMCustomScan(CustomScanState *node, +static void PgColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate); -static void ColumnarInitializeWorkerCustomScan(CustomScanState *node, +static void PgColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, void *coordinate); -static void ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, +static void PgColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainState *es); /* ExecScan helpers */ -static TupleTableSlot *ColumnarScanNext(ScanState *ss); -static bool ColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot); +static TupleTableSlot *PgColumnarScanNext(ScanState *ss); +static bool PgColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot); -static const CustomPathMethods columnar_path_methods = { +static const CustomPathMethods pgcolumnar_path_methods = { .CustomName = "ColumnarScan", - .PlanCustomPath = ColumnarPlanCustomPath, + .PlanCustomPath = PgColumnarPlanCustomPath, .ReparameterizeCustomPathByChild = NULL, }; -const CustomScanMethods columnar_scan_methods = { +const CustomScanMethods pgcolumnar_scan_methods = { .CustomName = "ColumnarScan", - .CreateCustomScanState = ColumnarCreateScanState, + .CreateCustomScanState = PgColumnarCreateScanState, }; -static const CustomExecMethods columnar_exec_methods = { +static const CustomExecMethods pgcolumnar_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginCustomScan, - .ExecCustomScan = ColumnarExecCustomScan, - .EndCustomScan = ColumnarEndCustomScan, - .ReScanCustomScan = ColumnarReScanCustomScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMCustomScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMCustomScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMCustomScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerCustomScan, - .ExplainCustomScan = ColumnarExplainCustomScan, + .BeginCustomScan = PgColumnarBeginCustomScan, + .ExecCustomScan = PgColumnarExecCustomScan, + .EndCustomScan = PgColumnarEndCustomScan, + .ReScanCustomScan = PgColumnarReScanCustomScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMCustomScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMCustomScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMCustomScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerCustomScan, + .ExplainCustomScan = PgColumnarExplainCustomScan, }; /* ------------------------------------------------------------------------- @@ -161,7 +161,7 @@ static const CustomExecMethods columnar_exec_methods = { * ------------------------------------------------------------------------- */ /* - * columnar_projected_columns + * pgcolumnar_projected_columns * Build the 0-based set of columns the plan actually references, from the * Vars in its target list and its restriction clauses. Returns NULL when * a whole-row or system column is requested, meaning "all columns", so the @@ -170,7 +170,7 @@ static const CustomExecMethods columnar_exec_methods = { * (spec 9). */ static Bitmapset * -columnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) +pgcolumnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) { Bitmapset *needed = NULL; Bitmapset *projected = NULL; @@ -211,12 +211,12 @@ columnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) } /* - * columnar_commute_strategy + * pgcolumnar_commute_strategy * The btree comparison strategy for "value op column" given the strategy * for "column op value", used when the constant is on the left. */ static StrategyNumber -columnar_commute_strategy(StrategyNumber s) +pgcolumnar_commute_strategy(StrategyNumber s) { switch (s) { @@ -236,7 +236,7 @@ columnar_commute_strategy(StrategyNumber s) } /* - * columnar_clause_to_scankey + * pgcolumnar_clause_to_scankey * Translate a single restriction clause of the form "column op const" * (or "const op column") into a scan key for chunk-group skipping, when * op is a btree comparison operator in the column type's default btree @@ -245,7 +245,7 @@ columnar_commute_strategy(StrategyNumber s) * executor still applies them as a filter, so results are unaffected. */ static bool -columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, +pgcolumnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, ScanKey key) { OpExpr *op; @@ -294,7 +294,7 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, /* * The stored per-chunk min/max are ordered under the column's own - * collation (that is what the writer used, columnar_write_state.c), and the + * collation (that is what the writer used, pgcolumnar_write_state.c), and the * reader evaluates the skip under that same collation. Only push a * predicate whose comparison uses that collation; otherwise a differently * collated comparison (for example an explicit COLLATE in the query) could @@ -315,12 +315,12 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, if (strat == InvalidStrategy) return false; if (!varOnLeft) - strat = columnar_commute_strategy(strat); + strat = pgcolumnar_commute_strategy(strat); if (strat == InvalidStrategy) return false; /* - * Fill the scan key directly. The reader (columnar_build_predicates) uses + * Fill the scan key directly. The reader (pgcolumnar_build_predicates) uses * sk_attno, sk_strategy, sk_subtype and sk_argument, and looks up the * column type's own comparison proc; it never calls sk_func, so we leave * that zeroed rather than build one for a possibly-cross-type operator. @@ -337,13 +337,13 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, } /* - * ColumnarBuildScanKeys + * PgColumnarBuildScanKeys * Build the scan-key array for chunk-group skipping from a plan's * restriction clauses (spec 9). Clauses that are not simple comparisons - * are skipped. Shared with the vectorized aggregate (columnar_vector.c). + * are skipped. Shared with the vectorized aggregate (pgcolumnar_vector.c). */ ScanKey -ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, +PgColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, int *nkeys) { ScanKey keys; @@ -357,7 +357,7 @@ ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, keys = (ScanKey) palloc0(sizeof(ScanKeyData) * list_length(qual)); foreach(lc, qual) { - if (columnar_clause_to_scankey((Node *) lfirst(lc), scanrelid, tupdesc, + if (pgcolumnar_clause_to_scankey((Node *) lfirst(lc), scanrelid, tupdesc, &keys[n])) n++; } @@ -367,14 +367,14 @@ ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, } /* - * ColumnarPlanCustomPath + * PgColumnarPlanCustomPath * Convert the CustomPath to a CustomScan plan node. The restriction * clauses become the scan's qual so the executor re-applies them (this is * what makes chunk-group skipping safe); custom_scan_tlist stays NIL so * the scan tuple is the full base-relation rowtype. */ static Plan * -ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, +PgColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans) { @@ -389,13 +389,13 @@ ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, /* carry the chosen projection name (gap 26), if any, into the plan */ cscan->custom_private = best_path->custom_private; cscan->custom_scan_tlist = NIL; - cscan->methods = &columnar_scan_methods; + cscan->methods = &pgcolumnar_scan_methods; return &cscan->scan.plan; } /* - * columnar_choose_projection + * pgcolumnar_choose_projection * Pick a projection that serves this scan better than the base: it must * cover every referenced column (so no base reconstruction is needed at * scan time) and its leading sort column must appear in a restriction @@ -404,7 +404,7 @@ ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, * reference disqualifies a projection scan. */ static char * -columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) +pgcolumnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) { uint64 storageId; Relation r; @@ -417,16 +417,16 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) int x; bool haveAdditional = false; - if (!columnar_enable_projection_scan) + if (!pgcolumnar_enable_projection_scan) return NULL; r = table_open(relid, AccessShareLock); - storageId = ColumnarStorageId(r); + storageId = PgColumnarStorageId(r); table_close(r, AccessShareLock); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) - if (((ColumnarProjection *) lfirst(lc))->projectionId > 0) + if (((PgColumnarProjection *) lfirst(lc))->projectionId > 0) haveAdditional = true; if (!haveAdditional) return NULL; @@ -447,7 +447,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); bool covers = true; bool skips; int y; @@ -494,7 +494,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) } /* - * columnar_index_correlation + * pgcolumnar_index_correlation * |correlation| of the index's leading key against heap (row-number) order, * read from pg_statistic exactly as btcostestimate does (selfuncs.c). A value * near 1 means rows an ordered index scan visits are already clustered into a @@ -507,7 +507,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) * key, no ANALYZE, no correlation slot. */ static double -columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) +pgcolumnar_index_correlation(IndexOptInfo *index, Oid heapRelid) { AttrNumber attno; Oid sortop; @@ -549,14 +549,14 @@ columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) } /* - * columnar_scan_decode_shape + * pgcolumnar_scan_decode_shape * How much a by-row-number fetch of this rel actually decodes: the number of * columns and their summed width. * * Not the columns the scan emits. The deferred index-fetch slot decodes the * attribute *prefix* 0..max-referenced, because slot_getsomeattrs asks for a - * prefix and cannot ask for a set (columnar_tableam.c, - * columnar_slot_decode_upto). A query referencing only a late column therefore + * prefix and cannot ask for a set (pgcolumnar_tableam.c, + * pgcolumnar_slot_decode_upto). A query referencing only a late column therefore * decodes every column before it, and sizing this from reltarget -- the * emitted columns -- understates the decode by the ratio of the prefix to the * projection (issue #363). @@ -571,7 +571,7 @@ columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) * unreferenced columns in the prefix are not in reltarget at all. */ static void -columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, +pgcolumnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, int *nprefix, double *prefixWidth) { Bitmapset *attrs = NULL; @@ -636,11 +636,11 @@ columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, } /* - * columnar_index_fetch_penalty + * pgcolumnar_index_fetch_penalty * extra cost to add to a heap-fetching columnar index scan for the row-group * decodes its per-row fetches force. Core's cost_index prices a heap fetch as * a page or two; a columnar fetch decodes the whole row group the row lives in - * (columnar_reader.c, ColumnarReadRowByNumber), which is why the planner picks + * (pgcolumnar_reader.c, PgColumnarReadRowByNumber), which is why the planner picks * an index scan for an unclustered ORDER BY and then runs for minutes (#355). * * A statement-scoped cache (issue #143) means a group is decoded once per scan, not @@ -663,10 +663,10 @@ columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, * decode of R rows across the columns the scan needs. */ static Cost -columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, +pgcolumnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, int nproj, double decodedWidth, bool tid_ordered) { - double R = (double) columnar_stripe_row_limit; + double R = (double) pgcolumnar_stripe_row_limit; double N = (rel->tuples > 0) ? rel->tuples : rows; double n_groups, pages_per_stripe, @@ -728,13 +728,13 @@ columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, /* * How far above one full scan a fetching index path may be priced (issue #376). - * See columnar_penalize_index_fetches for why this is a bound rather than a + * See pgcolumnar_penalize_index_fetches for why this is a bound rather than a * better model, and for the two measurements that fix the window it sits in. */ #define COLUMNAR_INDEX_FETCH_PENALTY_MAX_SCANS 20.0 /* - * columnar_full_scan_cost + * pgcolumnar_full_scan_cost * What one full scan of this relation costs. * * The seqscan's own number when core still has one, and the same work priced @@ -744,7 +744,7 @@ columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, * penalty below. */ static Cost -columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) +pgcolumnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) { QualCost qcost; double ntuples; @@ -761,7 +761,7 @@ columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) } /* - * columnar_path_order_cmp + * pgcolumnar_path_order_cmp * Order two paths the way add_path keeps rel->pathlist ordered. * * PG18 sorts by disabled_nodes and then total_cost; before that there is no @@ -770,7 +770,7 @@ columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) * wrong key, so it is spelled out per version rather than assumed. */ static int -columnar_path_order_cmp(const ListCell *a, const ListCell *b) +pgcolumnar_path_order_cmp(const ListCell *a, const ListCell *b) { const Path *pa = (const Path *) lfirst(a); const Path *pb = (const Path *) lfirst(b); @@ -787,7 +787,7 @@ columnar_path_order_cmp(const ListCell *a, const ListCell *b) } /* - * columnar_penalize_index_fetches + * pgcolumnar_penalize_index_fetches * Price the per-row heap fetch of the surviving index and bitmap paths * (#355), and restore the ordering add_path expects. * @@ -818,7 +818,7 @@ columnar_path_order_cmp(const ListCell *a, const ListCell *b) * planner models by fractioning (total - startup). */ static void -columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, +pgcolumnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, Cost fullScanCost) { int nproj; @@ -827,10 +827,10 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, bool mutated = false; ListCell *lc; - if (!columnar_enable_index_fetch_penalty) + if (!pgcolumnar_enable_index_fetch_penalty) return; - columnar_scan_decode_shape(rel, rti, relid, &nproj, &decodedWidth); + pgcolumnar_scan_decode_shape(rel, rti, relid, &nproj, &decodedWidth); /* * The penalty is bounded by a multiple of one full scan (issue #376). @@ -875,15 +875,15 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, if (p->pathtype == T_IndexScan) { IndexPath *ip = castNode(IndexPath, p); - double rho = columnar_index_correlation(ip->indexinfo, relid); + double rho = pgcolumnar_index_correlation(ip->indexinfo, relid); - add = columnar_index_fetch_penalty(rel, p->rows, rho, nproj, + add = pgcolumnar_index_fetch_penalty(rel, p->rows, rho, nproj, decodedWidth, false); } else if (p->pathtype == T_BitmapHeapScan) { /* a bitmap heap scan fetches in TID (row-number) order */ - add = columnar_index_fetch_penalty(rel, p->rows, 1.0, nproj, + add = pgcolumnar_index_fetch_penalty(rel, p->rows, 1.0, nproj, decodedWidth, true); } /* T_IndexOnlyScan and the custom scans do no heap fetch */ @@ -905,11 +905,11 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, } if (mutated) - list_sort(rel->pathlist, columnar_path_order_cmp); + list_sort(rel->pathlist, pgcolumnar_path_order_cmp); } /* - * ColumnarSetRelPathlist + * PgColumnarSetRelPathlist * set_rel_pathlist_hook: for a columnar base relation, replace the * sequential-scan path with the columnar custom scan and drop parallel * paths. Index and bitmap paths are left in place, so ordinary index @@ -918,7 +918,7 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, * steers the planner to an index scan exactly as before. */ static void -ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, +PgColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, RangeTblEntry *rte) { CustomPath *cpath; @@ -931,20 +931,20 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, if (prev_set_rel_pathlist_hook) prev_set_rel_pathlist_hook(root, rel, rti, rte); - if (!columnar_enable_custom_scan) + if (!pgcolumnar_enable_custom_scan) return; if (rte->rtekind != RTE_RELATION || rte->relkind != RELKIND_RELATION) return; if (rel->reloptkind != RELOPT_BASEREL) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; /* - * The custom scan is the scalar per-row path (ColumnarReadNextRow), so + * The custom scan is the scalar per-row path (PgColumnarReadNextRow), so * pushed-down predicates drive zone-map row-group and per-vector skipping * (native spec 7.1). Ungrouped aggregates are answered from the zone maps by - * the separate aggregate path (columnar_vector.c). + * the separate aggregate path (pgcolumnar_vector.c). */ /* find a non-parameterized seqscan path to inherit its costs from */ @@ -975,8 +975,8 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * doing this after the add_path calls meant the columnar path was judged * against index costs that had not yet been penalized, and freed. */ - columnar_penalize_index_fetches(rel, rti, rte->relid, - columnar_full_scan_cost(rel, seqpath)); + pgcolumnar_penalize_index_fetches(rel, rti, rte->relid, + pgcolumnar_full_scan_cost(rel, seqpath)); cpath = makeNode(CustomPath); cpath->path.pathtype = T_CustomScan; @@ -1011,7 +1011,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, */ cpath->path.startup_cost = (seqpath != NULL) ? seqpath->startup_cost : rel->baserestrictcost.startup; - cpath->path.total_cost = columnar_full_scan_cost(rel, seqpath); + cpath->path.total_cost = pgcolumnar_full_scan_cost(rel, seqpath); cpath->path.pathkeys = NIL; cpath->flags = 0; cpath->custom_paths = NIL; @@ -1025,7 +1025,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, */ cpath->custom_restrictinfo = rel->baserestrictinfo; #endif - cpath->methods = &columnar_path_methods; + cpath->methods = &pgcolumnar_path_methods; /* * Keep the costs before offering the path. add_path FREES a path it judges @@ -1049,7 +1049,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * result is correct whichever path wins (the executor re-applies the qual). */ { - char *projName = columnar_choose_projection(root, rel, rte->relid); + char *projName = pgcolumnar_choose_projection(root, rel, rte->relid); if (projName != NULL) { @@ -1079,7 +1079,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, #if PG_VERSION_NUM >= 170000 ppath->custom_restrictinfo = rel->baserestrictinfo; #endif - ppath->methods = &columnar_path_methods; + ppath->methods = &pgcolumnar_path_methods; add_path(rel, &ppath->path); } @@ -1135,7 +1135,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, #if PG_VERSION_NUM >= 170000 ppath->custom_restrictinfo = rel->baserestrictinfo; #endif - ppath->methods = &columnar_path_methods; + ppath->methods = &pgcolumnar_path_methods; add_partial_path(rel, &ppath->path); } } @@ -1146,13 +1146,13 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * ------------------------------------------------------------------------- */ /* - * ColumnarCreateScanState + * PgColumnarCreateScanState * Shared create-state callback for the one registered CustomScanMethods. A * scanrelid==0 plan is the vectorized aggregate upper node; anything else * is a base-relation columnar scan. */ static Node * -ColumnarCreateScanState(CustomScan *cscan) +PgColumnarCreateScanState(CustomScan *cscan) { if (cscan->scan.scanrelid == 0) { @@ -1163,28 +1163,28 @@ ColumnarCreateScanState(CustomScan *cscan) * path carries length 3. */ if (list_length(cscan->custom_private) == 5) - return ColumnarCreateGroupAggScanState(cscan); - return ColumnarCreateAggScanState(cscan); + return PgColumnarCreateGroupAggScanState(cscan); + return PgColumnarCreateAggScanState(cscan); } - return ColumnarCreateBaseScanState(cscan); + return PgColumnarCreateBaseScanState(cscan); } static Node * -ColumnarCreateBaseScanState(CustomScan *cscan) +PgColumnarCreateBaseScanState(CustomScan *cscan) { - ColumnarCustomScanState *cstate = - (ColumnarCustomScanState *) palloc0(sizeof(ColumnarCustomScanState)); + PgColumnarCustomScanState *cstate = + (PgColumnarCustomScanState *) palloc0(sizeof(PgColumnarCustomScanState)); cstate->css.ss.ps.type = T_CustomScanState; - cstate->css.methods = &columnar_exec_methods; + cstate->css.methods = &pgcolumnar_exec_methods; return (Node *) cstate; } /* the projection name the planner chose, or NULL for a base scan (gap 26) */ static char * -columnar_chosen_projection(CustomScan *cscan) +pgcolumnar_chosen_projection(CustomScan *cscan) { if (cscan->custom_private == NIL) return NULL; @@ -1192,7 +1192,7 @@ columnar_chosen_projection(CustomScan *cscan) } /* - * columnar_setup_projection_scan + * pgcolumnar_setup_projection_scan * Open a read on the named projection's storage with a synthetic * [rownumber, cols...] descriptor, build the base<->projection column map, * and translate the pushed-down scan keys to the projection's attnums so the @@ -1202,13 +1202,13 @@ columnar_chosen_projection(CustomScan *cscan) * since planning. */ static void -columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, +pgcolumnar_setup_projection_scan(PgColumnarCustomScanState *cstate, Relation rel, Snapshot snapshot, const char *projName) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); TupleDesc tableDesc = RelationGetDescr(rel); - List *projs = ColumnarListProjections(storageId); - ColumnarProjection *proj = NULL; + List *projs = PgColumnarListProjections(storageId); + PgColumnarProjection *proj = NULL; ListCell *lc; TupleDesc projTupdesc; ScanKey projKeys = NULL; @@ -1217,7 +1217,7 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projectionId > 0 && strcmp(p->name, projName) == 0) { @@ -1227,7 +1227,7 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, } if (proj == NULL) { - cstate->readState = ColumnarBeginRead(rel, snapshot, NULL, + cstate->readState = PgColumnarBeginRead(rel, snapshot, NULL, cstate->projectedColumns, cstate->nScanKeys, cstate->scanKeys); return; @@ -1268,20 +1268,20 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, } } - cstate->readState = ColumnarBeginReadWithStorage(rel, snapshot, + cstate->readState = PgColumnarBeginReadWithStorage(rel, snapshot, proj->projStorageId, projTupdesc, NULL, NULL, nProjKeys, projKeys); /* cache base liveness once so the per-row deletion test is a binary search, * not a per-row catalog scan */ - cstate->livenessCache = ColumnarBuildLivenessCache(rel, snapshot); + cstate->livenessCache = PgColumnarBuildLivenessCache(rel, snapshot); cstate->projScan = true; } static void -ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; CustomScan *cscan = (CustomScan *) node->ss.ps.plan; Relation rel = node->ss.ss_currentRelation; TupleDesc tupdesc = RelationGetDescr(rel); @@ -1294,13 +1294,13 @@ ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) * touches the catalog and data pages, is skipped when we will not execute. */ cstate->projectedColumns = - columnar_projected_columns(cscan, tupdesc->natts, &cstate->nProjected); + pgcolumnar_projected_columns(cscan, tupdesc->natts, &cstate->nProjected); cstate->scanKeys = - ColumnarBuildScanKeys(cscan->scan.plan.qual, cscan->scan.scanrelid, + PgColumnarBuildScanKeys(cscan->scan.plan.qual, cscan->scan.scanrelid, tupdesc, &cstate->nScanKeys); /* the projection the planner chose (gap 26); reported by EXPLAIN */ - cstate->projName = columnar_chosen_projection(cscan); + cstate->projName = pgcolumnar_chosen_projection(cscan); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) return; @@ -1310,36 +1310,36 @@ ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) * reader's catalog snapshot sees them (read-your-writes, spec 9), matching * the table AM's own scan_begin. */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); if (cstate->projName != NULL) { /* gap 26: read a covering projection instead of the base. */ - columnar_setup_projection_scan(cstate, rel, estate->es_snapshot, + pgcolumnar_setup_projection_scan(cstate, rel, estate->es_snapshot, cstate->projName); if (!cstate->projScan) cstate->projName = NULL; /* projection vanished; base fallback */ } else { - cstate->readState = ColumnarBeginRead(rel, estate->es_snapshot, NULL, + cstate->readState = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, cstate->projectedColumns, cstate->nScanKeys, cstate->scanKeys); } } /* - * ColumnarScanNext + * PgColumnarScanNext * ExecScan access method: fetch the next columnar row into the scan slot. * The row's synthetic item pointer (spec 6) is stored on the slot so an * UPDATE/DELETE above the scan can identify the row by its ctid. */ static TupleTableSlot * -columnar_projection_scan_next(ScanState *ss) +pgcolumnar_projection_scan_next(ScanState *ss) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) ss; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) ss; TupleTableSlot *slot = ss->ss_ScanTupleSlot; Relation rel = ss->ss_currentRelation; int natts = cstate->nTotalColumns; @@ -1350,14 +1350,14 @@ columnar_projection_scan_next(ScanState *ss) uint64 baseRow; int c; - if (!ColumnarReadNextRow(cstate->readState, cstate->projValues, + if (!PgColumnarReadNextRow(cstate->readState, cstate->projValues, cstate->projNulls, &projRowNum)) return NULL; /* deletes/visibility come from the base (gap 26): the projection stores * no delete vector, so filter by the stored base row number via the cache */ baseRow = (uint64) DatumGetInt64(cstate->projValues[0]); - if (!ColumnarLivenessCacheIsLive(cstate->livenessCache, baseRow)) + if (!PgColumnarLivenessCacheIsLive(cstate->livenessCache, baseRow)) continue; ExecClearTuple(slot); @@ -1377,60 +1377,60 @@ columnar_projection_scan_next(ScanState *ss) } } ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(baseRow, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(baseRow, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return slot; } } static TupleTableSlot * -ColumnarScanNext(ScanState *ss) +PgColumnarScanNext(ScanState *ss) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) ss; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) ss; TupleTableSlot *slot = ss->ss_ScanTupleSlot; uint64 rowNumber; if (cstate->projScan) - return columnar_projection_scan_next(ss); + return pgcolumnar_projection_scan_next(ss); ExecClearTuple(slot); - if (!ColumnarReadNextRow(cstate->readState, slot->tts_values, + if (!PgColumnarReadNextRow(cstate->readState, slot->tts_values, slot->tts_isnull, &rowNumber)) return NULL; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(ss->ss_currentRelation); return slot; } static bool -ColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot) +PgColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot) { return true; } static TupleTableSlot * -ColumnarExecCustomScan(CustomScanState *node) +PgColumnarExecCustomScan(CustomScanState *node) { return ExecScan(&node->ss, - (ExecScanAccessMtd) ColumnarScanNext, - (ExecScanRecheckMtd) ColumnarScanRecheck); + (ExecScanAccessMtd) PgColumnarScanNext, + (ExecScanRecheckMtd) PgColumnarScanRecheck); } static void -ColumnarReScanCustomScan(CustomScanState *node) +PgColumnarReScanCustomScan(CustomScanState *node) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->readState != NULL) { - ColumnarRescanRead(cstate->readState); + PgColumnarRescanRead(cstate->readState); /* a parallel rescan keeps sharing the same stripe counter */ if (cstate->parallelCounter != NULL) - ColumnarReadSetParallelCounter(cstate->readState, + PgColumnarReadSetParallelCounter(cstate->readState, cstate->parallelCounter); } @@ -1445,26 +1445,26 @@ ColumnarReScanCustomScan(CustomScanState *node) * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); cstate->parallelCounter = counter; if (cstate->readState != NULL) - ColumnarReadSetParallelCounter(cstate->readState, counter); + PgColumnarReadSetParallelCounter(cstate->readState, counter); } static void -ColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -1473,39 +1473,39 @@ ColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, } static void -ColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; cstate->parallelCounter = counter; if (cstate->readState != NULL) - ColumnarReadSetParallelCounter(cstate->readState, counter); + PgColumnarReadSetParallelCounter(cstate->readState, counter); } static void -ColumnarEndCustomScan(CustomScanState *node) +PgColumnarEndCustomScan(CustomScanState *node) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->readState != NULL) { - ColumnarEndRead(cstate->readState); + PgColumnarEndRead(cstate->readState); cstate->readState = NULL; } if (cstate->livenessCache != NULL) { - ColumnarFreeLivenessCache(cstate->livenessCache); + PgColumnarFreeLivenessCache(cstate->livenessCache); cstate->livenessCache = NULL; } } static void -ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, +PgColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->projName != NULL) ExplainPropertyText("Columnar Projection", cstate->projName, es); @@ -1518,8 +1518,8 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * Report what the scan pushes down, not what the planner handed it. * * cstate->nScanKeys is the count the planner produced, and it is the same - * whether or not pushdown is enabled. But columnar_enable_qual_pushdown - * gates columnar_build_predicates in ColumnarBeginRead, so with the setting + * whether or not pushdown is enabled. But pgcolumnar_enable_qual_pushdown + * gates pgcolumnar_build_predicates in PgColumnarBeginRead, so with the setting * off the reader builds no predicates and skips no chunk groups: nothing is * pushed down in any sense the scan acts on. Someone turning the setting off * to test a theory, and checking EXPLAIN to confirm it took effect, was told @@ -1530,7 +1530,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * so this one line meant something different from all of its neighbours. */ ExplainPropertyInteger("Columnar Pushed-Down Filters", NULL, - columnar_enable_qual_pushdown ? cstate->nScanKeys : 0, + pgcolumnar_enable_qual_pushdown ? cstate->nScanKeys : 0, es); if (cstate->readState != NULL) @@ -1539,7 +1539,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, uint64 groupsSkipped = 0; uint64 groupsTotal = 0; - ColumnarReadStats(cstate->readState, &groupsRead, &groupsSkipped, + PgColumnarReadStats(cstate->readState, &groupsRead, &groupsSkipped, &groupsTotal); ExplainPropertyInteger("Columnar Chunk Groups Total", NULL, @@ -1549,7 +1549,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainPropertyInteger("Columnar Chunk Groups Removed by Filter", NULL, (int64) groupsSkipped, es); ExplainPropertyInteger("Columnar Vectors Skipped", NULL, - (int64) ColumnarVectorsSkipped(cstate->readState), es); + (int64) PgColumnarVectorsSkipped(cstate->readState), es); } } @@ -1558,10 +1558,10 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * ------------------------------------------------------------------------- */ void -ColumnarCustomScanInit(void) +PgColumnarCustomScanInit(void) { - RegisterCustomScanMethods(&columnar_scan_methods); + RegisterCustomScanMethods(&pgcolumnar_scan_methods); prev_set_rel_pathlist_hook = set_rel_pathlist_hook; - set_rel_pathlist_hook = ColumnarSetRelPathlist; + set_rel_pathlist_hook = PgColumnarSetRelPathlist; } diff --git a/src/columnar_delete_vector.c b/src/columnar_delete_vector.c index 5637442..792288d 100644 --- a/src/columnar_delete_vector.c +++ b/src/columnar_delete_vector.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_delete_vector.c + * pgcolumnar_delete_vector.c * Delete and update marking for pgColumnar (spec 7.5, 9). Deletes do not * rewrite stripes; instead a bit is set in the columnar.delete_vector entry for * the affected chunk group. Update is delete-plus-insert, so it also uses @@ -57,8 +57,8 @@ typedef struct DeleteVectorBuffer DeleteVectorChunkBuffer *lastChunk; } DeleteVectorBuffer; -static MemoryContext ColumnarDeleteVectorContext = NULL; -static List *ColumnarDeleteVectorBuffers = NIL; +static MemoryContext PgColumnarDeleteVectorContext = NULL; +static List *PgColumnarDeleteVectorBuffers = NIL; static DeleteVectorBuffer *delete_vector_get_buffer(Relation rel, uint64 storageId); static NativeRowGroupMetadata *delete_vector_find_row_group(DeleteVectorBuffer *buf, @@ -74,7 +74,7 @@ static void delete_vector_flush_buffer(DeleteVectorBuffer *buf); * delete_vector_chunk_cmp * Total order over chunk-group buffers by (stripeId, chunkId, * startRowNumber). Flushing in this order makes every transaction acquire - * the per-chunk-group locks (columnar_metadata.c) in the same global + * the per-chunk-group locks (pgcolumnar_metadata.c) in the same global * order, so two concurrent deleters cannot form an AB-BA deadlock cycle. */ static int @@ -105,19 +105,19 @@ delete_vector_get_buffer(Relation rel, uint64 storageId) MemoryContext oldContext; DeleteVectorBuffer *buf; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { buf = (DeleteVectorBuffer *) lfirst(lc); if (buf->storageId == storageId && buf->subid == subid) return buf; } - if (ColumnarDeleteVectorContext == NULL) - ColumnarDeleteVectorContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarDeleteVectorContext == NULL) + PgColumnarDeleteVectorContext = AllocSetContextCreate(TopTransactionContext, "columnar delete vector", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); buf = palloc0(sizeof(DeleteVectorBuffer)); buf->relid = RelationGetRelid(rel); buf->storageId = storageId; @@ -126,7 +126,7 @@ delete_vector_get_buffer(Relation rel, uint64 storageId) buf->rowGroupCache = NIL; buf->lastChunk = NULL; buf->lastGroup = NULL; - ColumnarDeleteVectorBuffers = lappend(ColumnarDeleteVectorBuffers, buf); + PgColumnarDeleteVectorBuffers = lappend(PgColumnarDeleteVectorBuffers, buf); MemoryContextSwitchTo(oldContext); return buf; @@ -166,10 +166,10 @@ delete_vector_find_row_group(DeleteVectorBuffer *buf, uint64 rowNumber) if (attempt == 0) { MemoryContext oldContext = - MemoryContextSwitchTo(ColumnarDeleteVectorContext); - Snapshot snap = ColumnarCatalogSnapshot(GetActiveSnapshot()); + MemoryContextSwitchTo(PgColumnarDeleteVectorContext); + Snapshot snap = PgColumnarCatalogSnapshot(GetActiveSnapshot()); - buf->rowGroupCache = ColumnarReadRowGroupList(buf->storageId, snap); + buf->rowGroupCache = PgColumnarReadRowGroupList(buf->storageId, snap); buf->lastGroup = NULL; /* points into the list just replaced */ MemoryContextSwitchTo(oldContext); } @@ -206,7 +206,7 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, } } - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); chunk = palloc0(sizeof(DeleteVectorChunkBuffer)); chunk->stripeId = stripeId; chunk->chunkId = chunkId; @@ -223,7 +223,7 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, } /* - * ColumnarMarkRowDeleted + * PgColumnarMarkRowDeleted * Record that the row with the given 1-based row number is deleted, by * setting its bit in the in-memory delete buffer for its chunk group. * The mark targets the whole enclosing row group as one bitmap (chunk id @@ -231,9 +231,9 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, * delete_vector_find_row_group using its firstRowNumber and rowCount. */ void -ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) +PgColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); DeleteVectorBuffer *buf = delete_vector_get_buffer(rel, storageId); uint64 startRowNumber; uint64 endRowNumber; @@ -263,26 +263,26 @@ ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) * index-only scan never skips the fetch for a block with a dead row (gap 28). * A no-op unless a prior vacuum had marked the block visible. */ - ColumnarVMClearForRow(rel, rowNumber); + PgColumnarVMClearForRow(rel, rowNumber); } /* - * ColumnarDeleteVectorBufferedDeleted + * PgColumnarDeleteVectorBufferedDeleted * True when the row is marked deleted in an in-memory row-mask buffer that * has not yet been flushed to the catalog. The unique/primary-key check runs * as part of an insert (or the insert half of an update) and fetches a - * conflicting row through ColumnarReadRowByNumber before the delete of the + * conflicting row through PgColumnarReadRowByNumber before the delete of the * old row is flushed; consulting the buffer here lets a same-key UPDATE (the * old row is buffered-deleted) proceed. Checks every buffer for the relation, * across subtransactions. */ bool -ColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber) +PgColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber) { Oid relid = RelationGetRelid(rel); ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); ListCell *cc; @@ -350,7 +350,7 @@ delete_vector_flush_buffer(DeleteVectorBuffer *buf) rm.bitmap = chunk->mask; rm.bitmapLen = chunk->maskLen; - ColumnarUpsertDeleteVector(buf->storageId, &rm); + PgColumnarUpsertDeleteVector(buf->storageId, &rm); } if (pushedSnapshot) @@ -363,17 +363,17 @@ delete_vector_flush_buffer(DeleteVectorBuffer *buf) } /* - * ColumnarFlushDeleteVectorForRelation + * PgColumnarFlushDeleteVectorForRelation * Flush pending delete marks for one relation. Called at scan start so a * delete made earlier in this transaction is visible to a later scan. */ void -ColumnarFlushDeleteVectorForRelation(Relation rel) +PgColumnarFlushDeleteVectorForRelation(Relation rel) { Oid relid = RelationGetRelid(rel); ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); @@ -383,47 +383,47 @@ ColumnarFlushDeleteVectorForRelation(Relation rel) } /* - * ColumnarFlushAllDeleteVectors + * PgColumnarFlushAllDeleteVectors * Flush every pending delete buffer. Called at transaction pre-commit. */ void -ColumnarFlushAllDeleteVectors(void) +PgColumnarFlushAllDeleteVectors(void) { ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) delete_vector_flush_buffer((DeleteVectorBuffer *) lfirst(lc)); } /* - * ColumnarDiscardAllDeleteVectors + * PgColumnarDiscardAllDeleteVectors * Forget all pending delete buffers (transaction end). */ void -ColumnarDiscardAllDeleteVectors(void) +PgColumnarDiscardAllDeleteVectors(void) { - ColumnarDeleteVectorBuffers = NIL; - ColumnarDeleteVectorContext = NULL; + PgColumnarDeleteVectorBuffers = NIL; + PgColumnarDeleteVectorContext = NULL; } /* - * ColumnarDeleteVectorDiscardSubXact + * PgColumnarDeleteVectorDiscardSubXact * Drop delete buffers made in an aborting subtransaction. The catalog * rows they would have produced were never written (or, if a scan flushed * them, are made invisible by the subtransaction abort itself). */ void -ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) +PgColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarDeleteVectorBuffers == NIL) + if (PgColumnarDeleteVectorBuffers == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); - foreach(lc, ColumnarDeleteVectorBuffers) + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); @@ -432,20 +432,20 @@ ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) } MemoryContextSwitchTo(oldContext); - ColumnarDeleteVectorBuffers = kept; + PgColumnarDeleteVectorBuffers = kept; } /* - * ColumnarDeleteVectorPromoteSubXact + * PgColumnarDeleteVectorPromoteSubXact * On subtransaction commit, reassign its delete buffers to the parent so * they survive until the parent resolves. */ void -ColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent) +PgColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent) { ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); diff --git a/src/columnar_encoding.c b/src/columnar_encoding.c index ca43719..490c0f1 100644 --- a/src/columnar_encoding.c +++ b/src/columnar_encoding.c @@ -1,16 +1,16 @@ /*------------------------------------------------------------------------- * - * columnar_encoding.c + * pgcolumnar_encoding.c * Lightweight, type-aware value-stream encodings (I1) and the * compression-block abstraction they implement (I2). * * The encoding layer sits between the raw serialized value stream that the - * writer builds (a packed sequence of ColumnarEncodeValue outputs, spec 4) and - * the general-purpose block codec (columnar_compression.c, spec 5). An encoding + * writer builds (a packed sequence of PgColumnarEncodeValue outputs, spec 4) and + * the general-purpose block codec (pgcolumnar_compression.c, spec 5). An encoding * is a reversible transform of the raw value-stream BYTES: encode(raw) -> a * smaller encoded buffer, and decode(encoded) -> the byte-identical raw stream. * Because decode reconstructs the exact raw stream, every downstream consumer - * (per-value ColumnarDecodeValue, the vectorized group decoder, the min/max + * (per-value PgColumnarDecodeValue, the vectorized group decoder, the min/max * skip list) is unchanged; only the flush and load paths gain one step. * * The techniques come from the public column-store literature (see @@ -1162,7 +1162,7 @@ decode_alp(const char *enc, uint32 encLen, int w, uint32 n, uint32 rawLen, #define DICT_MAX_DISTINCT 1024 -static inline uint64 columnar_fnv1a(const char *p, uint32 n); +static inline uint64 pgcolumnar_fnv1a(const char *p, uint32 n); static bool encode_dict(const char *raw, uint32 rawLen, Form_pg_attribute att, uint32 n, @@ -1197,8 +1197,8 @@ encode_dict(const char *raw, uint32 rawLen, Form_pg_attribute att, uint32 n, { const char *vp = raw + pos; uint32 vlen = (w > 0) ? (uint32) w - : ColumnarVarSizeAnyUnaligned(vp); - uint64 h = columnar_fnv1a(vp, vlen); + : PgColumnarVarSizeAnyUnaligned(vp); + uint64 h = pgcolumnar_fnv1a(vp, vlen); uint32 s = (uint32) (h & (DICT_HASH_SLOTS - 1)); int code = -1; @@ -1708,7 +1708,7 @@ fsst_deserialize_table(const char *p, uint32 len, FsstTable *t) * probe below. Our own trivial implementation of the public FNV-1a formula, used * only to bucket values while counting distinct ones. */ static inline uint64 -columnar_fnv1a(const char *p, uint32 n) +pgcolumnar_fnv1a(const char *p, uint32 n) { uint64 h = UINT64CONST(1469598103934665603); uint32 i; @@ -1722,7 +1722,7 @@ columnar_fnv1a(const char *p, uint32 n) } /* - * ColumnarFsstDictWins + * PgColumnarFsstDictWins * Cheap pre-check for the FSST table build (issue #155): would dictionary * encoding win outright, making the costly FSST symbol-table build wasted * work? FSST is only attempted per vector when a cheaper encoding has not @@ -1737,7 +1737,7 @@ columnar_fnv1a(const char *p, uint32 n) * This is our own heuristic; FSST is the public scheme (VLDB 2020). */ bool -ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) +PgColumnarFsstDictWins(const char *corpus, uint32 corpusLen) { /* Open-addressing set of value hashes, capacity a power of two comfortably * above the distinct cap so the load factor at the early-exit stays low. */ @@ -1750,14 +1750,14 @@ ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) memset(used, 0, sizeof(used)); while (pos < corpusLen) { - uint32 vlen = ColumnarVarSizeAnyUnaligned(corpus + pos); + uint32 vlen = PgColumnarVarSizeAnyUnaligned(corpus + pos); uint64 h; uint32 s; if (vlen == 0 || pos + vlen > corpusLen) return false; /* malformed run: let the caller build normally */ - h = columnar_fnv1a(corpus + pos, vlen); + h = pgcolumnar_fnv1a(corpus + pos, vlen); s = (uint32) (h & (FSST_CARD_SLOTS - 1)); while (used[s] && slot[s] != h) s = (s + 1) & (FSST_CARD_SLOTS - 1); @@ -1774,13 +1774,13 @@ ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) } /* - * ColumnarFsstBuildChunkTable + * PgColumnarFsstBuildChunkTable * Build one FSST symbol table for a whole column chunk (E3b). The expensive * iterative table build runs once here; the serialized table is handed back * to be stored once per chunk and reused by every FSST vector. */ bool -ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut) { @@ -1809,7 +1809,7 @@ ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, * the raw length: the shared table's bytes are amortized across the chunk and * not charged here, so FSST wins per vector whenever the codes are smaller. * - * That test is only meaningful once ColumnarFsstHelpsCompressed has decided the + * That test is only meaningful once PgColumnarFsstHelpsCompressed has decided the * chunk should use FSST at all; it is what charges the table and what compares * against the compressed size actually written. See the comment on it below. */ @@ -1873,7 +1873,7 @@ encode_fsst_shared(const char *raw, uint32 rawLen, const char *table, * Would FSST still pay once the block compressor has had its turn? * * The per-vector win test above compares encoded lengths, but the bytes that - * reach disk are compressed afterwards (columnar_write_state.c compresses the + * reach disk are compressed afterwards (pgcolumnar_write_state.c compresses the * whole encoded stream). Those are different objectives, and for some shapes * they disagree sharply: FSST turns highly repetitive text into a stream of * high-entropy codes, which is smaller than the text but far less compressible @@ -1894,7 +1894,7 @@ encode_fsst_shared(const char *raw, uint32 rawLen, const char *table, * the per-vector encode is then skipped for every vector in the chunk. */ bool -ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, +PgColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, const char *table, uint32 tableLen, int compressionType, int compressionLevel) { @@ -1918,10 +1918,10 @@ ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, if (!encode_fsst_shared(corpus, corpusLen, table, tableLen, &codes, &codesLen)) return false; - ColumnarCompressValueStream(corpus, corpusLen, compressionType, + PgColumnarCompressValueStream(corpus, corpusLen, compressionType, compressionLevel, &plainComp, &plainCompLen, &usedType, &usedLevel); - ColumnarCompressValueStream(codes, codesLen, compressionType, + PgColumnarCompressValueStream(codes, codesLen, compressionType, compressionLevel, &codesComp, &codesCompLen, &usedType, &usedLevel); @@ -1935,7 +1935,7 @@ ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, * cannot underflow when plainCompLen is small. */ helps = (((uint64) codesCompLen + tableLen) * 100 - < (uint64) plainCompLen * (uint64) (100 - columnar_fsst_min_gain_percent)); + < (uint64) plainCompLen * (uint64) (100 - pgcolumnar_fsst_min_gain_percent)); pfree(codes); if (plainComp) @@ -2026,7 +2026,7 @@ decode_fsst_shared(const char *enc, uint32 encLen, const char *table, * ------------------------------------------------------------------------- */ void -ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width) { br->raw = raw; @@ -2036,7 +2036,7 @@ ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, } bool -ColumnarBlockNextRun(ColumnarBlockReader *br, const char **valBytes, +PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen) { const char *v; @@ -2058,7 +2058,7 @@ ColumnarBlockNextRun(ColumnarBlockReader *br, const char **valBytes, } const char * -ColumnarEncodingName(int encodingType) +PgColumnarEncodingName(int encodingType) { switch (encodingType) { @@ -2161,7 +2161,7 @@ build_sample(const char *raw, int w, uint32 n, uint32 want, uint32 *sampleN) } /* - * ColumnarEncodeChunk + * PgColumnarEncodeChunk * Choose and apply the best lightweight encoding for one chunk's raw value * stream. Returns the encoding code used and sets out and outLen to the * encoded buffer. When no encoding beats the raw size, returns NONE and @@ -2183,7 +2183,7 @@ build_sample(const char *raw, int w, uint32 n, uint32 want, uint32 *sampleN) * candidate. */ int -ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, +PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen) { @@ -2211,11 +2211,11 @@ ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, * cheap next to the three it replaces. With sampling off, or a chunk too * small to sample, every candidate is applied as before. */ - if (w > 0 && columnar_encoding_sample_rows >= ENCODE_SAMPLE_MIN) + if (w > 0 && pgcolumnar_encoding_sample_rows >= ENCODE_SAMPLE_MIN) { uint32 sampleN = 0; char *sample = build_sample(raw, w, n, - (uint32) columnar_encoding_sample_rows, + (uint32) pgcolumnar_encoding_sample_rows, &sampleN); if (sample != NULL) @@ -2428,12 +2428,12 @@ ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, } /* - * ColumnarDecodeChunk + * PgColumnarDecodeChunk * Reverse an encoding, reconstructing the byte-identical raw value stream * in cx. For NONE, returns the input pointer unchanged (no copy). */ char * -ColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, +PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx) @@ -2622,10 +2622,10 @@ selftest_rand(uint64 *state) return z ^ (z >> 31); } -PG_FUNCTION_INFO_V1(columnar_debug_encoding_selftest); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_encoding_selftest); Datum -columnar_debug_encoding_selftest(PG_FUNCTION_ARGS) +pgcolumnar_debug_encoding_selftest(PG_FUNCTION_ARGS) { ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; TupleDesc retdesc; diff --git a/src/columnar_flatbuffers.c b/src/columnar_flatbuffers.c index 858d9a1..b7c5510 100644 --- a/src/columnar_flatbuffers.c +++ b/src/columnar_flatbuffers.c @@ -1,7 +1,7 @@ /*------------------------------------------------------------------------- * - * columnar_flatbuffers.c - * A minimal FlatBuffers builder. See columnar_flatbuffers.h. + * pgcolumnar_flatbuffers.c + * A minimal FlatBuffers builder. See pgcolumnar_flatbuffers.h. * * Written fresh for pgColumnar from the public FlatBuffers format description. * @@ -20,7 +20,7 @@ * ------------------------------------------------------------------------- */ void -fb_init(FBB *b) +pgc_fb_init(FBB *b) { b->cap = 256; b->buf = palloc(b->cap); @@ -31,7 +31,7 @@ fb_init(FBB *b) } void -fb_grow(FBB *b, uint32 need) +pgc_fb_grow(FBB *b, uint32 need) { uint64 want; uint32 newcap; @@ -55,25 +55,25 @@ fb_grow(FBB *b, uint32 need) /* prepend n raw bytes (already in final order) */ void -fb_place(FBB *b, const void *src, uint32 n) +pgc_fb_place(FBB *b, const void *src, uint32 n) { - fb_grow(b, n); + pgc_fb_grow(b, n); b->tail += n; memcpy(b->buf + b->cap - b->tail, src, n); } void -fb_pad(FBB *b, uint32 n) +pgc_fb_pad(FBB *b, uint32 n) { if (n == 0) return; - fb_grow(b, n); + pgc_fb_grow(b, n); b->tail += n; memset(b->buf + b->cap - b->tail, 0, n); } uint32 -fb_offset(FBB *b) +pgc_fb_offset(FBB *b) { return b->tail; } @@ -81,71 +81,71 @@ fb_offset(FBB *b) /* align so that, after `additional` more bytes plus a `size`-aligned scalar are * written, the scalar lands aligned (relative to the eventually-aligned end). */ void -fb_prep(FBB *b, uint32 size, uint32 additional) +pgc_fb_prep(FBB *b, uint32 size, uint32 additional) { uint32 alignsize; if (size > b->minalign) b->minalign = size; alignsize = ((~(b->tail + additional)) + 1) & (size - 1); - fb_pad(b, alignsize); + pgc_fb_pad(b, alignsize); } void -fb_push_u8(FBB *b, uint8 v) +pgc_fb_push_u8(FBB *b, uint8 v) { - fb_prep(b, 1, 0); - fb_place(b, &v, 1); + pgc_fb_prep(b, 1, 0); + pgc_fb_place(b, &v, 1); } void -fb_push_i16(FBB *b, int16 v) +pgc_fb_push_i16(FBB *b, int16 v) { - fb_prep(b, 2, 0); - fb_place(b, &v, 2); + pgc_fb_prep(b, 2, 0); + pgc_fb_place(b, &v, 2); } void -fb_push_i32(FBB *b, int32 v) +pgc_fb_push_i32(FBB *b, int32 v) { - fb_prep(b, 4, 0); - fb_place(b, &v, 4); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &v, 4); } void -fb_push_i64(FBB *b, int64 v) +pgc_fb_push_i64(FBB *b, int64 v) { - fb_prep(b, 8, 0); - fb_place(b, &v, 8); + pgc_fb_prep(b, 8, 0); + pgc_fb_place(b, &v, 8); } /* prepend a uoffset that references object at `off` (bytes-from-end) */ void -fb_push_uoffset(FBB *b, uint32 off) +pgc_fb_push_uoffset(FBB *b, uint32 off) { uint32 v; - fb_prep(b, 4, 0); - v = (fb_offset(b) + 4) - off; - fb_place(b, &v, 4); + pgc_fb_prep(b, 4, 0); + v = (pgc_fb_offset(b) + 4) - off; + pgc_fb_place(b, &v, 4); } /* ---- vectors ---- */ void -fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align) +pgc_fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align) { - fb_prep(b, 4, elemSize * count); /* length prefix */ - fb_prep(b, align, elemSize * count); /* element alignment */ + pgc_fb_prep(b, 4, elemSize * count); /* length prefix */ + pgc_fb_prep(b, align, elemSize * count); /* element alignment */ } uint32 -fb_end_vector(FBB *b, uint32 count) +pgc_fb_end_vector(FBB *b, uint32 count) { - fb_prep(b, 4, 0); - fb_place(b, &count, 4); /* length prefix precedes the elements */ - return fb_offset(b); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &count, 4); /* length prefix precedes the elements */ + return pgc_fb_offset(b); } /* ---- tables ---- */ void -fb_start(FBB *b, int nslots) +pgc_fb_start(FBB *b, int nslots) { int i; @@ -153,66 +153,66 @@ fb_start(FBB *b, int nslots) b->nslots = nslots; for (i = 0; i < nslots; i++) b->vslot[i] = 0; - b->objectEnd = fb_offset(b); + b->objectEnd = pgc_fb_offset(b); } void -fb_slot(FBB *b, int i) +pgc_fb_slot(FBB *b, int i) { - b->vslot[i] = fb_offset(b); + b->vslot[i] = pgc_fb_offset(b); } void -fb_add_i16(FBB *b, int i, int16 val, int16 def) +pgc_fb_add_i16(FBB *b, int i, int16 val, int16 def) { if (val == def) return; - fb_push_i16(b, val); - fb_slot(b, i); + pgc_fb_push_i16(b, val); + pgc_fb_slot(b, i); } void -fb_add_i32(FBB *b, int i, int32 val, int32 def) +pgc_fb_add_i32(FBB *b, int i, int32 val, int32 def) { if (val == def) return; - fb_push_i32(b, val); - fb_slot(b, i); + pgc_fb_push_i32(b, val); + pgc_fb_slot(b, i); } void -fb_add_i64(FBB *b, int i, int64 val, int64 def) +pgc_fb_add_i64(FBB *b, int i, int64 val, int64 def) { if (val == def) return; - fb_push_i64(b, val); - fb_slot(b, i); + pgc_fb_push_i64(b, val); + pgc_fb_slot(b, i); } void -fb_add_bool(FBB *b, int i, bool val, bool def) +pgc_fb_add_bool(FBB *b, int i, bool val, bool def) { if (val == def) return; - fb_push_u8(b, val ? 1 : 0); - fb_slot(b, i); + pgc_fb_push_u8(b, val ? 1 : 0); + pgc_fb_slot(b, i); } void -fb_add_u8(FBB *b, int i, uint8 val, uint8 def) +pgc_fb_add_u8(FBB *b, int i, uint8 val, uint8 def) { if (val == def) return; - fb_push_u8(b, val); - fb_slot(b, i); + pgc_fb_push_u8(b, val); + pgc_fb_slot(b, i); } void -fb_add_offset(FBB *b, int i, uint32 off) +pgc_fb_add_offset(FBB *b, int i, uint32 off) { if (off == 0) return; - fb_push_uoffset(b, off); - fb_slot(b, i); + pgc_fb_push_uoffset(b, off); + pgc_fb_slot(b, i); } uint32 -fb_end(FBB *b) +pgc_fb_end(FBB *b) { uint32 objectOffset; uint32 vtOffset; @@ -223,50 +223,50 @@ fb_end(FBB *b) int32 zero = 0; /* soffset placeholder = table location */ - fb_prep(b, 4, 0); - fb_place(b, &zero, 4); - objectOffset = fb_offset(b); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &zero, 4); + objectOffset = pgc_fb_offset(b); /* vtable: field voffsets (high slot first), then objsize, then vtsize */ for (i = b->nslots - 1; i >= 0; i--) { int16 voff = b->vslot[i] ? (int16) (objectOffset - b->vslot[i]) : 0; - fb_place(b, &voff, 2); + pgc_fb_place(b, &voff, 2); } objsize = (int16) (objectOffset - b->objectEnd); - fb_place(b, &objsize, 2); + pgc_fb_place(b, &objsize, 2); vtsize = (int16) ((b->nslots + 2) * 2); - fb_place(b, &vtsize, 2); + pgc_fb_place(b, &vtsize, 2); - vtOffset = fb_offset(b); + vtOffset = pgc_fb_offset(b); soff = (int32) (vtOffset - objectOffset); memcpy(b->buf + b->cap - objectOffset, &soff, 4); return objectOffset; } void -fb_finish(FBB *b, uint32 root) +pgc_fb_finish(FBB *b, uint32 root) { - fb_prep(b, b->minalign, 4); - fb_push_uoffset(b, root); + pgc_fb_prep(b, b->minalign, 4); + pgc_fb_push_uoffset(b, root); } uint32 -fb_create_string(FBB *b, const char *s) +pgc_fb_create_string(FBB *b, const char *s) { uint32 n = (uint32) strlen(s); uint8 zero = 0; - fb_prep(b, 4, n + 1); - fb_place(b, &zero, 1); /* null terminator */ - fb_place(b, s, n); /* characters (s[0] ends lowest) */ + pgc_fb_prep(b, 4, n + 1); + pgc_fb_place(b, &zero, 1); /* null terminator */ + pgc_fb_place(b, s, n); /* characters (s[0] ends lowest) */ { uint32 len = n; - fb_place(b, &len, 4); /* length prefix (already aligned) */ + pgc_fb_place(b, &len, 4); /* length prefix (already aligned) */ } - return fb_offset(b); + return pgc_fb_offset(b); } /* ---- bounds-checked little-endian FlatBuffers reader ---- */ diff --git a/src/columnar_flatbuffers.h b/src/columnar_flatbuffers.h index 39577ee..13fbe33 100644 --- a/src/columnar_flatbuffers.h +++ b/src/columnar_flatbuffers.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_flatbuffers.h + * pgcolumnar_flatbuffers.h * A minimal FlatBuffers builder. * * FlatBuffers is the serialization format Arrow uses for its IPC metadata, and @@ -14,13 +14,13 @@ * captured right after the object is written. * * The short fb_ names are this module's namespace and are kept deliberately, - * unlike the ColumnarThrift* naming next door: the call sites are dense - * (fb_add_i32(b, 0, bits, 0) appears in runs of a dozen), and lengthening them + * unlike the PgColumnarThrift* naming next door: the call sites are dense + * (pgc_fb_add_i32(b, 0, bits, 0) appears in runs of a dozen), and lengthening them * would cost more in readability there than the consistency buys. * * What stays with Arrow, and why: * - * fb_arrow_type maps a column type onto Arrow's Type union. That is Arrow + * pgc_fb_arrow_type maps a column type onto Arrow's Type union. That is Arrow * semantics expressed through this builder, not part of it. * fbr_* the read-side scalar accessors raise Arrow's own * "malformed Arrow IPC file" on a short buffer. Moving them @@ -51,29 +51,29 @@ typedef struct FBB uint32 vslot[16]; } FBB; -extern void fb_init(FBB *b); -extern void fb_grow(FBB *b, uint32 need); -extern void fb_place(FBB *b, const void *src, uint32 n); -extern void fb_pad(FBB *b, uint32 n); -extern uint32 fb_offset(FBB *b); -extern void fb_prep(FBB *b, uint32 size, uint32 additional); -extern void fb_push_u8(FBB *b, uint8 v); -extern void fb_push_i16(FBB *b, int16 v); -extern void fb_push_i32(FBB *b, int32 v); -extern void fb_push_i64(FBB *b, int64 v); -extern void fb_push_uoffset(FBB *b, uint32 off); -extern void fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align); -extern uint32 fb_end_vector(FBB *b, uint32 count); -extern void fb_start(FBB *b, int nslots); -extern void fb_slot(FBB *b, int i); -extern void fb_add_i16(FBB *b, int i, int16 val, int16 def); -extern void fb_add_i32(FBB *b, int i, int32 val, int32 def); -extern void fb_add_i64(FBB *b, int i, int64 val, int64 def); -extern void fb_add_bool(FBB *b, int i, bool val, bool def); -extern void fb_add_u8(FBB *b, int i, uint8 val, uint8 def); -extern void fb_add_offset(FBB *b, int i, uint32 off); -extern uint32 fb_end(FBB *b); -extern void fb_finish(FBB *b, uint32 root); -extern uint32 fb_create_string(FBB *b, const char *s); +extern void pgc_fb_init(FBB *b); +extern void pgc_fb_grow(FBB *b, uint32 need); +extern void pgc_fb_place(FBB *b, const void *src, uint32 n); +extern void pgc_fb_pad(FBB *b, uint32 n); +extern uint32 pgc_fb_offset(FBB *b); +extern void pgc_fb_prep(FBB *b, uint32 size, uint32 additional); +extern void pgc_fb_push_u8(FBB *b, uint8 v); +extern void pgc_fb_push_i16(FBB *b, int16 v); +extern void pgc_fb_push_i32(FBB *b, int32 v); +extern void pgc_fb_push_i64(FBB *b, int64 v); +extern void pgc_fb_push_uoffset(FBB *b, uint32 off); +extern void pgc_fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align); +extern uint32 pgc_fb_end_vector(FBB *b, uint32 count); +extern void pgc_fb_start(FBB *b, int nslots); +extern void pgc_fb_slot(FBB *b, int i); +extern void pgc_fb_add_i16(FBB *b, int i, int16 val, int16 def); +extern void pgc_fb_add_i32(FBB *b, int i, int32 val, int32 def); +extern void pgc_fb_add_i64(FBB *b, int i, int64 val, int64 def); +extern void pgc_fb_add_bool(FBB *b, int i, bool val, bool def); +extern void pgc_fb_add_u8(FBB *b, int i, uint8 val, uint8 def); +extern void pgc_fb_add_offset(FBB *b, int i, uint32 off); +extern uint32 pgc_fb_end(FBB *b); +extern void pgc_fb_finish(FBB *b, uint32 root); +extern uint32 pgc_fb_create_string(FBB *b, const char *s); #endif /* PGCOLUMNAR_FLATBUFFERS_H */ diff --git a/src/columnar_index.c b/src/columnar_index.c index 7f95db6..e46d18f 100644 --- a/src/columnar_index.c +++ b/src/columnar_index.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_index.c + * pgcolumnar_index.c * Index maintenance for callers that insert rows without an executor. * * PostgreSQL puts index maintenance in the executor, not in the table access @@ -10,7 +10,7 @@ * the indexes silently stop describing the table and an index scan returns rows * that are not there and misses rows that are. * - * Two callers are in that position: the online rewrite in columnar_vacuum.c, + * Two callers are in that position: the online rewrite in pgcolumnar_vacuum.c, * which moves live rows into fresh groups, and the Arrow and Parquet importers, * which load rows from a file. The importers did not maintain indexes at all * (issue #153): an index scan over an imported table returned nothing, and a @@ -72,7 +72,7 @@ #include "nodes/parsenodes.h" /* - * ColumnarIndexInsertBegin + * PgColumnarIndexInsertBegin * Prepare to maintain the relation's indexes for a run of inserted rows. * * enforceConstraints selects the route: an importer gets the executor's index @@ -81,10 +81,10 @@ * insertion, and -- when enforcing -- must keep holding it until commit, since * a deferred constraint queued here is fired after the caller has returned. */ -ColumnarIndexInsertState * -ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) +PgColumnarIndexInsertState * +PgColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) { - ColumnarIndexInsertState *st = palloc0(sizeof(ColumnarIndexInsertState)); + PgColumnarIndexInsertState *st = palloc0(sizeof(PgColumnarIndexInsertState)); List *oids; int cap; ListCell *lc; @@ -154,7 +154,7 @@ ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) * double insert. * * Not reachable through CREATE INDEX CONCURRENTLY today, because - * columnar_index_validate_scan is unsupported and the build fails + * pgcolumnar_index_validate_scan is unsupported and the build fails * before the index becomes valid. It is reachable as debris: a failed * concurrent build leaves the index ready but invalid until someone * drops or reindexes it, and in that state an ordinary INSERT @@ -178,12 +178,12 @@ ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) } /* - * ColumnarIndexInsertRow + * PgColumnarIndexInsertRow * Put one row into every open index, under the row number's synthetic item * pointer. */ void -ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, +PgColumnarIndexInsertRow(PgColumnarIndexInsertState *st, Relation rel, Datum *values, bool *isnull, uint64 rowNumber) { int natts = RelationGetDescr(rel)->natts; @@ -194,7 +194,7 @@ ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, if (!st->enforcing && st->n == 0) return; - ColumnarRowNumberToItemPointer(rowNumber, &tid); + PgColumnarRowNumberToItemPointer(rowNumber, &tid); ExecClearTuple(st->slot); memcpy(st->slot->tts_values, values, natts * sizeof(Datum)); @@ -246,12 +246,12 @@ ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, } /* - * ColumnarIndexInsertEnd + * PgColumnarIndexInsertEnd * Close the indexes and free the state. The locks are held until the end of * the transaction, as index_close with RowExclusiveLock leaves them. */ void -ColumnarIndexInsertEnd(ColumnarIndexInsertState *st) +PgColumnarIndexInsertEnd(PgColumnarIndexInsertState *st) { int i; @@ -279,12 +279,12 @@ ColumnarIndexInsertEnd(ColumnarIndexInsertState *st) } /* - * ColumnarRelationHasIndexes + * PgColumnarRelationHasIndexes * Does this relation have any index at all? Used by the importers to skip * the machinery entirely on the common bulk-load-into-a-bare-table case. */ bool -ColumnarRelationHasIndexes(Relation rel) +PgColumnarRelationHasIndexes(Relation rel) { return RelationGetIndexList(rel) != NIL; } diff --git a/src/columnar_metadata.c b/src/columnar_metadata.c index f23c73a..84f22c0 100644 --- a/src/columnar_metadata.c +++ b/src/columnar_metadata.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_metadata.c + * pgcolumnar_metadata.c * Access to the "columnar" metadata catalog tables and the storage-id * sequence (spec 7). Most metadata are ordinary heap tables keyed by * storage id (the options and projection_declaration tables are keyed by @@ -118,17 +118,17 @@ #define Anum_delete_vector_deleted_count 4 #define Natts_delete_vector 4 -static Oid columnar_schema_oid(void); +static Oid pgcolumnar_schema_oid(void); static Relation open_columnar_table(const char *name, LOCKMODE lockmode); -static Oid columnar_index_oid(const char *name); +static Oid pgcolumnar_index_oid(const char *name); /* - * columnar_schema_oid + * pgcolumnar_schema_oid * OID of the "columnar" schema. It always exists once the extension is * installed. */ static Oid -columnar_schema_oid(void) +pgcolumnar_schema_oid(void) { return get_namespace_oid(COLUMNAR_SCHEMA_NAME, false); } @@ -136,7 +136,7 @@ columnar_schema_oid(void) static Relation open_columnar_table(const char *name, LOCKMODE lockmode) { - Oid nspOid = columnar_schema_oid(); + Oid nspOid = pgcolumnar_schema_oid(); Oid relOid = get_relname_relid(name, nspOid); if (!OidIsValid(relOid)) @@ -149,26 +149,26 @@ open_columnar_table(const char *name, LOCKMODE lockmode) } /* - * columnar_index_oid + * pgcolumnar_index_oid * The OID of one of the metadata indexes, by name. Returns InvalidOid when * it cannot be resolved, which callers pass through to systable_beginscan * as indexOK = false so the lookup degrades to a heap scan rather than * failing. */ static Oid -columnar_index_oid(const char *name) +pgcolumnar_index_oid(const char *name) { - return get_relname_relid(name, columnar_schema_oid()); + return get_relname_relid(name, pgcolumnar_schema_oid()); } /* - * ColumnarNextStorageId + * PgColumnarNextStorageId * Draw the next value from pgcolumnar.storageid_seq (spec 3, 7.6). */ uint64 -ColumnarNextStorageId(void) +PgColumnarNextStorageId(void) { - Oid nspOid = columnar_schema_oid(); + Oid nspOid = pgcolumnar_schema_oid(); Oid seqOid = get_relname_relid("storageid_seq", nspOid); int64 value; @@ -182,7 +182,7 @@ ColumnarNextStorageId(void) } /* - * ColumnarCatalogSnapshot + * PgColumnarCatalogSnapshot * Return a snapshot for reading the columnar metadata catalog that also * sees this transaction's own writes made in the current command (spec 9). * curcid only affects visibility of the current transaction's own tuples, @@ -190,7 +190,7 @@ ColumnarNextStorageId(void) * other transactions. */ Snapshot -ColumnarCatalogSnapshot(Snapshot base) +PgColumnarCatalogSnapshot(Snapshot base) { Snapshot copy; CommandId now; @@ -223,7 +223,7 @@ ColumnarCatalogSnapshot(Snapshot base) /* - * ColumnarComputeAllVisibleGroups + * PgColumnarComputeAllVisibleGroups * Return the chunk groups that are all-visible to every snapshot (gap 28 * phase 3): the covering stripe's insert xid is frozen or precedes * `oldestXmin` (so every current/future snapshot sees the insert), and the @@ -231,10 +231,10 @@ ColumnarCatalogSnapshot(Snapshot base) * under a dirty snapshot so a group being modified concurrently is excluded; * combined with clear-on-write (which removes a bit for any later * delete/insert), this keeps a set bit from ever covering a modified row. - * Returns a List of ColumnarRowRange * (one per all-visible group). + * Returns a List of PgColumnarRowRange * (one per all-visible group). */ List * -ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) +PgColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) { Relation grel = open_columnar_table("row_group", AccessShareLock); TupleDesc gtd = RelationGetDescr(grel); @@ -272,7 +272,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) List *rmList; ListCell *lc; bool hasDelete = false; - ColumnarRowRange *r; + PgColumnarRowRange *r; if (TransactionIdIsNormal(xmin) && !TransactionIdPrecedes(xmin, oldestXmin)) @@ -287,7 +287,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) if (rowCount == 0) continue; - rmList = ColumnarReadDeleteVectorList(storageId, groupNumber, &dirty); + rmList = PgColumnarReadDeleteVectorList(storageId, groupNumber, &dirty); foreach(lc, rmList) { if (((DeleteVectorMetadata *) lfirst(lc))->deletedCount > 0) @@ -299,7 +299,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) if (hasDelete) continue; - r = palloc(sizeof(ColumnarRowRange)); + r = palloc(sizeof(PgColumnarRowRange)); r->firstRowNumber = firstRow; r->rowCount = rowCount; ranges = lappend(ranges, r); @@ -312,7 +312,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) } /* - * ColumnarComputeFullyDeletedGroups + * PgColumnarComputeFullyDeletedGroups * Return the group numbers of row groups every one of whose rows is deleted * as-of oldestXmin -- i.e. the group's catalog row and every delete on it * committed before oldestXmin, so every live snapshot agrees the group is @@ -323,7 +323,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) * catalog version via heap MVCC. Returns a List of palloc'd uint64. */ List * -ColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin) +PgColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin) { Relation grel = open_columnar_table("row_group", AccessShareLock); TupleDesc gtd = RelationGetDescr(grel); @@ -461,7 +461,7 @@ read_row_group_range(uint64 storageId, uint64 groupNumber, } /* physical reclaim: split freed ranges on allocate and coalesce on free */ -bool columnar_reclaim_coalesce = true; +bool pgcolumnar_reclaim_coalesce = true; /* insert one free_space row (offset, length page-aligned, freed at freedXid) */ static void @@ -508,7 +508,7 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) uint64 origEnd = off + len; TransactionId freedXid = GetCurrentTransactionId(); - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) { Snapshot snap = RegisterSnapshot(GetLatestSnapshot()); ScanKeyData key[1]; @@ -570,13 +570,13 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) } insert_free_space_row(rel, td, storageId, off, len, freedXid); - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) CommandCounterIncrement(); table_close(rel, RowExclusiveLock); } /* - * ColumnarRetireGroup + * PgColumnarRetireGroup * Drop all catalog rows for one row group (row_group, column_chunk, * zone_map, bloom, delete_vector) in the current transaction (Phase F3a). The * storage row is left intact. Heap MVCC on these deletes keeps the group @@ -586,7 +586,7 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) * The caller must have verified the group is fully deleted as-of oldestXmin. */ void -ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) +PgColumnarRetireGroup(uint64 storageId, uint64 groupNumber) { uint64 fileOffset = 0; uint64 byteLength = 0; @@ -609,7 +609,7 @@ ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) } /* - * ColumnarAllocateFreeSpace + * PgColumnarAllocateFreeSpace * Try to satisfy a data reservation of dataLength bytes from a previously * freed range (Phase F physical reclaim). Returns true and sets *fileOffset * to a page-aligned freed range that is large enough AND whose freeing @@ -619,7 +619,7 @@ ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) * so no two callers race for the same row. */ bool -ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, +PgColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, TransactionId oldestXmin, uint64 *fileOffset) { Relation rel = open_columnar_table("free_space", RowExclusiveLock); @@ -677,7 +677,7 @@ ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, * same oldest-xmin gate still applies). Without this the tail of an * oversized freed range would leak until its group is re-freed. */ - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) { uint64 allocLen = COLUMNAR_PAGE_ROUND_UP(dataLength); @@ -721,7 +721,7 @@ reclaim_range_cmp(const void *a, const void *b) } /* - * ColumnarCheckFreeSpaceNoOverlap + * PgColumnarCheckFreeSpaceNoOverlap * Assert-only invariant check for physical reclaim: a storage's live * row-group footprints (data rounded up to whole pages) and its free_space * ranges must not overlap. An overlap would mean a reused block was handed @@ -729,7 +729,7 @@ reclaim_range_cmp(const void *a, const void *b) * the end of the online maintenance operations in assert builds. */ void -ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) +PgColumnarCheckFreeSpaceNoOverlap(uint64 storageId) { Relation rg; Relation fs; @@ -814,7 +814,7 @@ ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) #endif /* USE_ASSERT_CHECKING */ /* - * ColumnarTrailingFreeSpaceSafe + * PgColumnarTrailingFreeSpaceSafe * Physical end-truncation guard: return false if any free_space row for this * storage at or above liveEnd was freed at a transaction the oldest-xmin * horizon has NOT passed. Such a row is a recently retired group whose bytes @@ -824,7 +824,7 @@ ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) * the tail safe to drop. */ bool -ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, +PgColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, TransactionId oldestXmin) { Relation rel = open_columnar_table("free_space", AccessShareLock); @@ -862,14 +862,14 @@ ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, } /* - * ColumnarDeleteFreeSpaceAtOrAbove + * PgColumnarDeleteFreeSpaceAtOrAbove * Physical end-truncation: drop every free_space row for this storage whose * offset is at or above liveEnd. Those ranges are being physically removed * from the file, so they must no longer appear as reusable space. The caller * holds AccessExclusiveLock and has verified the tail is safe. */ void -ColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd) +PgColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd) { Relation rel = open_columnar_table("free_space", RowExclusiveLock); TupleDesc td = RelationGetDescr(rel); @@ -930,7 +930,7 @@ static void collect_footprints(uint64 storageId, Snapshot snap, FootRange **foots, int *nf, int *capf) { - List *rgs = ColumnarReadRowGroupList(storageId, snap); + List *rgs = PgColumnarReadRowGroupList(storageId, snap); ListCell *lc; foreach(lc, rgs) @@ -973,7 +973,7 @@ range_overlaps_footprint(uint64 start, uint64 end, FootRange *foots, int nf) } /* - * ColumnarReconcileFreeList + * PgColumnarReconcileFreeList * Delete any free_space row (for the base or any projection storage of this * relation) that overlaps a LIVE row-group footprint as-of the latest * committed state. @@ -988,15 +988,15 @@ range_overlaps_footprint(uint64 start, uint64 end, FootRange *foots, int nf) * with the trailing free rows restored; a later insert then places a live * group over one of those ranges. Running this at the start of every reuse * (compact_rewrite, recluster), under ShareUpdateExclusiveLock and before any - * ColumnarAllocateFreeSpace, drops such an entry before it can be handed out + * PgColumnarAllocateFreeSpace, drops such an entry before it can be handed out * on top of a live group. Inserts never reuse, so no stale entry is consumed * between the crash and the next reuse. */ void -ColumnarReconcileFreeList(Relation dataRel) +PgColumnarReconcileFreeList(Relation dataRel) { - uint64 base = ColumnarStorageId(dataRel); - List *projs = ColumnarListProjections(base); + uint64 base = PgColumnarStorageId(dataRel); + List *projs = PgColumnarListProjections(base); ListCell *lc; Snapshot snap; FootRange *foots; @@ -1012,7 +1012,7 @@ ColumnarReconcileFreeList(Relation dataRel) /* live footprints and the storage id set (base + distinct projections). Store * storage ids as heap uint64s, not stuffed into pointers, so an id > 2^32 is - * safe on 32-bit builds (matches columnar_end_truncation_storages). */ + * safe on 32-bit builds (matches pgcolumnar_end_truncation_storages). */ collect_footprints(base, snap, &foots, &nf, &capf); { uint64 *s = palloc(sizeof(uint64)); @@ -1022,7 +1022,7 @@ ColumnarReconcileFreeList(Relation dataRel) } foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != base) { @@ -1083,23 +1083,23 @@ ColumnarReconcileFreeList(Relation dataRel) } /* - * ColumnarRetireFullyDeletedGroups + * PgColumnarRetireFullyDeletedGroups * Online compaction (Phase F3a, lazy path): retire every row group that is * fully deleted as-of oldestXmin. Safe under ShareUpdateExclusiveLock, * concurrent with readers and writers. Returns the number of groups retired. */ int64 -ColumnarRetireFullyDeletedGroups(Relation rel) +PgColumnarRetireFullyDeletedGroups(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *groups = ColumnarComputeFullyDeletedGroups(storageId, oldestXmin); + uint64 storageId = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *groups = PgColumnarComputeFullyDeletedGroups(storageId, oldestXmin); ListCell *lc; int64 retired = 0; foreach(lc, groups) { - ColumnarRetireGroup(storageId, *(uint64 *) lfirst(lc)); + PgColumnarRetireGroup(storageId, *(uint64 *) lfirst(lc)); retired++; } return retired; @@ -1108,12 +1108,12 @@ ColumnarRetireFullyDeletedGroups(Relation rel) /* - * ColumnarReadDeleteVectorList + * PgColumnarReadDeleteVectorList * Read all delete_vector rows for a stripe (spec 7.5). Returns a list of * DeleteVectorMetadata* allocated in the current memory context. */ List * -ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot) +PgColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot) { Relation rel = open_columnar_table("delete_vector", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1163,14 +1163,14 @@ ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapsho } /* - * ColumnarStorageHasDeleteVector + * PgColumnarStorageHasDeleteVector * True when the storage has any delete_vector row, i.e. at least one delete has * been recorded. Used to decide whether the native zone-map-only aggregate * is valid (it is only correct when no rows are deleted) or must fall back to * a delete-applying scan (D6b). */ bool -ColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot) +PgColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot) { Relation rel = open_columnar_table("delete_vector", AccessShareLock); ScanKeyData key[1]; @@ -1218,7 +1218,7 @@ delete_vector_chunk_lock_key(uint64 storageId, uint64 stripeId, int chunkId) /* * delete_vector_lock_chunk_group * Take a transaction-scoped exclusive lock covering one chunk group's - * delete_vector tuple, so that the read-modify-write in ColumnarUpsertDeleteVector + * delete_vector tuple, so that the read-modify-write in PgColumnarUpsertDeleteVector * serializes against any concurrent deleter or updater touching the SAME * chunk group, while deletes to different chunk groups still proceed * concurrently. The lock is held until this transaction ends (commit or @@ -1247,11 +1247,11 @@ delete_vector_lock_chunk_group(uint64 storageId, uint64 stripeId, int chunkId) } /* - * ColumnarUpsertDeleteVector + * PgColumnarUpsertDeleteVector * Insert or replace the delete_vector row for one chunk group, identified by * (storage_id, group_number). If a row already exists it is replaced with the * merged bitmap carried in rm; otherwise a fresh row is inserted. Used at - * flush of the in-memory delete buffer (columnar_delete_vector.c), at most + * flush of the in-memory delete buffer (pgcolumnar_delete_vector.c), at most * once per chunk group per flush, so a single heap tuple is never updated * twice in the same command. * @@ -1287,20 +1287,20 @@ row_group_exists(uint64 storageId, uint64 groupNumber, Snapshot snapshot) } /* - * ColumnarLockChunkGroup + * PgColumnarLockChunkGroup * Take the transaction-scoped advisory lock for one chunk group (the whole - * row group, chunk id 0), the same lock ColumnarUpsertDeleteVector takes. Used by + * row group, chunk id 0), the same lock PgColumnarUpsertDeleteVector takes. Used by * online compaction (Phase F3b) so a rewrite of a group serializes with * concurrent deletes to that group. */ void -ColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber) +PgColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber) { delete_vector_lock_chunk_group(storageId, groupNumber, 0); } void -ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) +PgColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) { Relation rel; TupleDesc tupdesc; @@ -1332,13 +1332,13 @@ ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) bool exists; /* - * Latest committed state, with curcid advanced (ColumnarCatalogSnapshot) + * Latest committed state, with curcid advanced (PgColumnarCatalogSnapshot) * so a group this same transaction just flushed (pending writes flush * before delete vectors at pre-commit) is visible and does not look retired. * A group retired by a concurrent COMMITTED rewrite is gone here. */ PushActiveSnapshot(GetLatestSnapshot()); - latest = ColumnarCatalogSnapshot(GetActiveSnapshot()); + latest = PgColumnarCatalogSnapshot(GetActiveSnapshot()); exists = row_group_exists(storageId, rm->groupNumber, latest); PopActiveSnapshot(); if (!exists) @@ -1415,7 +1415,7 @@ ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) } /* - * ColumnarDeleteMetadata + * PgColumnarDeleteMetadata * Remove every metadata row for a storage id. Used when a columnar * table is dropped or truncated. */ @@ -1440,7 +1440,7 @@ delete_rows_by_storage_id(const char *tableName, AttrNumber storageAttno, } void -ColumnarDeleteMetadata(uint64 storageId) +PgColumnarDeleteMetadata(uint64 storageId) { delete_rows_by_storage_id("delete_vector", Anum_delete_vector_storage_id, storageId); /* native format catalog (PGCN v1); no-op rows for 2.2-line tables */ @@ -1455,20 +1455,20 @@ ColumnarDeleteMetadata(uint64 storageId) /* * Opt-in, default off. Set for its own session by a pgcolumnar.parallel_copy * loader (backing the pgcolumnar.bulk_parallel_writer GUC, registered in - * _PG_init) so ColumnarInsertNativeStorageRow can skip the storage-row creation + * _PG_init) so PgColumnarInsertNativeStorageRow can skip the storage-row creation * advisory lock when the row already exists committed. Ordinary writes never set * it, so the default write path is unchanged. */ -bool columnar_bulk_parallel_writer = false; +bool pgcolumnar_bulk_parallel_writer = false; /* - * ColumnarInsertNativeStorageRow, ColumnarInsertRowGroupRow, - * ColumnarInsertColumnChunkRow + * PgColumnarInsertNativeStorageRow, PgColumnarInsertRowGroupRow, + * PgColumnarInsertColumnChunkRow * Record the native-format catalog rows (PGCN v1, native spec 11). Called * by the native writer's flush. The 2.2-line writer does not use these. */ void -ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) +PgColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) { Relation rel = open_columnar_table("storage", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1496,7 +1496,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) */ /* * Bulk-parallel fast path (opt-in, default off). A pgcolumnar.parallel_copy - * loader sets columnar_bulk_parallel_writer for its own session; when the + * loader sets pgcolumnar_bulk_parallel_writer for its own session; when the * storage row already exists in the latest committed state we skip the advisory * lock entirely and return. That lock exists ONLY to serialize the first-writer * creation race -- once the row is committed there is nothing to wait for -- and @@ -1507,10 +1507,10 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * table's storage concurrently and 2PC-safely. Every ordinary write leaves the * flag false and takes the unchanged path below. */ - if (columnar_bulk_parallel_writer) + if (pgcolumnar_bulk_parallel_writer) { PushActiveSnapshot(GetLatestSnapshot()); - snapshot = ColumnarCatalogSnapshot(GetActiveSnapshot()); + snapshot = PgColumnarCatalogSnapshot(GetActiveSnapshot()); ScanKeyInit(&key[0], Anum_native_storage_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) s->storageId)); scan = systable_beginscan(rel, InvalidOid, false, snapshot, 1, key); @@ -1533,7 +1533,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) /* * Re-check under the lock against a fresh snapshot so the loser of a * cross-transaction race sees the winner's just-committed row (GetLatestSnapshot), - * with curcid advanced (ColumnarCatalogSnapshot) so a second flush in this same + * with curcid advanced (PgColumnarCatalogSnapshot) so a second flush in this same * transaction still sees the row this transaction already inserted. The latest * snapshot must be pushed active before it drives a heap visibility check: * PostgreSQL 18 asserts a scan snapshot is registered or active @@ -1541,7 +1541,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * count. Pop it once the existence scan is done. */ PushActiveSnapshot(GetLatestSnapshot()); - snapshot = ColumnarCatalogSnapshot(GetActiveSnapshot()); + snapshot = PgColumnarCatalogSnapshot(GetActiveSnapshot()); ScanKeyInit(&key[0], Anum_native_storage_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) s->storageId)); scan = systable_beginscan(rel, InvalidOid, false, snapshot, 1, key); @@ -1562,7 +1562,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) values[Anum_native_storage_row_group_limit - 1] = Int32GetDatum(s->rowGroupLimit); /* * A new storage starts unordered. An ordering rewrite sets this afterwards - * (ColumnarSetSortedThrough); an unsorted one leaves it NULL, which is what + * (PgColumnarSetSortedThrough); an unsorted one leaves it NULL, which is what * makes a rewrite reset the sort state with no invalidation step. */ nulls[Anum_native_storage_sorted_through - 1] = true; @@ -1575,7 +1575,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) } /* - * ColumnarSetSortedThrough + * PgColumnarSetSortedThrough * Record the row group number the last ordering rewrite ended at, so a * reader can tell how much of the layout is still ordered (issue #301). * @@ -1594,7 +1594,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * groups and reports no decay. */ void -ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) +PgColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) { Relation rel; TupleDesc tupdesc; @@ -1645,10 +1645,10 @@ ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) } /* - * ColumnarCheckNativeFormatVersion + * PgColumnarCheckNativeFormatVersion * Reject a native data format version this build does not understand, before * any bytes are decoded (issue #240). The physical metapage version is checked - * separately when the metapage is read (ColumnarReadMetapage); this is the + * separately when the metapage is read (PgColumnarReadMetapage); this is the * independent data-format stamp (pgcolumnar.storage.format_version), so a * future PGCN version that changes the encoding while keeping the metapage * layout is caught here rather than silently misread. @@ -1659,7 +1659,7 @@ ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) * nothing is left open across the error. */ void -ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) +PgColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) { Relation rel = open_columnar_table("storage", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1701,7 +1701,7 @@ ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) } void -ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) +PgColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) { Relation rel = open_columnar_table("row_group", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1726,7 +1726,7 @@ ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) } void -ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) +PgColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) { Relation rel = open_columnar_table("column_chunk", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1757,13 +1757,13 @@ ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) } /* - * ColumnarInsertZoneMapRow + * PgColumnarInsertZoneMapRow * Record one native zone-map row (Small Materialized Aggregate) for a vector * or for a whole column chunk (native spec 7.1, Phase D5). Called by the * native writer's flush. */ void -ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) +PgColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) { Relation rel = open_columnar_table("zone_map", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1813,11 +1813,11 @@ ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) } /* - * ColumnarInsertBloomRow + * PgColumnarInsertBloomRow * Record one per-column-chunk bloom filter (native spec 7.2, Phase D5b). */ void -ColumnarInsertBloomRow(const NativeBloomMetadata *b) +PgColumnarInsertBloomRow(const NativeBloomMetadata *b) { Relation rel = open_columnar_table("bloom", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1844,13 +1844,13 @@ ColumnarInsertBloomRow(const NativeBloomMetadata *b) } /* - * ColumnarReadBloomList + * PgColumnarReadBloomList * The per-column-chunk bloom filters of one row group (native spec 7.2, * Phase D5b). The caller indexes the result by column_index; the filter * bytes are copied into the current memory context. */ List * -ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("bloom", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1864,7 +1864,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_bloom_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("bloom_pkey"); + idxOid = pgcolumnar_index_oid("bloom_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -1895,7 +1895,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) } /* - * ColumnarReadBloomForColumn + * PgColumnarReadBloomForColumn * One column's bloom filter for one row group, or NULL when it has none * (issue #314). * @@ -1907,7 +1907,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) * column reads most of what it fetches for nothing. */ NativeBloomMetadata * -ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, +PgColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, int columnIndex, Snapshot snapshot) { Relation rel = open_columnar_table("bloom", AccessShareLock); @@ -1924,7 +1924,7 @@ ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); ScanKeyInit(&key[2], Anum_bloom_column_index, BTEqualStrategyNumber, F_INT2EQ, Int16GetDatum((int16) columnIndex)); - idxOid = columnar_index_oid("bloom_pkey"); + idxOid = pgcolumnar_index_oid("bloom_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 3, key); tuple = systable_getnext(scan); @@ -1954,14 +1954,14 @@ ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, } /* - * ColumnarReadZoneMapVectors + * PgColumnarReadZoneMapVectors * The per-vector zone maps (vector_index >= 0) of one row group, for * per-vector skipping (native spec 7.1, Phase D5b). Only min/max and the * vector/column indices are needed by the caller. The min/max bytes are * copied into the current memory context. */ List * -ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("zone_map", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1975,7 +1975,7 @@ ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapsh F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_zone_map_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("zone_map_pkey"); + idxOid = pgcolumnar_index_oid("zone_map_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2044,11 +2044,11 @@ row_group_cmp(const ListCell *a, const ListCell *b) } /* - * ColumnarReadRowGroupList + * PgColumnarReadRowGroupList * The native row groups of a storage, ordered by group number. */ List * -ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) +PgColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) { Relation rel = open_columnar_table("row_group", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2056,7 +2056,7 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) SysScanDesc scan; HeapTuple tuple; List *result = NIL; - Oid rgIdx = columnar_index_oid("row_group_pkey"); + Oid rgIdx = pgcolumnar_index_oid("row_group_pkey"); ScanKeyInit(&key[0], Anum_row_group_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) storageId)); @@ -2065,12 +2065,12 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) * rather than a heap scan of every storage's row groups in the database. * * This read is on the unique-check path: _bt_check_unique() reaches it - * through columnar_index_fetch_tuple() under its own on-stack SnapshotDirty + * through pgcolumnar_index_fetch_tuple() under its own on-stack SnapshotDirty * and reads xmin/xmax back out afterwards. A bare index scan here once * spun the check forever (test/unique_conc.sh scenario 7): when this storage * has nothing flushed the scan matches no rows, HeapTupleSatisfiesDirty() * never runs, and the uninitialised out-fields are read as a phantom xact to - * wait on. columnar_index_fetch_tuple() now resets those fields before the + * wait on. pgcolumnar_index_fetch_tuple() now resets those fields before the * fetch, the way HeapTupleSatisfiesDirty() does for a heap row, so the index * scan is safe: an in-progress group still sets xmin here and the check waits * on the real inserter. @@ -2102,13 +2102,13 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) } /* - * ColumnarReadColumnChunkList + * PgColumnarReadColumnChunkList * The native column chunks of one row group. The caller indexes the result * by column_index; the encoding descriptor bytes are copied into the * current memory context. */ List * -ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("column_chunk", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2122,7 +2122,7 @@ ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snaps F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_column_chunk_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("column_chunk_pkey"); + idxOid = pgcolumnar_index_oid("column_chunk_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2162,14 +2162,14 @@ ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snaps } /* - * ColumnarReadZoneMapList + * PgColumnarReadZoneMapList * The whole-chunk zone maps (vector_index -1) of one row group, for group * skipping (native spec 7.1, Phase D5b). The caller indexes the result by * column_index; the minimum/maximum bytes are copied into the current memory * context. Per-vector rows (vector_index >= 0) are skipped by this reader. */ List * -ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("zone_map", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2183,7 +2183,7 @@ ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_zone_map_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("zone_map_pkey"); + idxOid = pgcolumnar_index_oid("zone_map_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2250,12 +2250,12 @@ ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) * ------------------------------------------------------------------------- */ /* - * columnar_compression_from_name + * pgcolumnar_compression_from_name * Map a compression codec name to its code (spec 5). Returns -1 for an * unrecognized name so the caller can fall back to the instance default. */ static int -columnar_compression_from_name(const char *name) +pgcolumnar_compression_from_name(const char *name) { if (strcmp(name, "none") == 0) return COLUMNAR_COMPRESSION_NONE; @@ -2269,7 +2269,7 @@ columnar_compression_from_name(const char *name) } /* - * ColumnarReadOptions + * PgColumnarReadOptions * Load the per-table options row for a relation (spec 7.4) into *opts, * setting a per-field "set" flag for each column that is present (not * SQL NULL). Returns true when a row exists. The catalog is read with a @@ -2277,7 +2277,7 @@ columnar_compression_from_name(const char *name) * take effect for subsequent writes (spec 9). */ bool -ColumnarReadOptions(Oid relid, ColumnarOptions *opts) +PgColumnarReadOptions(Oid relid, PgColumnarOptions *opts) { Relation rel = open_columnar_table("options", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2288,10 +2288,10 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) Snapshot snapshot; bool found = false; - memset(opts, 0, sizeof(ColumnarOptions)); + memset(opts, 0, sizeof(PgColumnarOptions)); base = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - snapshot = ColumnarCatalogSnapshot(base); + snapshot = PgColumnarCatalogSnapshot(base); ScanKeyInit(&key[0], Anum_options_regclass, BTEqualStrategyNumber, F_OIDEQ, ObjectIdGetDatum(relid)); @@ -2328,7 +2328,7 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) d = heap_getattr(tuple, Anum_options_compression, tupdesc, &isnull); if (!isnull) { - int code = columnar_compression_from_name(NameStr(*DatumGetName(d))); + int code = pgcolumnar_compression_from_name(NameStr(*DatumGetName(d))); if (code >= 0) { @@ -2369,18 +2369,18 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) /* - * ColumnarReadSortBy + * PgColumnarReadSortBy * Load the declared sort_by column names for a relation (#288) from * pgcolumnar.options. Returns a List of pstrdup'd column-name strings in * the caller's memory context, or NIL when no sort key is declared (no * options row, or sort_by is SQL NULL, or the array is empty). Stored as * column NAMES, not attnums, so it survives dump/restore; the caller * resolves the names to attnums and validates them each apply. Read with - * the same command-id-advanced snapshot as ColumnarReadOptions so a + * the same command-id-advanced snapshot as PgColumnarReadOptions so a * sort_by set earlier in this transaction is visible. */ List * -ColumnarReadSortBy(Oid relid) +PgColumnarReadSortBy(Oid relid) { Relation rel = open_columnar_table("options", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2392,7 +2392,7 @@ ColumnarReadSortBy(Oid relid) List *names = NIL; base = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - snapshot = ColumnarCatalogSnapshot(base); + snapshot = PgColumnarCatalogSnapshot(base); ScanKeyInit(&key[0], Anum_options_regclass, BTEqualStrategyNumber, F_OIDEQ, ObjectIdGetDatum(relid)); @@ -2431,13 +2431,13 @@ ColumnarReadSortBy(Oid relid) /* - * ColumnarDeleteOptions + * PgColumnarDeleteOptions * Remove a relation's per-table options row, called when the table is * dropped. The options table is keyed by regclass (relation oid), not by - * storage id, so it is cleaned up separately from ColumnarDeleteMetadata. + * storage id, so it is cleaned up separately from PgColumnarDeleteMetadata. */ void -ColumnarDeleteOptions(Oid relid) +PgColumnarDeleteOptions(Oid relid) { Relation rel = open_columnar_table("options", RowExclusiveLock); ScanKeyData key[1]; @@ -2456,12 +2456,12 @@ ColumnarDeleteOptions(Oid relid) } /* - * ColumnarIsColumnarRelation + * PgColumnarIsColumnarRelation * Whether a relation uses the columnar table access method. The access * method oid is resolved once and cached. */ bool -ColumnarIsColumnarRelation(Oid relid) +PgColumnarIsColumnarRelation(Oid relid) { static Oid columnarAmOid = InvalidOid; @@ -2519,8 +2519,8 @@ int16_array_from_datum(Datum d, int *len) static int projection_cmp(const ListCell *a, const ListCell *b) { - const ColumnarProjection *pa = (const ColumnarProjection *) lfirst(a); - const ColumnarProjection *pb = (const ColumnarProjection *) lfirst(b); + const PgColumnarProjection *pa = (const PgColumnarProjection *) lfirst(a); + const PgColumnarProjection *pb = (const PgColumnarProjection *) lfirst(b); if (pa->projectionId < pb->projectionId) return -1; @@ -2530,7 +2530,7 @@ projection_cmp(const ListCell *a, const ListCell *b) } void -ColumnarInsertProjectionRow(const ColumnarProjection *proj) +PgColumnarInsertProjectionRow(const PgColumnarProjection *proj) { Relation rel = open_columnar_table("projection", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2559,7 +2559,7 @@ ColumnarInsertProjectionRow(const ColumnarProjection *proj) } /* - * ColumnarRecordProjectionDeclaration + * PgColumnarRecordProjectionDeclaration * Record the intent behind a projection: which relation, which name, and * which columns by NAME rather than by attnum (#266). * @@ -2573,7 +2573,7 @@ ColumnarInsertProjectionRow(const ColumnarProjection *proj) * not leave two declarations behind. */ void -ColumnarRecordProjectionDeclaration(Oid relid, const char *name, +PgColumnarRecordProjectionDeclaration(Oid relid, const char *name, ArrayType *columns, ArrayType *sortKey) { Relation rel; @@ -2582,7 +2582,7 @@ ColumnarRecordProjectionDeclaration(Oid relid, const char *name, bool nulls[Natts_projection_declaration]; HeapTuple tuple; - ColumnarDeleteProjectionDeclaration(relid, name); + PgColumnarDeleteProjectionDeclaration(relid, name); rel = open_columnar_table("projection_declaration", RowExclusiveLock); tupdesc = RelationGetDescr(rel); @@ -2603,12 +2603,12 @@ ColumnarRecordProjectionDeclaration(Oid relid, const char *name, } /* - * ColumnarDeleteProjectionDeclaration + * PgColumnarDeleteProjectionDeclaration * Forget the declaration for one projection. Called when it is dropped, and * before recording a replacement. */ void -ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) +PgColumnarDeleteProjectionDeclaration(Oid relid, const char *name) { Relation rel = open_columnar_table("projection_declaration", RowExclusiveLock); @@ -2637,7 +2637,7 @@ ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) } /* - * ColumnarDeleteProjectionDeclarationsForRel + * PgColumnarDeleteProjectionDeclarationsForRel * Forget every declaration for a relation. Called when the relation is * dropped (#304). * @@ -2649,7 +2649,7 @@ ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) * the same hook; this catalog needs the same treatment. */ void -ColumnarDeleteProjectionDeclarationsForRel(Oid relid) +PgColumnarDeleteProjectionDeclarationsForRel(Oid relid) { Relation rel = open_columnar_table("projection_declaration", RowExclusiveLock); @@ -2670,7 +2670,7 @@ ColumnarDeleteProjectionDeclarationsForRel(Oid relid) } List * -ColumnarListProjections(uint64 storageId) +PgColumnarListProjections(uint64 storageId) { Relation rel = open_columnar_table("projection", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2687,7 +2687,7 @@ ColumnarListProjections(uint64 storageId) scan = systable_beginscan(rel, InvalidOid, false, NULL, 1, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) { - ColumnarProjection *p = palloc0(sizeof(ColumnarProjection)); + PgColumnarProjection *p = palloc0(sizeof(PgColumnarProjection)); bool isnull; Datum d; @@ -2713,7 +2713,7 @@ ColumnarListProjections(uint64 storageId) } void -ColumnarDeleteProjectionRow(uint64 storageId, int projectionId) +PgColumnarDeleteProjectionRow(uint64 storageId, int projectionId) { Relation rel = open_columnar_table("projection", RowExclusiveLock); ScanKeyData key[2]; diff --git a/src/columnar_parallel_copy.c b/src/columnar_parallel_copy.c index 2cf794f..e8bd676 100644 --- a/src/columnar_parallel_copy.c +++ b/src/columnar_parallel_copy.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parallel_copy.c + * pgcolumnar_parallel_copy.c * Parallel bulk ingest for pgColumnar (#300). * * Splits a load across background workers, each running core COPY over a @@ -22,8 +22,8 @@ * - the single (non-partitioned) columnar table shape: N loaders write ONE shared * storage concurrently. Here parallelism does not come from distinct storage; * it comes from the coordinator pre-creating and committing the storage row so - * the loaders skip its creation lock, each writing via columnar_bulk_parallel_writer. - * - a standalone byte splitter (columnar_file_split_offsets) exposed to SQL: N+1 + * the loaders skip its creation lock, each writing via pgcolumnar_bulk_parallel_writer. + * - a standalone byte splitter (pgcolumnar_file_split_offsets) exposed to SQL: N+1 * line-aligned offsets, a diagnostic the parallel load itself no longer calls. * Text format only for now, numeric/date-time partition keys only (their text form * is escape-free); CSV and other key types are later phases (see the plan). @@ -223,10 +223,10 @@ pcopy_naive_offsets(const char *path, int workers) return offs; } -PG_FUNCTION_INFO_V1(columnar_file_split_offsets); +PG_FUNCTION_INFO_V1(pgcolumnar_file_split_offsets); /* - * columnar_file_split_offsets(path text, workers int) -> bigint[] + * pgcolumnar_file_split_offsets(path text, workers int) -> bigint[] * * Returns workers+1 ascending byte offsets [0 .. filesize] that split the file * into `workers` line-aligned ranges. off[0] is always 0 and off[workers] is @@ -241,7 +241,7 @@ PG_FUNCTION_INFO_V1(columnar_file_split_offsets); * PCOPY_MAX_WORKERS. */ Datum -columnar_file_split_offsets(PG_FUNCTION_ARGS) +pgcolumnar_file_split_offsets(PG_FUNCTION_ARGS) { char *path; int32 workers; @@ -363,7 +363,7 @@ typedef struct PcopyHeader int nworkers; bool single_table; /* target is one columnar table (not partitioned): * coordinator pre-creates the storage row and loaders - * set columnar_bulk_parallel_writer */ + * set pgcolumnar_bulk_parallel_writer */ char filename[MAXPGPATH]; /* coordinator -> function result channel */ pg_atomic_uint32 coord_state; /* PcopyCoordState */ @@ -777,7 +777,7 @@ pgcolumnar_parallel_copy_worker(Datum main_arg) * never contends, so the flag stays off there. */ if (hdr->single_table) - columnar_bulk_parallel_writer = true; + pgcolumnar_bulk_parallel_writer = true; PG_TRY(); { @@ -1017,7 +1017,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) * Single-table load: pre-create and COMMIT the storage catalog row before any * loader starts, in the coordinator's own top-level session (the SQL function * cannot commit). With the row committed, each loader -- which sets - * columnar_bulk_parallel_writer -- sees it and skips the storage-row creation + * pgcolumnar_bulk_parallel_writer -- sees it and skips the storage-row creation * lock, so N loaders write the one storage concurrently and 2PC-safely. The * coordinator itself leaves the flag off, so this uses the normal create path. */ @@ -1028,7 +1028,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) StartTransactionCommand(); /* * StartTransactionCommand does not push an active snapshot, but - * ColumnarEnsureStorageRow reads pgcolumnar.options/storage via + * PgColumnarEnsureStorageRow reads pgcolumnar.options/storage via * systable scans, and those visibility checks require a registered or * active snapshot. A normal backend has one from the executor; this * bgworker does not, so push one explicitly. Without it the scan runs @@ -1038,7 +1038,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) */ PushActiveSnapshot(GetTransactionSnapshot()); rel = table_open(hdr->relid, RowExclusiveLock); - ColumnarEnsureStorageRow(rel); + PgColumnarEnsureStorageRow(rel); table_close(rel, NoLock); PopActiveSnapshot(); CommitTransactionCommand(); @@ -1213,17 +1213,17 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) proc_exit(0); } -PG_FUNCTION_INFO_V1(columnar_parallel_copy); +PG_FUNCTION_INFO_V1(pgcolumnar_parallel_copy); /* - * columnar_parallel_copy(target regclass, filename text, workers int) + * pgcolumnar_parallel_copy(target regclass, filename text, workers int) * -> rows loaded. * * Atomic bulk load: launches the coordinator bgworker (which spawns the loaders, * 2-phase-commits them all or rolls them all back) and returns its total. */ Datum -columnar_parallel_copy(PG_FUNCTION_ARGS) +pgcolumnar_parallel_copy(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1272,7 +1272,7 @@ columnar_parallel_copy(PG_FUNCTION_ARGS) * partition-aligned byte ranges (this may lower `workers` to the partition * count), which requires the file sorted by the partition key. * - a single columnar table: the loaders write the one storage concurrently - * via columnar_bulk_parallel_writer (see below), so a naive record-aligned + * via pgcolumnar_bulk_parallel_writer (see below), so a naive record-aligned * byte split is enough and the file needs no ordering. * Any other target (e.g. a heap, or a partitioned table with non-columnar * partitions) is rejected. A naive split of one non-partitioned columnar table @@ -1308,13 +1308,13 @@ columnar_parallel_copy(PG_FUNCTION_ARGS) offs = pcopy_partition_aligned_offsets(target, path, &workers); single_table = false; } - else if (ColumnarIsColumnarRelation(relid)) + else if (PgColumnarIsColumnarRelation(relid)) { /* * A single columnar table: workers write the ONE storage concurrently * (distinct stripe/row-number reservations; the coordinator pre-creates * the storage row and the loaders skip its creation lock via - * columnar_bulk_parallel_writer). Any record-aligned byte split is + * pgcolumnar_bulk_parallel_writer). Any record-aligned byte split is * correct -- no partition key, so no sorted-input requirement. */ offs = pcopy_naive_offsets(path, workers); diff --git a/src/columnar_parallel_export.c b/src/columnar_parallel_export.c index f9ad60a..c92ad5a 100644 --- a/src/columnar_parallel_export.c +++ b/src/columnar_parallel_export.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parallel_export.c + * pgcolumnar_parallel_export.c * pgcolumnar.parallel_export_parquet: parallel Parquet export. * * N read-only background workers each write a disjoint slice of the source to @@ -22,7 +22,7 @@ * the dispatcher ships the oid-sorted leaf list so workers do not re-derive * it from the live catalog (which a concurrent ATTACH could desynchronise); * - a single columnar table: split by row-group index ranges; each worker - * restricts its read to its slice via ColumnarReadRestrictToGroups. + * restricts its read to its slice via PgColumnarReadRestrictToGroups. * * Cleanroom: public PostgreSQL APIs and this project's own code only. * @@ -334,8 +334,8 @@ pgcolumnar_parallel_export_worker(Datum main_arg) if (hdr->single_table) { Relation rel = table_open(hdr->relid, AccessShareLock); - List *groups = ColumnarReadRowGroupList(hdr->storageId, - ColumnarCatalogSnapshot(snap)); + List *groups = PgColumnarReadRowGroupList(hdr->storageId, + PgColumnarCatalogSnapshot(snap)); int ntake = me->endIdx - me->startIdx; /* always non-NULL, so an empty slice restricts to nothing (not all) */ uint64 *gnos = palloc(sizeof(uint64) * Max(ntake, 1)); @@ -353,7 +353,7 @@ pgcolumnar_parallel_export_worker(Datum main_arg) } idx++; } - rows = ColumnarWriteParquetFile(rel, snap, me->filepath, gnos, k); + rows = PgColumnarWriteParquetFile(rel, snap, me->filepath, gnos, k); table_close(rel, AccessShareLock); } else @@ -371,7 +371,7 @@ pgcolumnar_parallel_export_worker(Datum main_arg) char fp[MAXPGPATH]; snprintf(fp, sizeof(fp), "%s/part-%04d.parquet", hdr->dirpath, i); - rows += ColumnarWriteParquetFile(part, snap, fp, NULL, 0); + rows += PgColumnarWriteParquetFile(part, snap, fp, NULL, 0); table_close(part, AccessShareLock); MemoryContextSwitchTo(old); MemoryContextDelete(pctx); @@ -405,13 +405,13 @@ pgcolumnar_parallel_export_worker(Datum main_arg) } /* - * columnar_parallel_export_parquet + * pgcolumnar_parallel_export_parquet * SQL: pgcolumnar.parallel_export_parquet(target regclass, path text, * workers int DEFAULT NULL) -> bigint. */ -PG_FUNCTION_INFO_V1(columnar_parallel_export_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_parallel_export_parquet); Datum -columnar_parallel_export_parquet(PG_FUNCTION_ARGS) +pgcolumnar_parallel_export_parquet(PG_FUNCTION_ARGS) { Oid relid; char *dir; @@ -497,7 +497,7 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) { Relation part = table_open(leafoids[i], AccessShareLock); - if (!ColumnarIsColumnarRelation(leafoids[i])) + if (!PgColumnarIsColumnarRelation(leafoids[i])) { char nm[NAMEDATALEN]; @@ -507,7 +507,7 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("partition \"%s\" is not a columnar table", nm))); } - ColumnarParquetCheckExportable(part); + PgColumnarParquetCheckExportable(part); table_close(part, AccessShareLock); } single_table = false; @@ -515,13 +515,13 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) if (workers > npart) workers = npart; } - else if (ColumnarIsColumnarRelation(relid)) + else if (PgColumnarIsColumnarRelation(relid)) { List *groups; - ColumnarParquetCheckExportable(rel); - storageId = ColumnarStorageId(rel); - groups = ColumnarReadRowGroupList(storageId, ColumnarCatalogSnapshot(snap)); + PgColumnarParquetCheckExportable(rel); + storageId = PgColumnarStorageId(rel); + groups = PgColumnarReadRowGroupList(storageId, PgColumnarCatalogSnapshot(snap)); ntasks = list_length(groups); single_table = true; if (workers > Max(ntasks, 1)) diff --git a/src/columnar_parquet.c b/src/columnar_parquet.c index 663b524..db9fc60 100644 --- a/src/columnar_parquet.c +++ b/src/columnar_parquet.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet.c + * pgcolumnar_parquet.c * Parquet file export for pgColumnar (gap 27, piece 2). * * pgcolumnar.export_parquet(rel regclass, path text) writes a columnar table @@ -47,7 +47,7 @@ #include "utils/typcache.h" #include "utils/uuid.h" -PG_FUNCTION_INFO_V1(columnar_export_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_export_parquet); #define PARQUET_ROWGROUP_ROWS 65536 @@ -531,7 +531,7 @@ build_rle_levels(StringInfo out, const uint8 *levels, int64 n, int bit_width) int32 len; initStringInfo(&h); - ColumnarThriftPutVarint(&h, (uint64) ((ngroups << 1) | 1)); /* one bit-packed run */ + PgColumnarThriftPutVarint(&h, (uint64) ((ngroups << 1) | 1)); /* one bit-packed run */ for (i = 0; i < n; i++) { uint32 v = levels[i]; @@ -594,17 +594,17 @@ write_page_header(StringInfo out, int64 nrows, int32 body_size) int16 dlast = 0; /* PageHeader */ - ColumnarThriftPutI32Field(out, &last, 1, 0); /* type = DATA_PAGE */ - ColumnarThriftPutI32Field(out, &last, 2, body_size); /* uncompressed_page_size */ - ColumnarThriftPutI32Field(out, &last, 3, body_size); /* compressed_page_size */ + PgColumnarThriftPutI32Field(out, &last, 1, 0); /* type = DATA_PAGE */ + PgColumnarThriftPutI32Field(out, &last, 2, body_size); /* uncompressed_page_size */ + PgColumnarThriftPutI32Field(out, &last, 3, body_size); /* compressed_page_size */ /* field 5: data_page_header (struct) */ - ColumnarThriftPutField(out, &last, 5, TC_STRUCT); - ColumnarThriftPutI32Field(out, &dlast, 1, (int32) nrows); /* num_values */ - ColumnarThriftPutI32Field(out, &dlast, 2, PQ_ENC_PLAIN); /* encoding */ - ColumnarThriftPutI32Field(out, &dlast, 3, PQ_ENC_RLE); /* def level encoding */ - ColumnarThriftPutI32Field(out, &dlast, 4, PQ_ENC_RLE); /* rep level encoding */ - ColumnarThriftPutStop(out); /* end data_page_header */ - ColumnarThriftPutStop(out); /* end PageHeader */ + PgColumnarThriftPutField(out, &last, 5, TC_STRUCT); + PgColumnarThriftPutI32Field(out, &dlast, 1, (int32) nrows); /* num_values */ + PgColumnarThriftPutI32Field(out, &dlast, 2, PQ_ENC_PLAIN); /* encoding */ + PgColumnarThriftPutI32Field(out, &dlast, 3, PQ_ENC_RLE); /* def level encoding */ + PgColumnarThriftPutI32Field(out, &dlast, 4, PQ_ENC_RLE); /* rep level encoding */ + PgColumnarThriftPutStop(out); /* end data_page_header */ + PgColumnarThriftPutStop(out); /* end PageHeader */ } /* ---- FileMetaData footer ---- */ @@ -613,9 +613,9 @@ write_schema_element_root(StringInfo b, int ncols) { int16 last = 0; - ColumnarThriftPutStringField(b, &last, 4, "schema", 6); /* name */ - ColumnarThriftPutI32Field(b, &last, 5, ncols); /* num_children */ - ColumnarThriftPutStop(b); + PgColumnarThriftPutStringField(b, &last, 4, "schema", 6); /* name */ + PgColumnarThriftPutI32Field(b, &last, 5, ncols); /* num_children */ + PgColumnarThriftPutStop(b); } /* one leaf SchemaElement (a primitive) */ @@ -624,19 +624,19 @@ write_schema_leaf(StringInfo b, const char *name, PqLeaf *leaf, int repetition) { int16 last = 0; - ColumnarThriftPutI32Field(b, &last, 1, leaf->ptype); /* type */ + PgColumnarThriftPutI32Field(b, &last, 1, leaf->ptype); /* type */ if (leaf->ptype == PQ_FIXED_LEN_BYTE_ARRAY) - ColumnarThriftPutI32Field(b, &last, 2, leaf->typeLength); - ColumnarThriftPutI32Field(b, &last, 3, repetition); - ColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); + PgColumnarThriftPutI32Field(b, &last, 2, leaf->typeLength); + PgColumnarThriftPutI32Field(b, &last, 3, repetition); + PgColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); if (leaf->convType >= 0) - ColumnarThriftPutI32Field(b, &last, 6, leaf->convType); + PgColumnarThriftPutI32Field(b, &last, 6, leaf->convType); if (leaf->convType == PQ_CT_DECIMAL) { - ColumnarThriftPutI32Field(b, &last, 7, leaf->scale); - ColumnarThriftPutI32Field(b, &last, 8, leaf->precision); + PgColumnarThriftPutI32Field(b, &last, 7, leaf->scale); + PgColumnarThriftPutI32Field(b, &last, 8, leaf->precision); } - ColumnarThriftPutStop(b); + PgColumnarThriftPutStop(b); } /* one group SchemaElement (no physical type; has num_children) */ @@ -646,12 +646,12 @@ write_schema_group(StringInfo b, const char *name, int repetition, { int16 last = 0; - ColumnarThriftPutI32Field(b, &last, 3, repetition); - ColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); - ColumnarThriftPutI32Field(b, &last, 5, num_children); + PgColumnarThriftPutI32Field(b, &last, 3, repetition); + PgColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); + PgColumnarThriftPutI32Field(b, &last, 5, num_children); if (convType >= 0) - ColumnarThriftPutI32Field(b, &last, 6, convType); - ColumnarThriftPutStop(b); + PgColumnarThriftPutI32Field(b, &last, 6, convType); + PgColumnarThriftPutStop(b); } /* number of SchemaElements a top column contributes (excluding the root) */ @@ -711,27 +711,27 @@ write_column_chunk(StringInfo b, PqLeaf *c, PqColMeta *m) int16 mlast = 0; int p; - ColumnarThriftPutI64Field(b, &last, 2, m->dataPageOffset); /* file_offset */ - ColumnarThriftPutField(b, &last, 3, TC_STRUCT); /* meta_data */ - ColumnarThriftPutI32Field(b, &mlast, 1, c->ptype); /* type */ - ColumnarThriftPutField(b, &mlast, 2, TC_LIST); /* encodings [PLAIN, RLE] */ - ColumnarThriftPutListHeader(b, 2, TC_I32); - ColumnarThriftPutZigzag32(b, PQ_ENC_PLAIN); - ColumnarThriftPutZigzag32(b, PQ_ENC_RLE); - ColumnarThriftPutField(b, &mlast, 3, TC_LIST); /* path_in_schema */ - ColumnarThriftPutListHeader(b, c->pathlen, TC_BINARY); + PgColumnarThriftPutI64Field(b, &last, 2, m->dataPageOffset); /* file_offset */ + PgColumnarThriftPutField(b, &last, 3, TC_STRUCT); /* meta_data */ + PgColumnarThriftPutI32Field(b, &mlast, 1, c->ptype); /* type */ + PgColumnarThriftPutField(b, &mlast, 2, TC_LIST); /* encodings [PLAIN, RLE] */ + PgColumnarThriftPutListHeader(b, 2, TC_I32); + PgColumnarThriftPutZigzag32(b, PQ_ENC_PLAIN); + PgColumnarThriftPutZigzag32(b, PQ_ENC_RLE); + PgColumnarThriftPutField(b, &mlast, 3, TC_LIST); /* path_in_schema */ + PgColumnarThriftPutListHeader(b, c->pathlen, TC_BINARY); for (p = 0; p < c->pathlen; p++) { - ColumnarThriftPutVarint(b, (uint64) strlen(c->path[p])); + PgColumnarThriftPutVarint(b, (uint64) strlen(c->path[p])); appendBinaryStringInfo(b, c->path[p], strlen(c->path[p])); } - ColumnarThriftPutI32Field(b, &mlast, 4, 0); /* codec = UNCOMPRESSED */ - ColumnarThriftPutI64Field(b, &mlast, 5, m->numValues); /* num_values */ - ColumnarThriftPutI64Field(b, &mlast, 6, m->totalSize); /* total_uncompressed_size */ - ColumnarThriftPutI64Field(b, &mlast, 7, m->totalSize); /* total_compressed_size */ - ColumnarThriftPutI64Field(b, &mlast, 9, m->dataPageOffset); /* data_page_offset */ - ColumnarThriftPutStop(b); /* end ColumnMetaData */ - ColumnarThriftPutStop(b); /* end ColumnChunk */ + PgColumnarThriftPutI32Field(b, &mlast, 4, 0); /* codec = UNCOMPRESSED */ + PgColumnarThriftPutI64Field(b, &mlast, 5, m->numValues); /* num_values */ + PgColumnarThriftPutI64Field(b, &mlast, 6, m->totalSize); /* total_uncompressed_size */ + PgColumnarThriftPutI64Field(b, &mlast, 7, m->totalSize); /* total_compressed_size */ + PgColumnarThriftPutI64Field(b, &mlast, 9, m->dataPageOffset); /* data_page_offset */ + PgColumnarThriftPutStop(b); /* end ColumnMetaData */ + PgColumnarThriftPutStop(b); /* end ColumnChunk */ } static void @@ -740,13 +740,13 @@ write_row_group(StringInfo b, PqLeaf *leaves, int nleaves, PqRowGroup *rg) int16 last = 0; int i; - ColumnarThriftPutField(b, &last, 1, TC_LIST); /* columns */ - ColumnarThriftPutListHeader(b, nleaves, TC_STRUCT); + PgColumnarThriftPutField(b, &last, 1, TC_LIST); /* columns */ + PgColumnarThriftPutListHeader(b, nleaves, TC_STRUCT); for (i = 0; i < nleaves; i++) write_column_chunk(b, &leaves[i], &rg->cols[i]); - ColumnarThriftPutI64Field(b, &last, 2, rg->totalByteSize); - ColumnarThriftPutI64Field(b, &last, 3, rg->numRows); - ColumnarThriftPutStop(b); + PgColumnarThriftPutI64Field(b, &last, 2, rg->totalByteSize); + PgColumnarThriftPutI64Field(b, &last, 3, rg->numRows); + PgColumnarThriftPutStop(b); } /* initialize a scalar leaf for a given type; *ok=false if unsupported */ @@ -886,13 +886,13 @@ build_top_column(TopColumn *tc, const char *name, Oid typid, int32 typmod, } /* - * ColumnarParquetCheckExportable + * PgColumnarParquetCheckExportable * Ereport if rel cannot be exported to Parquet (a dropped column, or a * column type the writer does not support). Lets the parallel exporter fail * fast in the dispatcher, before it opens files or spawns workers. */ void -ColumnarParquetCheckExportable(Relation rel) +PgColumnarParquetCheckExportable(Relation rel) { TupleDesc tupdesc = RelationGetDescr(rel); int ntop = tupdesc->natts; @@ -933,16 +933,16 @@ ColumnarParquetCheckExportable(Relation rel) } /* - * ColumnarWriteParquetFile + * PgColumnarWriteParquetFile * Write rel's live rows to a Parquet file at filepath, under snapshot. * When restrictGroups is non-NULL, only those row groups are written (the * parallel exporter gives each worker a disjoint group slice). Returns rows * written. The caller owns rel (kept open) and the snapshot; on error this * ereports and the resource owner releases the lock. Shared by the serial - * columnar_export_parquet and the parallel exporter. + * pgcolumnar_export_parquet and the parallel exporter. */ int64 -ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, +PgColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, const uint64 *restrictGroups, int nRestrictGroups) { TupleDesc tupdesc; @@ -951,7 +951,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, PqLeaf *leaves; int nleaves = 0; int totalLeaves = 0; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *values; bool *nulls; uint64 rowNumber; @@ -1010,7 +1010,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, values = palloc(sizeof(Datum) * ntop); nulls = palloc(sizeof(bool) * ntop); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); /* * NULL restrictGroups means no restriction (the whole table). A non-NULL * list restricts to exactly those groups, so an empty list (nRestrictGroups @@ -1019,11 +1019,11 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, * earlier version did -- the entire table. */ if (restrictGroups != NULL) - ColumnarReadRestrictToGroups(readState, restrictGroups, nRestrictGroups); + PgColumnarReadRestrictToGroups(readState, restrictGroups, nRestrictGroups); for (;;) { - bool got = ColumnarReadNextRow(readState, values, nulls, &rowNumber); + bool got = PgColumnarReadNextRow(readState, values, nulls, &rowNumber); if (got) { @@ -1081,7 +1081,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, if (!got) break; } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); /* ---- FileMetaData footer ---- */ { @@ -1094,21 +1094,21 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, nschema += schema_count_for_top(&tops[i]); initStringInfo(&fmd); - ColumnarThriftPutI32Field(&fmd, &last, 1, 1); /* version */ + PgColumnarThriftPutI32Field(&fmd, &last, 1, 1); /* version */ /* schema list (2): root + the (possibly nested) elements per column */ - ColumnarThriftPutField(&fmd, &last, 2, TC_LIST); - ColumnarThriftPutListHeader(&fmd, nschema, TC_STRUCT); + PgColumnarThriftPutField(&fmd, &last, 2, TC_LIST); + PgColumnarThriftPutListHeader(&fmd, nschema, TC_STRUCT); write_schema_element_root(&fmd, ntop); for (i = 0; i < ntop; i++) write_top_schema(&fmd, &tops[i], leaves); - ColumnarThriftPutI64Field(&fmd, &last, 3, total); /* num_rows */ + PgColumnarThriftPutI64Field(&fmd, &last, 3, total); /* num_rows */ /* row_groups list (4) */ - ColumnarThriftPutField(&fmd, &last, 4, TC_LIST); - ColumnarThriftPutListHeader(&fmd, nrgs, TC_STRUCT); + PgColumnarThriftPutField(&fmd, &last, 4, TC_LIST); + PgColumnarThriftPutListHeader(&fmd, nrgs, TC_STRUCT); for (i = 0; i < nrgs; i++) write_row_group(&fmd, leaves, nleaves, &rgs[i]); - ColumnarThriftPutStringField(&fmd, &last, 6, "pgColumnar", 10); /* created_by */ - ColumnarThriftPutStop(&fmd); + PgColumnarThriftPutStringField(&fmd, &last, 6, "pgColumnar", 10); /* created_by */ + PgColumnarThriftPutStop(&fmd); fwrite(fmd.data, 1, fmd.len, f); footerLen = fmd.len; @@ -1126,12 +1126,12 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, } /* - * columnar_export_parquet + * pgcolumnar_export_parquet * SQL: pgcolumnar.export_parquet(rel regclass, path text) -> bigint. - * Thin wrapper over ColumnarWriteParquetFile for the whole table. + * Thin wrapper over PgColumnarWriteParquetFile for the whole table. */ Datum -columnar_export_parquet(PG_FUNCTION_ARGS) +pgcolumnar_export_parquet(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1152,7 +1152,7 @@ columnar_export_parquet(PG_FUNCTION_ARGS) path = text_to_cstring(PG_GETARG_TEXT_PP(1)); rel = table_open(relid, AccessShareLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessShareLock); ereport(ERROR, @@ -1162,7 +1162,7 @@ columnar_export_parquet(PG_FUNCTION_ARGS) } snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - total = ColumnarWriteParquetFile(rel, snapshot, path, NULL, 0); + total = PgColumnarWriteParquetFile(rel, snapshot, path, NULL, 0); table_close(rel, AccessShareLock); PG_RETURN_INT64(total); diff --git a/src/columnar_parquet_codec.c b/src/columnar_parquet_codec.c index 661bc95..9c1046b 100644 --- a/src/columnar_parquet_codec.c +++ b/src/columnar_parquet_codec.c @@ -1,7 +1,7 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_codec.c - * Parquet data-page decompression. See columnar_parquet_codec.h. + * pgcolumnar_parquet_codec.c + * Parquet data-page decompression. See pgcolumnar_parquet_codec.h. * * Written fresh for pgColumnar from the public Apache Parquet specification and * the public Snappy format description. @@ -143,7 +143,7 @@ snappy_raw_uncompress(const uint8 *in, size_t inlen, StringInfo out) * driven to allocate or inflate more than the header declares. */ bool -ColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, +PgColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, StringInfo scratch, const uint8 **out, size_t *outlen) { if (srclen > MaxAllocSize || usize > MaxAllocSize) diff --git a/src/columnar_parquet_codec.h b/src/columnar_parquet_codec.h index a7cd30b..f692e35 100644 --- a/src/columnar_parquet_codec.h +++ b/src/columnar_parquet_codec.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_codec.h + * pgcolumnar_parquet_codec.h * Parquet data-page decompression. * * One entry point that turns a compressed page into bytes, dispatching on the @@ -30,7 +30,7 @@ * false on malformed input, on a codec this build cannot decode, and on any * length that disagrees with the page header. */ -extern bool ColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, +extern bool PgColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, StringInfo scratch, const uint8 **out, size_t *outlen); diff --git a/src/columnar_parquet_format.h b/src/columnar_parquet_format.h index ec96729..fd2a4c7 100644 --- a/src/columnar_parquet_format.h +++ b/src/columnar_parquet_format.h @@ -1,13 +1,13 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_format.h + * pgcolumnar_parquet_format.h * Parquet and Thrift compact-protocol constants, shared by the reader and * the writer. * * These are values defined by the file format, not by this implementation, so * there is exactly one correct value for each and both directions must agree on - * it. They were previously declared twice, once in columnar_parquet.c and once - * in columnar_parquet_reader.c, with 21 names in common. The values did agree, + * it. They were previously declared twice, once in pgcolumnar_parquet.c and once + * in pgcolumnar_parquet_reader.c, with 21 names in common. The values did agree, * but nothing made them: a writer and a reader that disagree on a type code * produce a file that is silently wrong rather than one that fails to parse, * which is the worst shape a defect can take here. diff --git a/src/columnar_parquet_reader.c b/src/columnar_parquet_reader.c index 0051a69..ba56caf 100644 --- a/src/columnar_parquet_reader.c +++ b/src/columnar_parquet_reader.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_reader.c + * pgcolumnar_parquet_reader.c * Parquet file import: pgcolumnar.import_parquet(rel regclass, path text). * * A self-contained Parquet reader with no libparquet/libarrow dependency. It @@ -83,9 +83,9 @@ #include #endif -PG_FUNCTION_INFO_V1(columnar_import_parquet); -PG_FUNCTION_INFO_V1(columnar_parquet_schema); -PG_FUNCTION_INFO_V1(columnar_read_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_import_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_parquet_schema); +PG_FUNCTION_INFO_V1(pgcolumnar_read_parquet); PG_FUNCTION_INFO_V1(pgcolumnar_parquet_fdw_handler); PG_FUNCTION_INFO_V1(pgcolumnar_parquet_fdw_validator); @@ -197,7 +197,7 @@ parse_statistics(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) @@ -205,7 +205,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 1: /* max (deprecated): fallback only */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p && !ch->has_max) { @@ -218,7 +218,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 2: /* min (deprecated): fallback only */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p && !ch->has_min) { @@ -231,7 +231,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 5: /* max_value (preferred) */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) { @@ -244,7 +244,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 6: /* min_value (preferred) */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) { @@ -255,7 +255,7 @@ parse_statistics(TCReader *r, PqChunk *ch) break; } default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -284,34 +284,34 @@ parse_column_meta(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 4: /* codec */ - ch->codec = (int) ColumnarThriftZigzag(r); + ch->codec = (int) PgColumnarThriftZigzag(r); break; case 5: /* num_values */ - ch->num_values = ColumnarThriftZigzag(r); + ch->num_values = PgColumnarThriftZigzag(r); break; case 7: /* total_compressed_size */ - ch->total_compressed_size = ColumnarThriftZigzag(r); + ch->total_compressed_size = PgColumnarThriftZigzag(r); break; case 9: /* data_page_offset */ - ch->data_page_offset = ColumnarThriftZigzag(r); + ch->data_page_offset = PgColumnarThriftZigzag(r); break; case 11: /* dictionary_page_offset */ - ch->dict_page_offset = ColumnarThriftZigzag(r); + ch->dict_page_offset = PgColumnarThriftZigzag(r); break; case 12: /* statistics */ if (ft == TC_STRUCT) parse_statistics(r, ch); else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -328,13 +328,13 @@ parse_column_chunk(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if (fid == 3 && ft == TC_STRUCT) parse_column_meta(r, ch); else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } } @@ -347,7 +347,7 @@ parse_column_chunk(TCReader *r, PqChunk *ch) * This hand-walks nested Thrift unions and assumes the compact-protocol field * ordering the spec prescribes. A writer that reorders or partially populates the * union could desync the cursor -- but a desync fails the surrounding footer - * parse (ColumnarThriftField / bounds checks catch it), so the blast radius is "file + * parse (PgColumnarThriftField / bounds checks catch it), so the blast radius is "file * rejected", never a wrong value silently returned from a good decode. */ static void @@ -360,7 +360,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if ((fid == PQ_LT_TIME || fid == PQ_LT_TIMESTAMP) && ft == TC_STRUCT) @@ -373,7 +373,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int ift, ifid; - ColumnarThriftField(r, &ift, &ifid, &innerLast); + PgColumnarThriftField(r, &ift, &ifid, &innerLast); if (ift == TC_STOP || r->error) break; if (ifid == 2 && ift == TC_STRUCT) /* unit: union TimeUnit */ @@ -382,7 +382,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int uft, ufid; - ColumnarThriftField(r, &uft, &ufid, &uLast); + PgColumnarThriftField(r, &uft, &ufid, &uLast); if (uft != TC_STOP && !r->error) { switch (ufid) @@ -398,17 +398,17 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) break; } sc->is_timestamp = isTs; - ColumnarThriftSkip(r, uft); /* the unit member is an empty struct */ + PgColumnarThriftSkip(r, uft); /* the unit member is an empty struct */ /* consume the union's STOP */ - ColumnarThriftField(r, &uft, &ufid, &uLast); + PgColumnarThriftField(r, &uft, &ufid, &uLast); } } else - ColumnarThriftSkip(r, ift); /* isAdjustedToUTC and anything later */ + PgColumnarThriftSkip(r, ift); /* isAdjustedToUTC and anything later */ } } else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } } @@ -434,47 +434,47 @@ parse_schema_element(TCReader *r, PqSchemaCol *sc) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 1: /* type */ - sc->phys_type = (int) ColumnarThriftZigzag(r); + sc->phys_type = (int) PgColumnarThriftZigzag(r); break; case 2: /* type_length */ - sc->type_length = (int) ColumnarThriftZigzag(r); + sc->type_length = (int) PgColumnarThriftZigzag(r); break; case 3: /* repetition_type */ - sc->repetition = (int) ColumnarThriftZigzag(r); + sc->repetition = (int) PgColumnarThriftZigzag(r); break; case 4: /* name */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) sc->name = pnstrdup((const char *) p, n); break; } case 5: /* num_children */ - num_children = (int) ColumnarThriftZigzag(r); + num_children = (int) PgColumnarThriftZigzag(r); sc->num_children = num_children; break; case 6: /* converted_type */ - sc->converted_type = (int) ColumnarThriftZigzag(r); + sc->converted_type = (int) PgColumnarThriftZigzag(r); break; case 7: /* scale (DECIMAL) */ - sc->scale = (int) ColumnarThriftZigzag(r); + sc->scale = (int) PgColumnarThriftZigzag(r); break; case 8: /* precision (DECIMAL) */ - sc->precision = (int) ColumnarThriftZigzag(r); + sc->precision = (int) PgColumnarThriftZigzag(r); break; case 10: /* logicalType */ parse_logical_type(r, sc); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -525,7 +525,7 @@ walk_schema(PqFile *pf, int *cursor, int def, int rep, * num_children comes straight from the footer, so a schema that chains * group inside group descends here as deep as the file says. This is a * second unbounded recursion, reached only after the footer parses, so the - * guard on ColumnarThriftSkip does not cover it; without one, a crafted + * guard on PgColumnarThriftSkip does not cover it; without one, a crafted * schema SIGSEGVs the backend and restarts the cluster. */ check_stack_depth(); @@ -574,14 +574,14 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) int ft, fid; - ColumnarThriftField(&r, &ft, &fid, &lastId); + PgColumnarThriftField(&r, &ft, &fid, &lastId); if (ft == TC_STOP || r.error) break; if (fid == 2 && ft == TC_LIST) /* schema: list */ { int etype; - uint32 n = ColumnarThriftListHeader(&r, &etype); + uint32 n = PgColumnarThriftListHeader(&r, &etype); uint32 i; PqSchemaCol *tmp = palloc0(sizeof(PqSchemaCol) * Max(n, 1)); @@ -593,7 +593,7 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) else if (fid == 4 && ft == TC_LIST) /* row_groups */ { int etype; - uint32 n = ColumnarThriftListHeader(&r, &etype); + uint32 n = PgColumnarThriftListHeader(&r, &etype); uint32 i; pf->nrowgroups = n; @@ -610,13 +610,13 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) int rft, rfid; - ColumnarThriftField(&r, &rft, &rfid, &rgLast); + PgColumnarThriftField(&r, &rft, &rfid, &rgLast); if (rft == TC_STOP || r.error) break; if (rfid == 1 && rft == TC_LIST) /* columns */ { int cet; - uint32 cn = ColumnarThriftListHeader(&r, &cet); + uint32 cn = PgColumnarThriftListHeader(&r, &cet); uint32 ci; rg->chunks = palloc0(sizeof(PqChunk) * Max(cn, 1)); @@ -625,14 +625,14 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) parse_column_chunk(&r, &rg->chunks[ci]); } else if (rfid == 3) /* num_rows */ - rg->num_rows = ColumnarThriftZigzag(&r); + rg->num_rows = PgColumnarThriftZigzag(&r); else - ColumnarThriftSkip(&r, rft); + PgColumnarThriftSkip(&r, rft); } } } else - ColumnarThriftSkip(&r, ft); + PgColumnarThriftSkip(&r, ft); } if (r.error || pf->nelems < 1) @@ -1285,7 +1285,7 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if (!v2) @@ -1293,13 +1293,13 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) switch (fid) { case 1: - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); break; case 2: - h->encoding = (int) ColumnarThriftZigzag(r); + h->encoding = (int) PgColumnarThriftZigzag(r); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1308,22 +1308,22 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) switch (fid) { case 1: - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); break; case 4: - h->encoding = (int) ColumnarThriftZigzag(r); + h->encoding = (int) PgColumnarThriftZigzag(r); break; case 5: - h->def_levels_len = (int) ColumnarThriftZigzag(r); + h->def_levels_len = (int) PgColumnarThriftZigzag(r); break; case 6: - h->rep_levels_len = (int) ColumnarThriftZigzag(r); + h->rep_levels_len = (int) PgColumnarThriftZigzag(r); break; case 7: h->is_compressed = (ft == TC_BOOL_TRUE); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1343,19 +1343,19 @@ parse_page_header(TCReader *r, PqPageHeader *h) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 1: - h->type = (int) ColumnarThriftZigzag(r); + h->type = (int) PgColumnarThriftZigzag(r); break; case 2: - h->uncompressed_size = (int) ColumnarThriftZigzag(r); + h->uncompressed_size = (int) PgColumnarThriftZigzag(r); break; case 3: - h->compressed_size = (int) ColumnarThriftZigzag(r); + h->compressed_size = (int) PgColumnarThriftZigzag(r); break; case 5: /* DataPageHeader (v1) */ parse_data_page_header(r, h, false); @@ -1369,13 +1369,13 @@ parse_page_header(TCReader *r, PqPageHeader *h) int dft, dfid; - ColumnarThriftField(r, &dft, &dfid, &dl); + PgColumnarThriftField(r, &dft, &dfid, &dl); if (dft == TC_STOP || r->error) break; if (dfid == 1) - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); else - ColumnarThriftSkip(r, dft); + PgColumnarThriftSkip(r, dft); } break; } @@ -1383,7 +1383,7 @@ parse_page_header(TCReader *r, PqPageHeader *h) parse_data_page_header(r, h, true); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1687,7 +1687,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, const uint8 *end; int i; - if (!ColumnarParquetDecompress(ch->codec, praw, h.compressed_size, + if (!PgColumnarParquetDecompress(ch->codec, praw, h.compressed_size, h.uncompressed_size, &dec, &db, &dblen)) return false; dictCount = h.num_values; @@ -1755,7 +1755,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, size_t vusize = (h.uncompressed_size > levLen) ? (size_t) (h.uncompressed_size - levLen) : 0; - if (!ColumnarParquetDecompress(ch->codec, vraw, vrawlen, vusize, + if (!PgColumnarParquetDecompress(ch->codec, vraw, vrawlen, vusize, &dec, &valbuf, &vallen)) return false; } @@ -1771,7 +1771,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, size_t pblen; size_t off = 0; - if (!ColumnarParquetDecompress(ch->codec, praw, h.compressed_size, + if (!PgColumnarParquetDecompress(ch->codec, praw, h.compressed_size, h.uncompressed_size, &dec, &pb, &pblen)) return false; /* v1: repetition levels first, then definition levels, both @@ -2017,7 +2017,7 @@ pq_want_phys_for(Oid typid, const PqSchemaCol *sc) /* * Infer the PostgreSQL column type a Parquet leaf should map to. This is the - * inverse of the exporter's parquet_kind_for_type (columnar_parquet.c): it reads + * inverse of the exporter's parquet_kind_for_type (pgcolumnar_parquet.c): it reads * the physical type plus the ConvertedType annotation, so a round-tripped file * reports the source column types. It is tolerant of files other writers produce * (both the millis and micros time/timestamp variants, INT_8/INT_32 widths) so @@ -2561,7 +2561,7 @@ typedef struct PqInsertSinkArg { Relation rel; CommandId cid; - ColumnarIndexInsertState *indexes; /* NULL when the table has none */ + PgColumnarIndexInsertState *indexes; /* NULL when the table has none */ } PqInsertSinkArg; static void @@ -2579,9 +2579,9 @@ pq_insert_sink(TupleTableSlot *slot, void *arg) * assigned. */ if (a->indexes != NULL) - ColumnarIndexInsertRow(a->indexes, a->rel, slot->tts_values, + PgColumnarIndexInsertRow(a->indexes, a->rel, slot->tts_values, slot->tts_isnull, - ColumnarItemPointerToRowNumber(&slot->tts_tid)); + PgColumnarItemPointerToRowNumber(&slot->tts_tid)); } static void @@ -2885,7 +2885,7 @@ pq_read_file_into(const char *path, TupleDesc tupdesc, TupleTableSlot *slot, * pgcolumnar.import_parquet(rel regclass, path text) -> bigint */ Datum -columnar_import_parquet(PG_FUNCTION_ARGS) +pgcolumnar_import_parquet(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); char *path = text_to_cstring(PG_GETARG_TEXT_PP(1)); @@ -2912,8 +2912,8 @@ columnar_import_parquet(PG_FUNCTION_ARGS) sinkarg.rel = rel; sinkarg.cid = GetCurrentCommandId(true); - sinkarg.indexes = ColumnarRelationHasIndexes(rel) - ? ColumnarIndexInsertBegin(rel, true) : NULL; + sinkarg.indexes = PgColumnarRelationHasIndexes(rel) + ? PgColumnarIndexInsertBegin(rel, true) : NULL; /* insert every resolved file's rows; the per-file decode is bounded by * fileCtx. Each file is bound against the target's descriptor, so a directory * whose files disagree with the table errors rather than importing garbage. */ @@ -2932,7 +2932,7 @@ columnar_import_parquet(PG_FUNCTION_ARGS) MemoryContextDelete(fileCtx); if (sinkarg.indexes != NULL) - ColumnarIndexInsertEnd(sinkarg.indexes); + PgColumnarIndexInsertEnd(sinkarg.indexes); ExecDropSingleTupleTableSlot(slot); /* @@ -2964,7 +2964,7 @@ columnar_import_parquet(PG_FUNCTION_ARGS) * materialize-mode SRF. */ Datum -columnar_read_parquet(PG_FUNCTION_ARGS) +pgcolumnar_read_parquet(PG_FUNCTION_ARGS) { char *path = text_to_cstring(PG_GETARG_TEXT_PP(0)); ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; @@ -3040,7 +3040,7 @@ columnar_read_parquet(PG_FUNCTION_ARGS) * files are reported as their flattened leaf columns. */ Datum -columnar_parquet_schema(PG_FUNCTION_ARGS) +pgcolumnar_parquet_schema(PG_FUNCTION_ARGS) { char *path = text_to_cstring(PG_GETARG_TEXT_PP(0)); ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; @@ -3808,11 +3808,11 @@ pqfdw_compute_skip(ForeignScanState *node, PqFile *pf, * btree family will do: a given operator means the same comparison * in every family that lists it. */ - interp = ColumnarGetOpInterpretation(op->opno); + interp = PgColumnarGetOpInterpretation(op->opno); foreach(ic, interp) { - ColumnarOpInterpretation *o = (ColumnarOpInterpretation *) lfirst(ic); - int s = ColumnarOpInterpStrategy(o); + PgColumnarOpInterpretation *o = (PgColumnarOpInterpretation *) lfirst(ic); + int s = PgColumnarOpInterpStrategy(o); if (s >= BTLessStrategyNumber && s <= BTGreaterStrategyNumber) { diff --git a/src/columnar_projection.c b/src/columnar_projection.c index b60f838..dd22cea 100644 --- a/src/columnar_projection.c +++ b/src/columnar_projection.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_projection.c + * pgcolumnar_projection.c * DDL for multiple physical projections (gap 26, format 2.2). * * A projection is a named, ordered subset of a table's columns stored as its @@ -8,7 +8,7 @@ * identity space (the C-Store model; see design/gaps/26-*). The catalog * (pgcolumnar.projection) and the add/drop DDL are provided here: declaring a * projection allocates its storage id, records the catalog row, and back-fills - * the projection's storage from existing rows (ColumnarBackfillProjection). Read + * the projection's storage from existing rows (PgColumnarBackfillProjection). Read * paths (read_projection, reconstruct_via_projection) are also provided. * * projection_id 0 is the implicit base projection (all live columns, insert @@ -34,9 +34,9 @@ #include "utils/snapmgr.h" #include "utils/tuplestore.h" -PG_FUNCTION_INFO_V1(columnar_add_projection); -PG_FUNCTION_INFO_V1(columnar_drop_projection); -PG_FUNCTION_INFO_V1(columnar_read_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_add_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_drop_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_read_projection); /* * Collect the live (non-dropped) attribute numbers of a relation, in attnum @@ -118,11 +118,11 @@ static void record_base_projection(Relation rel, uint64 storageId, List *existing) { ListCell *lc; - ColumnarProjection base; + PgColumnarProjection base; foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projectionId == 0) return; @@ -136,7 +136,7 @@ record_base_projection(Relation rel, uint64 storageId, List *existing) base.sortKey = NULL; base.sortKeyLen = 0; base.columns = live_attnums(rel, &base.columnsLen); - ColumnarInsertProjectionRow(&base); + PgColumnarInsertProjectionRow(&base); } /* @@ -144,7 +144,7 @@ record_base_projection(Relation rel, uint64 storageId, List *existing) * Declare a projection: a named column subset sorted on sort_key. */ Datum -columnar_add_projection(PG_FUNCTION_ARGS) +pgcolumnar_add_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -154,7 +154,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) uint64 storageId; List *existing; ListCell *lc; - ColumnarProjection proj; + PgColumnarProjection proj; int nextId = 1; int i, j; @@ -169,7 +169,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) colsArr = PG_GETARG_ARRAYTYPE_P(2); sortArr = PG_ARGISNULL(3) ? NULL : PG_GETARG_ARRAYTYPE_P(3); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", @@ -191,13 +191,13 @@ columnar_add_projection(PG_FUNCTION_ARGS) * unaffected. (A CONCURRENTLY variant is future work.) */ rel = table_open(relid, ShareLock); - ColumnarRequireTableOwner(rel); - storageId = ColumnarStorageId(rel); + PgColumnarRequireTableOwner(rel); + storageId = PgColumnarStorageId(rel); - existing = ColumnarListProjections(storageId); + existing = PgColumnarListProjections(storageId); record_base_projection(rel, storageId, existing); /* re-read so the base row is included when picking the next id / name check */ - existing = ColumnarListProjections(storageId); + existing = PgColumnarListProjections(storageId); memset(&proj, 0, sizeof(proj)); proj.storageId = storageId; @@ -232,7 +232,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) /* name must be unique for this table; next id is max + 1 */ foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) ereport(ERROR, @@ -244,11 +244,11 @@ columnar_add_projection(PG_FUNCTION_ARGS) } proj.projectionId = nextId; - proj.projStorageId = ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&proj); + proj.projStorageId = PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&proj); /* populate the projection from the table's existing rows (gap 26 back-fill) */ - ColumnarBackfillProjection(rel, &proj); + PgColumnarBackfillProjection(rel, &proj); /* * Record the declaration behind it, by relation and column name, so a dump @@ -256,7 +256,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) * (#266). Written here rather than in the SQL binding so that a projection * cannot come into existence without one. */ - ColumnarRecordProjectionDeclaration(relid, projname, colsArr, + PgColumnarRecordProjectionDeclaration(relid, projname, colsArr, sortArr ? sortArr : construct_empty_array(TEXTOID)); @@ -269,7 +269,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) * Drop a declared projection. The base projection cannot be dropped. */ Datum -columnar_drop_projection(PG_FUNCTION_ARGS) +pgcolumnar_drop_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -288,20 +288,20 @@ columnar_drop_projection(PG_FUNCTION_ARGS) relid = PG_GETARG_OID(0); projname = text_to_cstring(PG_GETARG_TEXT_PP(1)); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, ShareUpdateExclusiveLock); - ColumnarRequireTableOwner(rel); - storageId = ColumnarStorageId(rel); - existing = ColumnarListProjections(storageId); + PgColumnarRequireTableOwner(rel); + storageId = PgColumnarStorageId(rel); + existing = PgColumnarListProjections(storageId); foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -331,11 +331,11 @@ columnar_drop_projection(PG_FUNCTION_ARGS) * deleted. */ if (targetStorageId != storageId) - ColumnarDeleteMetadata(targetStorageId); - ColumnarDeleteProjectionRow(storageId, targetId); + PgColumnarDeleteMetadata(targetStorageId); + PgColumnarDeleteProjectionRow(storageId, targetId); /* and forget the declaration, so a later rebuild does not resurrect it (#266) */ - ColumnarDeleteProjectionDeclaration(relid, projname); + PgColumnarDeleteProjectionDeclaration(relid, projname); table_close(rel, ShareUpdateExclusiveLock); PG_RETURN_VOID(); @@ -351,7 +351,7 @@ columnar_drop_projection(PG_FUNCTION_ARGS) * flushed first so rows written earlier in this transaction are visible. */ Datum -columnar_read_projection(PG_FUNCTION_ARGS) +pgcolumnar_read_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -360,7 +360,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) uint64 storageId; List *projs; ListCell *lc; - ColumnarProjection *proj = NULL; + PgColumnarProjection *proj = NULL; TupleDesc projTupdesc; TupleDesc retdesc; Tuplestorestate *tupstore; @@ -369,7 +369,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) int ncols; int i; Snapshot snap; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *rvals; bool *rnulls; uint64 projRowNum; @@ -387,20 +387,20 @@ columnar_read_projection(PG_FUNCTION_ARGS) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("set-valued function called in context that cannot accept a set"))); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, AccessShareLock); /* persist pending base + projection writes so this read sees them */ - ColumnarFlushWriteStateForRelation(relid); - storageId = ColumnarStorageId(rel); + PgColumnarFlushWriteStateForRelation(relid); + storageId = PgColumnarStorageId(rel); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -448,10 +448,10 @@ columnar_read_projection(PG_FUNCTION_ARGS) snap = GetActiveSnapshot(); rvals = palloc(sizeof(Datum) * (ncols + 1)); rnulls = palloc(sizeof(bool) * (ncols + 1)); - readState = ColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, + readState = PgColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, projTupdesc, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) + while (PgColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) { uint64 baseRow = (uint64) DatumGetInt64(rvals[0]); StringInfoData buf; @@ -464,7 +464,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) * base row to answer that decoded every column and threw all of it away * (issue #157). */ - if (!ColumnarRowIsLive(rel, snap, baseRow)) + if (!PgColumnarRowIsLive(rel, snap, baseRow)) continue; initStringInfo(&buf); @@ -484,7 +484,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) pfree(buf.data); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); table_close(rel, AccessShareLock); return (Datum) 0; @@ -500,9 +500,9 @@ columnar_read_projection(PG_FUNCTION_ARGS) * does not cover every referenced column. All live table columns are * rendered by their output functions and joined by '|'. */ -PG_FUNCTION_INFO_V1(columnar_reconstruct_via_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_reconstruct_via_projection); Datum -columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) +pgcolumnar_reconstruct_via_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -512,7 +512,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) uint64 storageId; List *projs; ListCell *lc; - ColumnarProjection *proj = NULL; + PgColumnarProjection *proj = NULL; TupleDesc projTupdesc; TupleDesc retdesc; Tuplestorestate *tupstore; @@ -523,7 +523,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) int tnatts; int i; Snapshot snap; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *rvals; bool *rnulls; Datum *basevals; @@ -544,21 +544,21 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("set-valued function called in context that cannot accept a set"))); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, AccessShareLock); - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); tableDesc = RelationGetDescr(rel); tnatts = tableDesc->natts; - storageId = ColumnarStorageId(rel); + storageId = PgColumnarStorageId(rel); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -637,10 +637,10 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) uncovered = bms_add_member(uncovered, i); } - readState = ColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, + readState = PgColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, projTupdesc, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) + while (PgColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) { uint64 baseRow = (uint64) DatumGetInt64(rvals[0]); StringInfoData buf; @@ -649,7 +649,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) bool first = true; /* fetch the base row: liveness, and only the columns not covered */ - if (!ColumnarReadRowByNumberCols(rel, snap, baseRow, basevals, + if (!PgColumnarReadRowByNumberCols(rel, snap, baseRow, basevals, basenulls, uncovered)) continue; @@ -688,7 +688,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) pfree(buf.data); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); table_close(rel, AccessShareLock); return (Datum) 0; diff --git a/src/columnar_reader.c b/src/columnar_reader.c index 87c9dc6..4a1ebe2 100644 --- a/src/columnar_reader.c +++ b/src/columnar_reader.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_reader.c + * pgcolumnar_reader.c * The columnar reader: a sequential scan that reads all columns of all * stripes and reconstructs rows (spec 4, 6). Also holds the value-stream * codec shared with the writer. @@ -49,7 +49,7 @@ typedef struct SkipPredicate Oid hashCollation; } SkipPredicate; -struct ColumnarReadState +struct PgColumnarReadState { Relation rel; Snapshot snapshot; @@ -161,13 +161,13 @@ struct ColumnarReadState MemoryContext skipContext; /* scratch for skip-list evaluation */ }; -static void columnar_build_predicates(ColumnarReadState *readState, +static void pgcolumnar_build_predicates(PgColumnarReadState *readState, int nkeys, ScanKey keys); -static int64 columnar_next_group_index(ColumnarReadState *readState); +static int64 pgcolumnar_next_group_index(PgColumnarReadState *readState); /* qsort comparator for the row group restriction set */ static int -columnar_uint64_cmp(const void *a, const void *b) +pgcolumnar_uint64_cmp(const void *a, const void *b) { uint64 x = *(const uint64 *) a; uint64 y = *(const uint64 *) b; @@ -176,13 +176,13 @@ columnar_uint64_cmp(const void *a, const void *b) } /* - * columnar_group_is_restricted_in + * pgcolumnar_group_is_restricted_in * Is this group number in the read state's restriction set? Binary search - * over the sorted array set by ColumnarReadRestrictToGroups. Only called + * over the sorted array set by PgColumnarReadRestrictToGroups. Only called * when restrictGroups is non-NULL. */ static bool -columnar_group_is_restricted_in(ColumnarReadState *rs, uint64 groupNumber) +pgcolumnar_group_is_restricted_in(PgColumnarReadState *rs, uint64 groupNumber) { int lo = 0; int hi = rs->numRestrictGroups - 1; @@ -206,13 +206,13 @@ columnar_group_is_restricted_in(ColumnarReadState *rs, uint64 groupNumber) * ------------------------------------------------------------------------- */ /* - * ColumnarEncodeValue + * PgColumnarEncodeValue * Append a non-null value to a column's value stream. Fixed-length * values are stored as their raw bytes; varlena values are detoasted * and stored with a full 4-byte header so the reader can size them. */ void -ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) +PgColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) { if (att->attbyval) { @@ -245,13 +245,13 @@ ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) } /* - * ColumnarDecodeValue + * PgColumnarDecodeValue * Read one value from a column's value stream, advancing *cursor. * By-reference values are copied into targetContext so they outlive the * stripe buffer's next reset. */ Datum -ColumnarDecodeValue(Form_pg_attribute att, char **cursor, +PgColumnarDecodeValue(Form_pg_attribute att, char **cursor, MemoryContext targetContext) { char *p = *cursor; @@ -272,7 +272,7 @@ ColumnarDecodeValue(Form_pg_attribute att, char **cursor, } else { - Size len = ColumnarVarSizeAnyUnaligned(p); + Size len = PgColumnarVarSizeAnyUnaligned(p); char *copy = MemoryContextAlloc(targetContext, len); memcpy(copy, p, len); @@ -287,23 +287,23 @@ ColumnarDecodeValue(Form_pg_attribute att, char **cursor, * sequential scan * ------------------------------------------------------------------------- */ -ColumnarReadState * -ColumnarBeginRead(Relation rel, Snapshot snapshot, +PgColumnarReadState * +PgColumnarBeginRead(Relation rel, Snapshot snapshot, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys) { - return ColumnarBeginReadWithStorage(rel, snapshot, ColumnarStorageId(rel), + return PgColumnarBeginReadWithStorage(rel, snapshot, PgColumnarStorageId(rel), RelationGetDescr(rel), parallelScan, projectedColumns, nkeys, keys); } -ColumnarReadState * -ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, +PgColumnarReadState * +PgColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, uint64 storageId, TupleDesc tupdesc, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys) { - ColumnarReadState *readState; + PgColumnarReadState *readState; MemoryContext readContext; MemoryContext oldContext; @@ -312,10 +312,10 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(readContext); - readState = palloc0(sizeof(ColumnarReadState)); + readState = palloc0(sizeof(PgColumnarReadState)); readState->rel = rel; readState->snapshot = snapshot; - readState->metaSnapshot = ColumnarCatalogSnapshot(snapshot); + readState->metaSnapshot = PgColumnarCatalogSnapshot(snapshot); readState->tupdesc = tupdesc; readState->natts = readState->tupdesc->natts; readState->storageId = storageId; @@ -323,10 +323,10 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* * Reject a native data format version this build does not understand before * any bytes are decoded (#240). The metapage version was already checked when - * ColumnarStorageId read the metapage; this is the independent data-format + * PgColumnarStorageId read the metapage; this is the independent data-format * stamp, catching a future encoding change that keeps the metapage layout. */ - ColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); + PgColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); /* * Resolve each column's missing value once, for stripes that predate an @@ -348,13 +348,13 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* * Flatten the projection (#338). A NULL bitmap means "all columns" -- that is - * how columnar_projected_columns reports a whole-row Var, any system column, + * how pgcolumnar_projected_columns reports a whole-row Var, any system column, * and a query referencing no column at all (count(*)), and it is what every * caller that does not compute a projection passes. */ readState->colWanted = palloc(sizeof(bool) * readState->natts); readState->allColumnsWanted = (projectedColumns == NULL || - !columnar_enable_column_projection); + !pgcolumnar_enable_column_projection); { int pc; @@ -380,8 +380,8 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, "columnar read skip", ALLOCSET_DEFAULT_SIZES); - if (columnar_enable_qual_pushdown) - columnar_build_predicates(readState, nkeys, keys); + if (pgcolumnar_enable_qual_pushdown) + pgcolumnar_build_predicates(readState, nkeys, keys); MemoryContextSwitchTo(oldContext); return readState; @@ -389,14 +389,14 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* - * columnar_build_predicates + * pgcolumnar_build_predicates * Translate the scan's ScanKeys into skip predicates for chunk-group * filtering (spec 9). Only simple, same-type btree comparison keys on an * orderable column are used; anything else is ignored, so skipping stays * conservative. Runs in readContext. */ static void -columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) +pgcolumnar_build_predicates(PgColumnarReadState *readState, int nkeys, ScanKey keys) { int i; int n = 0; @@ -448,13 +448,13 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) * For an equality predicate on a hashable column with a safe collation, * enable the bloom-filter probe (I7, gap 25), matching how the filter was * built. The scan key already matches the column collation (a - * differently collated predicate is not pushed; see ColumnarBuildScanKeys), + * differently collated predicate is not pushed; see PgColumnarBuildScanKeys), * so hashing the constant under the column collation is consistent. */ readState->predicates[n].hasHash = false; if (key->sk_strategy == BTEqualStrategyNumber && OidIsValid(tce->hash_proc_finfo.fn_oid) && - ColumnarCollationIsDeterministic(att->attcollation)) + PgColumnarCollationIsDeterministic(att->attcollation)) { readState->predicates[n].hasHash = true; fmgr_info_copy(&readState->predicates[n].hashFn, @@ -468,7 +468,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) } /* - * columnar_group_can_match + * pgcolumnar_group_can_match * Decide whether a chunk group could contain a row satisfying every * pushed-down predicate, using the stored per-chunk min/max (spec 9). A * return of false means the group can be skipped. Missing min/max, or a @@ -476,7 +476,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_setup_group + * pgcolumnar_setup_group * Position on a chunk group: decompress each projected column's value * stream into the group context and point the column cursors at the * decompressed bytes and the (uncompressed) exists bytes. Non-projected @@ -484,7 +484,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_position_group + * pgcolumnar_position_group * Advance from the current groupIndex to the next chunk group that could * match the pushed-down predicates, skipping groups whose min/max rule * them out (spec 9). Returns true when positioned on a readable group, @@ -492,19 +492,19 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_load_stripe + * pgcolumnar_load_stripe * Read a stripe's metadata and data into memory and position at its * first chunk group. */ /* - * columnar_read_start + * pgcolumnar_read_start * Lazily load the stripe list on the first fetch. For a parallel scan a * single worker claims the whole scan and the others see it exhausted, * which is a correct (if not parallel-accelerated) behaviour. */ static void -columnar_read_start(ColumnarReadState *readState) +pgcolumnar_read_start(PgColumnarReadState *readState) { if (readState->started) return; @@ -526,7 +526,7 @@ columnar_read_start(ColumnarReadState *readState) MemoryContext oldContext = MemoryContextSwitchTo(readState->readContext); readState->rowGroupList = - ColumnarReadRowGroupList(readState->storageId, + PgColumnarReadRowGroupList(readState->storageId, readState->metaSnapshot); readState->rowGroupIndex = 0; MemoryContextSwitchTo(oldContext); @@ -534,18 +534,18 @@ columnar_read_start(ColumnarReadState *readState) } /* - * columnar_native_decode_chunk + * pgcolumnar_native_decode_chunk * Reconstruct a native column chunk's raw present-value stream (D4) from its * encoding descriptor. The on-disk values region is the per-1024-value-vector * encoded streams concatenated, optionally block-compressed as a whole. This - * reverses the block codec, then decodes each vector with ColumnarDecodeChunk + * reverses the block codec, then decodes each vector with PgColumnarDecodeChunk * into one raw buffer byte-identical to what the writer buffered, so the * per-row producer walks it exactly as it walks the D2b baseline. Allocated * in the group context. The descriptor lengths are cross-checked so a corrupt * chunk cannot drive a decoder past its buffers. */ static char * -columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, +pgcolumnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, char *values, uint32 valuesLen, const char *desc, uint32 descLen, int blockCodec, uint32 **outVecRawLen, int *outVecCount) @@ -622,7 +622,7 @@ columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, /* reverse the block codec to recover the concatenated encoded region */ if (blockCodec != COLUMNAR_COMPRESSION_NONE) - encRegion = ColumnarDecompressValueStream(values, valuesLen, blockCodec, + encRegion = PgColumnarDecompressValueStream(values, valuesLen, blockCodec, (uint32) encTotal, decodeScratch); else @@ -657,7 +657,7 @@ columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, vecRawLen[v] = rawLen; if (rawLen > 0) { - char *rawVec = ColumnarDecodeChunk(encCursor, encLen, encType, + char *rawVec = PgColumnarDecodeChunk(encCursor, encLen, encType, att, valueCount, rawLen, sharedTable, sharedTableLen, decodeScratch); @@ -700,9 +700,9 @@ native_zone_excludes(SkipPredicate *pred, Form_pg_attribute att, return false; cur = (char *) z->minimum; - minv = ColumnarDecodeValue(att, &cur, cx); + minv = PgColumnarDecodeValue(att, &cur, cx); cur = (char *) z->maximum; - maxv = ColumnarDecodeValue(att, &cur, cx); + maxv = PgColumnarDecodeValue(att, &cur, cx); switch (pred->strategy) { @@ -734,16 +734,16 @@ native_zone_excludes(SkipPredicate *pred, Form_pg_attribute att, } /* - * columnar_native_group_can_match + * pgcolumnar_native_group_can_match * Decide whether a native row group could hold a row satisfying every * pushed-down predicate, using its whole-chunk zone maps (native spec 7.1, * Phase D5b). Returns false when the group can be skipped. Mirrors the 2.2 - * columnar_group_can_match, reading min/max from pgcolumnar.zone_map instead + * pgcolumnar_group_can_match, reading min/max from pgcolumnar.zone_map instead * of the 2.2 chunk catalog. A missing or non-orderable zone map is treated * conservatively as "may match". Runs in rs->skipContext (caller-reset). */ static bool -columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) +pgcolumnar_native_group_can_match(PgColumnarReadState *rs, uint64 groupNumber) { List *zones; NativeZoneMapMetadata **byCol; @@ -755,7 +755,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) if (rs->numPredicates == 0) return true; - zones = ColumnarReadZoneMapList(rs->storageId, groupNumber, rs->metaSnapshot); + zones = PgColumnarReadZoneMapList(rs->storageId, groupNumber, rs->metaSnapshot); byCol = palloc0(sizeof(NativeZoneMapMetadata *) * rs->natts); foreach(lc, zones) { @@ -779,7 +779,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) * columns that min/max cannot. */ if (pred->strategy == BTEqualStrategyNumber && - columnar_enable_bloom_filter && pred->hasHash) + pgcolumnar_enable_bloom_filter && pred->hasHash) { NativeBloomMetadata *b; @@ -811,7 +811,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) if (!bloomLookedUp[pred->attidx]) { byColBloom[pred->attidx] = - ColumnarReadBloomForColumn(rs->storageId, groupNumber, + PgColumnarReadBloomForColumn(rs->storageId, groupNumber, pred->attidx, rs->metaSnapshot); bloomLookedUp[pred->attidx] = true; } @@ -823,7 +823,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) FunctionCall1Coll(&pred->hashFn, pred->hashCollation, pred->compareValue)); - if (!ColumnarBloomProbe(b->filter, b->filterLen, h)) + if (!PgColumnarBloomProbe(b->filter, b->filterLen, h)) return false; } } @@ -833,7 +833,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) } /* - * columnar_native_build_skipvec + * pgcolumnar_native_build_skipvec * Build the per-vector skip flags for a loaded row group (native spec 7.1, * Phase D5b): vector v is skipped when any predicate's per-vector zone map * proves no row in it can match. Also fills rs->nativeVecStart with the @@ -844,7 +844,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) * min/max in rs->skipContext. */ static void -columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vecCount) +pgcolumnar_native_build_skipvec(PgColumnarReadState *rs, uint64 groupNumber, int vecCount) { List *zones; NativeZoneMapMetadata ***byColVec; @@ -863,7 +863,7 @@ columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vec if (rs->numPredicates == 0 || vecCount <= 0) return; - zones = ColumnarReadZoneMapVectors(rs->storageId, groupNumber, rs->metaSnapshot); + zones = PgColumnarReadZoneMapVectors(rs->storageId, groupNumber, rs->metaSnapshot); if (zones == NIL) return; /* legacy: no per-vector zone maps */ @@ -921,21 +921,21 @@ columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vec } /* a half-open span of the row group's bytes, used to build coalesced reads */ -typedef struct ColumnarByteRange +typedef struct PgColumnarByteRange { uint64 start; uint64 end; -} ColumnarByteRange; +} PgColumnarByteRange; /* - * columnar_byte_range_cmp + * pgcolumnar_byte_range_cmp * Order byte ranges by start offset, so adjacent ones can be coalesced. */ static int -columnar_byte_range_cmp(const void *a, const void *b) +pgcolumnar_byte_range_cmp(const void *a, const void *b) { - uint64 sa = ((const ColumnarByteRange *) a)->start; - uint64 sb = ((const ColumnarByteRange *) b)->start; + uint64 sa = ((const PgColumnarByteRange *) a)->start; + uint64 sb = ((const PgColumnarByteRange *) b)->start; if (sa < sb) return -1; @@ -945,21 +945,21 @@ columnar_byte_range_cmp(const void *a, const void *b) } /* - * columnar_native_read_projected + * pgcolumnar_native_read_projected * Read only the byte ranges the projected columns occupy (#338), rather * than the whole row group. * * Ranges that touch or overlap in the file are coalesced, so a projection * covering neighbouring columns costs one read rather than one per column. - * Chunks are written column-major (columnar_write_state.c), so in practice + * Chunks are written column-major (pgcolumnar_write_state.c), so in practice * a projection is a small number of runs. Everything lands at its natural * offset inside the full-size group buffer, leaving the rest untouched. */ static void -columnar_native_read_projected(ColumnarReadState *rs, +pgcolumnar_native_read_projected(PgColumnarReadState *rs, NativeRowGroupMetadata *rg, List *chunks) { - ColumnarByteRange *ranges; + PgColumnarByteRange *ranges; uint64 groupEnd = rg->fileOffset + rg->byteLength; uint64 minStart = groupEnd; uint64 maxEnd = rg->fileOffset; @@ -971,8 +971,8 @@ columnar_native_read_projected(ColumnarReadState *rs, if (chunks == NIL) return; - ranges = (ColumnarByteRange *) - palloc(sizeof(ColumnarByteRange) * list_length(chunks)); + ranges = (PgColumnarByteRange *) + palloc(sizeof(PgColumnarByteRange) * list_length(chunks)); foreach(lc, chunks) { @@ -1052,7 +1052,7 @@ columnar_native_read_projected(ColumnarReadState *rs, if (n == 0) return; - qsort(ranges, n, sizeof(ColumnarByteRange), columnar_byte_range_cmp); + qsort(ranges, n, sizeof(PgColumnarByteRange), pgcolumnar_byte_range_cmp); for (i = 0; i < n;) { @@ -1067,7 +1067,7 @@ columnar_native_read_projected(ColumnarReadState *rs, j++; } - ColumnarReadLogicalData(rs->rel, start, + PgColumnarReadLogicalData(rs->rel, start, rs->nativeBuffer + (start - rg->fileOffset), end - start); i = j; @@ -1077,7 +1077,7 @@ columnar_native_read_projected(ColumnarReadState *rs, } /* - * columnar_native_load_group + * pgcolumnar_native_load_group * Load the next native row group (PGCN v1, Phase D3): read the bytes of the * projected columns into the group context and set each such column's * validity-bitmap pointer and values cursor. Row groups the zone maps prove @@ -1089,7 +1089,7 @@ columnar_native_read_projected(ColumnarReadState *rs, * left unmaterialised and the row loop emits NULL for them. */ static bool -columnar_native_load_group(ColumnarReadState *rs) +pgcolumnar_native_load_group(PgColumnarReadState *rs) { MemoryContext oldContext; NativeRowGroupMetadata *rg; @@ -1102,7 +1102,7 @@ columnar_native_load_group(ColumnarReadState *rs) /* * Claim the next row group and advance past any the zone maps rule out * (native spec 7.1). Under a parallel custom scan each worker claims distinct - * groups from the shared counter (columnar_next_group_index), so a group is + * groups from the shared counter (pgcolumnar_next_group_index), so a group is * read by exactly one backend; serially it walks rowGroupIndex. Without the * counter every worker read every group and a parallel scan returned each row * once per participating backend (D6e). @@ -1110,7 +1110,7 @@ columnar_native_load_group(ColumnarReadState *rs) rg = NULL; for (;;) { - int64 gi = columnar_next_group_index(rs); + int64 gi = pgcolumnar_next_group_index(rs); bool match = true; if (gi < 0) @@ -1118,14 +1118,14 @@ columnar_native_load_group(ColumnarReadState *rs) rg = (NativeRowGroupMetadata *) list_nth(rs->rowGroupList, (int) gi); if (rs->restrictGroups != NULL && - !columnar_group_is_restricted_in(rs, rg->groupNumber)) + !pgcolumnar_group_is_restricted_in(rs, rg->groupNumber)) match = false; else if (rs->numPredicates > 0) { MemoryContext old = MemoryContextSwitchTo(rs->skipContext); MemoryContextReset(rs->skipContext); - match = columnar_native_group_can_match(rs, rg->groupNumber); + match = pgcolumnar_native_group_can_match(rs, rg->groupNumber); MemoryContextSwitchTo(old); } if (match) @@ -1145,7 +1145,7 @@ columnar_native_load_group(ColumnarReadState *rs) * per-column byte ranges the projected read needs. It is a catalog read and * touches none of the group's data pages. */ - chunks = ColumnarReadColumnChunkList(rs->storageId, rg->groupNumber, + chunks = PgColumnarReadColumnChunkList(rs->storageId, rg->groupNumber, rs->metaSnapshot); /* @@ -1159,10 +1159,10 @@ columnar_native_load_group(ColumnarReadState *rs) if (rg->byteLength > 0) { if (rs->allColumnsWanted) - ColumnarReadLogicalData(rs->rel, rg->fileOffset, rs->nativeBuffer, + PgColumnarReadLogicalData(rs->rel, rg->fileOffset, rs->nativeBuffer, rg->byteLength); else - columnar_native_read_projected(rs, rg, chunks); + pgcolumnar_native_read_projected(rs, rg, chunks); } rs->nativeValidity = palloc0(sizeof(char *) * rs->natts); @@ -1211,7 +1211,7 @@ columnar_native_load_group(ColumnarReadState *rs) /* D4: reconstruct the raw present-value stream from the descriptor */ rs->nativeValueCursor[cc->columnIndex] = - columnar_native_decode_chunk(rs->groupContext, att, base + validityBytes, + pgcolumnar_native_decode_chunk(rs->groupContext, att, base + validityBytes, (uint32) (cc->pageLength - validityBytes), cc->encodingDescriptor, cc->encodingDescriptorLen, @@ -1228,7 +1228,7 @@ columnar_native_load_group(ColumnarReadState *rs) * vector boundaries line up); a legacy baseline chunk disables it. */ if (allDescriptor) - columnar_native_build_skipvec(rs, rg->groupNumber, maxVecCount); + pgcolumnar_native_build_skipvec(rs, rg->groupNumber, maxVecCount); else { rs->nativeSkipVec = NULL; @@ -1240,12 +1240,12 @@ columnar_native_load_group(ColumnarReadState *rs) /* * Native delete visibility (D6b): combine this group's row-mask rows (keyed * by group number, one bit per row-in-group) into a single delete mask that - * columnar_native_next_row consults to skip deleted rows. + * pgcolumnar_native_next_row consults to skip deleted rows. */ rs->nativeDeleteMask = NULL; rs->nativeDeleteMaskLen = 0; { - List *maskList = ColumnarReadDeleteVectorList(rs->storageId, + List *maskList = PgColumnarReadDeleteVectorList(rs->storageId, rg->groupNumber, rs->metaSnapshot); ListCell *mlc; @@ -1276,7 +1276,7 @@ columnar_native_load_group(ColumnarReadState *rs) } /* - * columnar_native_skip_current_vector + * pgcolumnar_native_skip_current_vector * Per-vector skipping (native spec 7.1, D5b): when rowInGroup sits at the * start of a vector the zone maps rule out, step each column's value cursor * past that vector's decoded bytes and jump rowInGroup to the next vector, @@ -1284,7 +1284,7 @@ columnar_native_load_group(ColumnarReadState *rs) * caller re-checks bounds), false when the current row must be emitted. */ static bool -columnar_native_skip_current_vector(ColumnarReadState *rs) +pgcolumnar_native_skip_current_vector(PgColumnarReadState *rs) { int v = rs->nativeCurVec; int V = rs->nativeVectorCount; @@ -1309,14 +1309,14 @@ columnar_native_skip_current_vector(ColumnarReadState *rs) } /* - * columnar_native_next_row + * pgcolumnar_native_next_row * Native-format sequential row production (Phase D3). Decodes one row from * the current row group, reconstructing each column from its validity bit * and, when present, the next value on its cursor. Vectors the zone maps rule * out are stepped over without decoding (Phase D5b). */ static bool -columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, +pgcolumnar_native_next_row(PgColumnarReadState *rs, Datum *values, bool *nulls, uint64 *rowNumber) { MemoryContext oldContext; @@ -1341,7 +1341,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, if (rs->nativeGroup == NULL || rs->rowInGroup >= rs->groupRowCount) { - if (!columnar_native_load_group(rs)) + if (!pgcolumnar_native_load_group(rs)) { rs->exhausted = true; return false; @@ -1349,7 +1349,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, } if (rs->nativeSkipVec != NULL && - columnar_native_skip_current_vector(rs)) + pgcolumnar_native_skip_current_vector(rs)) continue; /* stepped past a ruled-out vector; re-check */ /* @@ -1394,7 +1394,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, { /* * Fast path (#289): inline the attbyval decode. This is exactly - * what ColumnarDecodeValue does for a by-value type -- one + * what PgColumnarDecodeValue does for a by-value type -- one * fetch_att and advance attlen -- but skips the out-of-line call * and its own attbyval branch, which is the per-row decode * dispatch #289 profiled as hot. It works for both baseline and @@ -1411,7 +1411,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, rs->nativeValueCursor[c] = p + att->attlen; } else - values[c] = ColumnarDecodeValue(att, + values[c] = PgColumnarDecodeValue(att, &rs->nativeValueCursor[c], rs->rowContext); nulls[c] = false; @@ -1441,37 +1441,37 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, } bool -ColumnarReadNextRow(ColumnarReadState *readState, Datum *values, bool *nulls, +PgColumnarReadNextRow(PgColumnarReadState *readState, Datum *values, bool *nulls, uint64 *rowNumber) { - columnar_read_start(readState); - return columnar_native_next_row(readState, values, nulls, rowNumber); + pgcolumnar_read_start(readState); + return pgcolumnar_native_next_row(readState, values, nulls, rowNumber); } /* ------------------------------------------------------------------------- * Batch-fold accessors (#289) * * These expose the current loaded group's decoded buffer so an ungrouped - * aggregate can fold it column-at-a-time, without columnar_native_next_row + * aggregate can fold it column-at-a-time, without pgcolumnar_native_next_row * producing one Datum tuple per row. Correctness is the caller's: it must walk * each column's validity bitmap to map a row to its packed value (nulls have no * slot), honor the delete mask, and step the per-column present index past a * ruled-out vector. Only fixed-width by-value columns can be read this way; the - * caller checks that from the tuple descriptor before using ColumnarReadFoldColumn. + * caller checks that from the tuple descriptor before using PgColumnarReadFoldColumn. * ------------------------------------------------------------------------- */ /* * Advance to the next row group to fold, loading it (and honoring restrict, - * parallel and zone-map group skipping via columnar_native_load_group). Returns + * parallel and zone-map group skipping via pgcolumnar_native_load_group). Returns * false at end of scan; on true, the accessors below describe the loaded group. */ bool -ColumnarReadFoldNextGroup(ColumnarReadState *readState) +PgColumnarReadFoldNextGroup(PgColumnarReadState *readState) { - columnar_read_start(readState); + pgcolumnar_read_start(readState); if (readState->exhausted) return false; - if (!columnar_native_load_group(readState)) + if (!pgcolumnar_native_load_group(readState)) { readState->exhausted = true; return false; @@ -1487,7 +1487,7 @@ ColumnarReadFoldNextGroup(ColumnarReadState *readState) * the vector count. */ void -ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, +PgColumnarReadFoldGroupInfo(PgColumnarReadState *readState, uint64 *nrows, const char **deleteMask, uint32 *deleteMaskLen, const bool **skipVec, const uint32 **vecStart, int *vectorCount) @@ -1509,7 +1509,7 @@ ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, * COLUMN); the caller then folds it from the missing value or falls back. */ bool -ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, +PgColumnarReadFoldColumn(PgColumnarReadState *readState, int attidx, const char **validity, const char **packed, int16 *attlen, const uint32 **vecRawLen) { @@ -1526,14 +1526,14 @@ ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, } /* - * columnar_next_group_index + * pgcolumnar_next_group_index * The next native row group to scan, or -1 when none remain. The native - * counterpart of columnar_next_stripe_index: a parallel custom scan claims + * counterpart of pgcolumnar_next_stripe_index: a parallel custom scan claims * it from the shared atomic so each worker reads distinct row groups (gap * 23, D6e); a serial scan walks rowGroupIndex. */ static int64 -columnar_next_group_index(ColumnarReadState *readState) +pgcolumnar_next_group_index(PgColumnarReadState *readState) { int ngroups = list_length(readState->rowGroupList); uint32 gi; @@ -1547,26 +1547,26 @@ columnar_next_group_index(ColumnarReadState *readState) } void -ColumnarReadSetParallelCounter(ColumnarReadState *readState, +PgColumnarReadSetParallelCounter(PgColumnarReadState *readState, pg_atomic_uint32 *counter) { readState->parallelCounter = counter; } /* - * ColumnarReadRestrictToGroups + * PgColumnarReadRestrictToGroups * Restrict this scan to the given row group numbers (issue #149). Groups * outside the set are skipped in the claim loop, so their bytes are never * read and their column chunks never decoded. The array is copied into the * read state's own context and sorted there, so the caller may free its own. * - * Must be called before the first ColumnarReadNextRow. Passing ngroups == 0 + * Must be called before the first PgColumnarReadNextRow. Passing ngroups == 0 * makes the scan return no rows, which is the honest reading of "restrict to * nothing" and is what the aggregate path relies on when every group is * clean. */ void -ColumnarReadRestrictToGroups(ColumnarReadState *readState, +PgColumnarReadRestrictToGroups(PgColumnarReadState *readState, const uint64 *groupNumbers, int ngroups) { MemoryContext oldContext; @@ -1582,7 +1582,7 @@ ColumnarReadRestrictToGroups(ColumnarReadState *readState, memcpy(readState->restrictGroups, groupNumbers, sizeof(uint64) * ngroups); qsort(readState->restrictGroups, ngroups, sizeof(uint64), - columnar_uint64_cmp); + pgcolumnar_uint64_cmp); } MemoryContextSwitchTo(oldContext); } @@ -1592,7 +1592,7 @@ ColumnarReadRestrictToGroups(ColumnarReadState *readState, * row number for deletion/visibility. The cache reads the base row-group list * and delete vectors once (at the scan's snapshot) into memory, then answers each * test with a binary search over row groups plus a bitmap probe. Consistent - * with the scan's fixed snapshot, the same way ColumnarBeginRead reads those + * with the scan's fixed snapshot, the same way PgColumnarBeginRead reads those * lists once at begin. * ------------------------------------------------------------------------- */ typedef struct LiveStripeEntry @@ -1605,7 +1605,7 @@ typedef struct LiveStripeEntry uint32 *maskLens; /* [chunkGroupCount] */ } LiveStripeEntry; -struct ColumnarLivenessCache +struct PgColumnarLivenessCache { LiveStripeEntry *stripes; /* sorted ascending by firstRowNumber */ int nstripes; @@ -1625,17 +1625,17 @@ livestripe_cmp(const void *a, const void *b) return 0; } -ColumnarLivenessCache * -ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) +PgColumnarLivenessCache * +PgColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) { - uint64 storageId = ColumnarStorageId(rel); - Snapshot metaSnapshot = ColumnarCatalogSnapshot(snapshot); + uint64 storageId = PgColumnarStorageId(rel); + Snapshot metaSnapshot = PgColumnarCatalogSnapshot(snapshot); MemoryContext ctx = AllocSetContextCreate(CurrentMemoryContext, "columnar liveness cache", ALLOCSET_DEFAULT_SIZES); MemoryContext oldContext = MemoryContextSwitchTo(ctx); - List *rgList = ColumnarReadRowGroupList(storageId, metaSnapshot); - ColumnarLivenessCache *cache = palloc0(sizeof(ColumnarLivenessCache)); + List *rgList = PgColumnarReadRowGroupList(storageId, metaSnapshot); + PgColumnarLivenessCache *cache = palloc0(sizeof(PgColumnarLivenessCache)); ListCell *lc; int i = 0; @@ -1643,7 +1643,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) * Each native row group is one liveness entry with a single whole-group * delete mask (the delete vector is keyed by group number, chunk id 0). Modeling * it as chunkGroupCount 1 with chunkRowCount == rowCount makes the shared - * ColumnarLivenessCacheIsLive map every row to chunk 0. + * PgColumnarLivenessCacheIsLive map every row to chunk 0. */ cache->ctx = ctx; cache->nstripes = list_length(rgList); @@ -1664,7 +1664,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) e->masks = palloc0(sizeof(char *) * 1); e->maskLens = palloc0(sizeof(uint32) * 1); - rml = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); + rml = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); foreach(mc, rml) { DeleteVectorMetadata *rm = (DeleteVectorMetadata *) lfirst(mc); @@ -1691,7 +1691,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) } bool -ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, uint64 rowNumber) +PgColumnarLivenessCacheIsLive(PgColumnarLivenessCache *cache, uint64 rowNumber) { int lo = 0; int hi = cache->nstripes - 1; @@ -1724,14 +1724,14 @@ ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, uint64 rowNumber) } void -ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) +PgColumnarFreeLivenessCache(PgColumnarLivenessCache *cache) { if (cache != NULL) MemoryContextDelete(cache->ctx); } /* - * ColumnarReadRowByNumber + * PgColumnarReadRowByNumber * Fetch a single row addressed by its row number (spec 6). Used by the * table AM's fetch-by-tid callback (UPDATE re-fetches the old row). Reads * only the one chunk group that holds the row and decodes each column up @@ -1742,7 +1742,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) /* ------------------------------------------------------------------------- * Statement-scoped decoded row-group cache (issue #143). * - * ColumnarReadRowByNumber() read and decoded a whole row group to return one + * PgColumnarReadRowByNumber() read and decoded a whole row group to return one * row, so fetching N rows out of one group cost N times the group: measured at * 878 ms for 5,000 rows, 4,452 ms for 10,000 and 19,211 ms for 20,000, all in a * single group. Every index scan, bitmap scan, and index-driven UPDATE or DELETE @@ -1773,7 +1773,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) * decoded column values are held here. * * Entries live in contexts under TopTransactionContext, so an abort or commit - * frees them without a hook; ColumnarDiscardFetchCache() clears the descriptors + * frees them without a hook; PgColumnarDiscardFetchCache() clears the descriptors * to match. * ------------------------------------------------------------------------- */ @@ -1804,7 +1804,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) * crossing the cap costs the overflow fraction rather than everything. */ -typedef struct ColumnarFetchGroup +typedef struct PgColumnarFetchGroup { MemoryContext cx; /* holds every pointer below; NULL when free */ uint64 storageId; @@ -1855,18 +1855,18 @@ typedef struct ColumnarFetchGroup MemoryContext *colCx; /* [natts] */ bool *overflow; /* [natts] */ uint64 lastUsed; -} ColumnarFetchGroup; +} PgColumnarFetchGroup; -static ColumnarFetchGroup columnarFetchCache[COLUMNAR_FETCH_CACHE_ENTRIES]; +static PgColumnarFetchGroup columnarFetchCache[COLUMNAR_FETCH_CACHE_ENTRIES]; static uint64 columnarFetchClock = 0; /* * Memoized native-format-version validation for the by-row-number fetch path - * (#240). columnar_fetch_row runs once per row, so checking the storage's + * (#240). pgcolumnar_fetch_row runs once per row, so checking the storage's * format_version against a catalog row on every call would be a per-row systable * scan on a hot path. The value is immutable for a storage, so one check per * (command, storageId) is enough; this single slot is cleared in - * ColumnarDiscardFetchCache, the same command/transaction boundary the fetch + * PgColumnarDiscardFetchCache, the same command/transaction boundary the fetch * cache resets on. */ static uint64 columnarFetchFmtOkStorageId = 0; @@ -1877,7 +1877,7 @@ static bool columnarFetchFmtOk = false; #define COLUMNAR_RANK_BLOCK_BYTES (COLUMNAR_RANK_BLOCK_ROWS / 8) /* - * columnar_build_rank_prefix + * pgcolumnar_build_rank_prefix * Cumulative count of set validity bits at each 64-row boundary, so the * number of present values before an arbitrary row can be had without * walking to it. Entry b counts the bits below row b * 64; the array has one @@ -1887,7 +1887,7 @@ static bool columnarFetchFmtOk = false; * chunk does not hold. */ static uint32 * -columnar_build_rank_prefix(const char *vbits, uint64 rowCount) +pgcolumnar_build_rank_prefix(const char *vbits, uint64 rowCount) { uint64 nblocks = (rowCount + COLUMNAR_RANK_BLOCK_ROWS - 1) / COLUMNAR_RANK_BLOCK_ROWS; @@ -1921,13 +1921,13 @@ columnar_build_rank_prefix(const char *vbits, uint64 rowCount) } /* - * columnar_rank_before + * pgcolumnar_rank_before * How many values are present in this column before row `row` of the group. * The block prefix plus at most eight byte lookups, in place of a loop over * every earlier row. */ static inline uint64 -columnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) +pgcolumnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) { uint64 blk = row / COLUMNAR_RANK_BLOCK_ROWS; uint64 rank = prefix[blk]; @@ -1944,14 +1944,14 @@ columnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) } /* - * columnar_build_val_offsets + * pgcolumnar_build_val_offsets * Byte offset of every present value in a decoded varying-length stream. * One pass over the values, paid once per column per cached group, in place * of a partial pass on every fetch. Fixed-length columns never call this: * their k-th value is at k * attlen and needs no table. */ static uint32 * -columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) +pgcolumnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) { uint32 *offsets = (uint32 *) palloc(sizeof(uint32) * (nvalues + 1)); char *cursor = rawBuf; @@ -1960,7 +1960,7 @@ columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) for (k = 0; k < nvalues; k++) { offsets[k] = (uint32) (cursor - rawBuf); - cursor += ColumnarVarSizeAnyUnaligned(cursor); + cursor += PgColumnarVarSizeAnyUnaligned(cursor); /* * A chunk holds as many values as chunk_group_row_limit allows, which is @@ -1977,7 +1977,7 @@ columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) /* drop one entry and everything it holds */ static void -columnar_fetch_entry_reset(ColumnarFetchGroup *e) +pgcolumnar_fetch_entry_reset(PgColumnarFetchGroup *e) { if (e->cx != NULL) MemoryContextDelete(e->cx); @@ -1985,13 +1985,13 @@ columnar_fetch_entry_reset(ColumnarFetchGroup *e) } /* - * ColumnarDiscardFetchCache + * PgColumnarDiscardFetchCache * Forget every cached group. The contexts hang off TopTransactionContext * and are already gone by the time this runs at transaction end, so this * only clears the descriptors that pointed at them. */ void -ColumnarDiscardFetchCache(void) +PgColumnarDiscardFetchCache(void) { memset(columnarFetchCache, 0, sizeof(columnarFetchCache)); columnarFetchClock = 0; @@ -2003,18 +2003,18 @@ ColumnarDiscardFetchCache(void) * Returns NULL when nothing should be cached, in which case the caller decodes * into its own scratch context exactly as before. */ -static ColumnarFetchGroup * -columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) +static PgColumnarFetchGroup * +pgcolumnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) { CommandId cid = GetCurrentCommandId(false); - ColumnarFetchGroup *victim = NULL; + PgColumnarFetchGroup *victim = NULL; int i; *hit = false; for (i = 0; i < COLUMNAR_FETCH_CACHE_ENTRIES; i++) { - ColumnarFetchGroup *e = &columnarFetchCache[i]; + PgColumnarFetchGroup *e = &columnarFetchCache[i]; if (e->cx == NULL) { @@ -2025,7 +2025,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) /* an entry from an earlier command can never be used again */ if (e->cid != cid) { - columnar_fetch_entry_reset(e); + pgcolumnar_fetch_entry_reset(e); if (victim == NULL) victim = e; continue; @@ -2049,7 +2049,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) oldest = columnarFetchCache[i].lastUsed; victim = &columnarFetchCache[i]; } - columnar_fetch_entry_reset(victim); + pgcolumnar_fetch_entry_reset(victim); } victim->cx = AllocSetContextCreate(TopTransactionContext, @@ -2063,7 +2063,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) } /* - * columnar_fetch_row + * pgcolumnar_fetch_row * Shared worker behind the three fetch entry points below. * * Which columns to decode is said two ways, and deliberately not one. @@ -2080,11 +2080,11 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) * without touching the group's bytes at all. */ static bool -columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, +pgcolumnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, bool allColumns, Bitmapset *needed, bool wantValues) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); TupleDesc tupdesc = RelationGetDescr(rel); int natts = tupdesc->natts; MemoryContext target = CurrentMemoryContext; @@ -2093,7 +2093,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, Snapshot metaSnapshot; List *rgList; NativeRowGroupMetadata *rg = NULL; - ColumnarFetchGroup *entry; + PgColumnarFetchGroup *entry; bool hit; int validityBytes; uint64 rowInGrp; @@ -2101,9 +2101,9 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, int c; /* - * columnar_fetch_row is called once per item pointer by the executor -- per + * pgcolumnar_fetch_row is called once per item pointer by the executor -- per * row on an index or bitmap scan, and per duplicate by _bt_check_unique() - * while it holds the index page (see columnar_metadata.c). None of those + * while it holds the index page (see pgcolumnar_metadata.c). None of those * callers checks for interrupts between fetches, and each fetch reads the * row-group list out of the catalog, so a statement that fetches many rows * spends its whole time in here. Without a check the loop is uncancellable @@ -2116,7 +2116,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, /* * Reject a native format_version this build does not understand before * decoding any bytes (#240). The scan-open paths check this in - * ColumnarBeginReadWithStorage / ColumnarBeginAggScan, but the by-row-number + * PgColumnarBeginReadWithStorage / PgColumnarBeginAggScan, but the by-row-number * fetch path -- an index scan returning a decoded column, UPDATE re-fetching * the old row, and the vacuum row reader -- reaches neither. Only an actual * decode is guarded (wantValues); a visibility-only probe decodes nothing and @@ -2126,7 +2126,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, if (wantValues && !(columnarFetchFmtOk && columnarFetchFmtOkStorageId == storageId)) { - ColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); + PgColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); columnarFetchFmtOkStorageId = storageId; columnarFetchFmtOk = true; } @@ -2135,7 +2135,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, ALLOCSET_SMALL_SIZES); oldContext = MemoryContextSwitchTo(tmp); - metaSnapshot = ColumnarCatalogSnapshot(snapshot); + metaSnapshot = PgColumnarCatalogSnapshot(snapshot); /* * Native (PGCN v1) fetch-by-row-number: find the row group covering the row @@ -2146,7 +2146,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * The row-group list is read per fetch and deliberately not cached: a group * flushed earlier in this same statement has to become visible here. */ - rgList = ColumnarReadRowGroupList(storageId, metaSnapshot); + rgList = PgColumnarReadRowGroupList(storageId, metaSnapshot); foreach(nlc, rgList) { NativeRowGroupMetadata *g = (NativeRowGroupMetadata *) lfirst(nlc); @@ -2183,11 +2183,11 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, } else { - List *maskList = ColumnarReadDeleteVectorList(storageId, + List *maskList = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); ListCell *mlc; - bool deleted = ColumnarDeleteVectorBufferedDeleted(rel, rowNumber); + bool deleted = PgColumnarDeleteVectorBufferedDeleted(rel, rowNumber); foreach(mlc, maskList) { @@ -2223,7 +2223,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * statement-scoped cache above. A miss fills the entry; a hit skips the read * and the decode entirely. */ - entry = columnar_fetch_group_slot(storageId, rg->groupNumber, &hit); + entry = pgcolumnar_fetch_group_slot(storageId, rg->groupNumber, &hit); /* * The geometry the entry was filled with has to match the group just read @@ -2241,8 +2241,8 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, entry->fileOffset != rg->fileOffset || entry->natts != natts)) { - columnar_fetch_entry_reset(entry); - entry = columnar_fetch_group_slot(storageId, rg->groupNumber, &hit); + pgcolumnar_fetch_entry_reset(entry); + entry = pgcolumnar_fetch_group_slot(storageId, rg->groupNumber, &hit); Assert(!hit); } @@ -2265,10 +2265,10 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, MemoryContextSwitchTo(tmp); if (rg->byteLength > 0) - ColumnarReadLogicalData(rel, rg->fileOffset, entry->groupBuffer, + PgColumnarReadLogicalData(rel, rg->fileOffset, entry->groupBuffer, rg->byteLength); - nchunks = ColumnarReadColumnChunkList(storageId, rg->groupNumber, + nchunks = PgColumnarReadColumnChunkList(storageId, rg->groupNumber, metaSnapshot); foreach(nlc, nchunks) { @@ -2384,7 +2384,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, rawBuf = base + validityBytes; else rawBuf = - columnar_native_decode_chunk(decCx, att, + pgcolumnar_native_decode_chunk(decCx, att, base + validityBytes, (uint32) (cc->pageLength - validityBytes), cc->encodingDescriptor, @@ -2409,7 +2409,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, { MemoryContext idxOld = MemoryContextSwitchTo(entry->cx); - entry->rankPrefix[c] = columnar_build_rank_prefix(vbits, + entry->rankPrefix[c] = pgcolumnar_build_rank_prefix(vbits, entry->rowCount); if (att->attlen < 0) { @@ -2418,7 +2418,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, COLUMNAR_RANK_BLOCK_ROWS; entry->valOffset[c] = - columnar_build_val_offsets(att, rawBuf, + pgcolumnar_build_val_offsets(att, rawBuf, entry->rankPrefix[c][nblocks]); } MemoryContextSwitchTo(idxOld); @@ -2439,14 +2439,14 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * how far into the group the row is, which is what made a cache hit * proportional to the row's position before. */ - present = columnar_rank_before(vbits, entry->rankPrefix[c], rowInGrp); + present = pgcolumnar_rank_before(vbits, entry->rankPrefix[c], rowInGrp); if (att->attlen > 0) cursor = rawBuf + present * (uint64) att->attlen; else cursor = rawBuf + entry->valOffset[c][present]; - values[c] = ColumnarDecodeValue(att, &cursor, target); + values[c] = PgColumnarDecodeValue(att, &cursor, target); nulls[c] = false; /* @@ -2464,7 +2464,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * * It is safe because nothing handed back points into the column: the * value returned was copied into the caller's context by the - * ColumnarDecodeValue call immediately above, and the position indexes + * PgColumnarDecodeValue call immediately above, and the position indexes * live in entry->cx rather than in the column's own context. */ if (justDecoded && entry->colCx[c] != NULL && @@ -2509,7 +2509,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * of keeping them. */ if (rg->byteLength > COLUMNAR_FETCH_CACHE_MAX_BYTES) - columnar_fetch_entry_reset(entry); + pgcolumnar_fetch_entry_reset(entry); MemoryContextSwitchTo(oldContext); MemoryContextDelete(tmp); @@ -2517,24 +2517,24 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, } /* - * ColumnarReadRowByNumber + * PgColumnarReadRowByNumber * Reconstruct every column of the row addressed by a row number. False when * the row is not visible. */ bool -ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, +PgColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls) { - return columnar_fetch_row(rel, snapshot, rowNumber, values, nulls, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, values, nulls, true, NULL, true); } /* - * ColumnarReadRowByNumberCols + * PgColumnarReadRowByNumberCols * Decode exactly the columns in `needed`; every other column reads as null. * An empty or NULL set therefore decodes nothing, which is what it says * rather than a silent "everything" -- for every column, call - * ColumnarReadRowByNumber, which takes no set and cannot be misread. + * PgColumnarReadRowByNumber, which takes no set and cannot be misread. * * Decoding every column whatever the caller wanted is not merely wasted * work on a wide table. The decoded bytes are measured against the fetch @@ -2543,30 +2543,30 @@ ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, * remove (issue #157). */ bool -ColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, +PgColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, Bitmapset *needed) { - return columnar_fetch_row(rel, snapshot, rowNumber, values, nulls, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, values, nulls, false, needed, true); } /* - * ColumnarRowIsLive + * PgColumnarRowIsLive * Is the row visible? Decodes nothing. * - * columnar_index_delete_tuples asks exactly this, once per candidate index + * pgcolumnar_index_delete_tuples asks exactly this, once per candidate index * tuple on a path nbtree drives during deletion, and answered it by * reconstructing every column and freeing the result unread. */ bool -ColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber) +PgColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber) { - return columnar_fetch_row(rel, snapshot, rowNumber, NULL, NULL, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, NULL, NULL, false, NULL, false); } void -ColumnarRescanRead(ColumnarReadState *readState) +PgColumnarRescanRead(PgColumnarReadState *readState) { MemoryContextReset(readState->stripeContext); readState->started = false; @@ -2589,18 +2589,18 @@ ColumnarRescanRead(ColumnarReadState *readState) } void -ColumnarEndRead(ColumnarReadState *readState) +PgColumnarEndRead(PgColumnarReadState *readState) { MemoryContextDelete(readState->readContext); } /* - * ColumnarReadStats + * PgColumnarReadStats * Report how many chunk groups the scan has read versus skipped by the * min/max skip lists (spec 9). Used by the custom scan's EXPLAIN output. */ void -ColumnarReadStats(ColumnarReadState *readState, uint64 *groupsRead, +PgColumnarReadStats(PgColumnarReadState *readState, uint64 *groupsRead, uint64 *groupsSkipped, uint64 *groupsTotal) { *groupsRead = readState->groupsRead; @@ -2609,12 +2609,12 @@ ColumnarReadStats(ColumnarReadState *readState, uint64 *groupsRead, } /* - * ColumnarVectorsSkipped + * PgColumnarVectorsSkipped * How many 1024-value vectors the native scan skipped within read row groups * via per-vector zone maps (native spec 7.1, D5b). Used by EXPLAIN. */ uint64 -ColumnarVectorsSkipped(ColumnarReadState *readState) +PgColumnarVectorsSkipped(PgColumnarReadState *readState) { return readState->vectorsSkipped; } diff --git a/src/columnar_storage.c b/src/columnar_storage.c index 0da5afe..2bce800 100644 --- a/src/columnar_storage.c +++ b/src/columnar_storage.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_storage.c + * pgcolumnar_storage.c * Physical storage layer for pgColumnar: metapage, the logical-to- * physical byte mapping, and the append-only reservation model. * @@ -35,30 +35,30 @@ /* the metapage struct lives right after the page header on block 0 */ #define COLUMNAR_METAPAGE_BLOCKNO 0 #define COLUMNAR_EMPTY_BLOCKNO 1 -#define ColumnarMetapagePointer(page) ((ColumnarMetapage *) PageGetContents(page)) +#define PgColumnarMetapagePointer(page) ((PgColumnarMetapage *) PageGetContents(page)) /* - * Stream/prefetch the block reads in ColumnarReadLogicalData via the read + * Stream/prefetch the block reads in PgColumnarReadLogicalData via the read * stream API (PostgreSQL 17+), which lets the PostgreSQL 18 asynchronous I/O * subsystem read ahead. On by default; off falls back to synchronous ReadBuffer. * On PostgreSQL 16 and earlier the streaming path is compiled out and this GUC * has no effect. */ -bool columnar_enable_read_stream = true; +bool pgcolumnar_enable_read_stream = true; #if PG_VERSION_NUM >= 170000 /* contiguous ascending block range for the read stream callback */ -typedef struct ColumnarBlockRange +typedef struct PgColumnarBlockRange { BlockNumber next; BlockNumber last; -} ColumnarBlockRange; +} PgColumnarBlockRange; static BlockNumber -columnar_read_stream_next(ReadStream *stream, void *private_data, +pgcolumnar_read_stream_next(ReadStream *stream, void *private_data, void *per_buffer_data) { - ColumnarBlockRange *range = (ColumnarBlockRange *) private_data; + PgColumnarBlockRange *range = (PgColumnarBlockRange *) private_data; if (range->next > range->last) return InvalidBlockNumber; @@ -67,7 +67,7 @@ columnar_read_stream_next(ReadStream *stream, void *private_data, #endif /* - * ColumnarWriteNewMetapage + * PgColumnarWriteNewMetapage * Initialize a freshly created relation's storage: block 0 holds the * metapage with the initial reserved values from spec 3, block 1 is * reserved and left empty. Written with a WAL full-page image and an @@ -75,17 +75,17 @@ columnar_read_stream_next(ReadStream *stream, void *private_data, * buffers here. */ void -ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, +PgColumnarWriteNewMetapage(const RelFileLocator *newrlocator, SMgrRelation srel, char persistence, uint64 storageId) { - Page page = ColumnarAllocPage(); - ColumnarMetapage *meta; + Page page = PgColumnarAllocPage(); + PgColumnarMetapage *meta; bool needsWAL = (persistence == RELPERSISTENCE_PERMANENT); /* block 0: metapage */ PageInit(page, BLCKSZ, 0); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->versionMajor = COLUMNAR_VERSION_MAJOR; meta->versionMinor = COLUMNAR_VERSION_MINOR; meta->storageId = storageId; @@ -94,7 +94,7 @@ ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, meta->reservedOffset = COLUMNAR_FIRST_LOGICAL_OFFSET; meta->unloggedReset = false; ((PageHeader) page)->pd_lower = - ((char *) meta - (char *) page) + sizeof(ColumnarMetapage); + ((char *) meta - (char *) page) + sizeof(PgColumnarMetapage); if (needsWAL) log_newpage(&COLUMNAR_SMGR_LOCATOR(srel), MAIN_FORKNUM, @@ -116,11 +116,11 @@ ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, } /* - * ColumnarReadMetapage + * PgColumnarReadMetapage * Read the metapage of an existing relation into *meta. */ void -ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) +PgColumnarReadMetapage(Relation rel, PgColumnarMetapage *meta) { Buffer buffer; Page page; @@ -128,7 +128,7 @@ ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_SHARE); page = BufferGetPage(buffer); - memcpy(meta, ColumnarMetapagePointer(page), sizeof(ColumnarMetapage)); + memcpy(meta, PgColumnarMetapagePointer(page), sizeof(PgColumnarMetapage)); UnlockReleaseBuffer(buffer); if (meta->versionMajor != COLUMNAR_VERSION_MAJOR) @@ -139,16 +139,16 @@ ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) } uint64 -ColumnarStorageId(Relation rel) +PgColumnarStorageId(Relation rel) { - ColumnarMetapage meta; + PgColumnarMetapage meta; - ColumnarReadMetapage(rel, &meta); + PgColumnarReadMetapage(rel, &meta); return meta.storageId; } /* - * ColumnarReserveRowNumbers + * PgColumnarReserveRowNumbers * Reserve a stripe id and a contiguous run of "rowCount" row numbers by * advancing the metapage's reservedStripeId and reservedRowNumber marks * (spec 2.2, 6). Returns the reserved stripe id and the first row number @@ -159,24 +159,24 @@ ColumnarStorageId(Relation rel) * at insert time. That is what lets an index carry a correct TID for a * freshly inserted row (spec 6, 9). The byte offset is reserved * separately, at flush, once the stripe's size is known - * (ColumnarReserveOffset). Row numbers not used by a short stripe are + * (PgColumnarReserveOffset). Row numbers not used by a short stripe are * simply left as a gap; row numbers need only be unique and stable. * * Serialized by the exclusive lock on the metapage buffer; no relation * extension lock is needed here because no data pages are extended. */ void -ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, +PgColumnarReserveRowNumbers(Relation rel, uint64 rowCount, uint64 *stripeId, uint64 *firstRowNumber) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); *stripeId = meta->reservedStripeId; *firstRowNumber = meta->reservedRowNumber; @@ -198,28 +198,28 @@ ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, } /* - * ColumnarReserveOffset + * PgColumnarReserveOffset * Reserve a page-aligned logical byte range of "dataLength" bytes for a * stripe's data and return its file offset (spec 2.1, 2.2). New * reservations start on a fresh page. * * The caller must already hold the relation extension lock and must - * write the reserved data immediately (via ColumnarWriteLogicalData) + * write the reserved data immediately (via PgColumnarWriteLogicalData) * before releasing it, so that reservation is serialized and the P_NEW * extends match the reserved blocks. */ void -ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) +PgColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; uint64 alignedOffset; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); /* align the start of this reservation up to a page boundary */ alignedOffset = ((meta->reservedOffset + COLUMNAR_BYTES_PER_PAGE - 1) / @@ -242,7 +242,7 @@ ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) } /* - * ColumnarAdvanceReservedOffset + * PgColumnarAdvanceReservedOffset * Increase the metapage highwater by addBytes without writing any data, * leaving a gap between the physical EOF and the new highwater. This is a * test hook for the gap-tolerant write path (it deliberately produces the @@ -250,16 +250,16 @@ ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) * Increase-only, so it can never make the highwater overlap live data. */ void -ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) +PgColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->reservedOffset += addBytes; @@ -277,24 +277,24 @@ ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) } /* - * ColumnarDebugSetMetapageVersion + * PgColumnarDebugSetMetapageVersion * Test-only: overwrite the metapage's stored physical format version. Reads - * go through ColumnarReadMetapage, which rejects a version it does not + * go through PgColumnarReadMetapage, which rejects a version it does not * understand; this lets a test plant a bad version and confirm that guard * fires cleanly. Reachable only via the SQL binding the format suite * creates, never from the shipped catalog. */ void -ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor) +PgColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->versionMajor = versionMajor; meta->versionMinor = versionMinor; @@ -313,7 +313,7 @@ ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versio } /* - * ColumnarSetReservedOffset + * PgColumnarSetReservedOffset * Lower the metapage highwater to newOffset (physical end-truncation). The * caller has computed newOffset as the end of all live data and holds an * exclusive lock on the relation, so no reservation can race. Future @@ -321,16 +321,16 @@ ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versio * the (now truncated) EOF. It is an error to raise the highwater here. */ void -ColumnarSetReservedOffset(Relation rel, uint64 newOffset) +PgColumnarSetReservedOffset(Relation rel, uint64 newOffset) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); Assert(newOffset <= meta->reservedOffset); meta->reservedOffset = newOffset; @@ -349,7 +349,7 @@ ColumnarSetReservedOffset(Relation rel, uint64 newOffset) } /* - * ColumnarTruncateMainFork + * PgColumnarTruncateMainFork * Physically truncate the relation's MAIN fork to newnblocks, returning the * trailing blocks to the OS (physical end-truncation). Scoped to the main * fork only: the visibility-map fork is indexed by row-number-derived blocks, @@ -361,7 +361,7 @@ ColumnarSetReservedOffset(Relation rel, uint64 newOffset) * range. */ void -ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) +PgColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) { SMgrRelation srel = RelationGetSmgr(rel); ForkNumber fork = MAIN_FORKNUM; @@ -414,7 +414,7 @@ ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) } /* - * ColumnarWriteLogicalData + * PgColumnarWriteLogicalData * Write a contiguous logical byte range starting at a page-aligned * logical offset, splitting it across the physical pages it maps to * (spec 2.1). Blocks past the current end of the relation are extended; @@ -431,7 +431,7 @@ ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) * relation extension lock, so the P_NEW extensions here are serialized. */ void -ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, +PgColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, char *data, uint64 length) { uint64 L = logicalOffset; @@ -521,12 +521,12 @@ ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, } /* - * ColumnarReadLogicalData + * PgColumnarReadLogicalData * Read a contiguous logical byte range into dest by walking the pages * it maps to (spec 2.1). */ void -ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, +PgColumnarReadLogicalData(Relation rel, uint64 logicalOffset, char *dest, uint64 length) { uint64 L = logicalOffset; @@ -537,7 +537,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, return; #if PG_VERSION_NUM >= 170000 - if (columnar_enable_read_stream) + if (pgcolumnar_enable_read_stream) { /* * The blocks map to a contiguous ascending range, so a read stream can @@ -546,7 +546,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, * so the same L/pageOffset walk drives the copy; the buffers are share- * locked here just as the synchronous path does. */ - ColumnarBlockRange range; + PgColumnarBlockRange range; ReadStream *stream; range.next = (BlockNumber) (L / COLUMNAR_BYTES_PER_PAGE); @@ -554,7 +554,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, stream = read_stream_begin_relation(READ_STREAM_SEQUENTIAL, NULL, rel, MAIN_FORKNUM, - columnar_read_stream_next, &range, 0); + pgcolumnar_read_stream_next, &range, 0); while (remaining > 0) { @@ -615,21 +615,21 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, } /* - * ColumnarResetMetapage + * PgColumnarResetMetapage * Reset the reserved high-water marks to their initial values, keeping * the storage id. Used by non-transactional truncate. */ void -ColumnarResetMetapage(Relation rel) +PgColumnarResetMetapage(Relation rel) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->reservedStripeId = 1; meta->reservedRowNumber = COLUMNAR_FIRST_ROW_NUMBER; @@ -652,7 +652,7 @@ ColumnarResetMetapage(Relation rel) * Row-number <-> item-pointer mapping (spec 6). Row number 0 is invalid. */ void -ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) +PgColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) { BlockNumber blockno; OffsetNumber offset; @@ -665,7 +665,7 @@ ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) } uint64 -ColumnarItemPointerToRowNumber(ItemPointer tid) +PgColumnarItemPointerToRowNumber(ItemPointer tid) { BlockNumber blockno = ItemPointerGetBlockNumber(tid); OffsetNumber offset = ItemPointerGetOffsetNumber(tid); diff --git a/src/columnar_tableam.c b/src/columnar_tableam.c index 08e1f19..259d30a 100644 --- a/src/columnar_tableam.c +++ b/src/columnar_tableam.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_tableam.c + * pgcolumnar_tableam.c * Table access method handler for pgColumnar and extension glue: * GUCs, the pre-commit flush hook, and drop-time metadata cleanup. * @@ -56,19 +56,19 @@ PG_MODULE_MAGIC; /* GUC-backed instance defaults (spec 8.3) */ -int columnar_stripe_row_limit = 150000; -int columnar_chunk_group_row_limit = 10000; -int columnar_encoding_sample_rows = 2048; +int pgcolumnar_stripe_row_limit = 150000; +int pgcolumnar_chunk_group_row_limit = 10000; +int pgcolumnar_encoding_sample_rows = 2048; -int columnar_compression = COLUMNAR_COMPRESSION_ZSTD; -int columnar_compression_level = 3; -int columnar_fsst_min_gain_percent = 5; -bool columnar_enable_qual_pushdown = true; -bool columnar_enable_column_projection = true; -bool columnar_enable_bloom_filter = true; +int pgcolumnar_compression = COLUMNAR_COMPRESSION_ZSTD; +int pgcolumnar_compression_level = 3; +int pgcolumnar_fsst_min_gain_percent = 5; +bool pgcolumnar_enable_qual_pushdown = true; +bool pgcolumnar_enable_column_projection = true; +bool pgcolumnar_enable_bloom_filter = true; /* value set for columnar.compression (spec 5, 8.3) */ -static const struct config_enum_entry columnar_compression_options[] = { +static const struct config_enum_entry pgcolumnar_compression_options[] = { {"none", COLUMNAR_COMPRESSION_NONE, false}, {"pglz", COLUMNAR_COMPRESSION_PGLZ, false}, {"lz4", COLUMNAR_COMPRESSION_LZ4, false}, @@ -77,7 +77,7 @@ static const struct config_enum_entry columnar_compression_options[] = { }; /* forward declaration of the AM routine so hooks can compare against it */ -static const TableAmRoutine columnar_am_methods; +static const TableAmRoutine pgcolumnar_am_methods; static object_access_hook_type prev_object_access_hook = NULL; static ProcessUtility_hook_type prev_process_utility_hook = NULL; @@ -89,7 +89,7 @@ static get_relation_info_hook_type prev_get_relation_info_hook = NULL; #endif /* cached OID of the "columnar" table access method (index-only-scan hook) */ -static Oid columnar_am_oid_cache = InvalidOid; +static Oid pgcolumnar_am_oid_cache = InvalidOid; /* our scan descriptor wraps the base scan and the reader state */ /* @@ -114,7 +114,7 @@ static Oid columnar_am_oid_cache = InvalidOid; * whole of every group core touches, which is what defeats the clustering trap. * A row is offered by exactly one block, so no row can be sampled twice. */ -typedef struct ColumnarAnalyzeState +typedef struct PgColumnarAnalyzeState { List *rowGroups; /* NativeRowGroupMetadata *, in row order */ Snapshot metaSnapshot; @@ -142,18 +142,18 @@ typedef struct ColumnarAnalyzeState * target was lowered, because the work is per row offered rather than per row * kept. */ - ColumnarReadState *rs; /* NULL until the first slice with rows */ + PgColumnarReadState *rs; /* NULL until the first slice with rows */ uint64 rsGroup; /* group number rs is restricted to */ bool rsHavePending; /* pendingRow/values hold an unconsumed row */ uint64 pendingRow; Datum *pendingValues; bool *pendingNulls; -} ColumnarAnalyzeState; +} PgColumnarAnalyzeState; -typedef struct ColumnarScanDescData +typedef struct PgColumnarScanDescData { TableScanDescData rs_base; - ColumnarReadState *readState; + PgColumnarReadState *readState; /* * The context the scan descriptor itself was allocated in. The read state @@ -162,11 +162,11 @@ typedef struct ColumnarScanDescData * instead and outlives the row that triggered it. */ MemoryContext scanContext; - ColumnarAnalyzeState *analyzeState; -} ColumnarScanDescData; -typedef struct ColumnarScanDescData *ColumnarScanDesc; + PgColumnarAnalyzeState *analyzeState; +} PgColumnarScanDescData; +typedef struct PgColumnarScanDescData *PgColumnarScanDesc; -PG_FUNCTION_INFO_V1(columnar_handler); +PG_FUNCTION_INFO_V1(pgcolumnar_handler); /* ------------------------------------------------------------------------- * slot / scan callbacks @@ -204,7 +204,7 @@ PG_FUNCTION_INFO_V1(columnar_handler); * slot_getsomeattrs never calls it. The full suite is run on an assert-enabled * build to keep that reasoning honest. */ -static TupleTableSlotOps ColumnarSlotOps; +static TupleTableSlotOps PgColumnarSlotOps; /* * A slot that can defer its decode (issue #157). @@ -225,7 +225,7 @@ static TupleTableSlotOps ColumnarSlotOps; * ExecStoreVirtualTuple, which sets tts_nvalid to the full count, so getsomeattrs * is never reached for those and their behaviour is unchanged. */ -typedef struct ColumnarSlot +typedef struct PgColumnarSlot { /* * VirtualTupleTableSlot, not TupleTableSlot, and it must come first. Every @@ -242,18 +242,18 @@ typedef struct ColumnarSlot Relation rel; Snapshot snapshot; uint64 rowNumber; -} ColumnarSlot; +} PgColumnarSlot; /* * The fields above must sit past everything the inherited callbacks touch. If * VirtualTupleTableSlot ever grows, this fails to compile rather than silently * aliasing. */ -StaticAssertDecl(offsetof(ColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlot), - "ColumnarSlot fields must not overlap VirtualTupleTableSlot"); +StaticAssertDecl(offsetof(PgColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlot), + "PgColumnarSlot fields must not overlap VirtualTupleTableSlot"); /* - * columnar_slot_decode_upto + * pgcolumnar_slot_decode_upto * Materialise attributes 0 .. natts-1 of a deferred slot. * * Decodes a prefix because that is what slot_getsomeattrs asks for, and the @@ -261,9 +261,9 @@ StaticAssertDecl(offsetof(ColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlo * column 2 of 41 decodes two columns, not forty-one. */ static void -columnar_slot_decode_upto(TupleTableSlot *slot, int natts) +pgcolumnar_slot_decode_upto(TupleTableSlot *slot, int natts) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; Bitmapset *needed = NULL; int i; @@ -279,10 +279,10 @@ columnar_slot_decode_upto(TupleTableSlot *slot, int natts) * reconstructs the whole row, which is correct if not lazy, and it is bounded * by what one transaction has buffered. */ - if (!ColumnarReadRowByNumberCols(cslot->rel, cslot->snapshot, + if (!PgColumnarReadRowByNumberCols(cslot->rel, cslot->snapshot, cslot->rowNumber, slot->tts_values, slot->tts_isnull, needed)) - (void) ColumnarBufferedRowByNumber(cslot->rel, cslot->rowNumber, + (void) PgColumnarBufferedRowByNumber(cslot->rel, cslot->rowNumber, slot->tts_values, slot->tts_isnull); bms_free(needed); @@ -297,9 +297,9 @@ columnar_slot_decode_upto(TupleTableSlot *slot, int natts) } static void -columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) +pgcolumnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; if (!cslot->deferred) { @@ -311,7 +311,7 @@ columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) elog(ERROR, "getsomeattrs on a columnar slot that was filled eagerly"); } - columnar_slot_decode_upto(slot, natts); + pgcolumnar_slot_decode_upto(slot, natts); } /* @@ -319,24 +319,24 @@ columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) * -- has to finish the decode first. */ static void -columnar_slot_force_full(TupleTableSlot *slot) +pgcolumnar_slot_force_full(TupleTableSlot *slot) { - ColumnarSlot *cslot; + PgColumnarSlot *cslot; /* * Callers hand us slots that are not ours. copyslot in particular takes a * source of any type -- the executor copies an ordinary virtual slot into a - * columnar one on every INSERT -- and casting that to ColumnarSlot reads + * columnar one on every INSERT -- and casting that to PgColumnarSlot reads * past the end of it, so the deferred flag is whatever happened to be in * the next word and the relation pointer behind it is garbage. That is a * segfault on the plainest INSERT there is, which is how it was found. */ - if (slot->tts_ops != &ColumnarSlotOps) + if (slot->tts_ops != &PgColumnarSlotOps) return; - cslot = (ColumnarSlot *) slot; + cslot = (PgColumnarSlot *) slot; if (cslot->deferred && slot->tts_nvalid < slot->tts_tupleDescriptor->natts) - columnar_slot_decode_upto(slot, slot->tts_tupleDescriptor->natts); + pgcolumnar_slot_decode_upto(slot, slot->tts_tupleDescriptor->natts); } /* @@ -345,9 +345,9 @@ columnar_slot_force_full(TupleTableSlot *slot) * are there. */ static void -columnar_slot_init(TupleTableSlot *slot) +pgcolumnar_slot_init(TupleTableSlot *slot) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; TTSOpsVirtual.init(slot); cslot->deferred = false; @@ -357,11 +357,11 @@ columnar_slot_init(TupleTableSlot *slot) } static void -columnar_slot_clear(TupleTableSlot *slot) +pgcolumnar_slot_clear(TupleTableSlot *slot) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; - Assert(slot->tts_ops == &ColumnarSlotOps); + Assert(slot->tts_ops == &PgColumnarSlotOps); cslot->deferred = false; cslot->rel = NULL; cslot->snapshot = NULL; @@ -370,37 +370,37 @@ columnar_slot_clear(TupleTableSlot *slot) } static void -columnar_slot_materialize(TupleTableSlot *slot) +pgcolumnar_slot_materialize(TupleTableSlot *slot) { - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); TTSOpsVirtual.materialize(slot); } static void -columnar_slot_copyslot(TupleTableSlot *dstslot, TupleTableSlot *srcslot) +pgcolumnar_slot_copyslot(TupleTableSlot *dstslot, TupleTableSlot *srcslot) { - columnar_slot_force_full(srcslot); + pgcolumnar_slot_force_full(srcslot); TTSOpsVirtual.copyslot(dstslot, srcslot); } static MinimalTuple -columnar_slot_copy_minimal_tuple(COLUMNAR_COPY_MINIMAL_TUPLE_ARGS) +pgcolumnar_slot_copy_minimal_tuple(COLUMNAR_COPY_MINIMAL_TUPLE_ARGS) { - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); return TTSOpsVirtual.copy_minimal_tuple COLUMNAR_COPY_MINIMAL_TUPLE_FWD(slot); } /* - * ColumnarSlotStoreDeferred + * PgColumnarSlotStoreDeferred * Point the slot at a row without decoding it. The caller has already * established that the row is visible. */ static void -ColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, +PgColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, Snapshot snapshot, uint64 rowNumber) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; ExecClearTuple(slot); cslot->deferred = true; @@ -413,13 +413,13 @@ ColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, } static HeapTuple -columnar_slot_copy_heap_tuple(TupleTableSlot *slot) +pgcolumnar_slot_copy_heap_tuple(TupleTableSlot *slot) { HeapTuple tuple; Assert(!TTS_EMPTY(slot)); - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); tuple = heap_form_tuple(slot->tts_tupleDescriptor, slot->tts_values, slot->tts_isnull); @@ -430,30 +430,30 @@ columnar_slot_copy_heap_tuple(TupleTableSlot *slot) } static const TupleTableSlotOps * -columnar_slot_callbacks(Relation relation) +pgcolumnar_slot_callbacks(Relation relation) { - return &ColumnarSlotOps; + return &PgColumnarSlotOps; } static TableScanDesc -columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, +pgcolumnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, ScanKey key, ParallelTableScanDesc pscan, uint32 flags) { - ColumnarScanDesc scan; + PgColumnarScanDesc scan; RelationIncrementReferenceCount(rel); /* * Persist any data and delete marks written earlier in this transaction so * they reach the catalog before this scan reads it. The reader consults the - * catalog with a command-id-advanced snapshot (ColumnarCatalogSnapshot), so + * catalog with a command-id-advanced snapshot (PgColumnarCatalogSnapshot), so * these become visible to this same scan: same-transaction read-your-writes * (spec 9). */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); - scan = (ColumnarScanDesc) palloc0(sizeof(ColumnarScanDescData)); + scan = (PgColumnarScanDesc) palloc0(sizeof(PgColumnarScanDescData)); scan->rs_base.rs_rd = rel; scan->rs_base.rs_snapshot = snapshot; scan->rs_base.rs_nkeys = nkeys; @@ -488,7 +488,7 @@ columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, } /* - * columnar_scan_read_state + * pgcolumnar_scan_read_state * The scan's reader, built on first use against the descriptor the caller * is asking for. * @@ -500,19 +500,19 @@ columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, * * The read state is allocated in the context the scan descriptor itself lives * in. The current context on first use is usually a per-tuple one that is reset - * before the scan ends, and ColumnarEndRead then frees an already-freed pointer. + * before the scan ends, and PgColumnarEndRead then frees an already-freed pointer. */ -static ColumnarReadState * -columnar_scan_read_state(ColumnarScanDesc scan, TupleDesc tupdesc) +static PgColumnarReadState * +pgcolumnar_scan_read_state(PgColumnarScanDesc scan, TupleDesc tupdesc) { if (scan->readState == NULL) { MemoryContext oldContext = MemoryContextSwitchTo(scan->scanContext); scan->readState = - ColumnarBeginReadWithStorage(scan->rs_base.rs_rd, + PgColumnarBeginReadWithStorage(scan->rs_base.rs_rd, scan->rs_base.rs_snapshot, - ColumnarStorageId(scan->rs_base.rs_rd), + PgColumnarStorageId(scan->rs_base.rs_rd), tupdesc, scan->rs_base.rs_parallel, NULL, scan->rs_base.rs_nkeys, @@ -524,17 +524,17 @@ columnar_scan_read_state(ColumnarScanDesc scan, TupleDesc tupdesc) } static void -columnar_scan_end(TableScanDesc sscan) +pgcolumnar_scan_end(TableScanDesc sscan) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; if (scan->readState != NULL) - ColumnarEndRead(scan->readState); + PgColumnarEndRead(scan->readState); if (scan->analyzeState != NULL) { if (scan->analyzeState->rs != NULL) - ColumnarEndRead(scan->analyzeState->rs); + PgColumnarEndRead(scan->analyzeState->rs); MemoryContextDelete(scan->analyzeState->cx); } @@ -547,48 +547,48 @@ columnar_scan_end(TableScanDesc sscan) } static void -columnar_scan_rescan(TableScanDesc sscan, ScanKey key, bool set_params, +pgcolumnar_scan_rescan(TableScanDesc sscan, ScanKey key, bool set_params, bool allow_strat, bool allow_sync, bool allow_pagemode) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; if (scan->readState != NULL) - ColumnarRescanRead(scan->readState); + PgColumnarRescanRead(scan->readState); } static bool -columnar_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, +pgcolumnar_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableSlot *slot) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; uint64 rowNumber; ExecClearTuple(slot); - if (!ColumnarReadNextRow(columnar_scan_read_state(scan, + if (!PgColumnarReadNextRow(pgcolumnar_scan_read_state(scan, slot->tts_tupleDescriptor), slot->tts_values, slot->tts_isnull, &rowNumber)) return false; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(scan->rs_base.rs_rd); return true; } /* ------------------------------------------------------------------------- - * parallel scan: single-worker claim (see columnar_reader.c) + * parallel scan: single-worker claim (see pgcolumnar_reader.c) * ------------------------------------------------------------------------- */ static Size -columnar_parallelscan_estimate(Relation rel) +pgcolumnar_parallelscan_estimate(Relation rel) { return sizeof(ParallelBlockTableScanDescData); } static Size -columnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) +pgcolumnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) { ParallelBlockTableScanDesc bpscan = (ParallelBlockTableScanDesc) pscan; @@ -603,7 +603,7 @@ columnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) } static void -columnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) +pgcolumnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) { ParallelBlockTableScanDesc bpscan = (ParallelBlockTableScanDesc) pscan; @@ -615,11 +615,11 @@ columnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) * ------------------------------------------------------------------------- */ static void -columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, +pgcolumnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate) { - ColumnarWriteState *writeState = ColumnarGetWriteState(rel); + PgColumnarWriteState *writeState = PgColumnarGetWriteState(rel); uint64 rowNumber; slot_getallattrs(slot); @@ -629,13 +629,13 @@ columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, * the executor runs its btree uniqueness check on this row, so the check * runs only after any conflicting transaction has committed and flushed. */ - ColumnarLockUniqueKeys(rel, slot); + PgColumnarLockUniqueKeys(rel, slot); - rowNumber = ColumnarWriteRow(writeState, rel, slot->tts_values, + rowNumber = PgColumnarWriteRow(writeState, rel, slot->tts_values, slot->tts_isnull); /* fan the row out to every additional projection of this table (gap 26) */ - ColumnarProjectionFanoutRow(rel, writeState, rowNumber, slot->tts_values, + PgColumnarProjectionFanoutRow(rel, writeState, rowNumber, slot->tts_values, slot->tts_isnull); /* @@ -643,23 +643,23 @@ columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, * index-only scan never skips the fetch for a block that just changed * (gap 28). A no-op unless a prior vacuum had marked the block visible. */ - ColumnarVMClearForRow(rel, rowNumber); + PgColumnarVMClearForRow(rel, rowNumber); /* * Publish the row's synthetic item pointer (spec 6) so the executor can * insert correct (index value, TID) entries into any indexes on this * relation and enforce unique constraints (spec 9). */ - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); } static void -columnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, +pgcolumnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate) { - ColumnarWriteState *writeState = ColumnarGetWriteState(rel); + PgColumnarWriteState *writeState = PgColumnarGetWriteState(rel); int i; for (i = 0; i < nslots; i++) @@ -667,27 +667,27 @@ columnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, uint64 rowNumber; slot_getallattrs(slots[i]); - ColumnarLockUniqueKeys(rel, slots[i]); /* issue #5 */ - rowNumber = ColumnarWriteRow(writeState, rel, slots[i]->tts_values, + PgColumnarLockUniqueKeys(rel, slots[i]); /* issue #5 */ + rowNumber = PgColumnarWriteRow(writeState, rel, slots[i]->tts_values, slots[i]->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, rowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, rowNumber, slots[i]->tts_values, slots[i]->tts_isnull); - ColumnarVMClearForRow(rel, rowNumber); /* gap 28: block changed */ - ColumnarRowNumberToItemPointer(rowNumber, &slots[i]->tts_tid); + PgColumnarVMClearForRow(rel, rowNumber); /* gap 28: block changed */ + PgColumnarRowNumberToItemPointer(rowNumber, &slots[i]->tts_tid); slots[i]->tts_tableOid = RelationGetRelid(rel); } } static void -columnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) +pgcolumnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) { /* * End of a bulk-load path (COPY, CREATE TABLE AS, ALTER TABLE rewrite). * Flush now, under this operation's subtransaction, so the buffer never * spans a later statement or savepoint boundary (spec 9). */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); } /* ------------------------------------------------------------------------- @@ -695,7 +695,7 @@ columnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) * ------------------------------------------------------------------------- */ static void -columnar_relation_set_new_filelocator(Relation rel, +pgcolumnar_relation_set_new_filelocator(Relation rel, const RelFileLocator *newrlocator, char persistence, TransactionId *freezeXid, @@ -712,19 +712,19 @@ columnar_relation_set_new_filelocator(Relation rel, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("unlogged columnar tables are not supported"))); - srel = ColumnarRelationCreateStorage(*newrlocator, persistence); - storageId = ColumnarNextStorageId(); - ColumnarWriteNewMetapage(newrlocator, srel, persistence, storageId); + srel = PgColumnarRelationCreateStorage(*newrlocator, persistence); + storageId = PgColumnarNextStorageId(); + PgColumnarWriteNewMetapage(newrlocator, srel, persistence, storageId); } static void -columnar_relation_nontransactional_truncate(Relation rel) +pgcolumnar_relation_nontransactional_truncate(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); - ColumnarDeleteMetadata(storageId); + PgColumnarDeleteMetadata(storageId); RelationTruncate(rel, 2); - ColumnarResetMetapage(rel); + PgColumnarResetMetapage(rel); } /* ------------------------------------------------------------------------- @@ -732,7 +732,7 @@ columnar_relation_nontransactional_truncate(Relation rel) * ------------------------------------------------------------------------- */ static uint64 -columnar_relation_size(Relation rel, ForkNumber forkNumber) +pgcolumnar_relation_size(Relation rel, ForkNumber forkNumber) { SMgrRelation srel = RelationGetSmgr(rel); @@ -748,19 +748,19 @@ columnar_relation_size(Relation rel, ForkNumber forkNumber) } static bool -columnar_relation_needs_toast_table(Relation rel) +pgcolumnar_relation_needs_toast_table(Relation rel) { /* the writer detoasts and stores values inline in the value stream */ return false; } static void -columnar_relation_estimate_size(Relation rel, int32 *attr_widths, +pgcolumnar_relation_estimate_size(Relation rel, int32 *attr_widths, BlockNumber *pages, double *tuples, double *allvisfrac) { BlockNumber nblocks = RelationGetNumberOfBlocks(rel); - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Snapshot snapshot; List *rowGroupList; ListCell *lc; @@ -773,7 +773,7 @@ columnar_relation_estimate_size(Relation rel, int32 *attr_widths, * planner from mis-costing scans (spec 6, 9). */ snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - rowGroupList = ColumnarReadRowGroupList(storageId, ColumnarCatalogSnapshot(snapshot)); + rowGroupList = PgColumnarReadRowGroupList(storageId, PgColumnarCatalogSnapshot(snapshot)); foreach(lc, rowGroupList) liveRows += (double) ((NativeRowGroupMetadata *) lfirst(lc))->rowCount; @@ -784,28 +784,28 @@ columnar_relation_estimate_size(Relation rel, int32 *attr_widths, } /* - * columnar_analyze_state + * pgcolumnar_analyze_state * The sampling state for this scan, built on first use. The row group list * is read once: ANALYZE holds ShareUpdateExclusiveLock, so no group is added * or retired under us, and re-reading it per block would put a catalog scan * in the middle of the sample loop. */ -static ColumnarAnalyzeState * -columnar_analyze_state(ColumnarScanDesc scan) +static PgColumnarAnalyzeState * +pgcolumnar_analyze_state(PgColumnarScanDesc scan) { - ColumnarAnalyzeState *st = scan->analyzeState; + PgColumnarAnalyzeState *st = scan->analyzeState; Relation rel = scan->rs_base.rs_rd; MemoryContext oldContext; if (st != NULL) return st; - st = palloc0(sizeof(ColumnarAnalyzeState)); + st = palloc0(sizeof(PgColumnarAnalyzeState)); st->cx = AllocSetContextCreate(CurrentMemoryContext, "columnar analyze", ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(st->cx); - st->metaSnapshot = ColumnarCatalogSnapshot(scan->rs_base.rs_snapshot); - st->rowGroups = ColumnarReadRowGroupList(ColumnarStorageId(rel), + st->metaSnapshot = PgColumnarCatalogSnapshot(scan->rs_base.rs_snapshot); + st->rowGroups = PgColumnarReadRowGroupList(PgColumnarStorageId(rel), st->metaSnapshot); st->values = palloc(sizeof(Datum) * RelationGetDescr(rel)->natts); st->nulls = palloc(sizeof(bool) * RelationGetDescr(rel)->natts); @@ -818,7 +818,7 @@ columnar_analyze_state(ColumnarScanDesc scan) } /* - * columnar_analyze_set_slice + * pgcolumnar_analyze_set_slice * Point the sampler at the rows a physical block stands for. * * The block's logical byte offset locates the row group it falls in; its @@ -830,7 +830,7 @@ columnar_analyze_state(ColumnarScanDesc scan) * as visited and offers nothing. */ static void -columnar_analyze_set_slice(ColumnarAnalyzeState *st, BlockNumber blockno) +pgcolumnar_analyze_set_slice(PgColumnarAnalyzeState *st, BlockNumber blockno) { uint64 logicalOffset; ListCell *lc; @@ -903,17 +903,17 @@ columnar_analyze_set_slice(ColumnarAnalyzeState *st, BlockNumber blockno) /* * The block comes from a read stream from PG17 and as a plain BlockNumber - * before that. columnar_compat.h supplies the parameter list and splits at the + * before that. pgcolumnar_compat.h supplies the parameter list and splits at the * same major; these two must agree, and when they did not, PG17 took the * pre-17 branch and failed to compile on a `blockno` its signature does not * have. */ #if PG_VERSION_NUM >= 170000 static bool -columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) +pgcolumnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; - ColumnarAnalyzeState *st = columnar_analyze_state(cscan); + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; + PgColumnarAnalyzeState *st = pgcolumnar_analyze_state(cscan); Buffer buf = read_stream_next_buffer(stream, NULL); if (!BufferIsValid(buf)) @@ -925,26 +925,26 @@ columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) * for are read through the fetch path instead. Release the pin at once rather * than holding it across the tuple loop as heap does. */ - columnar_analyze_set_slice(st, BufferGetBlockNumber(buf)); + pgcolumnar_analyze_set_slice(st, BufferGetBlockNumber(buf)); ReleaseBuffer(buf); return true; } #else static bool -columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) +pgcolumnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; - columnar_analyze_set_slice(columnar_analyze_state(cscan), blockno); + pgcolumnar_analyze_set_slice(pgcolumnar_analyze_state(cscan), blockno); return true; } #endif static bool -columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) +pgcolumnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; - ColumnarAnalyzeState *st = columnar_analyze_state(cscan); + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; + PgColumnarAnalyzeState *st = pgcolumnar_analyze_state(cscan); Relation rel = scan->rs_rd; TupleDesc tupdesc = RelationGetDescr(rel); uint64 sliceEnd = st->sliceFirstRow + st->sliceRows; @@ -959,9 +959,9 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) MemoryContext oldContext = MemoryContextSwitchTo(st->cx); if (st->rs != NULL) - ColumnarEndRead(st->rs); - st->rs = ColumnarBeginRead(rel, scan->rs_snapshot, NULL, NULL, 0, NULL); - ColumnarReadRestrictToGroups(st->rs, &st->sliceGroup, 1); + PgColumnarEndRead(st->rs); + st->rs = PgColumnarBeginRead(rel, scan->rs_snapshot, NULL, NULL, 0, NULL); + PgColumnarReadRestrictToGroups(st->rs, &st->sliceGroup, 1); st->rsGroup = st->sliceGroup; st->rsHavePending = false; MemoryContextSwitchTo(oldContext); @@ -981,7 +981,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) sizeof(Datum) * tupdesc->natts); memcpy(st->nulls, st->pendingNulls, sizeof(bool) * tupdesc->natts); } - else if (!ColumnarReadNextRow(st->rs, st->values, st->nulls, &rowNumber)) + else if (!PgColumnarReadNextRow(st->rs, st->values, st->nulls, &rowNumber)) { /* * The group is exhausted. Any rows of this slice not returned were @@ -1034,7 +1034,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) * is monotonic, so this is what makes the sorted order the physical order * -- which in turn is what makes the correlation statistic mean anything. */ - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); *liverows += 1; return true; @@ -1044,7 +1044,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) /* VACUUM: mark all-visible groups in the VM fork and retire fully-deleted * groups online, both under ShareUpdateExclusiveLock */ static void -columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, +pgcolumnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, BufferAccessStrategy bstrategy) { /* @@ -1055,7 +1055,7 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, * concurrent with readers and writers. The space-reclaiming rewrite stays in * columnar.vacuum (AccessExclusiveLock, the VACUUM-FULL analog). */ - ColumnarVMSetVisibleForRelation(rel); + PgColumnarVMSetVisibleForRelation(rel); /* * Online compaction (Phase F3a): retire row groups that are fully deleted @@ -1065,7 +1065,7 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, * so a plain VACUUM / autovacuum reclaims fully-deleted groups online without * the AccessExclusiveLock rewrite. */ - ColumnarRetireFullyDeletedGroups(rel); + PgColumnarRetireFullyDeletedGroups(rel); } /* ------------------------------------------------------------------------- @@ -1079,33 +1079,33 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, feature))) /* our index-fetch descriptor is just the base plus nothing extra */ -typedef struct ColumnarIndexFetchData +typedef struct PgColumnarIndexFetchData { IndexFetchTableData xs_base; -} ColumnarIndexFetchData; +} PgColumnarIndexFetchData; static struct IndexFetchTableData * -columnar_index_fetch_begin(COLUMNAR_INDEX_FETCH_BEGIN_ARGS) +pgcolumnar_index_fetch_begin(COLUMNAR_INDEX_FETCH_BEGIN_ARGS) { - ColumnarIndexFetchData *scan = palloc0(sizeof(ColumnarIndexFetchData)); + PgColumnarIndexFetchData *scan = palloc0(sizeof(PgColumnarIndexFetchData)); scan->xs_base.rel = rel; return &scan->xs_base; } static void -columnar_index_fetch_reset(struct IndexFetchTableData *scan) +pgcolumnar_index_fetch_reset(struct IndexFetchTableData *scan) { } static void -columnar_index_fetch_end(struct IndexFetchTableData *scan) +pgcolumnar_index_fetch_end(struct IndexFetchTableData *scan) { pfree(scan); } /* - * columnar_index_fetch_tuple + * pgcolumnar_index_fetch_tuple * Fetch the columnar row addressed by an index item pointer (spec 6) into * the slot. Returns false when the row is marked deleted in the delete vector * or does not exist, so an index scan never returns a deleted row and a @@ -1120,12 +1120,12 @@ columnar_index_fetch_end(struct IndexFetchTableData *scan) * check path). */ static bool -columnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, +pgcolumnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot, bool *call_again, bool *all_dead) { Relation rel = scan->rel; - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); /* columnar rows are 1:1 with item pointers: no chain, never dead here */ *call_again = false; @@ -1161,31 +1161,31 @@ columnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, * transaction's write buffer, and that reader reconstructs whole rows, so it * is stored eagerly. */ - if (ColumnarRowIsLive(rel, snapshot, rowNumber)) - ColumnarSlotStoreDeferred(slot, rel, snapshot, rowNumber); - else if (ColumnarBufferedRowByNumber(rel, rowNumber, + if (PgColumnarRowIsLive(rel, snapshot, rowNumber)) + PgColumnarSlotStoreDeferred(slot, rel, snapshot, rowNumber); + else if (PgColumnarBufferedRowByNumber(rel, rowNumber, slot->tts_values, slot->tts_isnull)) ExecStoreVirtualTuple(slot); else return false; - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return true; } /* - * columnar_tuple_fetch_row_version + * pgcolumnar_tuple_fetch_row_version * Fetch the row addressed by tid into slot (spec 6). Used by UPDATE, which * re-fetches the old row by its item pointer. Returns false when the row * does not exist or is marked deleted. */ static bool -columnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, +pgcolumnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot) { - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); ExecClearTuple(slot); @@ -1210,33 +1210,33 @@ columnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, * partial stripe to satisfy a read would fragment storage for the sake of * data already in hand. */ - if (!ColumnarReadRowByNumber(rel, snapshot, rowNumber, + if (!PgColumnarReadRowByNumber(rel, snapshot, rowNumber, slot->tts_values, slot->tts_isnull) && - !ColumnarBufferedRowByNumber(rel, rowNumber, + !PgColumnarBufferedRowByNumber(rel, rowNumber, slot->tts_values, slot->tts_isnull)) return false; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return true; } static bool -columnar_tuple_tid_valid(TableScanDesc scan, ItemPointer tid) +pgcolumnar_tuple_tid_valid(TableScanDesc scan, ItemPointer tid) { return true; } static void -columnar_tuple_get_latest_tid(TableScanDesc scan, ItemPointer tid) +pgcolumnar_tuple_get_latest_tid(TableScanDesc scan, ItemPointer tid) { COLUMNAR_UNSUPPORTED("get latest tid"); } static bool -columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, Snapshot snapshot) { /* stripes are visible per their metadata snapshot; slots are visible */ @@ -1244,9 +1244,9 @@ columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, } /* - * columnar_index_delete_tuples + * pgcolumnar_index_delete_tuples * Opportunistic index tuple deletion. An index entry is deletable exactly - * when its row is no longer visible, i.e. ColumnarReadRowByNumber cannot + * when its row is no longer visible, i.e. PgColumnarReadRowByNumber cannot * return it (deleted via the delete vector). Reporting deletability by actual * liveness is required for correctness: nbtree's deletion pass (including * bottom-up deletion of duplicate keys, which a same-key UPDATE produces) @@ -1266,14 +1266,14 @@ columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, * correct and always-safe answer. */ static TransactionId -columnar_compute_xid_horizon_for_tuples(Relation rel, ItemPointerData *tids, +pgcolumnar_compute_xid_horizon_for_tuples(Relation rel, ItemPointerData *tids, int nitems) { return InvalidTransactionId; } #else static TransactionId -columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) +pgcolumnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) { Snapshot snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); @@ -1282,16 +1282,16 @@ columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) for (i = 0; i < delstate->ndeltids; i++) { uint64 rowNumber = - ColumnarItemPointerToRowNumber(&delstate->deltids[i].tid); + PgColumnarItemPointerToRowNumber(&delstate->deltids[i].tid); /* - * Only liveness matters here, and ColumnarRowIsLive decodes nothing to + * Only liveness matters here, and PgColumnarRowIsLive decodes nothing to * answer it. This used to reconstruct every column of the row and then * free the result unread, once per candidate index tuple, on a path * nbtree drives during deletion (issue #157). */ delstate->status[delstate->deltids[i].id].knowndeletable = - !ColumnarRowIsLive(rel, snapshot, rowNumber); + !PgColumnarRowIsLive(rel, snapshot, rowNumber); } return InvalidTransactionId; @@ -1299,7 +1299,7 @@ columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) #endif static void -columnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate, uint32 specToken) @@ -1308,29 +1308,29 @@ columnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, } static void -columnar_tuple_complete_speculative(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_complete_speculative(Relation rel, TupleTableSlot *slot, uint32 specToken, bool succeeded) { COLUMNAR_UNSUPPORTED("speculative insert"); } /* - * columnar_tuple_delete + * pgcolumnar_tuple_delete * Mark the row addressed by tid as deleted in the delete vector (spec 9). The * stripe is not rewritten. The tid is the synthetic item pointer the scan * produced, which maps back to the row number. */ static TM_Result -columnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) +pgcolumnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) { - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); - ColumnarMarkRowDeleted(rel, rowNumber); + PgColumnarMarkRowDeleted(rel, rowNumber); return TM_Ok; } /* - * columnar_tuple_update + * pgcolumnar_tuple_update * Update is delete-plus-insert (spec 9): mark the old row deleted in the * delete vector and append the new tuple as a fresh row with a new row number. * The new row's item pointer is published on the slot and index @@ -1339,24 +1339,24 @@ columnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) * row is now marked deleted (spec 6, 9). */ static TM_Result -columnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) +pgcolumnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) { - uint64 oldRowNumber = ColumnarItemPointerToRowNumber(otid); - ColumnarWriteState *writeState; + uint64 oldRowNumber = PgColumnarItemPointerToRowNumber(otid); + PgColumnarWriteState *writeState; uint64 rowNumber; - ColumnarMarkRowDeleted(rel, oldRowNumber); + PgColumnarMarkRowDeleted(rel, oldRowNumber); - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); slot_getallattrs(slot); /* the new row version is a fresh insert: serialize its unique keys too */ - ColumnarLockUniqueKeys(rel, slot); /* issue #5 */ + PgColumnarLockUniqueKeys(rel, slot); /* issue #5 */ - rowNumber = ColumnarWriteRow(writeState, rel, slot->tts_values, + rowNumber = PgColumnarWriteRow(writeState, rel, slot->tts_values, slot->tts_isnull); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); *lockmode = LockTupleExclusive; @@ -1365,7 +1365,7 @@ columnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) } static TM_Result -columnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, +pgcolumnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot, CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy, uint8 flags, TM_FailureData *tmfd) @@ -1375,19 +1375,19 @@ columnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, } static void -columnar_relation_copy_data(Relation rel, const RelFileLocator *newrlocator) +pgcolumnar_relation_copy_data(Relation rel, const RelFileLocator *newrlocator) { COLUMNAR_UNSUPPORTED("relation copy (ALTER TABLE SET TABLESPACE)"); } static void -columnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) +pgcolumnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) { COLUMNAR_UNSUPPORTED("CLUSTER / VACUUM FULL"); } /* - * columnar_index_build_range_scan + * pgcolumnar_index_build_range_scan * Scan every live row of the columnar table and hand it to the index * build callback, so CREATE INDEX (btree or hash) works over a columnar * table (spec 9). Deleted rows (delete vector) are skipped by the reader, so @@ -1400,14 +1400,14 @@ columnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) * included in the build. */ static double -columnar_index_build_range_scan(Relation table_rel, Relation index_rel, +pgcolumnar_index_build_range_scan(Relation table_rel, Relation index_rel, struct IndexInfo *index_info, bool allow_sync, bool anyvisible, bool progress, BlockNumber start_blockno, BlockNumber numblocks, IndexBuildCallback callback, void *callback_state, TableScanDesc scan) { - ColumnarReadState *readState; + PgColumnarReadState *readState; bool ownReadState; EState *estate; ExprContext *econtext; @@ -1424,8 +1424,8 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, errmsg("columnar: partial-range index build is not supported"))); /* persist buffered rows and delete marks so the build sees them (spec 9) */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(table_rel)); - ColumnarFlushDeleteVectorForRelation(table_rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(table_rel)); + PgColumnarFlushDeleteVectorForRelation(table_rel); estate = CreateExecutorState(); econtext = GetPerTupleExprContext(estate); @@ -1439,7 +1439,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * Obtain the reader. A parallel index build passes the TableScanDesc it * opened with table_beginscan_parallel; that scan already holds a reader * bound to the shared parallel scan, whose single-participant claim (see - * columnar_read_start) makes exactly one participant read the whole table. + * pgcolumnar_read_start) makes exactly one participant read the whole table. * We must read through that reader, not a private one: a private full-table * reader in every participant would index every row once per participant, * producing duplicate (key, TID) entries. When no scan is supplied (a serial @@ -1455,7 +1455,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * indexing what the table is now, not what an in-flight rewrite is * converting away from. */ - readState = columnar_scan_read_state((ColumnarScanDesc) scan, + readState = pgcolumnar_scan_read_state((PgColumnarScanDesc) scan, RelationGetDescr(table_rel)); ownReadState = false; } @@ -1468,7 +1468,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, else snapshot = GetTransactionSnapshot(); - readState = ColumnarBeginRead(table_rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(table_rel, snapshot, NULL, NULL, 0, NULL); ownReadState = true; } @@ -1477,7 +1477,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, CHECK_FOR_INTERRUPTS(); ExecClearTuple(slot); - if (!ColumnarReadNextRow(readState, slot->tts_values, slot->tts_isnull, + if (!PgColumnarReadNextRow(readState, slot->tts_values, slot->tts_isnull, &rowNumber)) break; ExecStoreVirtualTuple(slot); @@ -1491,14 +1491,14 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, FormIndexDatum(index_info, slot, estate, indexValues, indexNulls); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); callback(index_rel, &slot->tts_tid, indexValues, indexNulls, true, callback_state); } if (ownReadState) - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(slot); FreeExecutorState(estate); @@ -1506,9 +1506,9 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * The table AM contract makes index_build_range_scan the owner of a scan the * caller supplied: it must end it, exactly as heapam_index_build_range_scan * calls table_endscan on the passed scan whether or not it created the scan - * itself. columnar_scan_begin took a relation reference (and, for a worker, + * itself. pgcolumnar_scan_begin took a relation reference (and, for a worker, * a registered snapshot) and created the reader used above; table_endscan - * runs columnar_scan_end, which ends that reader and releases the reference. + * runs pgcolumnar_scan_end, which ends that reader and releases the reference. * Omitting this leaked one relation reference per build participant, which * surfaced at commit as "resource was not closed: relation". */ @@ -1519,7 +1519,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, } static void -columnar_index_validate_scan(Relation table_rel, Relation index_rel, +pgcolumnar_index_validate_scan(Relation table_rel, Relation index_rel, struct IndexInfo *index_info, Snapshot snapshot, struct ValidateIndexState *state) { @@ -1527,7 +1527,7 @@ columnar_index_validate_scan(Relation table_rel, Relation index_rel, } static bool -columnar_scan_sample_next_block(TableScanDesc scan, +pgcolumnar_scan_sample_next_block(TableScanDesc scan, struct SampleScanState *scanstate) { COLUMNAR_UNSUPPORTED("TABLESAMPLE"); @@ -1535,7 +1535,7 @@ columnar_scan_sample_next_block(TableScanDesc scan, } static bool -columnar_scan_sample_next_tuple(TableScanDesc scan, +pgcolumnar_scan_sample_next_tuple(TableScanDesc scan, struct SampleScanState *scanstate, TupleTableSlot *slot) { @@ -1547,67 +1547,67 @@ columnar_scan_sample_next_tuple(TableScanDesc scan, * the routine * ------------------------------------------------------------------------- */ -static const TableAmRoutine columnar_am_methods = { +static const TableAmRoutine pgcolumnar_am_methods = { .type = T_TableAmRoutine, - .slot_callbacks = columnar_slot_callbacks, + .slot_callbacks = pgcolumnar_slot_callbacks, - .scan_begin = columnar_scan_begin, - .scan_end = columnar_scan_end, - .scan_rescan = columnar_scan_rescan, - .scan_getnextslot = columnar_scan_getnextslot, + .scan_begin = pgcolumnar_scan_begin, + .scan_end = pgcolumnar_scan_end, + .scan_rescan = pgcolumnar_scan_rescan, + .scan_getnextslot = pgcolumnar_scan_getnextslot, - .parallelscan_estimate = columnar_parallelscan_estimate, - .parallelscan_initialize = columnar_parallelscan_initialize, - .parallelscan_reinitialize = columnar_parallelscan_reinitialize, + .parallelscan_estimate = pgcolumnar_parallelscan_estimate, + .parallelscan_initialize = pgcolumnar_parallelscan_initialize, + .parallelscan_reinitialize = pgcolumnar_parallelscan_reinitialize, - .index_fetch_begin = columnar_index_fetch_begin, - .index_fetch_reset = columnar_index_fetch_reset, - .index_fetch_end = columnar_index_fetch_end, - .index_fetch_tuple = columnar_index_fetch_tuple, + .index_fetch_begin = pgcolumnar_index_fetch_begin, + .index_fetch_reset = pgcolumnar_index_fetch_reset, + .index_fetch_end = pgcolumnar_index_fetch_end, + .index_fetch_tuple = pgcolumnar_index_fetch_tuple, - .tuple_fetch_row_version = columnar_tuple_fetch_row_version, - .tuple_tid_valid = columnar_tuple_tid_valid, - .tuple_get_latest_tid = columnar_tuple_get_latest_tid, - .tuple_satisfies_snapshot = columnar_tuple_satisfies_snapshot, + .tuple_fetch_row_version = pgcolumnar_tuple_fetch_row_version, + .tuple_tid_valid = pgcolumnar_tuple_tid_valid, + .tuple_get_latest_tid = pgcolumnar_tuple_get_latest_tid, + .tuple_satisfies_snapshot = pgcolumnar_tuple_satisfies_snapshot, #if PG_VERSION_NUM < 140000 - .COLUMNAR_AM_INDEX_DELETE_FIELD = columnar_compute_xid_horizon_for_tuples, + .COLUMNAR_AM_INDEX_DELETE_FIELD = pgcolumnar_compute_xid_horizon_for_tuples, #else - .COLUMNAR_AM_INDEX_DELETE_FIELD = columnar_index_delete_tuples, + .COLUMNAR_AM_INDEX_DELETE_FIELD = pgcolumnar_index_delete_tuples, #endif - .tuple_insert = columnar_tuple_insert, - .tuple_insert_speculative = columnar_tuple_insert_speculative, - .tuple_complete_speculative = columnar_tuple_complete_speculative, - .multi_insert = columnar_multi_insert, - .tuple_delete = columnar_tuple_delete, - .tuple_update = columnar_tuple_update, - .tuple_lock = columnar_tuple_lock, - .finish_bulk_insert = columnar_finish_bulk_insert, - - .COLUMNAR_AM_SET_NEW_FILE_FIELD = columnar_relation_set_new_filelocator, - .relation_nontransactional_truncate = columnar_relation_nontransactional_truncate, - .relation_copy_data = columnar_relation_copy_data, - .relation_copy_for_cluster = columnar_relation_copy_for_cluster, - .relation_vacuum = columnar_relation_vacuum, - .scan_analyze_next_block = columnar_scan_analyze_next_block, - .scan_analyze_next_tuple = columnar_scan_analyze_next_tuple, - .index_build_range_scan = columnar_index_build_range_scan, - .index_validate_scan = columnar_index_validate_scan, - - .relation_size = columnar_relation_size, - .relation_needs_toast_table = columnar_relation_needs_toast_table, - - .relation_estimate_size = columnar_relation_estimate_size, - - .scan_sample_next_block = columnar_scan_sample_next_block, - .scan_sample_next_tuple = columnar_scan_sample_next_tuple, + .tuple_insert = pgcolumnar_tuple_insert, + .tuple_insert_speculative = pgcolumnar_tuple_insert_speculative, + .tuple_complete_speculative = pgcolumnar_tuple_complete_speculative, + .multi_insert = pgcolumnar_multi_insert, + .tuple_delete = pgcolumnar_tuple_delete, + .tuple_update = pgcolumnar_tuple_update, + .tuple_lock = pgcolumnar_tuple_lock, + .finish_bulk_insert = pgcolumnar_finish_bulk_insert, + + .COLUMNAR_AM_SET_NEW_FILE_FIELD = pgcolumnar_relation_set_new_filelocator, + .relation_nontransactional_truncate = pgcolumnar_relation_nontransactional_truncate, + .relation_copy_data = pgcolumnar_relation_copy_data, + .relation_copy_for_cluster = pgcolumnar_relation_copy_for_cluster, + .relation_vacuum = pgcolumnar_relation_vacuum, + .scan_analyze_next_block = pgcolumnar_scan_analyze_next_block, + .scan_analyze_next_tuple = pgcolumnar_scan_analyze_next_tuple, + .index_build_range_scan = pgcolumnar_index_build_range_scan, + .index_validate_scan = pgcolumnar_index_validate_scan, + + .relation_size = pgcolumnar_relation_size, + .relation_needs_toast_table = pgcolumnar_relation_needs_toast_table, + + .relation_estimate_size = pgcolumnar_relation_estimate_size, + + .scan_sample_next_block = pgcolumnar_scan_sample_next_block, + .scan_sample_next_tuple = pgcolumnar_scan_sample_next_tuple, }; Datum -columnar_handler(PG_FUNCTION_ARGS) +pgcolumnar_handler(PG_FUNCTION_ARGS) { - PG_RETURN_POINTER(&columnar_am_methods); + PG_RETURN_POINTER(&pgcolumnar_am_methods); } /* ------------------------------------------------------------------------- @@ -1615,23 +1615,23 @@ columnar_handler(PG_FUNCTION_ARGS) * ------------------------------------------------------------------------- */ static void -columnar_xact_callback(XactEvent event, void *arg) +pgcolumnar_xact_callback(XactEvent event, void *arg) { switch (event) { case XACT_EVENT_PRE_COMMIT: case XACT_EVENT_PARALLEL_PRE_COMMIT: case XACT_EVENT_PREPARE: - ColumnarFlushAllPendingWrites(); - ColumnarFlushAllDeleteVectors(); + PgColumnarFlushAllPendingWrites(); + PgColumnarFlushAllDeleteVectors(); break; case XACT_EVENT_COMMIT: case XACT_EVENT_ABORT: case XACT_EVENT_PARALLEL_COMMIT: case XACT_EVENT_PARALLEL_ABORT: - ColumnarDiscardAllPendingWrites(); - ColumnarDiscardAllDeleteVectors(); - ColumnarDiscardFetchCache(); + PgColumnarDiscardAllPendingWrites(); + PgColumnarDiscardAllDeleteVectors(); + PgColumnarDiscardFetchCache(); break; default: break; @@ -1643,18 +1643,18 @@ columnar_xact_callback(XactEvent event, void *arg) * ------------------------------------------------------------------------- */ static void -columnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, +pgcolumnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, SubTransactionId parentSubid, void *arg) { switch (event) { case SUBXACT_EVENT_ABORT_SUB: - ColumnarWriteStateDiscardSubXact(mySubid); - ColumnarDeleteVectorDiscardSubXact(mySubid); + PgColumnarWriteStateDiscardSubXact(mySubid); + PgColumnarDeleteVectorDiscardSubXact(mySubid); break; case SUBXACT_EVENT_COMMIT_SUB: - ColumnarWriteStatePromoteSubXact(mySubid, parentSubid); - ColumnarDeleteVectorPromoteSubXact(mySubid, parentSubid); + PgColumnarWriteStatePromoteSubXact(mySubid, parentSubid); + PgColumnarDeleteVectorPromoteSubXact(mySubid, parentSubid); break; default: break; @@ -1674,15 +1674,15 @@ columnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, * ------------------------------------------------------------------------- */ static void -columnar_executor_end(QueryDesc *queryDesc) +pgcolumnar_executor_end(QueryDesc *queryDesc) { if (prev_executor_end_hook) prev_executor_end_hook(queryDesc); else standard_ExecutorEnd(queryDesc); - ColumnarFlushAllPendingWrites(); - ColumnarFlushAllDeleteVectors(); + PgColumnarFlushAllPendingWrites(); + PgColumnarFlushAllDeleteVectors(); /* * The fetch cache is scoped to a statement, so release it here rather than @@ -1690,7 +1690,7 @@ columnar_executor_end(QueryDesc *queryDesc) * slot pins them for the rest of the transaction, and a session sitting idle * in transaction after one UPDATE holds them indefinitely. */ - ColumnarDiscardFetchCache(); + PgColumnarDiscardFetchCache(); } /* ------------------------------------------------------------------------- @@ -1699,7 +1699,7 @@ columnar_executor_end(QueryDesc *queryDesc) * ------------------------------------------------------------------------- */ /* - * columnar_reject_fk_to_columnar + * pgcolumnar_reject_fk_to_columnar * Raise on a foreign key whose referenced side is a columnar table. * * The referential-integrity check reads the referenced row with FOR KEY SHARE, @@ -1726,7 +1726,7 @@ columnar_executor_end(QueryDesc *queryDesc) * through the ordinary insert path. */ static void -columnar_reject_fk_to_columnar(Oid constraintId) +pgcolumnar_reject_fk_to_columnar(Oid constraintId) { Relation conRel; SysScanDesc scan; @@ -1763,7 +1763,7 @@ columnar_reject_fk_to_columnar(Oid constraintId) if (get_rel_relkind(referenced) != RELKIND_RELATION) return; - if (ColumnarIsColumnarRelation(referenced)) + if (PgColumnarIsColumnarRelation(referenced)) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("cannot create a foreign key referencing columnar table \"%s\"", @@ -1775,14 +1775,14 @@ columnar_reject_fk_to_columnar(Oid constraintId) } /* - * columnar_am_is_columnar + * pgcolumnar_am_is_columnar * Does this table access method oid resolve to this extension's routine? * * By handler identity rather than by name, so a second name for the same access * method is still recognised. */ static bool -columnar_am_is_columnar(Oid amoid) +pgcolumnar_am_is_columnar(Oid amoid) { HeapTuple tup; Form_pg_am amform; @@ -1810,15 +1810,15 @@ columnar_am_is_columnar(Oid amoid) } ReleaseSysCache(tup); - return GetTableAmRoutine(handler) == &columnar_am_methods; + return GetTableAmRoutine(handler) == &pgcolumnar_am_methods; } /* - * columnar_fk_referencing + * pgcolumnar_fk_referencing * Name of some foreign key whose referenced side is relid, or NULL. */ static char * -columnar_fk_referencing(Oid relid) +pgcolumnar_fk_referencing(Oid relid) { Relation conRel; SysScanDesc scan; @@ -1852,11 +1852,11 @@ columnar_fk_referencing(Oid relid) } /* - * columnar_reject_set_am_to_columnar + * pgcolumnar_reject_set_am_to_columnar * Refuse ALTER TABLE ... SET ACCESS METHOD to columnar when the relation is * the referenced side of a foreign key. * - * columnar_reject_fk_to_columnar closes this door where the constraint is + * pgcolumnar_reject_fk_to_columnar closes this door where the constraint is * created. The same unusable configuration is reachable from the other end, by * making the referenced table columnar after the constraint already exists: * @@ -1883,7 +1883,7 @@ columnar_fk_referencing(Oid relid) * every supported major it executes this same statement. */ static void -columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) +pgcolumnar_reject_set_am_to_columnar(AlterTableStmt *stmt) { Oid relid; ListCell *lc; @@ -1934,10 +1934,10 @@ columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) /* SET ACCESS METHOD DEFAULT leaves the name unset (PG17+) */ amname = cmd->name ? cmd->name : default_table_access_method; - if (!columnar_am_is_columnar(get_table_am_oid(amname, true))) + if (!pgcolumnar_am_is_columnar(get_table_am_oid(amname, true))) continue; - conname = columnar_fk_referencing(relid); + conname = pgcolumnar_fk_referencing(relid); if (conname == NULL) continue; @@ -1954,7 +1954,7 @@ columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) } static void -columnar_process_utility(PlannedStmt *pstmt, const char *queryString, +pgcolumnar_process_utility(PlannedStmt *pstmt, const char *queryString, bool readOnlyTree, ProcessUtilityContext context, ParamListInfo params, QueryEnvironment *queryEnv, DestReceiver *dest, QueryCompletion *qc) @@ -1963,7 +1963,7 @@ columnar_process_utility(PlannedStmt *pstmt, const char *queryString, /* read-only inspection, so readOnlyTree needs no copy of the tree */ if (parsetree != NULL && IsA(parsetree, AlterTableStmt)) - columnar_reject_set_am_to_columnar((AlterTableStmt *) parsetree); + pgcolumnar_reject_set_am_to_columnar((AlterTableStmt *) parsetree); if (prev_process_utility_hook) prev_process_utility_hook(pstmt, queryString, readOnlyTree, context, @@ -1974,14 +1974,14 @@ columnar_process_utility(PlannedStmt *pstmt, const char *queryString, } static void -columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, +pgcolumnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, int subId, void *arg) { if (prev_object_access_hook) prev_object_access_hook(access, classId, objectId, subId, arg); if (access == OAT_POST_CREATE && classId == ConstraintRelationId) - columnar_reject_fk_to_columnar(objectId); + pgcolumnar_reject_fk_to_columnar(objectId); if (access == OAT_DROP && classId == RelationRelationId && subId == 0) { @@ -1993,10 +1993,10 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, /* DROP already holds AccessExclusiveLock on the relation */ rel = relation_open(objectId, NoLock); - if (rel->rd_tableam == &columnar_am_methods) + if (rel->rd_tableam == &pgcolumnar_am_methods) { - uint64 storageId = ColumnarStorageId(rel); - List *projs = ColumnarListProjections(storageId); + uint64 storageId = PgColumnarStorageId(rel); + List *projs = PgColumnarListProjections(storageId); ListCell *lc; /* @@ -2005,20 +2005,20 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * projection's row groups, chunks, zone maps and bloom filters * behind with no relation to reach them from: metadata for a table * that no longer exists, accumulating one projection's worth per - * drop. This is the same loop columnar_vacuum.c runs when it + * drop. This is the same loop pgcolumnar_vacuum.c runs when it * rewrites into fresh storage. */ foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != storageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(storageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(storageId, p->projectionId); } - ColumnarDeleteMetadata(storageId); - ColumnarDeleteOptions(objectId); + PgColumnarDeleteMetadata(storageId); + PgColumnarDeleteOptions(objectId); /* * And the projection declarations, for the same reason and in the * same place (#304). A declaration left behind holds a regclass that @@ -2026,7 +2026,7 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * rebuild_projections() aborts on, taking every other table in the * database with it. */ - ColumnarDeleteProjectionDeclarationsForRel(objectId); + PgColumnarDeleteProjectionDeclarationsForRel(objectId); } relation_close(rel, NoLock); @@ -2052,13 +2052,13 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * ------------------------------------------------------------------------- */ static bool -columnar_relation_is_columnar(Oid relid) +pgcolumnar_relation_is_columnar(Oid relid) { - if (columnar_am_oid_cache == InvalidOid) - columnar_am_oid_cache = get_am_oid("pgcolumnar", true); + if (pgcolumnar_am_oid_cache == InvalidOid) + pgcolumnar_am_oid_cache = get_am_oid("pgcolumnar", true); - return OidIsValid(columnar_am_oid_cache) && - get_rel_relam(relid) == columnar_am_oid_cache; + return OidIsValid(pgcolumnar_am_oid_cache) && + get_rel_relam(relid) == pgcolumnar_am_oid_cache; } /* GUC: when on, allow the planner to build index-only-scan paths for columnar @@ -2067,22 +2067,22 @@ columnar_relation_is_columnar(Oid relid) * horizon accounts for open snapshots and every write clears the bit, both * WAL-logged), and a not-all-visible block always falls back to the * snapshot-checked fetch, so results are correct regardless. */ -bool columnar_enable_index_only_scan = true; +bool pgcolumnar_enable_index_only_scan = true; /* clear the "can return" flags of every index on a columnar relation */ static void -columnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) +pgcolumnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) { ListCell *lc; /* when index-only scans are enabled, leave the index canreturn flags intact * so the planner may choose an IOS; the VM fork (set by lazy vacuum) drives * whether the executor skips the fetch, and a not-all-visible block still - * falls back to columnar_index_fetch_tuple, so results are always correct. */ - if (columnar_enable_index_only_scan) + * falls back to pgcolumnar_index_fetch_tuple, so results are always correct. */ + if (pgcolumnar_enable_index_only_scan) return; - if (!OidIsValid(relid) || !columnar_relation_is_columnar(relid)) + if (!OidIsValid(relid) || !pgcolumnar_relation_is_columnar(relid)) return; foreach(lc, rel->indexlist) @@ -2100,24 +2100,24 @@ columnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) #if PG_VERSION_NUM >= 190000 static void -columnar_build_simple_rel(PlannerInfo *root, RelOptInfo *rel, +pgcolumnar_build_simple_rel(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte) { if (prev_build_simple_rel_hook) prev_build_simple_rel_hook(root, rel, rte); if (rte->rtekind == RTE_RELATION) - columnar_forbid_index_only_scan(rte->relid, rel); + pgcolumnar_forbid_index_only_scan(rte->relid, rel); } #else static void -columnar_get_relation_info(PlannerInfo *root, Oid relationObjectId, +pgcolumnar_get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, RelOptInfo *rel) { if (prev_get_relation_info_hook) prev_get_relation_info_hook(root, relationObjectId, inhparent, rel); - columnar_forbid_index_only_scan(relationObjectId, rel); + pgcolumnar_forbid_index_only_scan(relationObjectId, rel); } #endif @@ -2130,22 +2130,22 @@ _PG_init(void) { /* * Virtual slot behaviour, except that a copied heap tuple keeps the slot's - * item pointer. See the comment on columnar_slot_copy_heap_tuple. + * item pointer. See the comment on pgcolumnar_slot_copy_heap_tuple. */ - ColumnarSlotOps = TTSOpsVirtual; - ColumnarSlotOps.base_slot_size = sizeof(ColumnarSlot); - ColumnarSlotOps.copy_heap_tuple = columnar_slot_copy_heap_tuple; - ColumnarSlotOps.init = columnar_slot_init; - ColumnarSlotOps.getsomeattrs = columnar_slot_getsomeattrs; - ColumnarSlotOps.clear = columnar_slot_clear; - ColumnarSlotOps.materialize = columnar_slot_materialize; - ColumnarSlotOps.copyslot = columnar_slot_copyslot; - ColumnarSlotOps.copy_minimal_tuple = columnar_slot_copy_minimal_tuple; + PgColumnarSlotOps = TTSOpsVirtual; + PgColumnarSlotOps.base_slot_size = sizeof(PgColumnarSlot); + PgColumnarSlotOps.copy_heap_tuple = pgcolumnar_slot_copy_heap_tuple; + PgColumnarSlotOps.init = pgcolumnar_slot_init; + PgColumnarSlotOps.getsomeattrs = pgcolumnar_slot_getsomeattrs; + PgColumnarSlotOps.clear = pgcolumnar_slot_clear; + PgColumnarSlotOps.materialize = pgcolumnar_slot_materialize; + PgColumnarSlotOps.copyslot = pgcolumnar_slot_copyslot; + PgColumnarSlotOps.copy_minimal_tuple = pgcolumnar_slot_copy_minimal_tuple; DefineCustomIntVariable("pgcolumnar.stripe_row_limit", "Maximum number of rows per stripe.", NULL, - &columnar_stripe_row_limit, + &pgcolumnar_stripe_row_limit, 150000, 1000, INT_MAX, PGC_USERSET, @@ -2155,7 +2155,7 @@ _PG_init(void) DefineCustomIntVariable("pgcolumnar.chunk_group_row_limit", "Maximum number of rows per chunk group.", NULL, - &columnar_chunk_group_row_limit, + &pgcolumnar_chunk_group_row_limit, 10000, 100, INT_MAX, PGC_USERSET, @@ -2171,7 +2171,7 @@ _PG_init(void) "value below 128 is treated as 0, because a sample that " "small cannot rank candidates: every candidate's fixed " "header would exceed the sample itself.", - &columnar_encoding_sample_rows, + &pgcolumnar_encoding_sample_rows, 2048, 0, INT_MAX, PGC_USERSET, @@ -2181,9 +2181,9 @@ _PG_init(void) DefineCustomEnumVariable("pgcolumnar.compression", "Default compression codec for new chunks.", NULL, - &columnar_compression, + &pgcolumnar_compression, COLUMNAR_COMPRESSION_ZSTD, - columnar_compression_options, + pgcolumnar_compression_options, PGC_USERSET, 0, NULL, NULL, NULL); @@ -2191,7 +2191,7 @@ _PG_init(void) DefineCustomIntVariable("pgcolumnar.compression_level", "Compression level for the zstd codec.", NULL, - &columnar_compression_level, + &pgcolumnar_compression_level, 3, 1, 22, PGC_USERSET, @@ -2212,7 +2212,7 @@ _PG_init(void) "and saves roughly a third of their load time; where FSST " "wins clearly it changes nothing. Set to 0 to keep FSST on " "any win at all.", - &columnar_fsst_min_gain_percent, + &pgcolumnar_fsst_min_gain_percent, 5, 0, 99, PGC_USERSET, @@ -2222,7 +2222,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_qual_pushdown", "Push scan qualifiers down for chunk-group skipping.", NULL, - &columnar_enable_qual_pushdown, + &pgcolumnar_enable_qual_pushdown, true, PGC_USERSET, 0, @@ -2234,7 +2234,7 @@ _PG_init(void) "read and decoded, as before the projection was honored. " "Provided as an escape hatch and as the A/B oracle the " "projection tests compare against.", - &columnar_enable_column_projection, + &pgcolumnar_enable_column_projection, true, PGC_USERSET, 0, @@ -2243,7 +2243,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_custom_scan", "Use the columnar custom scan path for columnar tables.", NULL, - &columnar_enable_custom_scan, + &pgcolumnar_enable_custom_scan, true, PGC_USERSET, 0, @@ -2252,7 +2252,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_vectorization", "Use the vectorized aggregate fast path.", NULL, - &columnar_enable_vectorization, + &pgcolumnar_enable_vectorization, true, PGC_USERSET, 0, @@ -2261,7 +2261,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_group_vectorization", "Use the vectorized aggregate fast path for GROUP BY queries.", NULL, - &columnar_enable_group_vectorization, + &pgcolumnar_enable_group_vectorization, false, PGC_USERSET, 0, @@ -2272,7 +2272,7 @@ _PG_init(void) "aggregates with a filter or sum/avg over " "int8/float/numeric.", NULL, - &columnar_enable_ungrouped_vector_agg, + &pgcolumnar_enable_ungrouped_vector_agg, false, PGC_USERSET, 0, @@ -2283,7 +2283,7 @@ _PG_init(void) "each worker folds distinct row groups and emits a partial " "aggregate a core Finalize combines.", NULL, - &columnar_enable_parallel_vector_agg, + &pgcolumnar_enable_parallel_vector_agg, false, PGC_USERSET, 0, @@ -2296,7 +2296,7 @@ _PG_init(void) "not the planner's estimate: over the cap the query errors " "rather than falling back, since the plan is fixed by then. " "Raise it, or turn off pgcolumnar.enable_group_vectorization.", - &columnar_groupagg_max_groups, + &pgcolumnar_groupagg_max_groups, 1000000, 1, INT_MAX, PGC_USERSET, 0, @@ -2305,7 +2305,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_bloom_filter", "Skip chunk groups on equality using per-chunk bloom filters.", NULL, - &columnar_enable_bloom_filter, + &pgcolumnar_enable_bloom_filter, true, PGC_USERSET, 0, @@ -2316,7 +2316,7 @@ _PG_init(void) "adjacent freed ranges, so compaction reclaims space " "under fragmentation. Off reverts to whole-range reuse.", NULL, - &columnar_reclaim_coalesce, + &pgcolumnar_reclaim_coalesce, true, PGC_USERSET, 0, @@ -2327,7 +2327,7 @@ _PG_init(void) "trailing reclaimed blocks to the OS. Off (the default) " "makes truncate() a no-op.", NULL, - &columnar_enable_end_truncation, + &pgcolumnar_enable_end_truncation, false, PGC_SUSET, 0, @@ -2336,7 +2336,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_read_stream", "Prefetch block reads with the read stream API (PostgreSQL 17+).", NULL, - &columnar_enable_read_stream, + &pgcolumnar_enable_read_stream, true, PGC_USERSET, 0, @@ -2347,7 +2347,7 @@ _PG_init(void) "visibility-map fork (gap 28). On by default; set off to force " "a plain index scan.", NULL, - &columnar_enable_index_only_scan, + &pgcolumnar_enable_index_only_scan, true, PGC_USERSET, 0, @@ -2357,7 +2357,7 @@ _PG_init(void) "Let the planner scan a covering projection instead of the " "base table when one serves the query better (gap 26).", NULL, - &columnar_enable_projection_scan, + &pgcolumnar_enable_projection_scan, true, PGC_USERSET, 0, @@ -2370,7 +2370,7 @@ _PG_init(void) "lives in, so an unclustered ordered index scan can cost far " "more than core's per-page estimate. Off restores the " "unpenalized planner behaviour.", - &columnar_enable_index_fetch_penalty, + &pgcolumnar_enable_index_fetch_penalty, true, PGC_USERSET, 0, @@ -2385,7 +2385,7 @@ _PG_init(void) "when the creation lock guards nothing, so an ordinary write still " "behaves correctly. Off by default leaves the write path unchanged.", NULL, - &columnar_bulk_parallel_writer, + &pgcolumnar_bulk_parallel_writer, false, PGC_USERSET, GUC_NOT_IN_SAMPLE, @@ -2397,7 +2397,7 @@ _PG_init(void) "index key so overlapping same-key inserts conflict " "correctly (issue #5). Turning it off restores the " "prior racy behavior.", - &columnar_enable_unique_lock, + &pgcolumnar_enable_unique_lock, true, PGC_USERSET, 0, @@ -2414,7 +2414,7 @@ _PG_init(void) "backends inserting the same key must compute the " "same bucket, which they only do when they agree on " "this value.", - &columnar_unique_lock_buckets, + &pgcolumnar_unique_lock_buckets, 128, 1, 1048576, PGC_POSTMASTER, @@ -2423,17 +2423,17 @@ _PG_init(void) MarkGUCPrefixReserved("pgcolumnar"); - RegisterXactCallback(columnar_xact_callback, NULL); - RegisterSubXactCallback(columnar_subxact_callback, NULL); + RegisterXactCallback(pgcolumnar_xact_callback, NULL); + RegisterSubXactCallback(pgcolumnar_subxact_callback, NULL); prev_object_access_hook = object_access_hook; - object_access_hook = columnar_object_access; + object_access_hook = pgcolumnar_object_access; prev_process_utility_hook = ProcessUtility_hook; - ProcessUtility_hook = columnar_process_utility; + ProcessUtility_hook = pgcolumnar_process_utility; prev_executor_end_hook = ExecutorEnd_hook; - ExecutorEnd_hook = columnar_executor_end; + ExecutorEnd_hook = pgcolumnar_executor_end; /* * Forbid index-only scans on columnar tables. PG19 replaced @@ -2442,18 +2442,18 @@ _PG_init(void) */ #if PG_VERSION_NUM >= 190000 prev_build_simple_rel_hook = build_simple_rel_hook; - build_simple_rel_hook = columnar_build_simple_rel; + build_simple_rel_hook = pgcolumnar_build_simple_rel; #else prev_get_relation_info_hook = get_relation_info_hook; - get_relation_info_hook = columnar_get_relation_info; + get_relation_info_hook = pgcolumnar_get_relation_info; #endif /* register the custom scan provider and install the pathlist hook */ - ColumnarCustomScanInit(); + PgColumnarCustomScanInit(); /* install the vectorized-aggregate upper-path hook (spec 9) */ - ColumnarVectorInit(); + PgColumnarVectorInit(); /* register the unique-index cache invalidation callback (issue #5) */ - ColumnarUniqueInit(); + PgColumnarUniqueInit(); } diff --git a/src/columnar_thrift.c b/src/columnar_thrift.c index 718d49c..6fd48aa 100644 --- a/src/columnar_thrift.c +++ b/src/columnar_thrift.c @@ -1,9 +1,9 @@ /*------------------------------------------------------------------------- * - * columnar_thrift.c + * pgcolumnar_thrift.c * Thrift compact-protocol reader and writer. * - * See columnar_thrift.h. Nothing here knows about Parquet or about pgColumnar. + * See pgcolumnar_thrift.h. Nothing here knows about Parquet or about pgColumnar. * The two directions were previously in separate files, the reader inside the * Parquet import module and the writer inside the export module, which put the * encode and decode of the same wire format a thousand lines apart. @@ -25,7 +25,7 @@ * ------------------------------------------------------------------------- */ uint64 -ColumnarThriftVarint(TCReader *r) +PgColumnarThriftVarint(TCReader *r) { uint64 v = 0; int shift = 0; @@ -46,18 +46,18 @@ ColumnarThriftVarint(TCReader *r) } int64 -ColumnarThriftZigzag(TCReader *r) +PgColumnarThriftZigzag(TCReader *r) { - uint64 u = ColumnarThriftVarint(r); + uint64 u = PgColumnarThriftVarint(r); return (int64) (u >> 1) ^ -(int64) (u & 1); } /* read a binary/string field: returns pointer into the buffer and its length */ const uint8 * -ColumnarThriftBytes(TCReader *r, uint32 *outlen) +PgColumnarThriftBytes(TCReader *r, uint32 *outlen) { - uint64 n = ColumnarThriftVarint(r); + uint64 n = PgColumnarThriftVarint(r); const uint8 *p; /* @@ -88,7 +88,7 @@ ColumnarThriftBytes(TCReader *r, uint32 *outlen) * short-form delta encoding. */ void -ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) +PgColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) { uint8 b; @@ -108,13 +108,13 @@ ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) if ((b >> 4) != 0) *fid = *lastId + (b >> 4); /* short-form delta */ else - *fid = (int) ColumnarThriftZigzag(r); /* long form */ + *fid = (int) PgColumnarThriftZigzag(r); /* long form */ *lastId = *fid; } /* skip a value of the given compact type (for fields we do not consume) */ void -ColumnarThriftSkip(TCReader *r, int ftype) +PgColumnarThriftSkip(TCReader *r, int ftype) { /* * A crafted footer can nest structs (or lists of structs) to any depth, and @@ -136,7 +136,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) case TC_I16: case TC_I32: case TC_I64: - (void) ColumnarThriftZigzag(r); + (void) PgColumnarThriftZigzag(r); break; case TC_DOUBLE: r->pos += 8; @@ -145,7 +145,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) { uint32 n; - (void) ColumnarThriftBytes(r, &n); + (void) PgColumnarThriftBytes(r, &n); break; } case TC_LIST: @@ -165,9 +165,9 @@ ColumnarThriftSkip(TCReader *r, int ftype) size = (sizeType >> 4) & 0x0f; et = sizeType & 0x0f; if (size == 0x0f) - size = (uint32) ColumnarThriftVarint(r); + size = (uint32) PgColumnarThriftVarint(r); for (i = 0; i < size && !r->error; i++) - ColumnarThriftSkip(r, et); + PgColumnarThriftSkip(r, et); break; } case TC_STRUCT: @@ -179,10 +179,10 @@ ColumnarThriftSkip(TCReader *r, int ftype) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } break; } @@ -193,7 +193,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) /* list header: returns element count and element compact type */ uint32 -ColumnarThriftListHeader(TCReader *r, int *etype) +PgColumnarThriftListHeader(TCReader *r, int *etype) { uint8 b; uint32 size; @@ -208,14 +208,14 @@ ColumnarThriftListHeader(TCReader *r, int *etype) size = (b >> 4) & 0x0f; *etype = b & 0x0f; if (size == 0x0f) - size = (uint32) ColumnarThriftVarint(r); + size = (uint32) PgColumnarThriftVarint(r); return size; } /* ---- Thrift compact-protocol writer (into a StringInfo) ---- */ void -ColumnarThriftPutVarint(StringInfo b, uint64 v) +PgColumnarThriftPutVarint(StringInfo b, uint64 v) { while (v >= 0x80) { @@ -226,20 +226,20 @@ ColumnarThriftPutVarint(StringInfo b, uint64 v) } void -ColumnarThriftPutZigzag32(StringInfo b, int32 v) +PgColumnarThriftPutZigzag32(StringInfo b, int32 v) { - ColumnarThriftPutVarint(b, (uint32) ((v << 1) ^ (v >> 31))); + PgColumnarThriftPutVarint(b, (uint32) ((v << 1) ^ (v >> 31))); } void -ColumnarThriftPutZigzag64(StringInfo b, int64 v) +PgColumnarThriftPutZigzag64(StringInfo b, int64 v) { - ColumnarThriftPutVarint(b, (uint64) ((v << 1) ^ (v >> 63))); + PgColumnarThriftPutVarint(b, (uint64) ((v << 1) ^ (v >> 63))); } /* field header with delta-encoded id */ void -ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) +PgColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) { int delta = id - *lastId; @@ -248,49 +248,49 @@ ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) else { appendStringInfoChar(b, (char) type); - ColumnarThriftPutZigzag32(b, id); + PgColumnarThriftPutZigzag32(b, id); } *lastId = id; } void -ColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v) +PgColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v) { - ColumnarThriftPutField(b, lastId, id, TC_I32); - ColumnarThriftPutZigzag32(b, v); + PgColumnarThriftPutField(b, lastId, id, TC_I32); + PgColumnarThriftPutZigzag32(b, v); } void -ColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v) +PgColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v) { - ColumnarThriftPutField(b, lastId, id, TC_I64); - ColumnarThriftPutZigzag64(b, v); + PgColumnarThriftPutField(b, lastId, id, TC_I64); + PgColumnarThriftPutZigzag64(b, v); } void -ColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len) +PgColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len) { - ColumnarThriftPutField(b, lastId, id, TC_BINARY); - ColumnarThriftPutVarint(b, (uint64) len); + PgColumnarThriftPutField(b, lastId, id, TC_BINARY); + PgColumnarThriftPutVarint(b, (uint64) len); if (len > 0) appendBinaryStringInfo(b, s, len); } /* list header; caller then appends the elements */ void -ColumnarThriftPutListHeader(StringInfo b, int size, int elemType) +PgColumnarThriftPutListHeader(StringInfo b, int size, int elemType) { if (size < 15) appendStringInfoChar(b, (char) ((size << 4) | elemType)); else { appendStringInfoChar(b, (char) (0xF0 | elemType)); - ColumnarThriftPutVarint(b, (uint64) size); + PgColumnarThriftPutVarint(b, (uint64) size); } } void -ColumnarThriftPutStop(StringInfo b) +PgColumnarThriftPutStop(StringInfo b) { appendStringInfoChar(b, (char) TC_STOP); } diff --git a/src/columnar_thrift.h b/src/columnar_thrift.h index 0d60041..d25309c 100644 --- a/src/columnar_thrift.h +++ b/src/columnar_thrift.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_thrift.h + * pgcolumnar_thrift.h * Thrift compact-protocol reader and writer. * * The protocol is the container Parquet uses for its file metadata, and nothing @@ -33,24 +33,24 @@ typedef struct TCReader bool error; } TCReader; -extern uint64 ColumnarThriftVarint(TCReader *r); -extern int64 ColumnarThriftZigzag(TCReader *r); -extern const uint8 *ColumnarThriftBytes(TCReader *r, uint32 *outlen); -extern void ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId); -extern void ColumnarThriftSkip(TCReader *r, int ftype); -extern uint32 ColumnarThriftListHeader(TCReader *r, int *etype); +extern uint64 PgColumnarThriftVarint(TCReader *r); +extern int64 PgColumnarThriftZigzag(TCReader *r); +extern const uint8 *PgColumnarThriftBytes(TCReader *r, uint32 *outlen); +extern void PgColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId); +extern void PgColumnarThriftSkip(TCReader *r, int ftype); +extern uint32 PgColumnarThriftListHeader(TCReader *r, int *etype); /* ---- writer: appends into a StringInfo ---- */ -extern void ColumnarThriftPutVarint(StringInfo b, uint64 v); -extern void ColumnarThriftPutZigzag32(StringInfo b, int32 v); -extern void ColumnarThriftPutZigzag64(StringInfo b, int64 v); -extern void ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type); -extern void ColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v); -extern void ColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v); -extern void ColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, +extern void PgColumnarThriftPutVarint(StringInfo b, uint64 v); +extern void PgColumnarThriftPutZigzag32(StringInfo b, int32 v); +extern void PgColumnarThriftPutZigzag64(StringInfo b, int64 v); +extern void PgColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type); +extern void PgColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v); +extern void PgColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v); +extern void PgColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len); -extern void ColumnarThriftPutListHeader(StringInfo b, int size, int elemType); -extern void ColumnarThriftPutStop(StringInfo b); +extern void PgColumnarThriftPutListHeader(StringInfo b, int size, int elemType); +extern void PgColumnarThriftPutStop(StringInfo b); #endif /* PGCOLUMNAR_THRIFT_H */ diff --git a/src/columnar_unique.c b/src/columnar_unique.c index 8ae9e2a..5367659 100644 --- a/src/columnar_unique.c +++ b/src/columnar_unique.c @@ -1,20 +1,20 @@ /*------------------------------------------------------------------------- * - * columnar_unique.c + * pgcolumnar_unique.c * Concurrent unique-key insert serialization for pgColumnar (issue #5). * * A columnar row's data is invisible to other backends until its stripe is * flushed at statement end, but its btree index entry (with the eagerly * reserved synthetic TID) is written immediately. PostgreSQL's dirty-snapshot * uniqueness check (_bt_check_unique -> table_index_fetch_tuple_check) resolves - * that TID through columnar_index_fetch_tuple, which returns false for a row + * that TID through pgcolumnar_index_fetch_tuple, which returns false for a row * still buffered in another backend's private write state. So two transactions * inserting the same unique key in overlapping windows can both miss the * conflict and both commit (see design/ISSUE_5_ANALYSIS.md, section A). * * The fix here serializes inserters of the SAME unique key. Before a row is * handed back to the executor's index maintenance, the table AM insert paths - * call ColumnarLockUniqueKeys, which takes a transaction-scoped advisory lock + * call PgColumnarLockUniqueKeys, which takes a transaction-scoped advisory lock * (the same SET_LOCKTAG_ADVISORY primitive used by the issue #4 delete_vector lock) * keyed by the row's unique key value(s). Because the lock is held to commit, * when a second inserter finally acquires it the first inserter has either @@ -55,8 +55,8 @@ #include "utils/typcache.h" /* GUCs (spec 8.3): default on, bounded bucket count to bound the lock budget */ -bool columnar_enable_unique_lock = true; -int columnar_unique_lock_buckets = 128; +bool pgcolumnar_enable_unique_lock = true; +int pgcolumnar_unique_lock_buckets = 128; /* * Advisory-lock discriminator in locktag_field4. The issue #4 delete_vector lock @@ -116,7 +116,7 @@ static HTAB *RelUniqueCache = NULL; * (an invalidation on an index carries the index's relid, not the table's). */ static void -columnar_unique_invalidate(Datum arg, Oid relid) +pgcolumnar_unique_invalidate(Datum arg, Oid relid) { HASH_SEQ_STATUS status; RelUniqueCacheEntry *entry; @@ -136,7 +136,7 @@ columnar_unique_invalidate(Datum arg, Oid relid) } static void -columnar_unique_cache_init(void) +pgcolumnar_unique_cache_init(void) { HASHCTL ctl; @@ -160,7 +160,7 @@ columnar_unique_cache_init(void) * no hash support. */ static bool -columnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, +pgcolumnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, UniqueKeyCol *out) { Oid keyType = indexRel->rd_opcintype[c]; @@ -191,7 +191,7 @@ columnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, /* Build (or rebuild) the cache entry for rel. */ static RelUniqueCacheEntry * -columnar_unique_build(Relation rel) +pgcolumnar_unique_build(Relation rel) { Oid relid = RelationGetRelid(rel); MemoryContext cxt; @@ -202,7 +202,7 @@ columnar_unique_build(Relation rel) int nIndexes = 0; bool found; - columnar_unique_cache_init(); + pgcolumnar_unique_cache_init(); cxt = AllocSetContextCreate(CacheMemoryContext, "columnar unique index info", @@ -254,7 +254,7 @@ columnar_unique_build(Relation rel) for (c = 0; c < uidx->nKeyCols; c++) { - if (!columnar_key_col_hash(indexRel, c, cxt, &uidx->cols[c])) + if (!pgcolumnar_key_col_hash(indexRel, c, cxt, &uidx->cols[c])) { uidx->coarse = true; break; @@ -290,7 +290,7 @@ columnar_unique_build(Relation rel) } static RelUniqueCacheEntry * -columnar_unique_lookup(Relation rel) +pgcolumnar_unique_lookup(Relation rel) { Oid relid = RelationGetRelid(rel); RelUniqueCacheEntry *entry = NULL; @@ -301,7 +301,7 @@ columnar_unique_lookup(Relation rel) if (entry != NULL) return entry; - return columnar_unique_build(rel); + return pgcolumnar_unique_build(rel); } /* ------------------------------------------------------------------------- @@ -310,7 +310,7 @@ columnar_unique_lookup(Relation rel) /* splitmix64/murmur3 finalizer, matching delete_vector_chunk_lock_key */ static inline uint64 -columnar_hash_finalize(uint64 h) +pgcolumnar_hash_finalize(uint64 h) { h ^= h >> 33; h *= UINT64CONST(0xff51afd7ed558ccd); @@ -321,7 +321,7 @@ columnar_hash_finalize(uint64 h) } static void -columnar_acquire_key_lock(Oid indexOid, uint32 bucket) +pgcolumnar_acquire_key_lock(Oid indexOid, uint32 bucket) { LOCKTAG tag; @@ -333,7 +333,7 @@ columnar_acquire_key_lock(Oid indexOid, uint32 bucket) } /* - * ColumnarLockUniqueKeys + * PgColumnarLockUniqueKeys * For a row about to be inserted into rel (through the table AM), take a * transaction-scoped advisory lock for each applicable unique index's key, * so a concurrent inserter of an equal key serializes behind this one until @@ -353,7 +353,7 @@ columnar_acquire_key_lock(Oid indexOid, uint32 bucket) * coarse per-index lock (over-serializes, always correct). */ void -ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) +PgColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) { RelUniqueCacheEntry *entry; ExprContext *econtext; @@ -361,14 +361,14 @@ ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) uint32 numBuckets; int i; - if (!columnar_enable_unique_lock) + if (!pgcolumnar_enable_unique_lock) return; - entry = columnar_unique_lookup(rel); + entry = pgcolumnar_unique_lookup(rel); if (entry->nIndexes == 0) return; - numBuckets = (uint32) Max(1, columnar_unique_lock_buckets); + numBuckets = (uint32) Max(1, pgcolumnar_unique_lock_buckets); econtext = GetPerTupleExprContext(entry->estate); saveScanTuple = econtext->ecxt_scantuple; @@ -434,25 +434,25 @@ ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) combined = (combined ^ (uint64) h) * COLUMNAR_FNV_PRIME; } - combined = columnar_hash_finalize(combined); + combined = pgcolumnar_hash_finalize(combined); bucket = (uint32) (combined % (uint64) numBuckets); } MemoryContextSwitchTo(oldcxt); - columnar_acquire_key_lock(uidx->indexOid, bucket); + pgcolumnar_acquire_key_lock(uidx->indexOid, bucket); } econtext->ecxt_scantuple = saveScanTuple; } /* - * ColumnarUniqueInit + * PgColumnarUniqueInit * Register the relcache invalidation callback that keeps the per-relation * unique-index cache coherent across DDL. Called once from _PG_init. */ void -ColumnarUniqueInit(void) +PgColumnarUniqueInit(void) { - CacheRegisterRelcacheCallback(columnar_unique_invalidate, (Datum) 0); + CacheRegisterRelcacheCallback(pgcolumnar_unique_invalidate, (Datum) 0); } diff --git a/src/columnar_vacuum.c b/src/columnar_vacuum.c index 50adb8e..70720b8 100644 --- a/src/columnar_vacuum.c +++ b/src/columnar_vacuum.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_vacuum.c + * pgcolumnar_vacuum.c * Compaction, statistics, and storage-id lookup functions for pgColumnar * (spec 8.2, 9). columnar.vacuum rewrites a columnar table's live rows * into fresh, full stripes: this combines many small stripes into few and @@ -49,30 +49,30 @@ #include "utils/tuplestore.h" #include "utils/typcache.h" -PG_FUNCTION_INFO_V1(columnar_relation_storageid); -PG_FUNCTION_INFO_V1(columnar_vacuum); -PG_FUNCTION_INFO_V1(columnar_vacuum_sorted); -PG_FUNCTION_INFO_V1(columnar_cluster); -PG_FUNCTION_INFO_V1(columnar_compact); -PG_FUNCTION_INFO_V1(columnar_compact_rewrite); -PG_FUNCTION_INFO_V1(columnar_recluster); -PG_FUNCTION_INFO_V1(columnar_truncate); -PG_FUNCTION_INFO_V1(columnar_debug_advance_reserved_offset); -PG_FUNCTION_INFO_V1(columnar_debug_set_metapage_version); +PG_FUNCTION_INFO_V1(pgcolumnar_relation_storageid); +PG_FUNCTION_INFO_V1(pgcolumnar_vacuum); +PG_FUNCTION_INFO_V1(pgcolumnar_vacuum_sorted); +PG_FUNCTION_INFO_V1(pgcolumnar_cluster); +PG_FUNCTION_INFO_V1(pgcolumnar_compact); +PG_FUNCTION_INFO_V1(pgcolumnar_compact_rewrite); +PG_FUNCTION_INFO_V1(pgcolumnar_recluster); +PG_FUNCTION_INFO_V1(pgcolumnar_truncate); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_advance_reserved_offset); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_set_metapage_version); /* physical end-truncation opt-in (GUC), registered in _PG_init. Default off * until the abort/crash path is fully hardened and matrix-validated. */ -bool columnar_enable_end_truncation = false; +bool pgcolumnar_enable_end_truncation = false; /* - * ColumnarRequireTableOwner + * PgColumnarRequireTableOwner * Error unless the current user owns the relation (superusers pass). Every * maintenance and projection-DDL function gates on this: they rewrite data, * reclaim space, or take strong locks (truncate takes AccessExclusiveLock), * so they must be owner-only, like VACUUM and CLUSTER. */ void -ColumnarRequireTableOwner(Relation rel) +PgColumnarRequireTableOwner(Relation rel) { if (!COLUMNAR_TABLE_OWNERCHECK(RelationGetRelid(rel))) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, @@ -106,7 +106,7 @@ uint64_cmp(const void *a, const void *b) * rests on the protocol in design/PHASE_F3B_PLAN.md: the rewrite serializes with * deleters on the per-chunk-group advisory lock, and a delete that races a * rewrite of its group is aborted with a serialization failure by the - * conflict check in ColumnarUpsertDeleteVector. + * conflict check in PgColumnarUpsertDeleteVector. * ------------------------------------------------------------------------- */ /* the relation's ready indexes, opened once for a rewrite pass */ @@ -116,37 +116,37 @@ uint64_cmp(const void *a, const void *b) * on rel and has opened the indexes in ris. */ static int64 -rewrite_one_group(Relation rel, ColumnarIndexInsertState *ris, uint64 storageId, +rewrite_one_group(Relation rel, PgColumnarIndexInsertState *ris, uint64 storageId, uint64 groupNumber, uint64 firstRow, uint64 rowCount) { Oid relid = RelationGetRelid(rel); int natts = RelationGetDescr(rel)->natts; Datum *values = palloc(natts * sizeof(Datum)); bool *isnull = palloc(natts * sizeof(bool)); - ColumnarWriteState *ws; + PgColumnarWriteState *ws; Snapshot snap; - ColumnarReadState *rs; + PgColumnarReadState *rs; uint64 oldRn; int64 moved = 0; - /* serialize with concurrent deleters to this group (see ColumnarUpsertDeleteVector) */ - ColumnarLockChunkGroup(storageId, groupNumber); + /* serialize with concurrent deleters to this group (see PgColumnarUpsertDeleteVector) */ + PgColumnarLockChunkGroup(storageId, groupNumber); /* Read the group's live set under a snapshot taken after the lock, so every * delete committed before the lock is reflected. Register it (not just push * active): the reader derives a catalog snapshot copy from it, in - * ColumnarCatalogSnapshot, that must inherit a nonzero regd_count for PG18's + * PgColumnarCatalogSnapshot, that must inherit a nonzero regd_count for PG18's * heap-visibility assertion, which a merely-active GetLatestSnapshot does not * provide. */ snap = RegisterSnapshot(GetLatestSnapshot()); PushActiveSnapshot(snap); - ws = ColumnarGetWriteState(rel); + ws = PgColumnarGetWriteState(rel); /* * Stream the group rather than fetching its rows one at a time. * - * ColumnarReadRowByNumber decodes the whole group to return one value and + * PgColumnarReadRowByNumber decodes the whole group to return one value and * relies on the fetch cache to make the next call cheap. That cache holds * only what fits under COLUMNAR_FETCH_CACHE_MAX_BYTES, so a group whose * decoded form exceeds the cap re-decodes the columns that did not fit, once @@ -167,27 +167,27 @@ rewrite_one_group(Relation rel, ColumnarIndexInsertState *ris, uint64 storageId, * ANALYZE's sampler was moved off the same per-row fetch for the same * reason. */ - rs = ColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); - ColumnarReadRestrictToGroups(rs, &groupNumber, 1); + rs = PgColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); + PgColumnarReadRestrictToGroups(rs, &groupNumber, 1); - while (ColumnarReadNextRow(rs, values, isnull, &oldRn)) + while (PgColumnarReadNextRow(rs, values, isnull, &oldRn)) { uint64 newRn; CHECK_FOR_INTERRUPTS(); - newRn = ColumnarWriteRow(ws, rel, values, isnull); - ColumnarProjectionFanoutRow(rel, ws, newRn, values, isnull); - ColumnarIndexInsertRow(ris, rel, values, isnull, newRn); + newRn = PgColumnarWriteRow(ws, rel, values, isnull); + PgColumnarProjectionFanoutRow(rel, ws, newRn, values, isnull); + PgColumnarIndexInsertRow(ris, rel, values, isnull, newRn); moved++; } - ColumnarEndRead(rs); - ColumnarFlushWriteStateForRelation(relid); + PgColumnarEndRead(rs); + PgColumnarFlushWriteStateForRelation(relid); /* atomically (same transaction) the new group is now in the catalog; drop the * old one. Heap MVCC keeps the old group readable to older snapshots. */ - ColumnarRetireGroup(storageId, groupNumber); + PgColumnarRetireGroup(storageId, groupNumber); PopActiveSnapshot(); UnregisterSnapshot(snap); @@ -205,35 +205,35 @@ typedef struct RewriteCandidate } RewriteCandidate; /* - * columnar_rewrite_partial_groups + * pgcolumnar_rewrite_partial_groups * Rewrite up to maxGroups groups whose deleted fraction is at least * minDeletedFraction (and which are not fully dead -- F3a handles those). * maxGroups <= 0 means all. Returns the number of groups rewritten. */ static int64 -columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, +pgcolumnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, int maxGroups) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Oid relid = RelationGetRelid(rel); Snapshot snap; List *rgList; ListCell *lc; List *cands = NIL; - ColumnarIndexInsertState *ris; + PgColumnarIndexInsertState *ris; int64 rewritten = 0; /* persist own pending work so the group list and deletes are current */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* drop any free_space row overlapping a live group (the residual of a crash * in end-truncation's narrow window) before reusing anything */ - ColumnarReconcileFreeList(rel); + PgColumnarReconcileFreeList(rel); /* collect candidate groups first (do not mutate the catalog mid-scan) */ snap = RegisterSnapshot(GetLatestSnapshot()); - rgList = ColumnarReadRowGroupList(storageId, snap); + rgList = PgColumnarReadRowGroupList(storageId, snap); foreach(lc, rgList) { NativeRowGroupMetadata *rg = (NativeRowGroupMetadata *) lfirst(lc); @@ -243,7 +243,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, if (rg->rowCount == 0) continue; - rmList = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); + rmList = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); foreach(lc2, rmList) deleted += ((DeleteVectorMetadata *) lfirst(lc2))->deletedCount; @@ -264,7 +264,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, if (cands == NIL) return 0; - ris = ColumnarIndexInsertBegin(rel, false); + ris = PgColumnarIndexInsertBegin(rel, false); foreach(lc, cands) { RewriteCandidate *c = (RewriteCandidate *) lfirst(lc); @@ -275,14 +275,14 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, c->firstRow, c->rowCount); rewritten++; } - ColumnarIndexInsertEnd(ris); + PgColumnarIndexInsertEnd(ris); COLUMNAR_ASSERT_NO_OVERLAP(storageId); return rewritten; } /* - * columnar_compact_rewrite + * pgcolumnar_compact_rewrite * SQL: pgcolumnar.compact_rewrite(tablename regclass, * min_deleted_fraction float8 default 0.2, max_groups int default 0). * The lazy online space-reclaiming path (Phase F3b): rewrite partially @@ -314,7 +314,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, * An earlier version marked only an upper bound and tried to exclude the * below case by requiring the lowest live group to equal ours[0]. That * could not work (#342): the group list was read under the rewrite's own - * snapshot, taken before it read a row, and ColumnarCatalogSnapshot only + * snapshot, taken before it read a row, and PgColumnarCatalogSnapshot only * advances curcid rather than refreshing xmin/xmax, so a concurrent * inserter's group was invisible to the check whenever it committed. A * foreign group written just before the rewrite's first reservation was @@ -327,10 +327,10 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, */ static void record_online_sorted_extent(Relation rel, uint64 storageId, - ColumnarWriteState *writeState, int stripeMark) + PgColumnarWriteState *writeState, int stripeMark) { int nAll; - uint64 *all = ColumnarWriteStateStripeIds(writeState, &nAll); + uint64 *all = PgColumnarWriteStateStripeIds(writeState, &nAll); int nOurs = nAll - stripeMark; uint64 *ours; uint64 runEnd; @@ -354,7 +354,7 @@ record_online_sorted_extent(Relation rel, uint64 storageId, { int nProj = 0; - uint64 *projIds = ColumnarWriteStateProjStripeIds(writeState, &nProj); + uint64 *projIds = PgColumnarWriteStateProjStripeIds(writeState, &nProj); if (nProj > 0) { @@ -387,12 +387,12 @@ record_online_sorted_extent(Relation rel, uint64 storageId, runEnd = ours[i]; } - ColumnarSetSortedExtent(storageId, (int64) ours[0], (int64) runEnd); + PgColumnarSetSortedExtent(storageId, (int64) ours[0], (int64) runEnd); pfree(ours); } /* - * columnar_recluster_online + * pgcolumnar_recluster_online * Re-establish global Z-order clustering over the relation's live rows * online (Phase F3c): read all live rows under a snapshot taken after * advisory-locking every group, Morton-sort them, write them back as fresh @@ -402,9 +402,9 @@ record_online_sorted_extent(Relation rel, uint64 storageId, * conflict protocol. Returns the number of groups retired. */ static int64 -columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) +pgcolumnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); int natts = tupdesc->natts; @@ -425,23 +425,23 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) Oid byteaLt; Oid sortColl = InvalidOid; bool nullsFirst = false; - ColumnarReadState *readState; - ColumnarWriteState *writeState; - ColumnarIndexInsertState *ris; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; + PgColumnarIndexInsertState *ris; uint64 rowNumber; int stripeMark; /* persist own pending work so the group list and deletes are current */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* drop any free_space row overlapping a live group (the residual of a crash * in end-truncation's narrow window) before reusing anything */ - ColumnarReconcileFreeList(rel); + PgColumnarReconcileFreeList(rel); /* capture the current groups (retired at the end, after the new ones exist) */ listSnap = RegisterSnapshot(GetLatestSnapshot()); - rgList = ColumnarReadRowGroupList(storageId, listSnap); + rgList = PgColumnarReadRowGroupList(storageId, listSnap); oldGroups = palloc(sizeof(uint64) * (list_length(rgList) > 0 ? list_length(rgList) : 1)); foreach(lc, rgList) { @@ -463,7 +463,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) /* lock every group in ascending order (deadlock-safe), held to commit */ qsort(oldGroups, nGroups, sizeof(uint64), uint64_cmp); for (i = 0; i < nGroups; i++) - ColumnarLockChunkGroup(storageId, oldGroups[i]); + PgColumnarLockChunkGroup(storageId, oldGroups[i]); /* read all live rows into a Morton-keyed tuplesort (as in eager cluster). * Register the snapshot (not just push active) so the catalog snapshot copies @@ -492,8 +492,8 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) putSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsVirtual); augSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { bytea *zkey; @@ -509,41 +509,41 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) tuplesort_puttupleslot(tsort, putSlot); ExecClearTuple(putSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); ExecDropSingleTupleTableSlot(putSlot); tuplesort_performsort(tsort); /* write the sorted rows back as fresh groups, with online index maintenance */ - ris = ColumnarIndexInsertBegin(rel, false); - writeState = ColumnarGetWriteState(rel); + ris = PgColumnarIndexInsertBegin(rel, false); + writeState = PgColumnarGetWriteState(rel); /* * Note where this rewrite's own stripe reservations begin (#311). The write * state can already hold entries from earlier work in this transaction, so * only the tail from here on belongs to us. */ - stripeMark = ColumnarWriteStateStripeCount(writeState); + stripeMark = PgColumnarWriteStateStripeCount(writeState); while (tuplesort_gettupleslot(tsort, true, false, augSlot, NULL)) { uint64 newRn; CHECK_FOR_INTERRUPTS(); slot_getallattrs(augSlot); - newRn = ColumnarWriteRow(writeState, rel, augSlot->tts_values, + newRn = PgColumnarWriteRow(writeState, rel, augSlot->tts_values, augSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRn, augSlot->tts_values, + PgColumnarProjectionFanoutRow(rel, writeState, newRn, augSlot->tts_values, augSlot->tts_isnull); - ColumnarIndexInsertRow(ris, rel, augSlot->tts_values, + PgColumnarIndexInsertRow(ris, rel, augSlot->tts_values, augSlot->tts_isnull, newRn); } - ColumnarFlushWriteStateForRelation(relid); - ColumnarIndexInsertEnd(ris); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarIndexInsertEnd(ris); tuplesort_end(tsort); ExecDropSingleTupleTableSlot(augSlot); /* retire the old groups; heap MVCC keeps them readable to older snapshots */ for (i = 0; i < nGroups; i++) - ColumnarRetireGroup(storageId, oldGroups[i]); + PgColumnarRetireGroup(storageId, oldGroups[i]); /* record how far the reordered run reaches (#311) */ record_online_sorted_extent(rel, storageId, writeState, stripeMark); @@ -557,7 +557,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) } /* - * columnar_recluster + * pgcolumnar_recluster * SQL: pgcolumnar.recluster(tablename regclass, VARIADIC columns name[]). * The lazy online counterpart to cluster(): re-establish global Z-order * clustering under ShareUpdateExclusiveLock (concurrent reads and writes), @@ -565,7 +565,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) * number of groups reclustered. */ Datum -columnar_recluster(PG_FUNCTION_ARGS) +pgcolumnar_recluster(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); ArrayType *colArray; @@ -602,7 +602,7 @@ columnar_recluster(PG_FUNCTION_ARGS) /* the lazy lock: concurrent reads and writes during the recluster */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -611,7 +611,7 @@ columnar_recluster(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); tupdesc = RelationGetDescr(rel); atts = palloc(ncols * sizeof(AttrNumber)); @@ -654,14 +654,14 @@ columnar_recluster(PG_FUNCTION_ARGS) atts[i] = attno; } - reclustered = columnar_recluster_online(rel, ncols, atts); + reclustered = pgcolumnar_recluster_online(rel, ncols, atts); table_close(rel, NoLock); PG_RETURN_INT64(reclustered); } Datum -columnar_compact_rewrite(PG_FUNCTION_ARGS) +pgcolumnar_compact_rewrite(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); double minFrac = PG_ARGISNULL(1) ? 0.2 : PG_GETARG_FLOAT8(1); @@ -680,7 +680,7 @@ columnar_compact_rewrite(PG_FUNCTION_ARGS) rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -689,9 +689,9 @@ columnar_compact_rewrite(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - rewritten = columnar_rewrite_partial_groups(rel, minFrac, maxGroups); + rewritten = pgcolumnar_rewrite_partial_groups(rel, minFrac, maxGroups); table_close(rel, NoLock); PG_RETURN_INT64(rewritten); @@ -828,13 +828,13 @@ cluster_zorder_key(Datum *values, bool *isnull, AttrNumber *atts, int ncols, } /* - * columnar_relation_storageid + * pgcolumnar_relation_storageid * SQL: columnar.get_storage_id(regclass) -> bigint. Reads the relation's * metapage and returns its storage id (spec 3), so SQL-level functions * such as columnar.stats can join the metadata catalog by storage id. */ Datum -columnar_relation_storageid(PG_FUNCTION_ARGS) +pgcolumnar_relation_storageid(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; @@ -844,7 +844,7 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) if (rel == NULL) PG_RETURN_NULL(); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { relation_close(rel, AccessShareLock); ereport(ERROR, @@ -853,7 +853,7 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - storageId = ColumnarStorageId(rel); + storageId = PgColumnarStorageId(rel); relation_close(rel, AccessShareLock); PG_RETURN_INT64((int64) storageId); @@ -875,15 +875,15 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) static void record_sorted_extent(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); List *groups; uint64 lastGroup = 0; uint64 firstGroup = 0; bool haveGroup = false; ListCell *lc; - groups = ColumnarReadRowGroupList(storageId, - ColumnarCatalogSnapshot(GetActiveSnapshot())); + groups = PgColumnarReadRowGroupList(storageId, + PgColumnarCatalogSnapshot(GetActiveSnapshot())); foreach(lc, groups) { NativeRowGroupMetadata *rg = (NativeRowGroupMetadata *) lfirst(lc); @@ -897,11 +897,11 @@ record_sorted_extent(Relation rel) list_free_deep(groups); if (haveGroup) - ColumnarSetSortedExtent(storageId, (int64) firstGroup, (int64) lastGroup); + PgColumnarSetSortedExtent(storageId, (int64) firstGroup, (int64) lastGroup); } /* - * columnar_compact_relation + * pgcolumnar_compact_relation * Rewrite every live row of a columnar relation into fresh stripes. The * relation is already open with AccessExclusiveLock. * @@ -913,14 +913,14 @@ record_sorted_extent(Relation rel) * auto-maintained, so rows inserted afterward append in insert order. */ static void -columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) +pgcolumnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) { Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); uint64 oldStorageId; Snapshot snapshot; - ColumnarReadState *readState; - ColumnarWriteState *writeState; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; Tuplestorestate *tstore = NULL; Tuplesortstate *tsort = NULL; TupleTableSlot *readSlot; @@ -930,10 +930,10 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) List *oldProjs; /* persist any pending work so the read below sees it (spec 9) */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); - oldStorageId = ColumnarStorageId(rel); + oldStorageId = PgColumnarStorageId(rel); /* * Capture the table's projections (gap 26) before the storage swap. Compaction @@ -941,7 +941,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * compacted base; we re-record the definitions under the new storage id below * and the rewrite loop re-fans-out every live row into them. */ - oldProjs = ColumnarListProjections(oldStorageId); + oldProjs = PgColumnarListProjections(oldStorageId); /* * Take the read snapshot AFTER the AccessExclusiveLock the caller already @@ -950,7 +950,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * pre-lock snapshot's in-progress set, so it would be invisible to the row * enumeration below and silently discarded by the relfilenode swap -- data * loss. A fresh GetLatestSnapshot sees every commit as of now. Register it - * (not merely push active): ColumnarBeginRead derives a ColumnarCatalogSnapshot + * (not merely push active): PgColumnarBeginRead derives a PgColumnarCatalogSnapshot * copy that must inherit a nonzero regd_count for PG18's heap-visibility * assertion, exactly as the sibling rewrite/retire paths document. */ @@ -1002,8 +1002,8 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) readSlot = MakeSingleTupleTableSlot(tupdesc, &TTSOpsVirtual); writeSlot = MakeSingleTupleTableSlot(tupdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { CHECK_FOR_INTERRUPTS(); @@ -1014,7 +1014,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) tuplestore_puttupleslot(tstore, readSlot); ExecClearTuple(readSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); if (tsort != NULL) @@ -1027,8 +1027,8 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * still points at the old storage id) and remove the old metadata rows. */ RelationSetNewRelfilenumber(rel, rel->rd_rel->relpersistence); - ColumnarForgetWriteStateForRelation(relid); - ColumnarDeleteMetadata(oldStorageId); + PgColumnarForgetWriteStateForRelation(relid); + PgColumnarDeleteMetadata(oldStorageId); /* * Realign projections to the compacted base (gap 26): drop each old @@ -1040,31 +1040,31 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) */ if (oldProjs != NIL) { - uint64 newStorageId = ColumnarStorageId(rel); + uint64 newStorageId = PgColumnarStorageId(rel); ListCell *lc; foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != oldStorageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(oldStorageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(oldStorageId, p->projectionId); } foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); - ColumnarProjection np = *p; + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); + PgColumnarProjection np = *p; np.storageId = newStorageId; np.projStorageId = (p->projectionId == 0) ? newStorageId - : ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&np); + : PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&np); } } /* write the live rows back into the fresh storage, in sorted order if any */ - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); if (tsort != NULL) { while (tuplesort_gettupleslot(tsort, true, false, writeSlot, NULL)) @@ -1073,9 +1073,9 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) CHECK_FOR_INTERRUPTS(); slot_getallattrs(writeSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, writeSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, writeSlot->tts_values, writeSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, writeSlot->tts_values, writeSlot->tts_isnull); ExecClearTuple(writeSlot); } @@ -1088,14 +1088,14 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) CHECK_FOR_INTERRUPTS(); slot_getallattrs(writeSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, writeSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, writeSlot->tts_values, writeSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, writeSlot->tts_values, writeSlot->tts_isnull); ExecClearTuple(writeSlot); } } - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); /* * A sorted rewrite leaves the whole relation ordered, so record its extent. @@ -1117,22 +1117,22 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * their synthetic item pointers (spec 6). A relation with no indexes is a * no-op here. */ - ColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); + PgColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); PopActiveSnapshot(); UnregisterSnapshot(snapshot); } /* - * columnar_compact_relation_zorder + * pgcolumnar_compact_relation_zorder * Rewrite every live row of a columnar relation ordered by the Z-order * (Morton) code over atts[0..ncols-1] (Phase F2). Mirrors - * columnar_compact_relation, but sorts by a computed key carried as a + * pgcolumnar_compact_relation, but sorts by a computed key carried as a * trailing bytea column of an augmented tuple, so the sort still spills to * disk through tuplesort. The relation is already open AccessExclusiveLock. */ static void -columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) +pgcolumnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) { Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1140,8 +1140,8 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) AttrNumber zAtt = (AttrNumber) (natts + 1); uint64 oldStorageId; Snapshot snapshot; - ColumnarReadState *readState; - ColumnarWriteState *writeState; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; Tuplesortstate *tsort; TupleDesc augdesc; TupleTableSlot *readSlot; @@ -1156,15 +1156,15 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) int i; /* persist pending work so the read below sees it (spec 9) */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); - oldStorageId = ColumnarStorageId(rel); - oldProjs = ColumnarListProjections(oldStorageId); + oldStorageId = PgColumnarStorageId(rel); + oldProjs = PgColumnarListProjections(oldStorageId); /* fresh snapshot AFTER the AccessExclusiveLock, not the caller's pre-lock one, * so a row group committed during the lock wait is copied rather than dropped * by the swap (#295); registered for PG18's catalog-snapshot regd_count. Same - * reasoning as columnar_compact_relation. */ + * reasoning as pgcolumnar_compact_relation. */ snapshot = RegisterSnapshot(GetLatestSnapshot()); PushActiveSnapshot(snapshot); @@ -1190,8 +1190,8 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) putSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsVirtual); augSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { bytea *zkey; @@ -1207,57 +1207,57 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) tuplesort_puttupleslot(tsort, putSlot); ExecClearTuple(putSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); ExecDropSingleTupleTableSlot(putSlot); tuplesort_performsort(tsort); /* swap to fresh storage and drop old metadata (as in compact_relation) */ RelationSetNewRelfilenumber(rel, rel->rd_rel->relpersistence); - ColumnarForgetWriteStateForRelation(relid); - ColumnarDeleteMetadata(oldStorageId); + PgColumnarForgetWriteStateForRelation(relid); + PgColumnarDeleteMetadata(oldStorageId); /* realign projections to the compacted base (as in compact_relation) */ if (oldProjs != NIL) { - uint64 newStorageId = ColumnarStorageId(rel); + uint64 newStorageId = PgColumnarStorageId(rel); ListCell *lc; foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != oldStorageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(oldStorageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(oldStorageId, p->projectionId); } foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); - ColumnarProjection np = *p; + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); + PgColumnarProjection np = *p; np.storageId = newStorageId; np.projStorageId = (p->projectionId == 0) ? newStorageId - : ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&np); + : PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&np); } } /* write the live rows back in Z-order; the trailing key column is ignored */ - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); while (tuplesort_gettupleslot(tsort, true, false, augSlot, NULL)) { uint64 newRowNumber; CHECK_FOR_INTERRUPTS(); slot_getallattrs(augSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, augSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, augSlot->tts_values, augSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, augSlot->tts_values, augSlot->tts_isnull); ExecClearTuple(augSlot); } - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); /* Z-order is an order, so the same extent applies (see record_sorted_extent). */ record_sorted_extent(rel); @@ -1265,14 +1265,14 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) tuplesort_end(tsort); ExecDropSingleTupleTableSlot(augSlot); - ColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); + PgColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); PopActiveSnapshot(); UnregisterSnapshot(snapshot); } /* - * columnar_vacuum + * pgcolumnar_vacuum * SQL: columnar.vacuum(tablename regclass, stripe_count int default 0). * Compacts a columnar table by combining its stripes and reclaiming the * space of deleted rows (spec 8.2, 9). stripe_count is accepted for @@ -1281,14 +1281,14 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) * recent stripes" contract. */ Datum -columnar_vacuum(PG_FUNCTION_ARGS) +pgcolumnar_vacuum(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1297,9 +1297,9 @@ columnar_vacuum(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - columnar_compact_relation(rel, 0, NULL); + pgcolumnar_compact_relation(rel, 0, NULL); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1308,7 +1308,7 @@ columnar_vacuum(PG_FUNCTION_ARGS) } /* - * columnar_vacuum_sorted + * pgcolumnar_vacuum_sorted * SQL: columnar.vacuum_sorted(tablename regclass, VARIADIC sort_columns name[]). * Like columnar.vacuum, but rewrites the live rows physically sorted * ascending / NULLS LAST on the named columns, in order (gap 26, piece 1). @@ -1324,7 +1324,7 @@ columnar_vacuum(PG_FUNCTION_ARGS) * cluster() path is numeric-only. */ Datum -columnar_vacuum_sorted(PG_FUNCTION_ARGS) +pgcolumnar_vacuum_sorted(PG_FUNCTION_ARGS) { Oid relid; Relation rel; @@ -1344,7 +1344,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1353,7 +1353,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); /* * Collect the sort-column names. Explicit columns win; when none are given @@ -1387,7 +1387,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) if (colNames == NIL) { - colNames = ColumnarReadSortBy(relid); + colNames = PgColumnarReadSortBy(relid); fromPersisted = true; } @@ -1452,7 +1452,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) sortAtts[i++] = attno; } - columnar_compact_relation(rel, ncols, sortAtts); + pgcolumnar_compact_relation(rel, ncols, sortAtts); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1461,7 +1461,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) } /* - * columnar_cluster + * pgcolumnar_cluster * SQL: pgcolumnar.cluster(tablename regclass, VARIADIC columns name[]). * Physically reorders a columnar table by the Z-order (Morton) space-filling * curve over the named columns (Phase F2, spec 9). Unlike vacuum_sorted's @@ -1479,7 +1479,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) * path. */ Datum -columnar_cluster(PG_FUNCTION_ARGS) +pgcolumnar_cluster(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); ArrayType *colArray; @@ -1514,7 +1514,7 @@ columnar_cluster(PG_FUNCTION_ARGS) rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1523,7 +1523,7 @@ columnar_cluster(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); tupdesc = RelationGetDescr(rel); atts = palloc(ncols * sizeof(AttrNumber)); @@ -1569,7 +1569,7 @@ columnar_cluster(PG_FUNCTION_ARGS) atts[i] = attno; } - columnar_compact_relation_zorder(rel, ncols, atts); + pgcolumnar_compact_relation_zorder(rel, ncols, atts); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1578,7 +1578,7 @@ columnar_cluster(PG_FUNCTION_ARGS) } /* - * columnar_compact + * pgcolumnar_compact * SQL: pgcolumnar.compact(tablename regclass) -> bigint. The LAZY / online * maintenance path (Phase F3a): retire every row group that is fully deleted * as-of the oldest-xmin horizon, dropping its catalog rows so scans no longer @@ -1590,7 +1590,7 @@ columnar_cluster(PG_FUNCTION_ARGS) * Phase F3b. */ Datum -columnar_compact(PG_FUNCTION_ARGS) +pgcolumnar_compact(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; @@ -1604,7 +1604,7 @@ columnar_compact(PG_FUNCTION_ARGS) /* the lazy lock: concurrent reads and writes are allowed during compaction */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -1613,15 +1613,15 @@ columnar_compact(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); /* self-heal a truncate crash-residual so the no-overlap assert holds eagerly, - * even though compact does not reuse (see ColumnarReconcileFreeList) */ - ColumnarReconcileFreeList(rel); + * even though compact does not reuse (see PgColumnarReconcileFreeList) */ + PgColumnarReconcileFreeList(rel); - retired = ColumnarRetireFullyDeletedGroups(rel); + retired = PgColumnarRetireFullyDeletedGroups(rel); - COLUMNAR_ASSERT_NO_OVERLAP(ColumnarStorageId(rel)); + COLUMNAR_ASSERT_NO_OVERLAP(PgColumnarStorageId(rel)); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1630,16 +1630,16 @@ columnar_compact(PG_FUNCTION_ARGS) } /* - * columnar_end_truncation_storages + * pgcolumnar_end_truncation_storages * Collect the distinct storage ids that share this relation's file: the base * storage plus every projection's own storage. Returns a list of palloc'd * uint64. All must be considered when computing the safe truncation point, * because they all place data in the one shared file. */ static List * -columnar_end_truncation_storages(uint64 base) +pgcolumnar_end_truncation_storages(uint64 base) { - List *projs = ColumnarListProjections(base); + List *projs = PgColumnarListProjections(base); List *result = NIL; ListCell *lc; uint64 *b = palloc(sizeof(uint64)); @@ -1649,7 +1649,7 @@ columnar_end_truncation_storages(uint64 base) foreach(lc, projs) { - ColumnarProjection *pr = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *pr = (PgColumnarProjection *) lfirst(lc); ListCell *rc; bool dup = false; @@ -1671,7 +1671,7 @@ columnar_end_truncation_storages(uint64 base) } /* - * columnar_do_end_truncation + * pgcolumnar_do_end_truncation * Compute the safe truncation point and, if the trailing region is entirely * reclaimable, physically shrink the main fork. The caller holds * AccessExclusiveLock, so no reader or writer is concurrent. Returns the @@ -1691,7 +1691,7 @@ columnar_end_truncation_storages(uint64 base) * window (after the truncate, before the highwater is lowered) leaves * "highwater still high + free_space restored + file short", which the * gap-tolerant write path self-heals. The function also runs outside a - * transaction block (see columnar_truncate), so a user ROLLBACK cannot land + * transaction block (see pgcolumnar_truncate), so a user ROLLBACK cannot land * in the residual window between lowering the highwater and commit; and it * first purges any free_space row at or above the current highwater, which * under the exclusive lock is stale by definition and would otherwise be the @@ -1699,20 +1699,20 @@ columnar_end_truncation_storages(uint64 base) * live groups, none of which are in the truncated region. */ static int64 -columnar_do_end_truncation(Relation rel) +pgcolumnar_do_end_truncation(Relation rel) { - uint64 base = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *storages = columnar_end_truncation_storages(base); + uint64 base = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *storages = pgcolumnar_end_truncation_storages(base); ListCell *lc; uint64 liveEnd = COLUMNAR_FIRST_LOGICAL_OFFSET; - ColumnarMetapage meta; + PgColumnarMetapage meta; uint64 highwater; Snapshot snap; BlockNumber oldnblocks; BlockNumber truncBlock; - ColumnarReadMetapage(rel, &meta); + PgColumnarReadMetapage(rel, &meta); highwater = meta.reservedOffset; /* @@ -1725,15 +1725,15 @@ columnar_do_end_truncation(Relation rel) * the end-of-run no-overlap assert valid. */ foreach(lc, storages) - ColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), highwater); - ColumnarReconcileFreeList(rel); + PgColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), highwater); + PgColumnarReconcileFreeList(rel); /* highest live-data end across all storages, in the latest committed state */ snap = RegisterSnapshot(GetLatestSnapshot()); foreach(lc, storages) { uint64 sid = *(uint64 *) lfirst(lc); - List *rgs = ColumnarReadRowGroupList(sid, snap); + List *rgs = PgColumnarReadRowGroupList(sid, snap); ListCell *g; foreach(g, rgs) @@ -1756,16 +1756,16 @@ columnar_do_end_truncation(Relation rel) /* the trailing region must be entirely behind the oldest-xmin horizon */ foreach(lc, storages) - if (!ColumnarTrailingFreeSpaceSafe(*(uint64 *) lfirst(lc), liveEnd, + if (!PgColumnarTrailingFreeSpaceSafe(*(uint64 *) lfirst(lc), liveEnd, oldestXmin)) return 0; /* a recent retirement is in the tail; retry later */ /* drop the trailing free ranges, shrink the file, THEN lower the highwater */ foreach(lc, storages) - ColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), liveEnd); + PgColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), liveEnd); CommandCounterIncrement(); - ColumnarTruncateMainFork(rel, truncBlock); - ColumnarSetReservedOffset(rel, liveEnd); + PgColumnarTruncateMainFork(rel, truncBlock); + PgColumnarSetReservedOffset(rel, liveEnd); /* stale offset-keyed cache entries for this relation must go */ CacheInvalidateRelcacheByRelid(RelationGetRelid(rel)); @@ -1775,7 +1775,7 @@ columnar_do_end_truncation(Relation rel) } /* - * columnar_truncate + * pgcolumnar_truncate * SQL: pgcolumnar.truncate(regclass) -> bigint (blocks returned to the OS). * Physically shrinks a columnar table's file by dropping trailing blocks that * reclaim has freed. Opt-in (gated by pgcolumnar.enable_end_truncation) and @@ -1784,7 +1784,7 @@ columnar_do_end_truncation(Relation rel) * 0 without waiting if the table is busy, so it never blocks concurrent load. */ Datum -columnar_truncate(PG_FUNCTION_ARGS) +pgcolumnar_truncate(PG_FUNCTION_ARGS) { Oid relid; Relation rel; @@ -1813,7 +1813,7 @@ columnar_truncate(PG_FUNCTION_ARGS) /* serialize with other lazy maintenance (compact/recluster also take SUEL) */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -1822,11 +1822,11 @@ columnar_truncate(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - if (columnar_enable_end_truncation && + if (pgcolumnar_enable_end_truncation && ConditionalLockRelation(rel, AccessExclusiveLock)) - result = columnar_do_end_truncation(rel); + result = pgcolumnar_do_end_truncation(rel); /* keep the locks until end of transaction */ table_close(rel, NoLock); @@ -1834,14 +1834,14 @@ columnar_truncate(PG_FUNCTION_ARGS) } /* - * columnar_debug_advance_reserved_offset + * pgcolumnar_debug_advance_reserved_offset * SQL test hook: advance a columnar table's write highwater by N pages * without writing data, leaving a gap between the physical EOF and the * highwater so the next write exercises the gap-tolerant path. Not bound in * the shipped catalog; the gap test creates the SQL binding itself. */ Datum -columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) +pgcolumnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); int32 npages = PG_GETARG_INT32(1); @@ -1853,7 +1853,7 @@ columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) errmsg("npages must be non-negative"))); rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1862,21 +1862,21 @@ columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarAdvanceReservedOffset(rel, (uint64) npages * COLUMNAR_BYTES_PER_PAGE); + PgColumnarAdvanceReservedOffset(rel, (uint64) npages * COLUMNAR_BYTES_PER_PAGE); table_close(rel, NoLock); PG_RETURN_VOID(); } /* - * columnar_debug_set_metapage_version + * pgcolumnar_debug_set_metapage_version * Test hook: overwrite a columnar table's stored metapage format version so * a subsequent read exercises the unsupported-version rejection in - * ColumnarReadMetapage. Not bound in the shipped catalog; the format suite + * PgColumnarReadMetapage. Not bound in the shipped catalog; the format suite * creates the binding when it needs it (like the advance helper above). */ Datum -columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) +pgcolumnar_debug_set_metapage_version(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); int32 major = PG_GETARG_INT32(1); @@ -1884,7 +1884,7 @@ columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) Relation rel; rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1893,7 +1893,7 @@ columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarDebugSetMetapageVersion(rel, (uint32) major, (uint32) minor); + PgColumnarDebugSetMetapageVersion(rel, (uint32) major, (uint32) minor); table_close(rel, NoLock); PG_RETURN_VOID(); diff --git a/src/columnar_vector.c b/src/columnar_vector.c index 13d5f7f..eb81d38 100644 --- a/src/columnar_vector.c +++ b/src/columnar_vector.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_vector.c + * pgcolumnar_vector.c * Vectorized execution for pgColumnar (spec 9): a column-at-a-time filter * and vectorized aggregates over decoded chunk-group arrays. * @@ -23,8 +23,8 @@ * tests can assert that equality. * * The aggregate custom scan reuses the same registered CustomScanMethods as the - * base custom scan (so both show as "Custom Scan (ColumnarScan)"); the shared - * create-state callback in columnar_customscan.c dispatches to the aggregate + * base custom scan (so both show as "Custom Scan (PgColumnarScan)"); the shared + * create-state callback in pgcolumnar_customscan.c dispatches to the aggregate * variant when the plan is a scanrelid==0 upper node. * * Independent MIT implementation built from @@ -82,17 +82,17 @@ #include "utils/typcache.h" /* GUC: use the vectorized aggregate path (spec 8.3 scan control) */ -bool columnar_enable_vectorization = true; +bool pgcolumnar_enable_vectorization = true; /* * GUC: extend the vectorized aggregate to GROUP BY (#289). Default off while the * grouped path is built out incrementally; the ungrouped path is unaffected. * groupagg_max_groups caps the ACTUAL group count at execution time; exceeding - * it raises an error (see columnar_groupagg_lookup) rather than routing to core + * it raises an error (see pgcolumnar_groupagg_lookup) rather than routing to core * HashAgg. It is not a plan-time gate. */ -bool columnar_enable_group_vectorization = false; -int columnar_groupagg_max_groups = 1000000; +bool pgcolumnar_enable_group_vectorization = false; +int pgcolumnar_groupagg_max_groups = 1000000; /* * GUC: extend the ungrouped vectorized aggregate to the shapes a zone map cannot @@ -101,7 +101,7 @@ int columnar_groupagg_max_groups = 1000000; * over the columnar scan instead. Default off while the path is proven and * benchmarked; the metadata-answerable no-filter path is unaffected. */ -bool columnar_enable_ungrouped_vector_agg = false; +bool pgcolumnar_enable_ungrouped_vector_agg = false; /* * GUC: make the ungrouped vectorized batch fold parallel-aware (#289 phase 5/6). @@ -112,7 +112,7 @@ bool columnar_enable_ungrouped_vector_agg = false; * the batch-eligible count/sum/avg-float shapes take the parallel arm; everything * else keeps the serial node or the ordinary parallel core Agg. */ -bool columnar_enable_parallel_vector_agg = false; +bool pgcolumnar_enable_parallel_vector_agg = false; /* ------------------------------------------------------------------------- * shared column-at-a-time filter @@ -124,18 +124,18 @@ bool columnar_enable_parallel_vector_agg = false; * an array of these. The machinery that evaluated them over a decoded chunk * group had no call site and is deleted (issue #200). */ -typedef struct ColumnarVecPredicate +typedef struct PgColumnarVecPredicate { int attidx; /* 0-based column index */ bool varOnLeft; /* column op const, else const op column */ FmgrInfo opFn; /* the operator function (returns bool) */ Datum constValue; Oid collation; -} ColumnarVecPredicate; +} PgColumnarVecPredicate; /* - * columnar_clause_to_predicate + * pgcolumnar_clause_to_predicate * Turn one "column op const" (or "const op column") clause into a predicate * we can evaluate row by row. Requires a strict boolean operator and a * non-null constant, so that a null column value or a failed comparison @@ -143,8 +143,8 @@ typedef struct ColumnarVecPredicate * other clause. */ static bool -columnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, - ColumnarVecPredicate *pred) +pgcolumnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, + PgColumnarVecPredicate *pred) { OpExpr *op; Node *leftop; @@ -204,7 +204,7 @@ columnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, } static void -ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, +PgColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, int *nconvertible, bool *allConvertible) { ListCell *lc; @@ -217,9 +217,9 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, foreach(lc, qual) { - ColumnarVecPredicate scratch; + PgColumnarVecPredicate scratch; - if (columnar_clause_to_predicate((Node *) lfirst(lc), scanrelid, tupdesc, + if (pgcolumnar_clause_to_predicate((Node *) lfirst(lc), scanrelid, tupdesc, &scratch)) n++; else @@ -236,7 +236,7 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, * in the tree; both are deleted (issue #200). Recoverable from history if a * filtered aggregate path is ever built. What should not be recovered with them is their gating, which * was none: the scalar path's predicates are gated on - * pgcolumnar.enable_qual_pushdown inside ColumnarBeginRead and these never were, + * pgcolumnar.enable_qual_pushdown inside PgColumnarBeginRead and these never were, * so wiring them up as they stood would have filtered rows while EXPLAIN * reported no pushdown at all. */ @@ -246,7 +246,7 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, * vectorized aggregate: classification * ------------------------------------------------------------------------- */ -typedef enum ColumnarAggKind +typedef enum PgColumnarAggKind { COLUMNAR_AGG_COUNT_STAR, COLUMNAR_AGG_COUNT_COL, @@ -256,7 +256,7 @@ typedef enum ColumnarAggKind COLUMNAR_AGG_MAX, /* * Extended kinds used by the grouped path and the ungrouped scan-fold path - * (#289). The metadata-fold path (columnar_fill_native_metadata_agg) still + * (#289). The metadata-fold path (pgcolumnar_fill_native_metadata_agg) still * never produces these: a query using one is routed to scan-fold instead. */ COLUMNAR_AGG_SUM_INT8, /* sum(int8) -> numeric */ @@ -265,11 +265,11 @@ typedef enum ColumnarAggKind COLUMNAR_AGG_AVG_INT8, /* avg(int8) -> numeric */ COLUMNAR_AGG_AVG_FLOAT, /* avg(float4/float8) -> float8 */ COLUMNAR_AGG_AVG_NUMERIC /* avg(numeric) -> numeric */ -} ColumnarAggKind; +} PgColumnarAggKind; -typedef struct ColumnarAggSpec +typedef struct PgColumnarAggSpec { - ColumnarAggKind kind; + PgColumnarAggKind kind; int attidx; /* 0-based column, or -1 for count(*) */ Oid inputType; /* column type (min/max/sum/avg) */ @@ -288,10 +288,10 @@ typedef struct ColumnarAggSpec float8 fsxx; /* avg(float): Youngs-Cramer Sxx, for overflow parity */ Datum nsum; /* numeric running total (in resultContext) */ bool nsumSet; /* nsum initialized */ -} ColumnarAggSpec; +} PgColumnarAggSpec; /* - * columnar_classify_aggref + * pgcolumnar_classify_aggref * Decide whether an Aggref is one we can compute vectorized, and if so fill * its spec. expectedVarno is the scan relation's range-table index at plan * time (to check the argument Var), or a negative value at execution time @@ -299,8 +299,8 @@ typedef struct ColumnarAggSpec * scalar fallback. */ static bool -columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, - bool allowPartial, ColumnarAggSpec *spec) +pgcolumnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, + bool allowPartial, PgColumnarAggSpec *spec) { char *name; Oid nsp; @@ -427,7 +427,7 @@ columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, } /* - * columnar_group_key_unsupported_walker + * pgcolumnar_group_key_unsupported_walker * Reject any node in a candidate GROUP BY key the grouped path cannot * evaluate against a bare base-relation slot: aggregates, grouping-set * constructs, window functions, sublinks/subplans, external parameters, @@ -437,7 +437,7 @@ columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, * to the volatility and varno checks the caller also applies. */ static bool -columnar_group_key_unsupported_walker(Node *node, void *context) +pgcolumnar_group_key_unsupported_walker(Node *node, void *context) { if (node == NULL) return false; @@ -465,12 +465,12 @@ columnar_group_key_unsupported_walker(Node *node, void *context) break; } - return expression_tree_walker(node, columnar_group_key_unsupported_walker, + return expression_tree_walker(node, pgcolumnar_group_key_unsupported_walker, context); } /* - * columnar_classify_group_keys + * pgcolumnar_classify_group_keys * Decide whether every GROUP BY key can be computed and grouped by the * grouped vectorized path, and if so return copies of the key expressions * (original varnos, one per grouping column). A key must be computable from @@ -481,7 +481,7 @@ columnar_group_key_unsupported_walker(Node *node, void *context) * false (add no path, run the ordinary Agg) on anything unsupported. */ static bool -columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, +pgcolumnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, List **keysOut) { Query *parse = root->parse; @@ -512,7 +512,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, return false; if (expression_returns_set(expr)) return false; - if (columnar_group_key_unsupported_walker(expr, NULL)) + if (pgcolumnar_group_key_unsupported_walker(expr, NULL)) return false; type = exprType(expr); @@ -524,7 +524,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, return false; if (!OidIsValid(tce->eq_opr_finfo.fn_oid)) return false; - if (OidIsValid(coll) && !ColumnarCollationIsDeterministic(coll)) + if (OidIsValid(coll) && !PgColumnarCollationIsDeterministic(coll)) return false; keys = lappend(keys, copyObject(expr)); @@ -540,7 +540,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, * vectorized aggregate: executor state * ------------------------------------------------------------------------- */ -typedef struct ColumnarAggScanState +typedef struct PgColumnarAggScanState { CustomScanState css; @@ -548,7 +548,7 @@ typedef struct ColumnarAggScanState List *quals; /* restriction clauses (original varnos) */ Index scanrelid; /* their range-table index */ - ColumnarAggSpec *specs; + PgColumnarAggSpec *specs; int naggs; int npreds; /* pushed-down predicate count, for EXPLAIN */ @@ -593,16 +593,16 @@ typedef struct ColumnarAggScanState uint64 groupsRead; uint64 groupsSkipped; uint64 groupsTotal; -} ColumnarAggScanState; +} PgColumnarAggScanState; -static const CustomExecMethods columnar_agg_exec_methods; -static const CustomExecMethods columnar_agg_parallel_exec_methods; +static const CustomExecMethods pgcolumnar_agg_exec_methods; +static const CustomExecMethods pgcolumnar_agg_parallel_exec_methods; /* ------------------------------------------------------------------------- * grouped vectorized aggregate (#289): executor state * * Fires for SELECT , agg(col) ... [WHERE ...] GROUP BY over a - * single columnar relation. The reader (ColumnarReadNextRow) applies WHERE + * single columnar relation. The reader (PgColumnarReadNextRow) applies WHERE * pushdown for group/vector skipping; each surviving row is rechecked against * the full WHERE, its group keys are evaluated, and it is scattered into an * open-addressing hash table whose per-group accumulators fold in scan order -- @@ -611,7 +611,7 @@ static const CustomExecMethods columnar_agg_parallel_exec_methods; * scale, and deterministic-collation text all group exactly as core does. * ------------------------------------------------------------------------- */ -typedef struct ColumnarGroupKey +typedef struct PgColumnarGroupKey { Expr *expr; /* key expression (original varnos) */ ExprState *exprState; /* evaluates it against the base slot */ @@ -621,18 +621,18 @@ typedef struct ColumnarGroupKey bool byval; FmgrInfo hashFn; /* type hash function */ FmgrInfo eqFn; /* type equality operator function */ -} ColumnarGroupKey; +} PgColumnarGroupKey; -typedef struct ColumnarGroupEntry +typedef struct PgColumnarGroupEntry { uint32 hash; bool used; Datum *keys; /* nkeys key values, in keyContext */ bool *keyNulls; /* nkeys null flags */ - ColumnarAggSpec *specs; /* naggs accumulators, in specContext */ -} ColumnarGroupEntry; + PgColumnarAggSpec *specs; /* naggs accumulators, in specContext */ +} PgColumnarGroupEntry; -typedef struct ColumnarGroupAggScanState +typedef struct PgColumnarGroupAggScanState { CustomScanState css; @@ -641,10 +641,10 @@ typedef struct ColumnarGroupAggScanState Index scanrelid; /* their range-table index */ int nkeys; - ColumnarGroupKey *keys; + PgColumnarGroupKey *keys; int naggs; - ColumnarAggSpec *aggTemplate; /* classified once; copied per new group */ + PgColumnarAggSpec *aggTemplate; /* classified once; copied per new group */ int nout; /* output tuple width */ int *outMap; /* per output pos: >=0 key index, else agg -(v)-1 */ @@ -653,10 +653,10 @@ typedef struct ColumnarGroupAggScanState TupleTableSlot *baseSlot; /* holds each read row for key/qual eval */ ExprState *whereState; /* residual WHERE recheck, or NULL */ - ColumnarGroupEntry *entries; /* open-addressing table (power-of-two) */ + PgColumnarGroupEntry *entries; /* open-addressing table (power-of-two) */ int capacity; int nGroups; - int maxGroups; /* GUC cap enforced at execution (columnar_groupagg_lookup) */ + int maxGroups; /* GUC cap enforced at execution (pgcolumnar_groupagg_lookup) */ MemoryContext keyContext; /* copied key Datums */ MemoryContext specContext; /* per-group specs + running min/max/numeric */ @@ -686,13 +686,13 @@ typedef struct ColumnarGroupAggScanState uint64 groupsRead; uint64 groupsSkipped; uint64 groupsTotal; -} ColumnarGroupAggScanState; +} PgColumnarGroupAggScanState; -static const CustomExecMethods columnar_groupagg_exec_methods; -static const CustomExecMethods columnar_groupagg_parallel_exec_methods; -static void ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, +static const CustomExecMethods pgcolumnar_groupagg_exec_methods; +static const CustomExecMethods pgcolumnar_groupagg_parallel_exec_methods; +static void PgColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra); -static bool columnar_batch_shape_eligible(ColumnarAggScanState *state, +static bool pgcolumnar_batch_shape_eligible(PgColumnarAggScanState *state, TupleDesc tupdesc, ScanKey *keysOut, int *nkeysOut); @@ -701,7 +701,7 @@ static bool columnar_batch_shape_eligible(ColumnarAggScanState *state, * ------------------------------------------------------------------------- */ static Plan * -ColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, +PgColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans) { CustomScan *cscan = makeNode(CustomScan); @@ -714,29 +714,29 @@ ColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, cscan->custom_exprs = NIL; cscan->custom_private = best_path->custom_private; cscan->custom_scan_tlist = tlist; /* defines the output tuple shape */ - cscan->methods = &columnar_scan_methods; /* shared registered methods */ + cscan->methods = &pgcolumnar_scan_methods; /* shared registered methods */ return &cscan->scan.plan; } -static const CustomPathMethods columnar_agg_path_methods = { +static const CustomPathMethods pgcolumnar_agg_path_methods = { .CustomName = "ColumnarAgg", - .PlanCustomPath = ColumnarPlanAggPath, + .PlanCustomPath = PgColumnarPlanAggPath, .ReparameterizeCustomPathByChild = NULL, }; static create_upper_paths_hook_type prev_create_upper_paths_hook = NULL; /* - * columnar_agg_metadata_answerable + * pgcolumnar_agg_metadata_answerable * True when this aggregate kind is answerable from whole-chunk zone maps * with no data scan: count, count(col), sum/avg over int2/int4 (the zone * stores an int sum), min and max. The extended kinds (sum/avg over * int8/float/numeric) are not, so a query using one must scan and fold. See - * columnar_fill_native_metadata_agg. + * pgcolumnar_fill_native_metadata_agg. */ static bool -columnar_agg_metadata_answerable(ColumnarAggKind kind) +pgcolumnar_agg_metadata_answerable(PgColumnarAggKind kind) { switch (kind) { @@ -759,7 +759,7 @@ columnar_agg_metadata_answerable(ColumnarAggKind kind) } /* - * columnar_parallel_agg_ok + * pgcolumnar_parallel_agg_ok * Kinds whose transition state is a plain, non-internal value the batch fold * already holds and a core Finalize can combine (#289 phase 5/6): count(*), * count(col), and sum/avg over int2/int4/float4/float8. The transition types: @@ -769,7 +769,7 @@ columnar_agg_metadata_answerable(ColumnarAggKind kind) * and stay on the serial node or the ordinary core Agg. */ static bool -columnar_parallel_agg_ok(ColumnarAggKind kind) +pgcolumnar_parallel_agg_ok(PgColumnarAggKind kind) { switch (kind) { @@ -786,15 +786,15 @@ columnar_parallel_agg_ok(ColumnarAggKind kind) } /* - * ColumnarCreateUpperPaths - * create_upper_paths_hook: for a plain SELECT agg(col) FROM columnar_table + * PgColumnarCreateUpperPaths + * create_upper_paths_hook: for a plain SELECT agg(col) FROM pgcolumnar_table * [WHERE simple quals] with no grouping or HAVING, add a custom path that * computes the aggregates vectorized. Every aggregate, column type and * filter clause must be fully supported, or we add nothing and the ordinary * Agg plan runs, so results are never at risk. */ static void -ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, +PgColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra) { @@ -805,7 +805,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, ListCell *lc; int naggs; int i; - ColumnarAggSpec *specs; + PgColumnarAggSpec *specs; List *quals; int npreds; bool allConvertible; @@ -819,7 +819,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (stage != UPPERREL_GROUP_AGG) return; - if (!columnar_enable_vectorization || !columnar_enable_custom_scan) + if (!pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) return; if (!parse->hasAggs) @@ -835,14 +835,14 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, parse->havingQual != NULL || parse->distinctClause != NIL || parse->hasWindowFuncs || parse->hasTargetSRFs) { - if (columnar_enable_group_vectorization && + if (pgcolumnar_enable_group_vectorization && parse->groupClause != NIL && parse->groupingSets == NIL && parse->havingQual == NULL && parse->distinctClause == NIL && !parse->hasWindowFuncs && !parse->hasTargetSRFs) - ColumnarTryGroupAggPath(root, input_rel, output_rel, extra); + PgColumnarTryGroupAggPath(root, input_rel, output_rel, extra); return; } @@ -860,7 +860,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (rte == NULL || rte->rtekind != RTE_RELATION || rte->relkind != RELKIND_RELATION) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; relid = rte->relid; @@ -868,7 +868,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, naggs = list_length(tlist); if (naggs == 0) return; - specs = (ColumnarAggSpec *) palloc0(sizeof(ColumnarAggSpec) * naggs); + specs = (PgColumnarAggSpec *) palloc0(sizeof(PgColumnarAggSpec) * naggs); i = 0; foreach(lc, tlist) { @@ -876,7 +876,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (!IsA(expr, Aggref)) return; - if (!columnar_classify_aggref((Aggref *) expr, (int) input_rel->relid, + if (!pgcolumnar_classify_aggref((Aggref *) expr, (int) input_rel->relid, true, false, &specs[i])) return; i++; @@ -893,7 +893,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, needsScan = (quals != NIL); for (i = 0; i < naggs; i++) - if (!columnar_agg_metadata_answerable(specs[i].kind)) + if (!pgcolumnar_agg_metadata_answerable(specs[i].kind)) needsScan = true; if (needsScan) @@ -907,7 +907,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, Bitmapset *whereAtts = NULL; int m = -1; - if (!columnar_enable_ungrouped_vector_agg) + if (!pgcolumnar_enable_ungrouped_vector_agg) return; /* @@ -937,7 +937,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, Relation rel = table_open(relid, AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); - ColumnarCountConvertibleQuals(quals, input_rel->relid, tupdesc, + PgColumnarCountConvertibleQuals(quals, input_rel->relid, tupdesc, &npreds, &allConvertible); table_close(rel, AccessShareLock); if (!allConvertible) @@ -1007,8 +1007,8 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, } else { - ColumnarOptions opts; - int limit = columnar_stripe_row_limit; + PgColumnarOptions opts; + int limit = pgcolumnar_stripe_row_limit; double rows = input_rel->tuples; double ngroups; double dirtyFraction = 0.0; @@ -1024,7 +1024,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * model getting a right answer, and it stops being harmless as soon as the * clamp is not what decides. */ - if (ColumnarReadOptions(relid, &opts) && + if (PgColumnarReadOptions(relid, &opts) && opts.stripeRowLimitSet && opts.stripeRowLimit > 0) limit = opts.stripeRowLimit; if (limit <= 0) @@ -1057,8 +1057,8 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, { Relation frel = table_open(relid, AccessShareLock); - if (ColumnarStorageHasDeleteVector(ColumnarStorageId(frel), - ColumnarCatalogSnapshot(snap))) + if (PgColumnarStorageHasDeleteVector(PgColumnarStorageId(frel), + PgColumnarCatalogSnapshot(snap))) dirtyFraction = 0.25; table_close(frel, AccessShareLock); } @@ -1092,12 +1092,12 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, copyObject(quals), makeConst(OIDOID, -1, InvalidOid, sizeof(Oid), ObjectIdGetDatum(relid), false, true)); - cpath->methods = &columnar_agg_path_methods; + cpath->methods = &pgcolumnar_agg_path_methods; /* * Parallel arm (#289 phase 5/6). When the shape is one whose transition state * the fold already holds and can be combined by a core Finalize - * (columnar_parallel_agg_ok), and the base relation has a partial (parallel) + * (pgcolumnar_parallel_agg_ok), and the base relation has a partial (parallel) * path, add a parallel-aware partial version of this node under Gather -> * Finalize Aggregate. Each worker claims distinct row groups through the shared * gap-23 counter and emits one per-worker transition-state tuple; the core @@ -1110,7 +1110,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * cheap Gather cost by #133, would wrongly out-cost the genuinely parallel * plan. Opt-in while it is proven and benchmarked. */ - if (needsScan && columnar_enable_parallel_vector_agg) + if (needsScan && pgcolumnar_enable_parallel_vector_agg) { GroupPathExtraData *gpe = (GroupPathExtraData *) extra; bool parallelOk = (gpe != NULL && @@ -1120,7 +1120,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, input_rel->partial_pathlist != NIL); for (i = 0; parallelOk && i < naggs; i++) - if (!columnar_parallel_agg_ok(specs[i].kind)) + if (!pgcolumnar_parallel_agg_ok(specs[i].kind)) parallelOk = false; if (parallelOk) @@ -1175,7 +1175,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, copyObject(quals), makeConst(OIDOID, -1, InvalidOid, sizeof(Oid), ObjectIdGetDatum(relid), false, true)); - ppath->methods = &columnar_agg_path_methods; + ppath->methods = &pgcolumnar_agg_path_methods; /* * Gather this partial node directly rather than add_partial_path'ing @@ -1202,7 +1202,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, } /* - * columnar_groupagg_outmap + * pgcolumnar_groupagg_outmap * Map one target list onto this node's output: per position, the group-key * index (>= 0) or the aggregate index encoded as -(index + 1). False when an * entry is neither a supported aggregate nor a bare reference to a group key, @@ -1216,7 +1216,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * allowPartial admits the INITIAL_SERIAL aggrefs of the second. */ static bool -columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, +pgcolumnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, bool allowPartial, List **outMapOut, int *naggsOut) { List *outMap = NIL; @@ -1229,9 +1229,9 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, if (IsA(oexpr, Aggref)) { - ColumnarAggSpec spec; + PgColumnarAggSpec spec; - if (!columnar_classify_aggref((Aggref *) oexpr, (int) scanrelid, + if (!pgcolumnar_classify_aggref((Aggref *) oexpr, (int) scanrelid, true, allowPartial, &spec)) return false; outMap = lappend(outMap, makeInteger(-(aggIdx + 1))); @@ -1274,13 +1274,13 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, } /* - * ColumnarTryGroupAggPath + * PgColumnarTryGroupAggPath * Add a grouped vectorized aggregate path (#289) when the query is one we * can answer exactly: a single columnar base relation, an optional WHERE, * every output entry either a supported aggregate or a bare reference to a * supported GROUP BY key. On anything unsupported it adds nothing and the * ordinary Agg plan runs. The group-count cap is enforced only at - * execution (columnar_groupagg_lookup). + * execution (pgcolumnar_groupagg_lookup). * * When the shape also qualifies for the parallel arm (#349) this adds * Finalize HashAggregate -> Gather -> parallel-aware partial node instead of @@ -1288,7 +1288,7 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, * displacing a parallel plan with a single-threaded one. */ static void -ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, +PgColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra) { RangeTblEntry *rte; @@ -1324,12 +1324,12 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ if (rte->inh) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; relid = rte->relid; /* every GROUP BY key must be one we can evaluate and group exactly */ - if (!columnar_classify_group_keys(root, input_rel, &groupKeys)) + if (!pgcolumnar_classify_group_keys(root, input_rel, &groupKeys)) return; /* @@ -1337,7 +1337,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * one of the group keys. An output expression built on top of a key (a * function of a grouping column) is not handled here and forces the fallback. */ - if (!columnar_groupagg_outmap(output_rel->reltarget->exprs, groupKeys, + if (!pgcolumnar_groupagg_outmap(output_rel->reltarget->exprs, groupKeys, input_rel->relid, false, &outMap, &naggs)) return; @@ -1349,7 +1349,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * path on exactly the large tables it helps. The unbounded-hash-table guard * is the execution-time cap on the actual group count * (pgcolumnar.groupagg_max_groups), which errors with guidance -- see - * columnar_groupagg_lookup -- rather than silently declining the feature. + * pgcolumnar_groupagg_lookup -- rather than silently declining the feature. */ groupExprs = groupKeys; dNumGroups = estimate_num_groups(root, groupExprs, @@ -1439,7 +1439,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * Nothing serial survived, which is the common case rather than the rare * one: add_path drops the serial columnar scan once a Gather over the * partial path dominates it. Cost the scan this node performs directly, - * the same way ColumnarSetRelPathlist costs its own fallback, instead of + * the same way PgColumnarSetRelPathlist costs its own fallback, instead of * borrowing whatever path happened to survive. */ QualCost qcost = input_rel->baserestrictcost; @@ -1529,7 +1529,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, ObjectIdGetDatum(relid), false, true), groupKeys, outMap); - cpath->methods = &columnar_agg_path_methods; + cpath->methods = &pgcolumnar_agg_path_methods; /* * Parallel arm (#349). The serial node above folds vectors instead of @@ -1544,7 +1544,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * tuple per group. A core Finalize re-aggregates across workers by key, * exactly as it does for an ordinary parallel grouped aggregate. */ - if (columnar_enable_parallel_vector_agg) + if (pgcolumnar_enable_parallel_vector_agg) { GroupPathExtraData *gpe = (GroupPathExtraData *) extra; bool parallelOk = (gpe != NULL && @@ -1561,16 +1561,16 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ foreach(lc, output_rel->reltarget->exprs) { - ColumnarAggSpec spec; + PgColumnarAggSpec spec; if (!parallelOk) break; if (!IsA(lfirst(lc), Aggref)) continue; - if (!columnar_classify_aggref((Aggref *) lfirst(lc), + if (!pgcolumnar_classify_aggref((Aggref *) lfirst(lc), (int) input_rel->relid, true, false, &spec) || - !columnar_parallel_agg_ok(spec.kind)) + !pgcolumnar_parallel_agg_ok(spec.kind)) parallelOk = false; } @@ -1604,7 +1604,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ if (partialScan != NULL && partialScan->parallel_workers >= 1 && pgr->reltarget != NULL && - columnar_groupagg_outmap(pgr->reltarget->exprs, groupKeys, + pgcolumnar_groupagg_outmap(pgr->reltarget->exprs, groupKeys, input_rel->relid, true, &partialMap, &partialAggs)) { @@ -1649,7 +1649,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, ObjectIdGetDatum(relid), false, true), groupKeys, partialMap); - ppath->methods = &columnar_agg_path_methods; + ppath->methods = &pgcolumnar_agg_path_methods; /* * Gather this partial node directly rather than add_partial_path'ing @@ -1698,10 +1698,10 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * ------------------------------------------------------------------------- */ Node * -ColumnarCreateAggScanState(CustomScan *cscan) +PgColumnarCreateAggScanState(CustomScan *cscan) { - ColumnarAggScanState *state = - (ColumnarAggScanState *) palloc0(sizeof(ColumnarAggScanState)); + PgColumnarAggScanState *state = + (PgColumnarAggScanState *) palloc0(sizeof(PgColumnarAggScanState)); int naggs = list_length(cscan->custom_scan_tlist); ListCell *lc; int i = 0; @@ -1730,19 +1730,19 @@ ColumnarCreateAggScanState(CustomScan *cscan) state->isPartial = true; } state->css.methods = state->isPartial - ? &columnar_agg_parallel_exec_methods - : &columnar_agg_exec_methods; + ? &pgcolumnar_agg_parallel_exec_methods + : &pgcolumnar_agg_exec_methods; /* rebuild the aggregate specs from the output tuple's aggregates */ state->naggs = naggs; - state->specs = (ColumnarAggSpec *) palloc0(sizeof(ColumnarAggSpec) * naggs); + state->specs = (PgColumnarAggSpec *) palloc0(sizeof(PgColumnarAggSpec) * naggs); foreach(lc, cscan->custom_scan_tlist) { TargetEntry *tle = (TargetEntry *) lfirst(lc); /* classified successfully at plan time; -1 skips the varno check. * allowPartial accepts the partial arm's INITIAL_SERIAL aggrefs. */ - (void) columnar_classify_aggref((Aggref *) tle->expr, -1, true, true, + (void) pgcolumnar_classify_aggref((Aggref *) tle->expr, -1, true, true, &state->specs[i]); i++; } @@ -1750,20 +1750,20 @@ ColumnarCreateAggScanState(CustomScan *cscan) /* * Scan-fold mode (#289) matches the planner's routing: a residual filter, or * any aggregate a zone map cannot answer. The metadata-answerable no-filter - * case keeps the zone-map path in ColumnarExecAggScan. + * case keeps the zone-map path in PgColumnarExecAggScan. */ state->scanFold = (state->quals != NIL); for (i = 0; i < naggs; i++) - if (!columnar_agg_metadata_answerable(state->specs[i].kind)) + if (!pgcolumnar_agg_metadata_answerable(state->specs[i].kind)) state->scanFold = true; return (Node *) state; } static void -ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; Relation rel; TupleDesc tupdesc; bool allConvertible; @@ -1784,7 +1784,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) */ if (state->scanFold) state->batchEligible = - columnar_batch_shape_eligible(state, tupdesc, NULL, NULL); + pgcolumnar_batch_shape_eligible(state, tupdesc, NULL, NULL); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) { @@ -1794,20 +1794,20 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) /* * Guard the native format version before folding any aggregate (#240). The - * plain read path checks it in ColumnarBeginReadWithStorage, but the + * plain read path checks it in PgColumnarBeginReadWithStorage, but the * zone-map-only aggregate path answers count/min/max from metadata without * ever opening a read state, so the check must also sit here -- otherwise an * unsupported-format table answers from bytes this build may not decode - * correctly. ColumnarStorageId reads the metapage, so its version is checked + * correctly. PgColumnarStorageId reads the metapage, so its version is checked * here too. */ - ColumnarCheckNativeFormatVersion(ColumnarStorageId(rel), + PgColumnarCheckNativeFormatVersion(PgColumnarStorageId(rel), RelationGetRelationName(rel)); /* finish setting up min/max comparison info now that we have the tupdesc */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->kind == COLUMNAR_AGG_MIN || spec->kind == COLUMNAR_AGG_MAX) { @@ -1834,7 +1834,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) * stops being true without anyone noticing. Computing it costs one walk of * an empty list. */ - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, &state->npreds, &allConvertible); /* @@ -1883,7 +1883,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) * arithmetic is bit-for-bit what float4pl/float8pl compute. */ static inline float8 -columnar_float8_pl(float8 a, float8 b) +pgcolumnar_float8_pl(float8 a, float8 b) { float8 r = a + b; @@ -1895,7 +1895,7 @@ columnar_float8_pl(float8 a, float8 b) } static inline float4 -columnar_float4_pl(float4 a, float4 b) +pgcolumnar_float4_pl(float4 a, float4 b) { float4 r = a + b; @@ -1907,13 +1907,13 @@ columnar_float4_pl(float4 a, float4 b) } /* - * columnar_apply_one + * pgcolumnar_apply_one * Fold one value (or a null) into an aggregate accumulator. This is the * reference per-row semantics, shared by the ungrouped scan path - * (columnar_native_scan_agg) and the grouped path (columnar_groupagg_build). + * (pgcolumnar_native_scan_agg) and the grouped path (pgcolumnar_groupagg_build). */ static void -columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, +pgcolumnar_apply_one(MemoryContext resultContext, PgColumnarAggSpec *spec, Datum val, bool isnull) { switch (spec->kind) @@ -1969,10 +1969,10 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, ? (float8) DatumGetFloat4(val) : DatumGetFloat8(val); else if (spec->inputType == FLOAT4OID) - spec->fsum = (float8) columnar_float4_pl((float4) spec->fsum, + spec->fsum = (float8) pgcolumnar_float4_pl((float4) spec->fsum, DatumGetFloat4(val)); else - spec->fsum = columnar_float8_pl(spec->fsum, + spec->fsum = pgcolumnar_float8_pl(spec->fsum, DatumGetFloat8(val)); spec->sawValue = true; } @@ -2098,7 +2098,7 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, /* - * columnar_agg_emit_partial + * pgcolumnar_agg_emit_partial * A parallel partial node (#289 phase 5/6) emits each aggregate's transition * state, not its finalized value, for a core Finalize Aggregate to combine. * The transition types match core's own partial aggregate exactly, so @@ -2107,7 +2107,7 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, * re-derives and re-checks the Youngs-Cramer Sxx we pass through. */ static Datum -columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) +pgcolumnar_agg_emit_partial(PgColumnarAggSpec *spec, bool *isnull) { *isnull = false; @@ -2198,7 +2198,7 @@ columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) /* * The parallel arm is gated to the kinds above - * (columnar_parallel_agg_ok), so this is unreachable; fail loudly rather + * (pgcolumnar_parallel_agg_ok), so this is unreachable; fail loudly rather * than emit a value of the wrong transition type. */ elog(ERROR, "columnar parallel partial: unsupported aggregate kind %d", @@ -2208,12 +2208,12 @@ columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) } /* - * columnar_agg_finalize + * pgcolumnar_agg_finalize * Turn one accumulator into its output Datum, reproducing PostgreSQL's * aggregate result types and empty-input behaviour exactly. */ static Datum -columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) +pgcolumnar_agg_finalize(PgColumnarAggSpec *spec, bool *isnull) { *isnull = false; @@ -2302,7 +2302,7 @@ columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) } /* - * columnar_group_deleted_count + * pgcolumnar_group_deleted_count * How many of this row group's rows are deleted, under the given catalog * snapshot. A group can have several delete_vector rows, whose bitmaps * overlap, so they are OR'd before counting rather than summed -- summing @@ -2311,7 +2311,7 @@ columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) * Bits past the group's row count are ignored. */ static uint64 -columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, +pgcolumnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, Snapshot snap) { uint32 want = (uint32) ((rg->rowCount + 7) / 8); @@ -2321,7 +2321,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, uint64 deleted = 0; uint32 b; - rml = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); + rml = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); if (rml == NIL) return 0; @@ -2356,7 +2356,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, } /* - * columnar_fill_native_metadata_agg + * pgcolumnar_fill_native_metadata_agg * Answer an ungrouped, unfiltered aggregate over a native (PGCN v1) table * from its whole-chunk zone maps (native spec 7.1, D5b): count(*) from * row-group row counts, count(col) and the avg count from value_count, sum @@ -2379,7 +2379,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, * data pages even when the group has deletes. */ static uint64 * -columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) +pgcolumnar_fill_native_metadata_agg(PgColumnarAggScanState *state, int *ndirty) { EState *estate = state->css.ss.ps.state; Relation rel; @@ -2410,11 +2410,11 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) } } - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); rel = table_open(state->relid, AccessShareLock); tupdesc = RelationGetDescr(rel); - snap = ColumnarCatalogSnapshot(estate->es_snapshot); - storageId = ColumnarStorageId(rel); + snap = PgColumnarCatalogSnapshot(estate->es_snapshot); + storageId = PgColumnarStorageId(rel); /* * One storage-wide probe first. When nothing is deleted no group can have @@ -2422,9 +2422,9 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * keeps a clean table at exactly the catalog traffic it had before this * change, which for a count(*) is the row group list and nothing else. */ - anyDeletes = ColumnarStorageHasDeleteVector(storageId, snap); + anyDeletes = PgColumnarStorageHasDeleteVector(storageId, snap); - groups = ColumnarReadRowGroupList(storageId, snap); + groups = PgColumnarReadRowGroupList(storageId, snap); dirty = palloc(sizeof(uint64) * (list_length(groups) > 0 ? list_length(groups) : 1)); *ndirty = 0; @@ -2437,7 +2437,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) int a; if (anyDeletes) - deleted = columnar_group_deleted_count(storageId, rg, snap); + deleted = pgcolumnar_group_deleted_count(storageId, rg, snap); if (deleted > 0) { @@ -2464,7 +2464,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) if (needZones) { - List *zones = ColumnarReadZoneMapList(storageId, + List *zones = PgColumnarReadZoneMapList(storageId, rg->groupNumber, snap); ListCell *zc; @@ -2520,7 +2520,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; NativeZoneMapMetadata *z = (byCol != NULL && spec->attidx >= 0 && spec->attidx < tupdesc->natts) @@ -2562,7 +2562,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) MemoryContextSwitchTo(state->resultContext); char *cur = (spec->kind == COLUMNAR_AGG_MIN) ? (char *) z->minimum : (char *) z->maximum; - Datum v = ColumnarDecodeValue(att, &cur, + Datum v = PgColumnarDecodeValue(att, &cur, state->resultContext); if (!spec->sawValue) @@ -2608,7 +2608,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * once per row. For a full-scan ungrouped aggregate that per-row tax is the whole * cost. The batch fold instead walks each loaded group's packed value streams, * evaluates a pushable WHERE inline, and folds each surviving value through the - * same columnar_apply_one -- so accumulators are byte-identical to the row path, + * same pgcolumnar_apply_one -- so accumulators are byte-identical to the row path, * floats included -- with none of the per-row Datum, context, or executor cost. * It runs only when every aggregate and the whole WHERE are batch-eligible; a * false return means it folded nothing and the caller runs the always-correct @@ -2619,7 +2619,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * -0.0 == 0.0, matching float8_cmp_internal so an inline compare equals the * operator ExecQual would call. */ static inline int -columnar_batch_float_cmp(double a, double b) +pgcolumnar_batch_float_cmp(double a, double b) { if (isnan(a)) return isnan(b) ? 0 : 1; @@ -2631,7 +2631,7 @@ columnar_batch_float_cmp(double a, double b) /* Whether one fixed-width numeric column value (known non-null) satisfies one * btree scan key. */ static bool -columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) +pgcolumnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) { int c; @@ -2644,9 +2644,9 @@ columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) case INT8OID: { int64 a = DatumGetInt64(val), b = DatumGetInt64(arg); c = (a < b) ? -1 : (a > b) ? 1 : 0; break; } case FLOAT4OID: - c = columnar_batch_float_cmp((double) DatumGetFloat4(val), (double) DatumGetFloat4(arg)); break; + c = pgcolumnar_batch_float_cmp((double) DatumGetFloat4(val), (double) DatumGetFloat4(arg)); break; case FLOAT8OID: - c = columnar_batch_float_cmp(DatumGetFloat8(val), DatumGetFloat8(arg)); break; + c = pgcolumnar_batch_float_cmp(DatumGetFloat8(val), DatumGetFloat8(arg)); break; default: return false; } @@ -2663,17 +2663,17 @@ columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) /* A fixed-width by-value numeric column type the batch fold can read directly. */ static bool -columnar_batch_type_ok(Oid typ) +pgcolumnar_batch_type_ok(Oid typ) { return typ == INT2OID || typ == INT4OID || typ == INT8OID || typ == FLOAT4OID || typ == FLOAT8OID; } -/* An aggregate kind the batch fold accumulates (folded via columnar_apply_one +/* An aggregate kind the batch fold accumulates (folded via pgcolumnar_apply_one * over a fixed-width by-value numeric column, or count). int8/numeric sum/avg and * min/max stay on the row path. */ static bool -columnar_batch_agg_ok(ColumnarAggKind kind) +pgcolumnar_batch_agg_ok(PgColumnarAggKind kind) { switch (kind) { @@ -2690,7 +2690,7 @@ columnar_batch_agg_ok(ColumnarAggKind kind) } /* - * columnar_batch_shape_eligible + * pgcolumnar_batch_shape_eligible * Whether this aggregate's shape can use the batch fold: every aggregate is * batch-accumulable, the whole WHERE converts to scan keys (no residual), and * every key is a supported btree comparison on a batch-readable column. When @@ -2699,7 +2699,7 @@ columnar_batch_agg_ok(ColumnarAggKind kind) * Begin can report it in EXPLAIN before execution. */ static bool -columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, +pgcolumnar_batch_shape_eligible(PgColumnarAggScanState *state, TupleDesc tupdesc, ScanKey *keysOut, int *nkeysOut) { ScanKey keys; @@ -2711,15 +2711,15 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, bool ok = true; for (a = 0; a < state->naggs; a++) - if (!columnar_batch_agg_ok(state->specs[a].kind)) + if (!pgcolumnar_batch_agg_ok(state->specs[a].kind)) return false; - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, &npred, &allConvertible); if (!allConvertible) return false; - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nkeys); + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nkeys); for (k = 0; k < nkeys; k++) { ScanKey key = &keys[k]; @@ -2729,7 +2729,7 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, if (key->sk_flags != 0 || attidx < 0 || attidx >= tupdesc->natts) { ok = false; break; } coltype = TupleDescAttr(tupdesc, attidx)->atttypid; - if (!columnar_batch_type_ok(coltype)) + if (!pgcolumnar_batch_type_ok(coltype)) { ok = false; break; } if (key->sk_subtype != InvalidOid && key->sk_subtype != coltype) { ok = false; break; } @@ -2750,14 +2750,14 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, /* Reset the accumulators to their initial state (for a clean fall-back). */ static void -columnar_agg_specs_reset(ColumnarAggScanState *state) +pgcolumnar_agg_specs_reset(PgColumnarAggScanState *state) { int a; MemoryContextReset(state->resultContext); for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; spec->count = 0; spec->sum = 0; @@ -2771,13 +2771,13 @@ columnar_agg_specs_reset(ColumnarAggScanState *state) } /* - * columnar_native_batch_fold + * pgcolumnar_native_batch_fold * Fold the whole scan column-at-a-time. Returns false (having folded and * reset nothing that the caller cannot redo) when the shape is not eligible * or a group is missing a needed column, so the caller runs the row path. */ static bool -columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, +pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel, TupleDesc tupdesc) { EState *estate = state->css.ss.ps.state; @@ -2787,7 +2787,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int k; int col; int natts = tupdesc->natts; - ColumnarReadState *rs; + PgColumnarReadState *rs; const char **cvalidity = (const char **) palloc0(sizeof(char *) * natts); const char **cpacked = (const char **) palloc0(sizeof(char *) * natts); int16 *cattlen = (int16 *) palloc0(sizeof(int16) * natts); @@ -2796,7 +2796,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, Datum *cval = (Datum *) palloc0(sizeof(Datum) * natts); bool *cisnull = (bool *) palloc0(sizeof(bool) * natts); - if (!columnar_batch_shape_eligible(state, tupdesc, &keys, &nkeys)) + if (!pgcolumnar_batch_shape_eligible(state, tupdesc, &keys, &nkeys)) return false; col = -1; @@ -2814,11 +2814,11 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * * Per-vector skipping WITHIN a surviving group is a separate refinement the * fold does not take: it would need to step the present index past a skipped - * vector. That is safe to leave out because columnar_native_load_group builds + * vector. That is safe to leave out because pgcolumnar_native_load_group builds * the packed present-value stream whole regardless of the skip vector, so * walking all rows and advancing the present index over each is correct. */ - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, nkeys, keys); /* @@ -2830,14 +2830,14 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * InitializeDSM callback runs even leader-only), so a NULL here is a bug. */ if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); else if (state->isPartial) { - ColumnarEndRead(rs); + PgColumnarEndRead(rs); elog(ERROR, "parallel columnar aggregate ran without a shared group counter"); } - while (ColumnarReadFoldNextGroup(rs)) + while (PgColumnarReadFoldNextGroup(rs)) { uint64 nrows; const char *dmask; @@ -2847,7 +2847,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int vcount; uint64 r; - ColumnarReadFoldGroupInfo(rs, &nrows, &dmask, &dlen, + PgColumnarReadFoldGroupInfo(rs, &nrows, &dmask, &dlen, &skipVec, &vecStart, &vcount); for (col = 0; col < natts; col++) @@ -2860,7 +2860,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, cpresent[col] = 0; if (!cneeded[col]) continue; - if (!ColumnarReadFoldColumn(rs, col, &vbits, &pk, &al, &vrl)) + if (!PgColumnarReadFoldColumn(rs, col, &vbits, &pk, &al, &vrl)) { /* * The column is absent from this group (a later ADD COLUMN). The @@ -2874,8 +2874,8 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * cleanly rather than return a wrong answer. Rare (an old row group * predating an ADD COLUMN); turn the parallel GUC off for the table. */ - ColumnarEndRead(rs); - columnar_agg_specs_reset(state); + PgColumnarEndRead(rs); + pgcolumnar_agg_specs_reset(state); if (state->isPartial) elog(ERROR, "parallel columnar aggregate cannot fold a relation " "with a column added after some row groups; " @@ -2921,7 +2921,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int attidx = key->sk_attno - 1; if (cisnull[attidx] || - !columnar_batch_key_pass(TupleDescAttr(tupdesc, attidx)->atttypid, + !pgcolumnar_batch_key_pass(TupleDescAttr(tupdesc, attidx)->atttypid, cval[attidx], key->sk_strategy, key->sk_argument)) { @@ -2934,29 +2934,29 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->resultContext, spec, + pgcolumnar_apply_one(state->resultContext, spec, cval[spec->attidx], cisnull[spec->attidx]); else - columnar_apply_one(state->resultContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->resultContext, spec, (Datum) 0, true); } } } - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; state->batchFolded = true; - ColumnarEndRead(rs); + PgColumnarEndRead(rs); return true; } /* - * columnar_native_scan_agg + * pgcolumnar_native_scan_agg * Fold an ungrouped, unfiltered aggregate over a native table by scanning it - * one row at a time (ColumnarReadNextRow applies the delete mask), for the + * one row at a time (PgColumnarReadNextRow applies the delete mask), for the * case where the zone-map-only path cannot be used because the storage has * deletes (D6b). No quals: the upper-path hook only adds the native agg path * when there is no filter. @@ -2966,7 +2966,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * already folded from their zone maps by the caller. */ static void -columnar_native_scan_agg(ColumnarAggScanState *state, +pgcolumnar_native_scan_agg(PgColumnarAggScanState *state, const uint64 *restrictGroups, int nRestrictGroups) { EState *estate = state->css.ss.ps.state; @@ -2974,7 +2974,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, Relation rel = table_open(state->relid, AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); Bitmapset *projected; - ColumnarReadState *rs; + PgColumnarReadState *rs; ScanKey keys = NULL; int nScanKeys = 0; Datum *values = (Datum *) palloc(sizeof(Datum) * tupdesc->natts); @@ -2999,8 +2999,8 @@ columnar_native_scan_agg(ColumnarAggScanState *state, projected = bms_make_singleton(0); /* count(*) only: one column */ } - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* * Batch fold when the shape allows it (#289): the whole scan folds @@ -3009,7 +3009,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, * row path. */ if (state->scanFold && restrictGroups == NULL && - columnar_native_batch_fold(state, rel, tupdesc)) + pgcolumnar_native_batch_fold(state, rel, tupdesc)) { table_close(rel, AccessShareLock); return; @@ -3017,7 +3017,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, /* * The row path reached here because the shape is not batch-foldable (e.g. a - * NULL test or a non-btree filter): columnar_native_batch_fold returned false + * NULL test or a non-btree filter): pgcolumnar_native_batch_fold returned false * before claiming any group, so the shared counter is untouched and a parallel * partial node can fold correctly here too -- each worker just claims distinct * groups through the same atomic and applies the WHERE recheck per row. (The @@ -3027,21 +3027,21 @@ columnar_native_scan_agg(ColumnarAggScanState *state, /* push the WHERE down for group and vector pruning; the recheck is exact */ if (state->scanFold && state->quals != NIL) - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nScanKeys); - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, projected, nScanKeys, keys); if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); if (restrictGroups != NULL) - ColumnarReadRestrictToGroups(rs, restrictGroups, nRestrictGroups); + PgColumnarReadRestrictToGroups(rs, restrictGroups, nRestrictGroups); /* columns outside the projection stay null in the recheck slot */ if (state->whereState != NULL) memset(state->baseSlot->tts_isnull, true, sizeof(bool) * tupdesc->natts); - while (ColumnarReadNextRow(rs, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(rs, values, nulls, &rowNumber)) { /* * Recheck the whole WHERE per row: the scan keys only prune groups and @@ -3066,31 +3066,31 @@ columnar_native_scan_agg(ColumnarAggScanState *state, for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->resultContext, spec, + pgcolumnar_apply_one(state->resultContext, spec, values[spec->attidx], nulls[spec->attidx]); else - columnar_apply_one(state->resultContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->resultContext, spec, (Datum) 0, true); } } if (state->scanFold) { - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; } - ColumnarEndRead(rs); + PgColumnarEndRead(rs); table_close(rel, AccessShareLock); } static TupleTableSlot * -ColumnarExecAggScan(CustomScanState *node) +PgColumnarExecAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; TupleTableSlot *scanSlot = node->ss.ss_ScanTupleSlot; ExprContext *econtext = node->ss.ps.ps_ExprContext; TupleTableSlot *result; @@ -3116,24 +3116,24 @@ ColumnarExecAggScan(CustomScanState *node) * a zone map that counts the rows this transaction has already removed. */ frel = table_open(state->relid, AccessShareLock); - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); if (state->scanFold) { /* * A filter, or a sum/avg no zone map answers (#289): scan every row once - * and fold it. columnar_native_scan_agg builds the scan keys, rechecks the + * and fold it. pgcolumnar_native_scan_agg builds the scan keys, rechecks the * WHERE, and captures the EXPLAIN stats. */ - columnar_native_scan_agg(state, NULL, 0); + pgcolumnar_native_scan_agg(state, NULL, 0); } else { - dirtyGroups = columnar_fill_native_metadata_agg(state, &nDirtyGroups); + dirtyGroups = pgcolumnar_fill_native_metadata_agg(state, &nDirtyGroups); if (nDirtyGroups > 0) - columnar_native_scan_agg(state, dirtyGroups, nDirtyGroups); + pgcolumnar_native_scan_agg(state, dirtyGroups, nDirtyGroups); state->haveStats = false; } @@ -3145,8 +3145,8 @@ ColumnarExecAggScan(CustomScanState *node) ExecClearTuple(scanSlot); for (a = 0; a < state->naggs; a++) scanSlot->tts_values[a] = state->isPartial - ? columnar_agg_emit_partial(&state->specs[a], &scanSlot->tts_isnull[a]) - : columnar_agg_finalize(&state->specs[a], &scanSlot->tts_isnull[a]); + ? pgcolumnar_agg_emit_partial(&state->specs[a], &scanSlot->tts_isnull[a]) + : pgcolumnar_agg_finalize(&state->specs[a], &scanSlot->tts_isnull[a]); ExecStoreVirtualTuple(scanSlot); /* @@ -3166,21 +3166,21 @@ ColumnarExecAggScan(CustomScanState *node) } static void -ColumnarEndAggScan(CustomScanState *node) +PgColumnarEndAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; if (state->baseSlot != NULL) ExecDropSingleTupleTableSlot(state->baseSlot); state->baseSlot = NULL; - /* the reader is ended inside ColumnarExecAggScan; the memory contexts are + /* the reader is ended inside PgColumnarExecAggScan; the memory contexts are * children of es_query_cxt and freed with it */ } static void -ColumnarReScanAggScan(CustomScanState *node) +PgColumnarReScanAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; int a; state->done = false; @@ -3189,7 +3189,7 @@ ColumnarReScanAggScan(CustomScanState *node) MemoryContextReset(state->resultContext); for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; spec->count = 0; spec->sum = 0; @@ -3210,9 +3210,9 @@ ColumnarReScanAggScan(CustomScanState *node) } static void -ColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) +PgColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; ExplainPropertyInteger("Columnar Vectorized Aggregates", NULL, state->naggs, es); @@ -3233,13 +3233,13 @@ ColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) } } -static const CustomExecMethods columnar_agg_exec_methods = { +static const CustomExecMethods pgcolumnar_agg_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginAggScan, - .ExecCustomScan = ColumnarExecAggScan, - .EndCustomScan = ColumnarEndAggScan, - .ReScanCustomScan = ColumnarReScanAggScan, - .ExplainCustomScan = ColumnarExplainAggScan, + .BeginCustomScan = PgColumnarBeginAggScan, + .ExecCustomScan = PgColumnarExecAggScan, + .EndCustomScan = PgColumnarEndAggScan, + .ReScanCustomScan = PgColumnarReScanAggScan, + .ExplainCustomScan = PgColumnarExplainAggScan, }; /* ------------------------------------------------------------------------- @@ -3251,20 +3251,20 @@ static const CustomExecMethods columnar_agg_exec_methods = { * and passes its address as `coordinate` to the DSM/worker init callbacks. Our * agg node opens its reader lazily during Exec, strictly after both DSM-init and * Worker-init, so the callbacks only need to record the counter on the state; - * ColumnarBeginRead wiring happens at fold time. + * PgColumnarBeginRead wiring happens at fold time. * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMAggScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMAggScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); @@ -3277,17 +3277,17 @@ ColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, * rows the leader deleted earlier in this transaction. The Exec-time flush in * every backend then only ever flushes its own (empty) buffers. */ - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); { Relation frel = table_open(state->relid, AccessShareLock); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); } } static void -ColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -3297,26 +3297,26 @@ ColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, } static void -ColumnarInitializeWorkerAggScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerAggScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; state->parallelCounter = counter; } -static const CustomExecMethods columnar_agg_parallel_exec_methods = { +static const CustomExecMethods pgcolumnar_agg_parallel_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginAggScan, - .ExecCustomScan = ColumnarExecAggScan, - .EndCustomScan = ColumnarEndAggScan, - .ReScanCustomScan = ColumnarReScanAggScan, - .ExplainCustomScan = ColumnarExplainAggScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMAggScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMAggScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMAggScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerAggScan, + .BeginCustomScan = PgColumnarBeginAggScan, + .ExecCustomScan = PgColumnarExecAggScan, + .EndCustomScan = PgColumnarEndAggScan, + .ReScanCustomScan = PgColumnarReScanAggScan, + .ExplainCustomScan = PgColumnarExplainAggScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMAggScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMAggScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMAggScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerAggScan, }; /* ------------------------------------------------------------------------- @@ -3324,10 +3324,10 @@ static const CustomExecMethods columnar_agg_parallel_exec_methods = { * ------------------------------------------------------------------------- */ Node * -ColumnarCreateGroupAggScanState(CustomScan *cscan) +PgColumnarCreateGroupAggScanState(CustomScan *cscan) { - ColumnarGroupAggScanState *state = - (ColumnarGroupAggScanState *) palloc0(sizeof(ColumnarGroupAggScanState)); + PgColumnarGroupAggScanState *state = + (PgColumnarGroupAggScanState *) palloc0(sizeof(PgColumnarGroupAggScanState)); List *groupKeys; List *outMapList; ListCell *lc; @@ -3354,8 +3354,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) state->isPartial = true; } state->css.methods = state->isPartial - ? &columnar_groupagg_parallel_exec_methods - : &columnar_groupagg_exec_methods; + ? &pgcolumnar_groupagg_parallel_exec_methods + : &pgcolumnar_groupagg_exec_methods; /* custom_private: rti, quals, relid, group-key exprs, output map (length 5) */ state->scanrelid = (Index) intVal(linitial(cscan->custom_private)); @@ -3366,8 +3366,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) outMapList = (List *) list_nth(cscan->custom_private, 4); state->nkeys = list_length(groupKeys); - state->keys = (ColumnarGroupKey *) - palloc0(sizeof(ColumnarGroupKey) * Max(state->nkeys, 1)); + state->keys = (PgColumnarGroupKey *) + palloc0(sizeof(PgColumnarGroupKey) * Max(state->nkeys, 1)); i = 0; foreach(lc, groupKeys) state->keys[i++].expr = (Expr *) lfirst(lc); @@ -3377,8 +3377,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) if (IsA(((TargetEntry *) lfirst(lc))->expr, Aggref)) naggs++; state->naggs = naggs; - state->aggTemplate = (ColumnarAggSpec *) - palloc0(sizeof(ColumnarAggSpec) * Max(naggs, 1)); + state->aggTemplate = (PgColumnarAggSpec *) + palloc0(sizeof(PgColumnarAggSpec) * Max(naggs, 1)); i = 0; foreach(lc, cscan->custom_scan_tlist) { @@ -3387,7 +3387,7 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) if (IsA(tle->expr, Aggref)) { /* allowPartial accepts the parallel arm's INITIAL_SERIAL aggrefs */ - (void) columnar_classify_aggref((Aggref *) tle->expr, -1, true, true, + (void) pgcolumnar_classify_aggref((Aggref *) tle->expr, -1, true, true, &state->aggTemplate[i]); i++; } @@ -3399,7 +3399,7 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) foreach(lc, outMapList) state->outMap[i++] = intVal(lfirst(lc)); - state->maxGroups = columnar_groupagg_max_groups; + state->maxGroups = pgcolumnar_groupagg_max_groups; state->capacity = 0; state->nGroups = 0; state->entries = NULL; @@ -3408,9 +3408,9 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) } static void -ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; Relation rel; TupleDesc basedesc; Bitmapset *proj = NULL; @@ -3444,7 +3444,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) * Count pushable filters for EXPLAIN before the EXPLAIN-only early return, so * a plain EXPLAIN reports the real pushed-down filter count instead of 0. */ - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, basedesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, basedesc, &state->npreds, &allConvertible); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) @@ -3454,7 +3454,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) } /* guard the native format before decoding any value (#240) */ - ColumnarCheckNativeFormatVersion(ColumnarStorageId(rel), + PgColumnarCheckNativeFormatVersion(PgColumnarStorageId(rel), RelationGetRelationName(rel)); /* a virtual slot holding each read row for key and qual evaluation */ @@ -3464,7 +3464,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) /* group-key ExprStates and their hash/equality machinery */ for (k = 0; k < state->nkeys; k++) { - ColumnarGroupKey *key = &state->keys[k]; + PgColumnarGroupKey *key = &state->keys[k]; Oid type = exprType((Node *) key->expr); TypeCacheEntry *tce = lookup_type_cache(type, TYPECACHE_HASH_PROC_FINFO | @@ -3487,7 +3487,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) /* finish min/max comparison setup on the per-agg template */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->aggTemplate[a]; + PgColumnarAggSpec *spec = &state->aggTemplate[a]; if (spec->kind == COLUMNAR_AGG_MIN || spec->kind == COLUMNAR_AGG_MAX) { @@ -3525,14 +3525,14 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) } /* - * columnar_groupagg_keys_equal + * pgcolumnar_groupagg_keys_equal * Whether a probing row's keys match a stored group's, by SQL grouping * semantics: two nulls are equal, and non-nulls compare with the key type's * equality operator (with collation) -- exactly how core groups. */ static bool -columnar_groupagg_keys_equal(ColumnarGroupAggScanState *state, - ColumnarGroupEntry *e, +pgcolumnar_groupagg_keys_equal(PgColumnarGroupAggScanState *state, + PgColumnarGroupEntry *e, Datum *keyvals, bool *keynulls) { int k; @@ -3552,17 +3552,17 @@ columnar_groupagg_keys_equal(ColumnarGroupAggScanState *state, } /* - * columnar_groupagg_grow + * pgcolumnar_groupagg_grow * Double the open-addressing table and reinsert live entries. Entry structs * (and the key/spec pointers they carry) move by value; the pointed-at key * Datums and accumulators stay put in their own contexts. */ static void -columnar_groupagg_grow(ColumnarGroupAggScanState *state) +pgcolumnar_groupagg_grow(PgColumnarGroupAggScanState *state) { int oldCap = state->capacity; int newCap = (oldCap <= 0) ? 1024 : oldCap * 2; - ColumnarGroupEntry *newEntries; + PgColumnarGroupEntry *newEntries; MemoryContext old; int i; @@ -3578,15 +3578,15 @@ columnar_groupagg_grow(ColumnarGroupAggScanState *state) * via pgcolumnar.groupagg_max_groups. */ old = MemoryContextSwitchTo(state->hashContext); - newEntries = (ColumnarGroupEntry *) + newEntries = (PgColumnarGroupEntry *) MemoryContextAllocExtended(state->hashContext, - sizeof(ColumnarGroupEntry) * (Size) newCap, + sizeof(PgColumnarGroupEntry) * (Size) newCap, MCXT_ALLOC_HUGE | MCXT_ALLOC_ZERO); MemoryContextSwitchTo(old); for (i = 0; i < oldCap; i++) { - ColumnarGroupEntry *e = &state->entries[i]; + PgColumnarGroupEntry *e = &state->entries[i]; uint32 idx; if (!e->used) @@ -3604,23 +3604,23 @@ columnar_groupagg_grow(ColumnarGroupAggScanState *state) } /* - * columnar_groupagg_lookup + * pgcolumnar_groupagg_lookup * Find the group for this row's keys, inserting a fresh one (with the key * Datums copied into keyContext and accumulators seeded from the template) * when it is new. */ -static ColumnarGroupEntry * -columnar_groupagg_lookup(ColumnarGroupAggScanState *state, +static PgColumnarGroupEntry * +pgcolumnar_groupagg_lookup(PgColumnarGroupAggScanState *state, Datum *keyvals, bool *keynulls) { uint32 hash = 0; uint32 idx; int k; - ColumnarGroupEntry *e; + PgColumnarGroupEntry *e; /* grow before probing so the index is computed against the final table */ if ((int64) (state->nGroups + 1) * 10 >= (int64) state->capacity * 7) - columnar_groupagg_grow(state); + pgcolumnar_groupagg_grow(state); for (k = 0; k < state->nkeys; k++) { @@ -3642,7 +3642,7 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, if (!e->used) break; if (e->hash == hash && - columnar_groupagg_keys_equal(state, e, keyvals, keynulls)) + pgcolumnar_groupagg_keys_equal(state, e, keyvals, keynulls)) return e; idx = (idx + 1) & (uint32) (state->capacity - 1); } @@ -3682,10 +3682,10 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, { MemoryContext oldc = MemoryContextSwitchTo(state->specContext); - e->specs = (ColumnarAggSpec *) - palloc(sizeof(ColumnarAggSpec) * Max(state->naggs, 1)); + e->specs = (PgColumnarAggSpec *) + palloc(sizeof(PgColumnarAggSpec) * Max(state->naggs, 1)); memcpy(e->specs, state->aggTemplate, - sizeof(ColumnarAggSpec) * state->naggs); + sizeof(PgColumnarAggSpec) * state->naggs); MemoryContextSwitchTo(oldc); } state->nGroups++; @@ -3693,14 +3693,14 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, } /* - * columnar_groupagg_build + * pgcolumnar_groupagg_build * Scan the relation once and fold every surviving row into its group. The * reader prunes groups and vectors with the pushed-down WHERE; each row is * rechecked against the whole WHERE, its keys evaluated, and its values * folded in scan order so accumulators match the scalar Agg byte for byte. */ static void -columnar_groupagg_build(ColumnarGroupAggScanState *state) +pgcolumnar_groupagg_build(PgColumnarGroupAggScanState *state) { EState *estate = state->css.ss.ps.state; ExprContext *econtext = state->css.ss.ps.ps_ExprContext; @@ -3711,17 +3711,17 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) bool *nulls = (bool *) palloc(sizeof(bool) * natts); Datum *keyvals = (Datum *) palloc(sizeof(Datum) * Max(state->nkeys, 1)); bool *keynulls = (bool *) palloc(sizeof(bool) * Max(state->nkeys, 1)); - ColumnarReadState *rs; + PgColumnarReadState *rs; ScanKey keys; int nScanKeys = 0; uint64 rowNumber; - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(rel); - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, basedesc, + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, basedesc, &nScanKeys); - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, nScanKeys, keys); /* @@ -3731,17 +3731,17 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) * case the reader walks every group as before. */ if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); /* columns outside the projection stay null in the base slot */ memset(state->baseSlot->tts_isnull, true, sizeof(bool) * natts); - while (ColumnarReadNextRow(rs, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(rs, values, nulls, &rowNumber)) { int x; int k; int a; - ColumnarGroupEntry *e; + PgColumnarGroupEntry *e; ResetExprContext(econtext); @@ -3770,46 +3770,46 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) keyvals[k] = ExecEvalExprSwitchContext(state->keys[k].exprState, econtext, &keynulls[k]); - e = columnar_groupagg_lookup(state, keyvals, keynulls); + e = pgcolumnar_groupagg_lookup(state, keyvals, keynulls); /* fold this row's values into the group's accumulators */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &e->specs[a]; + PgColumnarAggSpec *spec = &e->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->specContext, spec, + pgcolumnar_apply_one(state->specContext, spec, values[spec->attidx], nulls[spec->attidx]); else - columnar_apply_one(state->specContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->specContext, spec, (Datum) 0, true); } } - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; - ColumnarEndRead(rs); + PgColumnarEndRead(rs); table_close(rel, AccessShareLock); } static TupleTableSlot * -ColumnarExecGroupAggScan(CustomScanState *node) +PgColumnarExecGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; TupleTableSlot *scanSlot = node->ss.ss_ScanTupleSlot; ExprContext *econtext = node->ss.ps.ps_ExprContext; if (!state->started) { - columnar_groupagg_build(state); + pgcolumnar_groupagg_build(state); state->started = true; state->emitPos = 0; } while (state->emitPos < state->capacity) { - ColumnarGroupEntry *e = &state->entries[state->emitPos++]; + PgColumnarGroupEntry *e = &state->entries[state->emitPos++]; int p; if (!e->used) @@ -3839,9 +3839,9 @@ ColumnarExecGroupAggScan(CustomScanState *node) * expects -- not one empty group. */ scanSlot->tts_values[p] = state->isPartial - ? columnar_agg_emit_partial(&e->specs[a], + ? pgcolumnar_agg_emit_partial(&e->specs[a], &scanSlot->tts_isnull[p]) - : columnar_agg_finalize(&e->specs[a], + : pgcolumnar_agg_finalize(&e->specs[a], &scanSlot->tts_isnull[p]); } } @@ -3859,9 +3859,9 @@ ColumnarExecGroupAggScan(CustomScanState *node) } static void -ColumnarEndGroupAggScan(CustomScanState *node) +PgColumnarEndGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; if (state->baseSlot != NULL) ExecDropSingleTupleTableSlot(state->baseSlot); @@ -3870,9 +3870,9 @@ ColumnarEndGroupAggScan(CustomScanState *node) } static void -ColumnarReScanGroupAggScan(CustomScanState *node) +PgColumnarReScanGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; state->started = false; state->emitPos = 0; @@ -3886,10 +3886,10 @@ ColumnarReScanGroupAggScan(CustomScanState *node) } static void -ColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, +PgColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; ExplainPropertyInteger("Columnar Vectorized Group Keys", NULL, state->nkeys, es); @@ -3909,13 +3909,13 @@ ColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, } } -static const CustomExecMethods columnar_groupagg_exec_methods = { +static const CustomExecMethods pgcolumnar_groupagg_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginGroupAggScan, - .ExecCustomScan = ColumnarExecGroupAggScan, - .EndCustomScan = ColumnarEndGroupAggScan, - .ReScanCustomScan = ColumnarReScanGroupAggScan, - .ExplainCustomScan = ColumnarExplainGroupAggScan, + .BeginCustomScan = PgColumnarBeginGroupAggScan, + .ExecCustomScan = PgColumnarExecGroupAggScan, + .EndCustomScan = PgColumnarEndGroupAggScan, + .ReScanCustomScan = PgColumnarReScanGroupAggScan, + .ExplainCustomScan = PgColumnarExplainGroupAggScan, }; /* ------------------------------------------------------------------------- @@ -3924,23 +3924,23 @@ static const CustomExecMethods columnar_groupagg_exec_methods = { * The same shared pg_atomic_uint32 the ungrouped partial node uses (gap 23), * handing out row-group indices so each worker folds distinct groups. These * mirror the ungrouped four and cannot share their bodies: those cast the node - * to ColumnarAggScanState, and the grouped node is a different struct. + * to PgColumnarAggScanState, and the grouped node is a different struct. * * The grouped node opens its reader lazily in Exec, strictly after both DSM-init * and Worker-init, so the callbacks need only record the counter on the state. * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); @@ -3961,17 +3961,17 @@ ColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, * a future path that reaches here with unflushed state would silently give * workers a stale view; no test covers its removal, and none claims to. */ - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); { Relation frel = table_open(state->relid, AccessShareLock); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); } } static void -ColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -3981,26 +3981,26 @@ ColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt } static void -ColumnarInitializeWorkerGroupAggScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerGroupAggScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; state->parallelCounter = counter; } -static const CustomExecMethods columnar_groupagg_parallel_exec_methods = { +static const CustomExecMethods pgcolumnar_groupagg_parallel_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginGroupAggScan, - .ExecCustomScan = ColumnarExecGroupAggScan, - .EndCustomScan = ColumnarEndGroupAggScan, - .ReScanCustomScan = ColumnarReScanGroupAggScan, - .ExplainCustomScan = ColumnarExplainGroupAggScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMGroupAggScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMGroupAggScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMGroupAggScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerGroupAggScan, + .BeginCustomScan = PgColumnarBeginGroupAggScan, + .ExecCustomScan = PgColumnarExecGroupAggScan, + .EndCustomScan = PgColumnarEndGroupAggScan, + .ReScanCustomScan = PgColumnarReScanGroupAggScan, + .ExplainCustomScan = PgColumnarExplainGroupAggScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMGroupAggScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMGroupAggScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMGroupAggScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerGroupAggScan, }; /* ------------------------------------------------------------------------- @@ -4008,8 +4008,8 @@ static const CustomExecMethods columnar_groupagg_parallel_exec_methods = { * ------------------------------------------------------------------------- */ void -ColumnarVectorInit(void) +PgColumnarVectorInit(void) { prev_create_upper_paths_hook = create_upper_paths_hook; - create_upper_paths_hook = ColumnarCreateUpperPaths; + create_upper_paths_hook = PgColumnarCreateUpperPaths; } diff --git a/src/columnar_visibilitymap.c b/src/columnar_visibilitymap.c index 5070850..2433e8a 100644 --- a/src/columnar_visibilitymap.c +++ b/src/columnar_visibilitymap.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_visibilitymap.c + * pgcolumnar_visibilitymap.c * Visibility-map fork maintenance for pgColumnar index-only scans * (gap 28 direction 1, phase 1: the load-bearing prototype). * @@ -46,8 +46,8 @@ #include "storage/procarray.h" #include "utils/rel.h" -PG_FUNCTION_INFO_V1(columnar_vm_selftest); -PG_FUNCTION_INFO_V1(columnar_vm_is_visible); +PG_FUNCTION_INFO_V1(pgcolumnar_vm_selftest); +PG_FUNCTION_INFO_V1(pgcolumnar_vm_is_visible); /* * Visibility-map on-disk layout. These mirror the private macros in @@ -67,14 +67,14 @@ PG_FUNCTION_INFO_V1(columnar_vm_is_visible); (((x) % COLUMNAR_VM_BLOCKS_PER_BYTE) * COLUMNAR_VM_BITS_PER_BLOCK) /* - * ColumnarVMSetVisible + * PgColumnarVMSetVisible * Mark the synthetic block `blk` all-visible in the relation's VM fork, * WAL-logged. Idempotent: a no-op if the bit is already set. This writes * only the VM fork -- there is no heap page to flag -- which is why it does * not go through visibilitymap_set(). */ void -ColumnarVMSetVisible(Relation rel, BlockNumber blk) +PgColumnarVMSetVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; Page page; @@ -108,12 +108,12 @@ ColumnarVMSetVisible(Relation rel, BlockNumber blk) } /* - * ColumnarVMClearVisible + * PgColumnarVMClearVisible * Clear the all-visible (and all-frozen) bits for `blk`, WAL-logged. Used * by write paths so a modified range is never reported all-visible. */ void -ColumnarVMClearVisible(Relation rel, BlockNumber blk) +PgColumnarVMClearVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; Page page; @@ -153,28 +153,28 @@ ColumnarVMClearVisible(Relation rel, BlockNumber blk) } /* - * ColumnarVMClearForRow + * PgColumnarVMClearForRow * Clear the all-visible bit for the synthetic block that holds `rowNumber`. - * The block is computed the same way ColumnarRowNumberToItemPointer derives + * The block is computed the same way PgColumnarRowNumberToItemPointer derives * a TID block, so it matches the block the index-only-scan executor probes. * Called by every write path (insert/delete/update) so a modified block is * never left all-visible. Cheap when no bit is set (a short-circuit read). */ void -ColumnarVMClearForRow(Relation rel, uint64 rowNumber) +PgColumnarVMClearForRow(Relation rel, uint64 rowNumber) { BlockNumber blk = (BlockNumber) (rowNumber / COLUMNAR_VALID_ITEMPOINTER_OFFSETS); - ColumnarVMClearVisible(rel, blk); + PgColumnarVMClearVisible(rel, blk); } /* - * ColumnarVMIsVisible + * PgColumnarVMIsVisible * True if `blk` is marked all-visible in the VM fork. Thin wrapper over the * stock reader (the same call the index-only-scan executor makes). */ bool -ColumnarVMIsVisible(Relation rel, BlockNumber blk) +PgColumnarVMIsVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; uint8 status = visibilitymap_get_status(rel, blk, &vmbuf); @@ -188,8 +188,8 @@ ColumnarVMIsVisible(Relation rel, BlockNumber blk) static int rowrange_cmp(const void *a, const void *b) { - uint64 fa = ((const ColumnarRowRange *) a)->firstRowNumber; - uint64 fb = ((const ColumnarRowRange *) b)->firstRowNumber; + uint64 fa = ((const PgColumnarRowRange *) a)->firstRowNumber; + uint64 fb = ((const PgColumnarRowRange *) b)->firstRowNumber; if (fa < fb) return -1; @@ -199,7 +199,7 @@ rowrange_cmp(const void *a, const void *b) } /* - * ColumnarVMSetVisibleForRelation + * PgColumnarVMSetVisibleForRelation * Lazy vacuum step (gap 28 phase 3): mark all-visible chunk groups in the * VM fork. Computes the all-visible groups (stripe committed past the * oldest-xmin horizon, no committed-or-in-progress deletes), merges @@ -211,25 +211,25 @@ rowrange_cmp(const void *a, const void *b) * and clear-on-write removes any bit for a row changed after this runs. */ void -ColumnarVMSetVisibleForRelation(Relation rel) +PgColumnarVMSetVisibleForRelation(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *groups = ColumnarComputeAllVisibleGroups(storageId, oldestXmin); + uint64 storageId = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *groups = PgColumnarComputeAllVisibleGroups(storageId, oldestXmin); uint64 K = COLUMNAR_VALID_ITEMPOINTER_OFFSETS; int n = list_length(groups); - ColumnarRowRange *arr; + PgColumnarRowRange *arr; ListCell *lc; int i; if (n == 0) return; - arr = palloc(sizeof(ColumnarRowRange) * n); + arr = palloc(sizeof(PgColumnarRowRange) * n); i = 0; foreach(lc, groups) - arr[i++] = *(ColumnarRowRange *) lfirst(lc); - qsort(arr, n, sizeof(ColumnarRowRange), rowrange_cmp); + arr[i++] = *(PgColumnarRowRange *) lfirst(lc); + qsort(arr, n, sizeof(PgColumnarRowRange), rowrange_cmp); i = 0; while (i < n) @@ -254,7 +254,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) b = (BlockNumber) ((lo + K - 1) / K); bend = (BlockNumber) (hi / K); for (; b < bend; b++) - ColumnarVMSetVisible(rel, b); + PgColumnarVMSetVisible(rel, b); i = m; } @@ -263,7 +263,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) } /* - * columnar_vm_selftest(rel regclass, blk int) -> bool + * pgcolumnar_vm_selftest(rel regclass, blk int) -> bool * Phase-1 proof: set the all-visible bit for a synthetic block on a * columnar relation, then read it back through the backend's own * visibilitymap_get_status. Returns true iff the round trip succeeds, @@ -271,7 +271,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) * reader the index-only-scan executor uses. */ Datum -columnar_vm_selftest(PG_FUNCTION_ARGS) +pgcolumnar_vm_selftest(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); BlockNumber blk = (BlockNumber) PG_GETARG_INT32(1); @@ -281,9 +281,9 @@ columnar_vm_selftest(PG_FUNCTION_ARGS) rel = table_open(relid, RowExclusiveLock); - before = ColumnarVMIsVisible(rel, blk); - ColumnarVMSetVisible(rel, blk); - after = ColumnarVMIsVisible(rel, blk); + before = PgColumnarVMIsVisible(rel, blk); + PgColumnarVMSetVisible(rel, blk); + after = PgColumnarVMIsVisible(rel, blk); table_close(rel, RowExclusiveLock); @@ -292,18 +292,18 @@ columnar_vm_selftest(PG_FUNCTION_ARGS) } /* - * columnar_vm_is_visible(rel regclass, blk int) -> bool + * pgcolumnar_vm_is_visible(rel regclass, blk int) -> bool * Read-only probe: is the synthetic block marked all-visible in the VM * fork? Used by the phase-3 tests to check that lazy vacuum sets bits for * all-visible groups and clear-on-write removes them for modified ones. */ Datum -columnar_vm_is_visible(PG_FUNCTION_ARGS) +pgcolumnar_vm_is_visible(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); BlockNumber blk = (BlockNumber) PG_GETARG_INT32(1); Relation rel = table_open(relid, AccessShareLock); - bool vis = ColumnarVMIsVisible(rel, blk); + bool vis = PgColumnarVMIsVisible(rel, blk); table_close(rel, AccessShareLock); PG_RETURN_BOOL(vis); diff --git a/src/columnar_write_state.c b/src/columnar_write_state.c index c94e4eb..e7700f0 100644 --- a/src/columnar_write_state.c +++ b/src/columnar_write_state.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_write_state.c + * pgcolumnar_write_state.c * The columnar writer: batch rows into chunk groups and stripes, and * flush a stripe (data pages + catalog rows) when it fills or at * transaction pre-commit (spec 4, 9). @@ -73,21 +73,21 @@ * predicate could take the shortcut, because it does not build stored bounds * that a later scan trusts; this one does, so it cannot. */ -typedef enum ColumnarFastCmp +typedef enum PgColumnarFastCmp { COLUMNAR_FASTCMP_NONE = 0, COLUMNAR_FASTCMP_I16, COLUMNAR_FASTCMP_I32, COLUMNAR_FASTCMP_I64 -} ColumnarFastCmp; +} PgColumnarFastCmp; /* per-column, per-write-state facts needed for the min/max skip list */ -typedef struct ColumnarColumnDef +typedef struct PgColumnarColumnDef { bool orderable; /* type has a default btree comparison proc */ FmgrInfo cmpFn; /* the comparison proc, when orderable */ Oid collation; /* collation to compare under */ - ColumnarFastCmp fastCmp; /* direct comparison, when the type allows */ + PgColumnarFastCmp fastCmp; /* direct comparison, when the type allows */ /* * int2/int4 column: its exact sum fits an int64 accumulator, so the zone map @@ -104,7 +104,7 @@ typedef struct ColumnarColumnDef bool bloomable; FmgrInfo hashFn; Oid hashCollation; /* collation to hash under (InvalidOid if none) */ -} ColumnarColumnDef; +} PgColumnarColumnDef; /* one column's two streams within one chunk group */ typedef struct ColumnChunkBuffer @@ -146,7 +146,7 @@ typedef struct ChunkGroupBuffer ColumnChunkBuffer *columns; /* array [natts] */ } ChunkGroupBuffer; -struct ColumnarWriteState +struct PgColumnarWriteState { Oid relid; SubTransactionId subid; /* subtransaction that owns the buffer */ @@ -159,7 +159,7 @@ struct ColumnarWriteState bool bloomEnabled; /* columnar.enable_bloom_filter at open time */ int encodeEffort; /* per-table encode_effort at open time */ uint64 storageId; - ColumnarColumnDef *colDefs; /* array [natts], in writeContext */ + PgColumnarColumnDef *colDefs; /* array [natts], in writeContext */ MemoryContext writeContext; /* lives for the transaction */ MemoryContext stripeContext; /* reset after each stripe flush */ @@ -204,28 +204,28 @@ struct ColumnarWriteState * lookup that builds the list. */ bool projInited; - List *projWriters; /* list of ColumnarProjWriter * */ + List *projWriters; /* list of PgColumnarProjWriter * */ }; -/* per-backend registry of pending write states, in ColumnarWriteContext */ -static MemoryContext ColumnarWriteContext = NULL; -static List *ColumnarWriteStates = NIL; +/* per-backend registry of pending write states, in PgColumnarWriteContext */ +static MemoryContext PgColumnarWriteContext = NULL; +static List *PgColumnarWriteStates = NIL; -static void columnar_flush_row_group(ColumnarWriteState *writeState); -static void flush_ws_projections(ColumnarWriteState *writeState); -static ChunkGroupBuffer *columnar_start_chunk_group(ColumnarWriteState *writeState); +static void pgcolumnar_flush_row_group(PgColumnarWriteState *writeState); +static void flush_ws_projections(PgColumnarWriteState *writeState); +static ChunkGroupBuffer *pgcolumnar_start_chunk_group(PgColumnarWriteState *writeState); static uint64 *grow_uint64_array(uint64 *arr, int oldSize, int newSize); -static void columnar_init_col_defs(ColumnarWriteState *writeState); +static void pgcolumnar_init_col_defs(PgColumnarWriteState *writeState); /* - * columnar_cmp_value + * pgcolumnar_cmp_value * Compare two values of a column under its ordering, taking the direct * route when the type allows one and fmgr otherwise. The integer kinds * reproduce their btree comparison exactly, so which route is taken can * never change the answer. */ static inline int32 -columnar_cmp_value(ColumnarColumnDef *def, Datum a, Datum b) +pgcolumnar_cmp_value(PgColumnarColumnDef *def, Datum a, Datum b) { switch (def->fastCmp) { @@ -258,18 +258,18 @@ columnar_cmp_value(ColumnarColumnDef *def, Datum a, Datum b) } /* - * columnar_init_col_defs + * pgcolumnar_init_col_defs * Allocate and fill writeState->colDefs: for each column, resolve the btree * comparison proc (for the per-chunk min/max skip list, spec 7.2) and the * hash proc (for the per-chunk bloom filter, I7). Shared by the base writer * and the projection writer so both carry skip metadata. */ static void -columnar_init_col_defs(ColumnarWriteState *writeState) +pgcolumnar_init_col_defs(PgColumnarWriteState *writeState) { int c; - writeState->colDefs = palloc0(sizeof(ColumnarColumnDef) * writeState->natts); + writeState->colDefs = palloc0(sizeof(PgColumnarColumnDef) * writeState->natts); for (c = 0; c < writeState->natts; c++) { @@ -286,7 +286,7 @@ columnar_init_col_defs(ColumnarWriteState *writeState) { writeState->colDefs[c].orderable = true; fmgr_info_copy(&writeState->colDefs[c].cmpFn, - &tce->cmp_proc_finfo, ColumnarWriteContext); + &tce->cmp_proc_finfo, PgColumnarWriteContext); writeState->colDefs[c].collation = att->attcollation; /* @@ -327,38 +327,38 @@ columnar_init_col_defs(ColumnarWriteState *writeState) */ if (writeState->bloomEnabled && OidIsValid(tce->hash_proc_finfo.fn_oid) && - ColumnarCollationIsDeterministic(att->attcollation)) + PgColumnarCollationIsDeterministic(att->attcollation)) { writeState->colDefs[c].bloomable = true; fmgr_info_copy(&writeState->colDefs[c].hashFn, - &tce->hash_proc_finfo, ColumnarWriteContext); + &tce->hash_proc_finfo, PgColumnarWriteContext); writeState->colDefs[c].hashCollation = att->attcollation; } } } /* - * ColumnarWriteStateStripeCount + * PgColumnarWriteStateStripeCount * How many stripe reservations this write state has taken so far (#311). * * A caller that wants to know which groups IT wrote records this before - * its writes and takes the tail afterwards, because ColumnarGetWriteState + * its writes and takes the tail afterwards, because PgColumnarGetWriteState * can hand back a state that already holds another statement's entries. */ int -ColumnarWriteStateStripeCount(ColumnarWriteState *ws) +PgColumnarWriteStateStripeCount(PgColumnarWriteState *ws) { return ws->nReservedStripeIds; } /* - * ColumnarWriteStateStripeIds + * PgColumnarWriteStateStripeIds * The stripe ids this write state has reserved, in reservation order. * Points at the write state's own array, which stays valid until the * transaction ends. */ uint64 * -ColumnarWriteStateStripeIds(ColumnarWriteState *ws, int *n) +PgColumnarWriteStateStripeIds(PgColumnarWriteState *ws, int *n) { *n = ws->nReservedStripeIds; return ws->reservedStripeIds; @@ -378,17 +378,17 @@ grow_uint64_array(uint64 *arr, int oldSize, int newSize) } /* - * ColumnarGetWriteState + * PgColumnarGetWriteState * Find or create the pending write state for a relation. */ -ColumnarWriteState * -ColumnarGetWriteState(Relation rel) +PgColumnarWriteState * +PgColumnarGetWriteState(Relation rel) { Oid relid = RelationGetRelid(rel); SubTransactionId subid = GetCurrentSubTransactionId(); ListCell *lc; MemoryContext oldContext; - ColumnarWriteState *writeState; + PgColumnarWriteState *writeState; /* * A write state is keyed by (relation, subtransaction) so that a buffer @@ -396,9 +396,9 @@ ColumnarGetWriteState(Relation rel) * rollback of a subtransaction a simple matter of dropping its buffers * (spec 9). */ - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - writeState = (ColumnarWriteState *) lfirst(lc); + writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid || writeState->subid != subid) continue; @@ -418,23 +418,23 @@ ColumnarGetWriteState(Relation rel) return writeState; if (writeState->stripeRowCount > 0) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - ColumnarWriteStates = list_delete_ptr(ColumnarWriteStates, writeState); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + PgColumnarWriteStates = list_delete_ptr(PgColumnarWriteStates, writeState); MemoryContextSwitchTo(oldContext); break; } - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); - writeState = palloc0(sizeof(ColumnarWriteState)); + writeState = palloc0(sizeof(PgColumnarWriteState)); writeState->relid = relid; writeState->subid = subid; /* CopyConstr (not CopyEntry) so attgenerated is preserved: the flush skips @@ -442,19 +442,19 @@ ColumnarGetWriteState(Relation rel) * CreateTupleDescCopy would clear. */ writeState->tupdesc = CreateTupleDescCopyConstr(RelationGetDescr(rel)); writeState->natts = writeState->tupdesc->natts; - writeState->stripeRowLimit = columnar_stripe_row_limit; - writeState->chunkGroupRowLimit = columnar_chunk_group_row_limit; - writeState->compressionType = columnar_compression; - writeState->compressionLevel = columnar_compression_level; + writeState->stripeRowLimit = pgcolumnar_stripe_row_limit; + writeState->chunkGroupRowLimit = pgcolumnar_chunk_group_row_limit; + writeState->compressionType = pgcolumnar_compression; + writeState->compressionLevel = pgcolumnar_compression_level; /* * Whether to build bloom filters at all, captured here with the other * write-time settings rather than consulted per row, so one stripe is * written under one decision. */ - writeState->bloomEnabled = columnar_enable_bloom_filter; + writeState->bloomEnabled = pgcolumnar_enable_bloom_filter; writeState->encodeEffort = COLUMNAR_ENCODE_EFFORT_FULL; - writeState->storageId = ColumnarStorageId(rel); + writeState->storageId = PgColumnarStorageId(rel); /* * Per-table options (spec 7.4) override the instance-wide GUC defaults for @@ -463,9 +463,9 @@ ColumnarGetWriteState(Relation rel) * inserts (spec 9). */ { - ColumnarOptions opts; + PgColumnarOptions opts; - if (ColumnarReadOptions(relid, &opts)) + if (PgColumnarReadOptions(relid, &opts)) { if (opts.stripeRowLimitSet) writeState->stripeRowLimit = opts.stripeRowLimit; @@ -480,12 +480,12 @@ ColumnarGetWriteState(Relation rel) } } - columnar_init_col_defs(writeState); + pgcolumnar_init_col_defs(writeState); - writeState->stripeContext = AllocSetContextCreate(ColumnarWriteContext, + writeState->stripeContext = AllocSetContextCreate(PgColumnarWriteContext, "columnar stripe", ALLOCSET_DEFAULT_SIZES); - writeState->writeContext = ColumnarWriteContext; + writeState->writeContext = PgColumnarWriteContext; writeState->chunkGroups = NIL; writeState->currentGroup = NULL; writeState->stripeRowCount = 0; @@ -496,7 +496,7 @@ ColumnarGetWriteState(Relation rel) writeState->nReservedStripeIds = 0; writeState->reservedStripeIdsSize = 0; - ColumnarWriteStates = lappend(ColumnarWriteStates, writeState); + PgColumnarWriteStates = lappend(PgColumnarWriteStates, writeState); MemoryContextSwitchTo(oldContext); @@ -504,31 +504,31 @@ ColumnarGetWriteState(Relation rel) } /* - * ColumnarEnsureStorageRow + * PgColumnarEnsureStorageRow * Create the native storage catalog row for `rel` if it does not exist, * with exactly the metadata a normal flush would record. Used by * pgcolumnar.parallel_copy's coordinator to pre-create and commit the storage * row before launching concurrent loaders, so each loader (with - * columnar_bulk_parallel_writer set) sees it committed and skips the - * storage-row creation lock. Idempotent -- ColumnarInsertNativeStorageRow + * pgcolumnar_bulk_parallel_writer set) sees it committed and skips the + * storage-row creation lock. Idempotent -- PgColumnarInsertNativeStorageRow * returns if the row already exists. */ void -ColumnarEnsureStorageRow(Relation rel) +PgColumnarEnsureStorageRow(Relation rel) { NativeStorageMetadata s; - ColumnarOptions opts; - int stripeRowLimit = columnar_stripe_row_limit; + PgColumnarOptions opts; + int stripeRowLimit = pgcolumnar_stripe_row_limit; - if (ColumnarReadOptions(RelationGetRelid(rel), &opts) && opts.stripeRowLimitSet) + if (PgColumnarReadOptions(RelationGetRelid(rel), &opts) && opts.stripeRowLimitSet) stripeRowLimit = opts.stripeRowLimit; - s.storageId = ColumnarStorageId(rel); + s.storageId = PgColumnarStorageId(rel); s.relationOid = RelationGetRelid(rel); s.formatVersion = COLUMNAR_NATIVE_VERSION_MAJOR; s.vectorLength = COLUMNAR_NATIVE_VECTOR_LENGTH; s.rowGroupLimit = stripeRowLimit; - ColumnarInsertNativeStorageRow(&s); + PgColumnarInsertNativeStorageRow(&s); } /* @@ -592,7 +592,7 @@ buffered_build_offsets(ColumnChunkBuffer *col, Form_pg_attribute att, * Bounded by chunk_group_row_limit, which is user-settable, so this pass * is a cancellation point for the same reason the decoders are. It is * also on the unique-check fetch path, which had no interrupt check at - * all before #220 and still has none below columnar_fetch_row. + * all before #220 and still has none below pgcolumnar_fetch_row. */ CHECK_FOR_INTERRUPTS(); @@ -600,19 +600,19 @@ buffered_build_offsets(ColumnChunkBuffer *col, Form_pg_attribute att, (uint32) (cursor - col->valueStream.data); if (col->existsStream.data[i]) - (void) ColumnarDecodeValue(att, &cursor, scratch); + (void) PgColumnarDecodeValue(att, &cursor, scratch); } MemoryContextDelete(scratch); } /* - * columnar_start_chunk_group + * pgcolumnar_start_chunk_group * Begin a new chunk group inside the current stripe, allocated in the * stripe memory context. */ static ChunkGroupBuffer * -columnar_start_chunk_group(ColumnarWriteState *writeState) +pgcolumnar_start_chunk_group(PgColumnarWriteState *writeState) { MemoryContext oldContext = MemoryContextSwitchTo(writeState->stripeContext); ChunkGroupBuffer *group = palloc0(sizeof(ChunkGroupBuffer)); @@ -636,14 +636,14 @@ columnar_start_chunk_group(ColumnarWriteState *writeState) } /* - * ColumnarWriteRow + * PgColumnarWriteRow * Append one row to the current stripe, opening a new chunk group when * the current one is full and flushing the stripe when it reaches the * stripe row limit. Returns the stable 1-based row number assigned to the * row (spec 6), so the caller can set the row's item pointer for indexing. */ uint64 -ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, +PgColumnarWriteRow(PgColumnarWriteState *writeState, Relation rel, Datum *values, bool *nulls) { ChunkGroupBuffer *group = writeState->currentGroup; @@ -660,7 +660,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, */ if (!writeState->haveReservation) { - ColumnarReserveRowNumbers(rel, (uint64) writeState->stripeRowLimit, + PgColumnarReserveRowNumbers(rel, (uint64) writeState->stripeRowLimit, &writeState->stripeId, &writeState->stripeFirstRowNumber); writeState->haveReservation = true; @@ -693,7 +693,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, if (group == NULL || group->rowCount >= (uint64) writeState->chunkGroupRowLimit) - group = columnar_start_chunk_group(writeState); + group = pgcolumnar_start_chunk_group(writeState); for (c = 0; c < writeState->natts; c++) { @@ -715,7 +715,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, else { appendStringInfoChar(&col->existsStream, 1); - ColumnarEncodeValue(&col->valueStream, att, values[c]); + PgColumnarEncodeValue(&col->valueStream, att, values[c]); col->valueCount++; /* accumulate the value's hash for the per-chunk bloom filter (I7) */ @@ -738,7 +738,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, /* maintain the per-chunk min/max for orderable types */ if (writeState->colDefs[c].orderable) { - ColumnarColumnDef *def = &writeState->colDefs[c]; + PgColumnarColumnDef *def = &writeState->colDefs[c]; MemoryContext oldContext = MemoryContextSwitchTo(writeState->stripeContext); @@ -760,7 +760,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, * of a serial or timestamp column produces -- cost one * comparison per value instead of two. */ - int32 cmpMax = columnar_cmp_value(def, values[c], + int32 cmpMax = pgcolumnar_cmp_value(def, values[c], col->maxValue); if (cmpMax > 0) @@ -770,7 +770,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, col->maxValue = datumCopy(values[c], att->attbyval, att->attlen); } - else if (columnar_cmp_value(def, values[c], + else if (pgcolumnar_cmp_value(def, values[c], col->minValue) < 0) { if (!att->attbyval) @@ -789,13 +789,13 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, writeState->stripeRowCount++; if (writeState->stripeRowCount >= (uint64) writeState->stripeRowLimit) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); return rowNumber; } /* - * ColumnarBufferedRowByNumber + * PgColumnarBufferedRowByNumber * Reconstruct a single row that is still held in an unflushed write buffer, * addressed by its row number (spec 6). Returns true and fills values/nulls * (by-reference values copied into the current memory context) when the row @@ -809,16 +809,16 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, * to call while the caller holds an index buffer lock. */ bool -ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, +PgColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, Datum *values, bool *nulls) { Oid relid = RelationGetRelid(rel); MemoryContext target = CurrentMemoryContext; ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *ws = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *ws = (PgColumnarWriteState *) lfirst(lc); uint64 offset; uint64 accumulated; ListCell *glc; @@ -876,7 +876,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, if (existsBytes[posInGroup]) { - values[c] = ColumnarDecodeValue(att, &cursor, target); + values[c] = PgColumnarDecodeValue(att, &cursor, target); nulls[c] = false; } else @@ -894,7 +894,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, } /* - * columnar_flush_row_group + * pgcolumnar_flush_row_group * Native-format (PGCN v1) flush. Lay out the accumulated rows as one row * group: each column is a column chunk of [validity bitmap][values], where * the validity bitmap is one bit per row (LSB-first) and the values are the @@ -905,7 +905,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, * column_chunk, zone_map, bloom). */ static void -columnar_flush_row_group(ColumnarWriteState *writeState) +pgcolumnar_flush_row_group(PgColumnarWriteState *writeState) { MemoryContext flushContext; MemoryContext oldContext; @@ -988,7 +988,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) char *finalData; uint32 finalLen; int blockCodec = COLUMNAR_COMPRESSION_NONE; - ColumnarColumnDef *def = &writeState->colDefs[c]; + PgColumnarColumnDef *def = &writeState->colDefs[c]; int vec = 0; bool chunkHasMinMax = false; Datum chunkMin = (Datum) 0; @@ -1090,8 +1090,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) */ if (corpus.len > 0 && writeState->encodeEffort != COLUMNAR_ENCODE_EFFORT_FAST && - !ColumnarFsstDictWins(corpus.data, (uint32) corpus.len)) - ColumnarFsstBuildChunkTable(corpus.data, sampleLen, att, + !PgColumnarFsstDictWins(corpus.data, (uint32) corpus.len)) + PgColumnarFsstBuildChunkTable(corpus.data, sampleLen, att, &fsstTable, &fsstTableLen); /* @@ -1112,7 +1112,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) * imprecise but inverted, so no margin on the sample would be safe. */ if (fsstTable != NULL && - !ColumnarFsstHelpsCompressed(corpus.data, (uint32) corpus.len, + !PgColumnarFsstHelpsCompressed(corpus.data, (uint32) corpus.len, fsstTable, fsstTableLen, writeState->compressionType, writeState->compressionLevel)) @@ -1137,7 +1137,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) uint32 entryValueCount; uint32 entryRawLen; - encType = ColumnarEncodeChunk(col->valueStream.data, + encType = PgColumnarEncodeChunk(col->valueStream.data, col->valueStream.len, att, col->valueCount, fsstTable, fsstTableLen, &encData, &encLen); @@ -1178,8 +1178,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) initStringInfo(&mn); initStringInfo(&mx); - ColumnarEncodeValue(&mn, att, col->minValue); - ColumnarEncodeValue(&mx, att, col->maxValue); + PgColumnarEncodeValue(&mn, att, col->minValue); + PgColumnarEncodeValue(&mx, att, col->maxValue); z->hasMinMax = true; z->minimum = mn.data; z->minimumLen = (uint32) mn.len; @@ -1249,8 +1249,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) initStringInfo(&mn); initStringInfo(&mx); - ColumnarEncodeValue(&mn, att, chunkMin); - ColumnarEncodeValue(&mx, att, chunkMax); + PgColumnarEncodeValue(&mn, att, chunkMin); + PgColumnarEncodeValue(&mx, att, chunkMax); z->hasMinMax = true; z->minimum = mn.data; z->minimumLen = (uint32) mn.len; @@ -1279,7 +1279,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) col->hashBuf.len); } if (hashes.len > 0 && - ColumnarBloomBuild((const uint32 *) hashes.data, + PgColumnarBloomBuild((const uint32 *) hashes.data, hashes.len / sizeof(uint32), &bloom, &bloomLen)) { @@ -1305,7 +1305,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) int usedType; int usedLevel; - ColumnarCompressValueStream(encoded->data, encoded->len, + PgColumnarCompressValueStream(encoded->data, encoded->len, writeState->compressionType, writeState->compressionLevel, &compData, &compLen, @@ -1343,14 +1343,14 @@ columnar_flush_row_group(ColumnarWriteState *writeState) reusedOffset = false; if (dataLength > 0 && CheckRelationLockedByMe(rel, ShareUpdateExclusiveLock, false)) - reusedOffset = ColumnarAllocateFreeSpace(ColumnarStorageId(rel), dataLength, - ColumnarOldestXmin(rel), &fileOffset); + reusedOffset = PgColumnarAllocateFreeSpace(PgColumnarStorageId(rel), dataLength, + PgColumnarOldestXmin(rel), &fileOffset); LockRelationForExtension(rel, ExclusiveLock); if (!reusedOffset) - ColumnarReserveOffset(rel, dataLength, &fileOffset); + PgColumnarReserveOffset(rel, dataLength, &fileOffset); if (dataLength > 0) - ColumnarWriteLogicalData(rel, fileOffset, data->data, dataLength); + PgColumnarWriteLogicalData(rel, fileOffset, data->data, dataLength); UnlockRelationForExtension(rel, ExclusiveLock); { @@ -1361,7 +1361,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) s.formatVersion = COLUMNAR_NATIVE_VERSION_MAJOR; s.vectorLength = COLUMNAR_NATIVE_VECTOR_LENGTH; s.rowGroupLimit = writeState->stripeRowLimit; - ColumnarInsertNativeStorageRow(&s); + PgColumnarInsertNativeStorageRow(&s); } { NativeRowGroupMetadata rg; @@ -1372,7 +1372,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) rg.rowCount = rowCount; rg.byteLength = dataLength; rg.firstRowNumber = writeState->stripeFirstRowNumber; - ColumnarInsertRowGroupRow(&rg); + PgColumnarInsertRowGroupRow(&rg); } for (c = 0; c < natts; c++) { @@ -1391,12 +1391,12 @@ columnar_flush_row_group(ColumnarWriteState *writeState) cc.blockCodec = chunkBlockCodec[c]; cc.pageOffset = fileOffset + chunkOffset[c]; cc.pageLength = chunkLength[c]; - ColumnarInsertColumnChunkRow(&cc); + PgColumnarInsertColumnChunkRow(&cc); } foreach(lc, zoneRows) - ColumnarInsertZoneMapRow((NativeZoneMapMetadata *) lfirst(lc)); + PgColumnarInsertZoneMapRow((NativeZoneMapMetadata *) lfirst(lc)); foreach(lc, bloomRows) - ColumnarInsertBloomRow((NativeBloomMetadata *) lfirst(lc)); + PgColumnarInsertBloomRow((NativeBloomMetadata *) lfirst(lc)); table_close(rel, RowExclusiveLock); @@ -1421,7 +1421,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) * table's relation file and row-number space. On insert, the projected columns * plus the base row number are buffered; at flush the batch is sorted on the * projection's sort key and written as a stripe to the projection's storage, - * reusing the base stripe encoder (ColumnarWriteRow + columnar_flush_row_group). + * reusing the base stripe encoder (PgColumnarWriteRow + pgcolumnar_flush_row_group). * The base row number is stored as a leading int8 column so the projection can * be joined back to the base; deletes/visibility come from the base delete_vector, so * only INSERT fans out (see design/gaps/26-IMPL-projections-phase2-plan.md). @@ -1434,7 +1434,7 @@ typedef struct ProjRow bool *nulls; } ProjRow; -typedef struct ColumnarProjWriter +typedef struct PgColumnarProjWriter { uint64 projStorageId; int ncols; /* number of projection columns (K) */ @@ -1455,14 +1455,14 @@ typedef struct ColumnarProjWriter int nrows; MemoryContext ctx; /* persists: struct arrays, projTupdesc */ MemoryContext rowCtx; /* reset after each stripe flush: row datums */ - ColumnarWriteState *innerWs; /* reused stripe encoder for this projection */ -} ColumnarProjWriter; + PgColumnarWriteState *innerWs; /* reused stripe encoder for this projection */ +} PgColumnarProjWriter; /* - * ColumnarWriteStateProjStripeIds + * PgColumnarWriteStateProjStripeIds * The stripe ids this write state's projection fan-out drew (#345). * * A projection writes through its own inner write state but reserves from - * the BASE relation's stripe counter, because ColumnarWriteRow is called + * the BASE relation's stripe counter, because PgColumnarWriteRow is called * with the base relation (see flush_proj_writer). Its groups are recorded * under the projection's own storage id, so they never appear in the base * relation's row group list. @@ -1478,7 +1478,7 @@ typedef struct ColumnarProjWriter * state has no projection writers. */ uint64 * -ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) +PgColumnarWriteStateProjStripeIds(PgColumnarWriteState *ws, int *n) { ListCell *lc; uint64 *ids = NULL; @@ -1491,7 +1491,7 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) foreach(lc, ws->projWriters) { - ColumnarProjWriter *w = (ColumnarProjWriter *) lfirst(lc); + PgColumnarProjWriter *w = (PgColumnarProjWriter *) lfirst(lc); if (w->innerWs != NULL) total += w->innerWs->nReservedStripeIds; @@ -1502,7 +1502,7 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) ids = (uint64 *) palloc(sizeof(uint64) * total); foreach(lc, ws->projWriters) { - ColumnarProjWriter *w = (ColumnarProjWriter *) lfirst(lc); + PgColumnarProjWriter *w = (PgColumnarProjWriter *) lfirst(lc); int i; if (w->innerWs == NULL) @@ -1516,28 +1516,28 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) /* - * columnar_build_write_state + * pgcolumnar_build_write_state * Allocate a standalone stripe encoder for the given tuple descriptor and - * storage id, not registered in ColumnarWriteStates. Used for a + * storage id, not registered in PgColumnarWriteStates. Used for a * projection's inner writer; carries the same per-chunk min/max and bloom * skip metadata as the base writer so a sorted projection gives tight * min/max ranges for the planner (gap 26). */ -static ColumnarWriteState * -columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, +static PgColumnarWriteState * +pgcolumnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, int stripeRowLimit, int chunkGroupRowLimit, int compType, int compLevel) { MemoryContext oldContext; - ColumnarWriteState *ws; + PgColumnarWriteState *ws; - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); - ws = palloc0(sizeof(ColumnarWriteState)); + ws = palloc0(sizeof(PgColumnarWriteState)); ws->relid = relid; ws->subid = GetCurrentSubTransactionId(); ws->tupdesc = CreateTupleDescCopy(srcTupdesc); @@ -1552,7 +1552,7 @@ columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, * relation. Leaving this unset would zero it, silently dropping bloom * filters from projections while the setting was on. */ - ws->bloomEnabled = columnar_enable_bloom_filter; + ws->bloomEnabled = pgcolumnar_enable_bloom_filter; /* * And under the same encode_effort as its base, for the same reason: a @@ -1561,17 +1561,17 @@ columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, */ ws->encodeEffort = COLUMNAR_ENCODE_EFFORT_FULL; { - ColumnarOptions opts; + PgColumnarOptions opts; - if (ColumnarReadOptions(relid, &opts) && opts.encodeEffortSet) + if (PgColumnarReadOptions(relid, &opts) && opts.encodeEffortSet) ws->encodeEffort = opts.encodeEffort; } ws->storageId = storageId; - columnar_init_col_defs(ws); /* min/max + bloom skip metadata for projections */ - ws->stripeContext = AllocSetContextCreate(ColumnarWriteContext, + pgcolumnar_init_col_defs(ws); /* min/max + bloom skip metadata for projections */ + ws->stripeContext = AllocSetContextCreate(PgColumnarWriteContext, "columnar proj stripe", ALLOCSET_DEFAULT_SIZES); - ws->writeContext = ColumnarWriteContext; + ws->writeContext = PgColumnarWriteContext; ws->chunkGroups = NIL; ws->currentGroup = NULL; ws->stripeRowCount = 0; @@ -1587,7 +1587,7 @@ proj_row_cmp(const void *a, const void *b, void *arg) { const ProjRow *ra = (const ProjRow *) a; const ProjRow *rb = (const ProjRow *) b; - ColumnarProjWriter *w = (ColumnarProjWriter *) arg; + PgColumnarProjWriter *w = (PgColumnarProjWriter *) arg; int i; for (i = 0; i < w->nsort; i++) @@ -1617,7 +1617,7 @@ proj_row_cmp(const void *a, const void *b, void *arg) * stripe to the projection's storage, then reset the buffer. */ static void -flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) +flush_proj_writer(PgColumnarProjWriter *w, Relation tableRel) { int i; @@ -1628,17 +1628,17 @@ flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) qsort_arg(w->rows, w->nrows, sizeof(ProjRow), proj_row_cmp, w); if (w->innerWs == NULL) - w->innerWs = columnar_build_write_state(RelationGetRelid(tableRel), + w->innerWs = pgcolumnar_build_write_state(RelationGetRelid(tableRel), w->projTupdesc, w->projStorageId, w->stripeRowLimit, w->chunkGroupRowLimit, w->compType, w->compLevel); for (i = 0; i < w->nrows; i++) - ColumnarWriteRow(w->innerWs, tableRel, w->rows[i].values, w->rows[i].nulls); + PgColumnarWriteRow(w->innerWs, tableRel, w->rows[i].values, w->rows[i].nulls); if (w->innerWs->stripeRowCount > 0) - columnar_flush_row_group(w->innerWs); + pgcolumnar_flush_row_group(w->innerWs); MemoryContextReset(w->rowCtx); w->nrows = 0; @@ -1646,24 +1646,24 @@ flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) /* * build_proj_writer - * Construct a ColumnarProjWriter for one projection catalog row. + * Construct a PgColumnarProjWriter for one projection catalog row. */ -static ColumnarProjWriter * -build_proj_writer(Relation rel, const ColumnarProjection *proj, +static PgColumnarProjWriter * +build_proj_writer(Relation rel, const PgColumnarProjection *proj, int stripeRowLimit, int chunkGroupRowLimit, int compType, int compLevel) { TupleDesc tableDesc = RelationGetDescr(rel); MemoryContext ctx; MemoryContext oldContext; - ColumnarProjWriter *w; + PgColumnarProjWriter *w; int i; - ctx = AllocSetContextCreate(ColumnarWriteContext, "columnar proj writer", + ctx = AllocSetContextCreate(PgColumnarWriteContext, "columnar proj writer", ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(ctx); - w = palloc0(sizeof(ColumnarProjWriter)); + w = palloc0(sizeof(PgColumnarProjWriter)); w->projStorageId = proj->projStorageId; w->ncols = proj->columnsLen; w->stripeRowLimit = stripeRowLimit; @@ -1734,7 +1734,7 @@ build_proj_writer(Relation rel, const ColumnarProjection *proj, * insert fan-out and the add-projection back-fill. */ static void -append_proj_row(ColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, +append_proj_row(PgColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, uint64 rowNumber, Datum *values, bool *nulls) { MemoryContext oldContext = MemoryContextSwitchTo(w->rowCtx); @@ -1770,14 +1770,14 @@ append_proj_row(ColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, } /* - * ColumnarProjectionFanoutRow + * PgColumnarProjectionFanoutRow * Buffer a freshly inserted row into each additional projection of the - * relation. rowNumber is the base row number returned by ColumnarWriteRow. + * relation. rowNumber is the base row number returned by PgColumnarWriteRow. * The projection writers hang off the base write state, so they share its * (relid, subid) lifecycle. */ void -ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, +PgColumnarProjectionFanoutRow(Relation rel, PgColumnarWriteState *baseWs, uint64 rowNumber, Datum *values, bool *nulls) { TupleDesc tableDesc = RelationGetDescr(rel); @@ -1785,13 +1785,13 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, if (!baseWs->projInited) { - List *projs = ColumnarListProjections(baseWs->storageId); - MemoryContext oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + List *projs = PgColumnarListProjections(baseWs->storageId); + MemoryContext oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); ListCell *pc; foreach(pc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(pc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(pc); if (p->projectionId == 0) continue; /* base projection is the table itself */ @@ -1810,12 +1810,12 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, return; foreach(lc, baseWs->projWriters) - append_proj_row((ColumnarProjWriter *) lfirst(lc), rel, tableDesc, + append_proj_row((PgColumnarProjWriter *) lfirst(lc), rel, tableDesc, rowNumber, values, nulls); } /* - * ColumnarBackfillProjection + * PgColumnarBackfillProjection * Populate a newly declared projection from the table's existing live rows * (gap 26): scan the base and buffer-sort-flush each row into the * projection's storage. Called by add_projection so a projection added to a @@ -1823,28 +1823,28 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, * concurrent writers (ShareLock) so no row is missed. */ void -ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) +PgColumnarBackfillProjection(Relation rel, const PgColumnarProjection *proj) { TupleDesc tableDesc = RelationGetDescr(rel); Oid relid = RelationGetRelid(rel); - int stripeRowLimit = columnar_stripe_row_limit; - int chunkGroupRowLimit = columnar_chunk_group_row_limit; - int compType = columnar_compression; - int compLevel = columnar_compression_level; - ColumnarOptions opts; - ColumnarProjWriter *w; - ColumnarReadState *readState; + int stripeRowLimit = pgcolumnar_stripe_row_limit; + int chunkGroupRowLimit = pgcolumnar_chunk_group_row_limit; + int compType = pgcolumnar_compression; + int compLevel = pgcolumnar_compression_level; + PgColumnarOptions opts; + PgColumnarProjWriter *w; + PgColumnarReadState *readState; Snapshot snapshot; Datum *values; bool *nulls; uint64 rowNumber; - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - if (ColumnarReadOptions(relid, &opts)) + if (PgColumnarReadOptions(relid, &opts)) { if (opts.stripeRowLimitSet) stripeRowLimit = opts.stripeRowLimit; @@ -1857,8 +1857,8 @@ ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) } /* flush any pending base writes so the scan sees this transaction's rows */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); w = build_proj_writer(rel, proj, stripeRowLimit, chunkGroupRowLimit, compType, compLevel); @@ -1867,58 +1867,58 @@ ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) values = palloc(sizeof(Datum) * tableDesc->natts); nulls = palloc(sizeof(bool) * tableDesc->natts); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, values, nulls, &rowNumber)) + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, values, nulls, &rowNumber)) append_proj_row(w, rel, tableDesc, rowNumber, values, nulls); - ColumnarEndRead(readState); + PgColumnarEndRead(readState); flush_proj_writer(w, rel); } /* Flush all projection writers hanging off a base write state. */ static void -flush_ws_projections(ColumnarWriteState *ws) +flush_ws_projections(PgColumnarWriteState *ws) { ListCell *lc; Relation rel; bool any = false; foreach(lc, ws->projWriters) - if (((ColumnarProjWriter *) lfirst(lc))->nrows > 0) + if (((PgColumnarProjWriter *) lfirst(lc))->nrows > 0) any = true; if (!any) return; rel = table_open(ws->relid, RowExclusiveLock); foreach(lc, ws->projWriters) - flush_proj_writer((ColumnarProjWriter *) lfirst(lc), rel); + flush_proj_writer((PgColumnarProjWriter *) lfirst(lc), rel); table_close(rel, RowExclusiveLock); } /* - * ColumnarFlushWriteStateForRelation + * PgColumnarFlushWriteStateForRelation * Flush any pending partial stripe for a single relation. Used at scan * start so data written earlier in this transaction is persisted. */ void -ColumnarFlushWriteStateForRelation(Oid relid) +PgColumnarFlushWriteStateForRelation(Oid relid) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid) continue; if (writeState->stripeRowCount > 0) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); } } /* - * ColumnarForgetWriteStateForRelation + * PgColumnarForgetWriteStateForRelation * Drop the cached write state for a relation without flushing it. Used * after the relation's storage is swapped (columnar.vacuum): the cached * state holds the old storage id, so it must be discarded and a fresh one @@ -1926,102 +1926,102 @@ ColumnarFlushWriteStateForRelation(Oid relid) * buffered rows still needed persisting. */ void -ColumnarForgetWriteStateForRelation(Oid relid) +PgColumnarForgetWriteStateForRelation(Oid relid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarWriteStates == NIL) + if (PgColumnarWriteStates == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - foreach(lc, ColumnarWriteStates) + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid) kept = lappend(kept, writeState); } MemoryContextSwitchTo(oldContext); - ColumnarWriteStates = kept; + PgColumnarWriteStates = kept; } /* - * ColumnarFlushAllPendingWrites + * PgColumnarFlushAllPendingWrites * Flush every pending write state. Called at transaction pre-commit. */ void -ColumnarFlushAllPendingWrites(void) +PgColumnarFlushAllPendingWrites(void) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); } } /* - * ColumnarDiscardAllPendingWrites + * PgColumnarDiscardAllPendingWrites * Forget all pending write states. The backing memory is freed with the * transaction context, so we only clear our static references. */ void -ColumnarDiscardAllPendingWrites(void) +PgColumnarDiscardAllPendingWrites(void) { - ColumnarWriteStates = NIL; - ColumnarWriteContext = NULL; + PgColumnarWriteStates = NIL; + PgColumnarWriteContext = NULL; } /* - * ColumnarWriteStateDiscardSubXact + * PgColumnarWriteStateDiscardSubXact * Drop buffered (unflushed) writes made in an aborting subtransaction. * Stripes already flushed by that subtransaction are made invisible by * the subtransaction abort itself (their catalog rows), so only the * in-memory buffers need discarding here (spec 9). */ void -ColumnarWriteStateDiscardSubXact(SubTransactionId subid) +PgColumnarWriteStateDiscardSubXact(SubTransactionId subid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarWriteStates == NIL) + if (PgColumnarWriteStates == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - foreach(lc, ColumnarWriteStates) + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->subid != subid) kept = lappend(kept, writeState); } MemoryContextSwitchTo(oldContext); - ColumnarWriteStates = kept; + PgColumnarWriteStates = kept; } /* - * ColumnarWriteStatePromoteSubXact + * PgColumnarWriteStatePromoteSubXact * On subtransaction commit, reassign its buffered writes to the parent so * they are flushed when the parent (eventually the top transaction) * commits. */ void -ColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent) +PgColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->subid == subid) writeState->subid = parent; diff --git a/test/analyze_stats.sh b/test/analyze_stats.sh index 6c3f5bb..9f1d79a 100755 --- a/test/analyze_stats.sh +++ b/test/analyze_stats.sh @@ -334,7 +334,7 @@ check "ANALYZE on a wide table is not many times a full scan of it" \ # by number, and each fetch decodes the whole row group the row lives in. When the # ordering column is unclustered those rows are scattered across every group, so the # scan decodes the table many times over -- but core prices the fetch as a page or -# two and picks the index to avoid a sort. columnar_index_fetch_penalty adds the +# two and picks the index to avoid a sort. pgcolumnar_index_fetch_penalty adds the # decode cost, so a sort over the scan wins instead. Measured on the bench, an # unclustered ORDER BY that took minutes on the index dropped to seconds once it # sorted. @@ -443,7 +443,7 @@ check "the penalty is applied before the columnar path is offered, so it can sti # The checks above vary how many rows a plan fetches. This varies *which column* it # references, holding everything else fixed -- same row count, same emitted width, # same plan shape. The deferred index-fetch slot decodes the attribute prefix -# 0..max-referenced (columnar_tableam.c, columnar_slot_decode_upto), so referencing +# 0..max-referenced (columnar_tableam.c, pgcolumnar_slot_decode_upto), so referencing # a late column decodes every column before it. # # Sizing that decode from rel->reltarget->width cannot see the difference: it is diff --git a/test/audit.sh b/test/audit.sh index 55c72bf..04f8943 100755 --- a/test/audit.sh +++ b/test/audit.sh @@ -29,7 +29,7 @@ # group yet; if that fetch cannot answer, _bt_doinsert retries forever. # Cheap standalone cover for what unique_conc scenario 7 exercises. # 4. CREATE INDEX must not leak a relation reference. A parallel index build -# opens a TableScanDesc per participant through columnar_scan_begin (which +# opens a TableScanDesc per participant through pgcolumnar_scan_begin (which # takes a relation reference); the index_build_range_scan callback owns that # scan and must end it, or each participant leaks a reference that surfaces # at transaction commit as "resource was not closed: relation". The callback diff --git a/test/build_san.sh b/test/build_san.sh index b25a7e0..406cb2d 100644 --- a/test/build_san.sh +++ b/test/build_san.sh @@ -16,7 +16,7 @@ # 2. clang, not gcc. This looks like a free choice and is not. The defect this # gate exists to catch (#225) is a 4-byte varlena header read at an unaligned # address, and the read sits behind a 1-byte-header branch that tests the same -# byte (ColumnarVarSizeAnyUnaligned). gcc's -fsanitize=alignment silently +# byte (PgColumnarVarSizeAnyUnaligned). gcc's -fsanitize=alignment silently # drops the check on that guarded load at -O1 -- verified: the load happens # 45,000 times on a low-cardinality-text INSERT, gcc reports none, clang # reports it. A gcc build here would run clean and prove nothing, which is the diff --git a/test/decode_interrupts.sh b/test/decode_interrupts.sh index a043259..b67429f 100755 --- a/test/decode_interrupts.sh +++ b/test/decode_interrupts.sh @@ -109,8 +109,8 @@ check "the stride is not so large that a check never lands" \ # leave a whole group load or a whole run of skipped rows uninterruptible. RD="$SRC/columnar_reader.c" check "the group load checks for interrupts per column chunk" \ - "$(awk '/^columnar_native_load_group\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" + "$(awk '/^pgcolumnar_native_load_group\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" check "the row loop checks for interrupts" \ - "$(awk '/^columnar_native_next_row\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" + "$(awk '/^pgcolumnar_native_next_row\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" pgc_summary diff --git a/test/encode_invariants.sh b/test/encode_invariants.sh index a262883..d54d1ff 100755 --- a/test/encode_invariants.sh +++ b/test/encode_invariants.sh @@ -47,7 +47,7 @@ ROWS="${PGC_ENCINV_ROWS:-4096}" # directly and compares each against a reference implementation of the algorithm # it replaced, which is the promise the rewrites actually made: the same bytes. psql_run "CREATE FUNCTION pgcolumnar.debug_encoding_selftest() - RETURNS SETOF text AS 'pgcolumnar', 'columnar_debug_encoding_selftest' + RETURNS SETOF text AS 'pgcolumnar', 'pgcolumnar_debug_encoding_selftest' LANGUAGE C;" >/dev/null 2>&1 selftest="$(q "SELECT * FROM pgcolumnar.debug_encoding_selftest();")" diff --git a/test/extension_upgrade.sh b/test/extension_upgrade.sh new file mode 100755 index 0000000..efb2c58 --- /dev/null +++ b/test/extension_upgrade.sh @@ -0,0 +1,165 @@ +#!/usr/bin/env bash +# +# pgColumnar extension-upgrade gate (#382). +# +# The failure this exists to catch is invisible to a build and to CI. Every SQL-callable +# function records its C link name in pg_proc.prosrc at CREATE EXTENSION time. Rename a +# link name in C and the old rows point at a symbol the new library no longer exports. +# Replacing only the shared library, which is exactly what a package upgrade does, then +# leaves the extension inert rather than degraded: reading an existing columnar table +# fails with "could not find function columnar_handler", and so does creating one. +# +# It compiles, it links, every suite passes on a fresh install, and every existing +# install is broken. That combination is why this needs its own gate. +# +# Not in run_all_versions.sh, for the same reason pg_upgrade.sh is not: the matrix builds +# one tree per invocation and this needs two builds of the extension at once. It is a +# second gate beside the matrix, run explicitly. +# +# Usage: +# test/extension_upgrade.sh [PG_CONFIG] [OLD_REF] +# +# OLD_REF defaults to the previous release, and may be any ref that still has the old +# link names. The old build is made in a throwaway clone, so the working tree is never +# checked out from under the caller. +set -uo pipefail + +PG_CONFIG=${1:-pg_config} +OLD_REF=${2:-v1.0-alpha} +SRCDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) + +# The cluster binds a port, so it must come from the band portlib.sh carves BELOW the +# kernel's ephemeral range, probed rather than assumed free. Picking out of the +# ephemeral range means the kernel can hand the same port to something else between +# the choice and the bind. harness_selftest enforces this, and caught it here. +. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh" + +command -v "$PG_CONFIG" >/dev/null 2>&1 || { echo "FATAL: $PG_CONFIG not found"; exit 1; } +BINDIR=$($PG_CONFIG --bindir) +SHAREDIR=$($PG_CONFIG --sharedir) +PGMAJ=$($PG_CONFIG --version | grep -oE '[0-9]+' | head -1) + +TMP=$(mktemp -d /tmp/pgc-extupg.XXXXXX) +DATA=$TMP/data +PORT=$(pgc_pick_free_port "$PGC_AUX_PORT_LO" "$PGC_AUX_PORT_HI" "$$") +LOG=$TMP/server.log +fail=0 + +cleanup () { + runuser -u postgres -- "$BINDIR/pg_ctl" -D "$DATA" -w stop >/dev/null 2>&1 + rm -rf "$TMP" +} +trap cleanup EXIT + +runpg () { runuser -u postgres -- "$@"; } +q () { runpg "$BINDIR/psql" -h /tmp -p "$PORT" -d extupg -X -Atc "$1" 2>&1; } + +echo "== extension_upgrade: PG$PGMAJ, old ref $OLD_REF" + +# ---- 1. build and install the old extension, from a throwaway clone ------------------ +# A missing ref must fail, not skip. This gate is invoked deliberately, and a skip that +# exits 0 would let it go inert the moment someone clones without tags. That is the same +# shape as the bug it exists to catch: everything green, nothing checked. +if ! git -C "$SRCDIR" rev-parse --verify -q "$OLD_REF^{commit}" >/dev/null 2>&1; then + echo " FAIL $OLD_REF is not present. Fetch tags, or pass an explicit ref:" + echo " git fetch --tags && test/extension_upgrade.sh $PG_CONFIG " + exit 1 +fi +git clone -q --shared "$SRCDIR" "$TMP/old" || { echo "FATAL: clone failed"; exit 1; } +git -C "$TMP/old" checkout -q --detach "$OLD_REF" || { echo "FATAL: checkout $OLD_REF failed"; exit 1; } +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" clean >/dev/null 2>&1 +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_old.log" 2>&1 \ + || { echo "FAIL old build"; tail -20 "$TMP/build_old.log"; exit 1; } +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ + || { echo "FAIL old install"; exit 1; } + +chown -R postgres "$TMP" 2>/dev/null +runpg "$BINDIR/initdb" -D "$DATA" --locale=C -U postgres >/dev/null 2>&1 \ + || { echo "FAIL initdb"; exit 1; } +{ + echo "shared_preload_libraries='pgcolumnar'" + echo "port=$PORT" +} >> "$DATA/postgresql.conf" +runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w start >/dev/null 2>&1 \ + || { echo "FAIL start"; tail -10 "$LOG"; exit 1; } +runpg "$BINDIR/createdb" -h /tmp -p "$PORT" extupg >/dev/null 2>&1 + +q "CREATE EXTENSION pgcolumnar" >/dev/null +q "CREATE TABLE t (id int, v text) USING pgcolumnar" >/dev/null +q "INSERT INTO t SELECT g, 'v'||g FROM generate_series(1,1000) g" >/dev/null +before_rows=$(q "SELECT count(*) FROM t") +old_ver=$(q "SELECT extversion FROM pg_extension WHERE extname='pgcolumnar'") +echo " old install: $before_rows rows, extversion $old_ver" +[ "$before_rows" = "1000" ] || { echo "FAIL old install did not store rows"; exit 1; } + +# ---- 2. install the tree under test over it, library and scripts --------------------- +# Clean first. This tree may have last been built against another major, and make +# would happily relink those objects into a .so this server cannot load. That is how +# this gate first failed: a pg19 build silently relinked for pg18, the postmaster +# refused to start, and every check below reported a connection error instead. +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" clean >/dev/null 2>&1 +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_new.log" 2>&1 \ + || { echo "FAIL new build"; tail -20 "$TMP/build_new.log"; exit 1; } +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ + || { echo "FAIL new install"; exit 1; } +if ! runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w restart >/dev/null 2>&1; then + echo " FAIL server did not come back after installing the new build" + tail -15 "$LOG" + exit 1 +fi + +# ---- 3. upgrade, and require that it be available at all ---------------------------- +# If the link names did not move, nothing is broken here and the upgrade is a no-op. If +# they did move, ALTER EXTENSION UPDATE is the only route back that keeps user tables, +# so its absence is itself the failure. +new_default=$(grep -oE "default_version = '[^']+'" "$SRCDIR/pgcolumnar.control" | sed "s/.*'\(.*\)'/\1/") +if [ "$new_default" != "$old_ver" ] && \ + [ ! -f "$SHAREDIR/extension/pgcolumnar--$old_ver--$new_default.sql" ]; then + echo " FAIL default_version moved $old_ver -> $new_default with no" + echo " pgcolumnar--$old_ver--$new_default.sql, so an existing install cannot upgrade" + fail=1 +fi + +upd=$(q "ALTER EXTENSION pgcolumnar UPDATE") +case "$upd" in + *ERROR*) echo " FAIL ALTER EXTENSION UPDATE: $upd"; fail=1 ;; + *) echo " ok ALTER EXTENSION UPDATE -> $(q "SELECT extversion FROM pg_extension WHERE extname='pgcolumnar'")" ;; +esac + +# ---- 4. the extension must work on the upgraded install ----------------------------- +chk () { + local label=$1 want=$2 got + got=$(q "$3") + if [ "$got" = "$want" ]; then + echo " PASS $label: $got" + else + echo " FAIL $label: got [$got] want [$want]" + fail=1 + fi +} +chk "existing rows still readable" "1000" "SELECT count(*) FROM t" +q "INSERT INTO t VALUES (0,'x')" >/dev/null +chk "insert into an existing table" "1001" "SELECT count(*) FROM t" +q "CREATE TABLE t2 (a int) USING pgcolumnar" >/dev/null +q "INSERT INTO t2 SELECT generate_series(1,3)" >/dev/null +chk "new columnar table creatable" "3" "SELECT count(*) FROM t2" +chk "access method still bound" "pgcolumnar" "SELECT amname FROM pg_am WHERE amname='pgcolumnar'" +chk "maintenance function callable" "" "SELECT pgcolumnar.vacuum('t')" + +# Every C function's recorded link name must resolve in the library we just installed. +# This is the general form of the bug, so it catches the next rename as well as this one. +missing=$(q "SELECT string_agg(p.proname||' -> '||p.prosrc, ', ') + FROM pg_proc p + JOIN pg_depend d ON d.objid = p.oid AND d.deptype = 'e' + JOIN pg_extension e ON e.oid = d.refobjid AND e.extname = 'pgcolumnar' + WHERE p.prolang = (SELECT oid FROM pg_language WHERE lanname='c') + AND p.prosrc !~ '^pgcolumnar'") +if [ -n "$missing" ]; then + echo " FAIL link names left outside the pgcolumnar namespace: $missing" + fail=1 +else + echo " PASS every C function's link name is namespaced" +fi + +echo "== extension_upgrade: $([ $fail -eq 0 ] && echo PASS || echo FAIL)" +exit $fail diff --git a/test/fsst_margin.sh b/test/fsst_margin.sh index e7457ec..88793fb 100755 --- a/test/fsst_margin.sh +++ b/test/fsst_margin.sh @@ -2,7 +2,7 @@ # # The FSST keep/drop cost margin (#155, #271). # -# ColumnarFsstHelpsCompressed used to keep FSST on any compressed win at all, +# PgColumnarFsstHelpsCompressed used to keep FSST on any compressed win at all, # however small, and the per-vector FSST encode is a dominant cost of a text or # varlena load. pgcolumnar.fsst_min_gain_percent requires the win to clear a # margin before FSST is kept. diff --git a/test/fuzz_parquet.sh b/test/fuzz_parquet.sh index 4fdf412..277b46c 100755 --- a/test/fuzz_parquet.sh +++ b/test/fuzz_parquet.sh @@ -10,7 +10,7 @@ # a malformed file makes the backend raise an ERROR, never die. # # That is the property #210 broke. A crafted footer reached unbounded recursion -# in ColumnarThriftSkip, and a crafted schema chain reached a second one in +# in PgColumnarThriftSkip, and a crafted schema chain reached a second one in # walk_schema, either of which takes down the whole cluster rather than the # session. Both were found by reading code. This suite is the machine that was # missing. diff --git a/test/harness_selftest.sh b/test/harness_selftest.sh index 6853a0e..ca73439 100755 --- a/test/harness_selftest.sh +++ b/test/harness_selftest.sh @@ -180,7 +180,7 @@ RUNNER="$TESTDIR/run_all_versions.sh" # not a suite the ordinary five-major matrix can carry. not_a_suite() { case "$1" in - lib|portlib|run_all_versions|build_all_versions|devloop|rebuild|native_scale|build_san|run_san|run_coverage|pg_upgrade) return 0 ;; + lib|portlib|run_all_versions|build_all_versions|devloop|rebuild|native_scale|build_san|run_san|run_coverage|pg_upgrade|extension_upgrade) return 0 ;; *) return 1 ;; esac } diff --git a/test/index_only.sh b/test/index_only.sh index a0fe6b2..a34d6fe 100644 --- a/test/index_only.sh +++ b/test/index_only.sh @@ -59,7 +59,7 @@ check "iv row count" "$(q "SELECT count(*) FROM iv;")" "50000" # Freshly written groups are never all-visible until a vacuum runs. check "before vacuum: block 50 not all-visible" "$(q "SELECT pgcolumnar.vm_is_visible('iv', 50);")" "f" -# Plain VACUUM (ShareUpdateExclusiveLock) drives columnar_relation_vacuum, which +# Plain VACUUM (ShareUpdateExclusiveLock) drives pgcolumnar_relation_vacuum, which # marks the all-visible groups. psql_run "VACUUM iv;" check "after vacuum: block 10 all-visible" "$(q "SELECT pgcolumnar.vm_is_visible('iv', 10);")" "t" diff --git a/test/native_cancel.sh b/test/native_cancel.sh index f8c867d..57d259b 100755 --- a/test/native_cancel.sh +++ b/test/native_cancel.sh @@ -4,7 +4,7 @@ # # The executor checks for interrupts once per tuple it receives, which is no help # while a scan is doing work without producing a tuple. The expensive case is -# loading a row group: columnar_native_load_group() reads and decodes every +# loading a row group: pgcolumnar_native_load_group() reads and decodes every # column chunk of the group before the row loop can iterate once, and a vector # holds up to pgcolumnar.chunk_group_row_limit values, which is user-settable and # unbounded. Without interrupt checks inside that load, statement_timeout, @@ -82,7 +82,7 @@ check "the short timeout is what fired" \ # Which guard this actually proves, established by removing each one rather than # from the description above: it is COLUMNAR_DECODE_INTERRUPT in # columnar_encoding.c, the per-value decode-loop check on a 65536 stride, not the -# per-column-chunk CHECK_FOR_INTERRUPTS in columnar_native_load_group(). Deleting +# per-column-chunk CHECK_FOR_INTERRUPTS in pgcolumnar_native_load_group(). Deleting # the per-chunk check leaves this suite green, because a two-column group reaches # it only twice; disabling the decode-loop macro makes cancel converge on full # (151 ms against 150) and fails this check, which is the behaviour the paragraph diff --git a/test/native_fastdecode.sh b/test/native_fastdecode.sh index 9eb9e2a..961689a 100755 --- a/test/native_fastdecode.sh +++ b/test/native_fastdecode.sh @@ -2,11 +2,11 @@ # # pgColumnar #289: fast decode of attbyval fixed-width columns. # -# The read path inlines the by-value decode in columnar_native_next_row: for an -# attbyval column it does the same fetch_att + advance ColumnarDecodeValue does, +# The read path inlines the by-value decode in pgcolumnar_native_next_row: for an +# attbyval column it does the same fetch_att + advance PgColumnarDecodeValue does, # but without the out-of-line call and its own attbyval branch (the per-row # decode dispatch #289 profiled as hot). By-reference (uuid) and varlena (text, -# numeric) columns keep the ColumnarDecodeValue path and serve as controls. This +# numeric) columns keep the PgColumnarDecodeValue path and serve as controls. This # test proves the inlined values are identical to the call path across every # byval fixed type, every NULL pattern, every encoding, per-vector skipping, # deletes and ADD COLUMN, plus adversarial bit patterns. diff --git a/test/native_fetch_cache.sh b/test/native_fetch_cache.sh index e6b3c8d..820f226 100755 --- a/test/native_fetch_cache.sh +++ b/test/native_fetch_cache.sh @@ -2,7 +2,7 @@ # # pgColumnar fetch-by-row-number cache (issue #143). # -# ColumnarReadRowByNumber() used to read and decode a whole row group per row +# PgColumnarReadRowByNumber() used to read and decode a whole row group per row # returned, so fetching N rows out of one group cost N times the group. A # statement-scoped cache of the decoded group removes the repeat. # @@ -94,7 +94,7 @@ check "a hit re-checks the group geometry it was filled with" \ "$(grep -cE 'entry->fileOffset != rg->fileOffset' "$SRC/columnar_reader.c")" "1" check "the cache is released at executor end, not only at transaction end" \ - "$(grep -c 'ColumnarDiscardFetchCache' "$SRC/columnar_tableam.c")" "2" + "$(grep -c 'PgColumnarDiscardFetchCache' "$SRC/columnar_tableam.c")" "2" # --- #353: a wide group's decode scratch must not blow the fetch cap ---------- # The by-row-number decode allocated its intermediates -- the decompressed region diff --git a/test/native_fetch_interrupt.sh b/test/native_fetch_interrupt.sh index a9b962f..67cdcad 100755 --- a/test/native_fetch_interrupt.sh +++ b/test/native_fetch_interrupt.sh @@ -2,7 +2,7 @@ # # pgColumnar fetch-path interrupt guard (#212). # -# columnar_fetch_row is reached once per candidate item pointer by +# pgcolumnar_fetch_row is reached once per candidate item pointer by # _bt_check_unique() during a unique INSERT, and each call reads the row-group # list out of the catalog. Before #212 that path had no CHECK_FOR_INTERRUPTS -- # the three checks already in columnar_reader.c are all on the scan/decode path, @@ -30,21 +30,21 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" -# The body of columnar_fetch_row, from its definition to its closing brace. -body="$(awk '/^columnar_fetch_row\(/{p=1} p{print} p&&/^}/{exit}' "$SRC/columnar_reader.c")" +# The body of pgcolumnar_fetch_row, from its definition to its closing brace. +body="$(awk '/^pgcolumnar_fetch_row\(/{p=1} p{print} p&&/^}/{exit}' "$SRC/columnar_reader.c")" # The guard has to be in this function, not merely in the file. The interrupt # checks elsewhere in columnar_reader.c are on the scan/decode path, which the # unique liveness fetch never enters, so counting the file would pass vacuously. -check "columnar_fetch_row carries an interrupt check" \ +check "pgcolumnar_fetch_row carries an interrupt check" \ "$(printf '%s\n' "$body" | grep -c 'CHECK_FOR_INTERRUPTS' | awk '{print ($1>=1)?"yes":"no"}')" \ "yes" # And it has to run before the per-fetch catalog read it guards, or the fetch # does its work before ever reaching a cancellation point -- which is the state -# #212 was in. Assert the check precedes the ColumnarReadRowGroupList call. +# #212 was in. Assert the check precedes the PgColumnarReadRowGroupList call. cfi_line="$(printf '%s\n' "$body" | grep -n 'CHECK_FOR_INTERRUPTS' | head -1 | cut -d: -f1)" -rgl_line="$(printf '%s\n' "$body" | grep -n 'ColumnarReadRowGroupList' | head -1 | cut -d: -f1)" +rgl_line="$(printf '%s\n' "$body" | grep -n 'PgColumnarReadRowGroupList' | head -1 | cut -d: -f1)" check "the interrupt check precedes the per-fetch catalog read" \ "$( [ -n "$cfi_line" ] && [ -n "$rgl_line" ] && [ "$cfi_line" -lt "$rgl_line" ] && echo yes || echo no )" \ "yes" diff --git a/test/native_fetch_position.sh b/test/native_fetch_position.sh index af988ae..1680ef5 100755 --- a/test/native_fetch_position.sh +++ b/test/native_fetch_position.sh @@ -228,7 +228,7 @@ check "doubling the row group does not double the cost of the same fetches" \ SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" check "the rank comes from a prefix rather than a loop over earlier rows" \ - "$(grep -c 'present = columnar_rank_before' "$SRC/columnar_reader.c")" "1" + "$(grep -c 'present = pgcolumnar_rank_before' "$SRC/columnar_reader.c")" "1" check "a varying-length column reaches its value through an offset table" \ "$(grep -c 'entry->valOffset\[c\]\[present\]' "$SRC/columnar_reader.c")" "1" diff --git a/test/native_fetch_projection.sh b/test/native_fetch_projection.sh index 8283133..0e3db3a 100755 --- a/test/native_fetch_projection.sh +++ b/test/native_fetch_projection.sh @@ -140,22 +140,22 @@ check "and its values are right with nothing decoded from the base" \ SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" check "the visibility-only caller decodes nothing" \ - "$(grep -c 'ColumnarRowIsLive(rel, snap, baseRow)' "$SRC/columnar_projection.c")" "1" + "$(grep -c 'PgColumnarRowIsLive(rel, snap, baseRow)' "$SRC/columnar_projection.c")" "1" check "the reconstruct caller asks only for uncovered columns" \ - "$(grep -c 'ColumnarReadRowByNumberCols(rel, snap, baseRow' "$SRC/columnar_projection.c")" "1" + "$(grep -c 'PgColumnarReadRowByNumberCols(rel, snap, baseRow' "$SRC/columnar_projection.c")" "1" # Scoped to the function rather than counting a string across the file: the # string appears legitimately elsewhere now that the index fetch also asks only # for liveness, and a whole-file count turned that correct second use into a # failure. -deltuples="$(awk '/^columnar_index_delete_tuples\(/,/^}/' "$SRC/columnar_tableam.c")" +deltuples="$(awk '/^pgcolumnar_index_delete_tuples\(/,/^}/' "$SRC/columnar_tableam.c")" check "index deletion asks whether the row is live" \ - "$(case "$deltuples" in *ColumnarRowIsLive*) echo yes ;; *) echo no ;; esac)" "yes" + "$(case "$deltuples" in *PgColumnarRowIsLive*) echo yes ;; *) echo no ;; esac)" "yes" check "and does not decode the row to find out" \ - "$(case "$deltuples" in *ColumnarReadRowByNumber*) echo "no (still decodes)" ;; + "$(case "$deltuples" in *PgColumnarReadRowByNumber*) echo "no (still decodes)" ;; *) echo yes ;; esac)" "yes" # and the convention that made an empty set mean its opposite stays gone: the diff --git a/test/native_format.sh b/test/native_format.sh index 6146e60..8c2636d 100755 --- a/test/native_format.sh +++ b/test/native_format.sh @@ -5,7 +5,7 @@ # Two layers carry a version. The native data format stamps a major version into # pgcolumnar.storage.format_version (PGCN v1); the physical metapage stamps # versionMajor/versionMinor into block 0. Only the metapage version is checked on -# read -- ColumnarReadMetapage rejects a version it does not understand -- so that +# read -- PgColumnarReadMetapage rejects a version it does not understand -- so that # guard is the thing standing between a future, incompatible layout and a silent # misread of old bytes. This suite pins both stamps and proves the guard fires. # @@ -14,7 +14,7 @@ # accidental bump goes red here rather than shipping unnoticed; # 2. a metapage version this build does not understand is REJECTED with a clean # error and a surviving backend -- not misread as valid data. A test-only -# hook (columnar_debug_set_metapage_version, bound here rather than shipped, +# hook (pgcolumnar_debug_set_metapage_version, bound here rather than shipped, # like the gap suite's advance hook) plants the bad version; # 3. within a version, a diverse-typed table round-trips byte-for-byte against a # heap mirror, so "same version" genuinely means "same data back". @@ -29,7 +29,7 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" # Bind the internal test hook (deliberately not in the shipped catalog): it # overwrites the metapage version so we can confirm the read-side guard. psql_run "CREATE FUNCTION pgcolumnar.debug_set_metapage_version(regclass, int, int) - RETURNS void AS 'pgcolumnar', 'columnar_debug_set_metapage_version' + RETURNS void AS 'pgcolumnar', 'pgcolumnar_debug_set_metapage_version' LANGUAGE C;" # Error text (stderr) of a failing statement; empty when it succeeds. Used to @@ -71,7 +71,7 @@ check "table reads at the version this build wrote" "$(q 'SELECT count(*) FROM b # Pin the metapage major -- the version that is actually enforced on read, and # the one this suite exists to guard (format_version above is stamped but not -# checked). ColumnarReadMetapage rejects on versionMajor != COLUMNAR_VERSION_MAJOR, +# checked). PgColumnarReadMetapage rejects on versionMajor != COLUMNAR_VERSION_MAJOR, # so re-stamping the current major must be a no-op the read accepts. Bump # COLUMNAR_VERSION_MAJOR by accident and this goes red; on a deliberate bump, # change the 2 below in the same commit. @@ -95,7 +95,7 @@ check "backend survives the rejection" "$(q 'SELECT 1;')" "1" # --- 4: native format_version is now enforced on read too (#240 decision) ----- # The metapage version above guards the physical layout; format_version is the # independent data-format stamp. It used to be written and never read; it is now a -# read-side guard (ColumnarCheckNativeFormatVersion at scan open), so a future +# read-side guard (PgColumnarCheckNativeFormatVersion at scan open), so a future # PGCN version that keeps the metapage layout but changes the encoding is rejected # rather than misread. A catalog UPDATE stands in for that future version -- the # value is read from pgcolumnar.storage, so no on-disk bytes need forging. @@ -106,8 +106,8 @@ check "table reads at native format_version 1" "$(q 'SELECT count(*) FROM fv;')" fvsid="$(storage_id_of fv)" psql_run "UPDATE pgcolumnar.storage SET format_version = 99 WHERE storage_id = $fvsid;" # Three decode shapes must all reject. A seq scan opens a read state -# (ColumnarBeginReadWithStorage); the zone-map-only aggregate answers from metadata -# without one (ColumnarBeginAggScan); and an index-scan fetch of a non-key column +# (PgColumnarBeginReadWithStorage); the zone-map-only aggregate answers from metadata +# without one (PgColumnarBeginAggScan); and an index-scan fetch of a non-key column # decodes through the by-row-number fetch path, which reaches neither scan-open # guard -- it is the case the first cut missed. All are keyed to the same catalog # UPDATE standing in for a future format version. diff --git a/test/native_gap.sh b/test/native_gap.sh index 4c898d7..155a47e 100644 --- a/test/native_gap.sh +++ b/test/native_gap.sh @@ -7,7 +7,7 @@ # leaves (the file was shortened but the highwater was not). A write whose target # block is beyond EOF must fill the gap with empty pages and succeed, so the state # self-heals on the next write. This suite forces that state with a test-only hook -# (columnar_debug_advance_reserved_offset, bound here rather than shipped) and +# (pgcolumnar_debug_advance_reserved_offset, bound here rather than shipped) and # asserts writes across the gap succeed, data stays correct against a heap mirror, # the file physically materializes the gap, and the table is fully usable after. # @@ -20,7 +20,7 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" # bind the internal test hook (deliberately not in the shipped catalog). psql_run "CREATE FUNCTION pgcolumnar.debug_advance_reserved_offset(regclass, int) - RETURNS void AS 'pgcolumnar', 'columnar_debug_advance_reserved_offset' + RETURNS void AS 'pgcolumnar', 'pgcolumnar_debug_advance_reserved_offset' LANGUAGE C;" psql_run "CREATE TABLE h (id int, v text);" diff --git a/test/native_groupagg.sh b/test/native_groupagg.sh index fab035e..5327d4f 100644 --- a/test/native_groupagg.sh +++ b/test/native_groupagg.sh @@ -4,7 +4,7 @@ # # The grouped path fires for SELECT , agg(col) ... [WHERE ...] GROUP BY # over one columnar relation. It reads each surviving row with -# ColumnarReadNextRow (WHERE pushed down for group/vector skipping), rechecks the +# PgColumnarReadNextRow (WHERE pushed down for group/vector skipping), rechecks the # full WHERE, evaluates the group keys, and scatters the row into an # open-addressing hash table whose per-group accumulators fold in scan order. # diff --git a/test/native_index.sh b/test/native_index.sh index 58d761a..2c59088 100644 --- a/test/native_index.sh +++ b/test/native_index.sh @@ -2,7 +2,7 @@ # # pgColumnar native fetch-by-row-number (Phase D6a): index scan, bitmap scan and # unique/primary-key enforcement on a native-format (PGCN v1) table. All route -# through ColumnarReadRowByNumber, which before D6a had only a 2.2 path (index +# through PgColumnarReadRowByNumber, which before D6a had only a 2.2 path (index # scans returned 0 rows, unique was silently unenforced). This suite proves parity # with a heap mirror under a forced index path. Index-only scan (visibility map) # is D6c; native projection storage is D6d; delete/update is D6b. @@ -26,7 +26,7 @@ check "row count" "$(q 'SELECT count(*) FROM n;')" "8000" # Force the index path (seqscan off) in a single session (two -c, quiet so the SET # tag is not printed), and confirm scalar results match the heap oracle. Filtered # aggregates fall back from the zone-map path to an index scan that fetches each -# row via ColumnarReadRowByNumber. +# row via PgColumnarReadRowByNumber. iscan() { env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \ -d "$PGC_DB" -q -At -c "SET enable_seqscan=off" -c "$1" 2>/dev/null diff --git a/test/native_parquet_stack.sh b/test/native_parquet_stack.sh index 3d62a43..1269b8d 100755 --- a/test/native_parquet_stack.sh +++ b/test/native_parquet_stack.sh @@ -11,7 +11,7 @@ # The two recursions are independent and each needs its own guard -- one does not # cover the other, because they are reached at different points: # -# 1. ColumnarThriftSkip recurses through nested structs (and lists of structs). +# 1. PgColumnarThriftSkip recurses through nested structs (and lists of structs). # Every unrecognised metadata field is skipped through it, so it is reached # straight from the footer bytes, before any schema is interpreted. A footer # that is just N struct openers nests N deep. @@ -24,7 +24,7 @@ # Both are guarded with check_stack_depth(), which turns the crash into a caught # ERROR (SQLSTATE 54001, statement_too_complex). That each guard is load-bearing # for its own vector was proven by removal at the depths that actually overflow an -# 8 MB stack: with only the ColumnarThriftSkip guard present a schema chain still +# 8 MB stack: with only the PgColumnarThriftSkip guard present a schema chain still # SIGSEGVs, and with only the walk_schema guard present a nested-struct footer # still SIGSEGVs. # @@ -71,7 +71,7 @@ def wrap(meta): # Vector 1: N nested struct openers. 0x1c is a compact-protocol field header # meaning (delta 1, type TC_STRUCT) -- "open a struct". N of them nest N deep -# through ColumnarThriftSkip. This never parses a schema; it crashes on the +# through PgColumnarThriftSkip. This never parses a schema; it crashes on the # top-level field skip. def nested(n): return wrap(b'\x1c' * n) diff --git a/test/native_reclaim_cycles.sh b/test/native_reclaim_cycles.sh index 9068970..153e58c 100644 --- a/test/native_reclaim_cycles.sh +++ b/test/native_reclaim_cycles.sh @@ -3,7 +3,7 @@ # pgColumnar physical reclaim, repeated compact_rewrite cycles (Phase F). # # Regression guard for the free-space allocator self-conflict fixed in PR #84: -# ColumnarAllocateFreeSpace consumed a free_space row without a +# PgColumnarAllocateFreeSpace consumed a free_space row without a # CommandCounterIncrement, so a compact_rewrite that wrote MORE THAN ONE group in # one command (many allocations) re-selected the just-consumed row and died with # "tuple already updated by self". It only fires once reusable free space exists diff --git a/test/native_reclaim_frag.sh b/test/native_reclaim_frag.sh index 0e94d51..76f1bc3 100644 --- a/test/native_reclaim_frag.sh +++ b/test/native_reclaim_frag.sh @@ -12,7 +12,7 @@ # GUC on they collapse to FEWER free_space rows than with it off (direct, # deterministic evidence the coalesce path runs); and # * coalesce is never worse than whole-range reuse for the final file size. -# The assert-build no-overlap validator (ColumnarCheckFreeSpaceNoOverlap) runs at +# The assert-build no-overlap validator (PgColumnarCheckFreeSpaceNoOverlap) runs at # the end of every compaction here, so this also exercises the tiling invariant # on variable-size ranges. # diff --git a/test/native_reclaim_reconcile.sh b/test/native_reclaim_reconcile.sh index 6393510..b94a5f4 100644 --- a/test/native_reclaim_reconcile.sh +++ b/test/native_reclaim_reconcile.sh @@ -4,7 +4,7 @@ # transactional, so normal retirement is atomic. The one seam is physical # end-truncation: a crash in its narrow window can leave a free_space row that # overlaps a live group (the highwater was lowered but the row's delete rolled -# back, and a later insert placed a live group there). ColumnarReconcileFreeList +# back, and a later insert placed a live group there). PgColumnarReconcileFreeList # runs at the start of every reuse op (compact_rewrite, recluster) and drops any # free_space row overlapping a live row-group footprint, so it cannot be handed # out on top of a live group. diff --git a/test/native_rewrite.sh b/test/native_rewrite.sh index e4d6260..9fec35b 100755 --- a/test/native_rewrite.sh +++ b/test/native_rewrite.sh @@ -95,13 +95,13 @@ check "compact_rewrite holds no AccessExclusiveLock" "${locks##*|}" "0" # --------------------------------------------------------------------------- # What makes the statement-scoped fetch cache safe (#143, #148). # -# ColumnarReadRowByNumber caches a decoded row group keyed by +# PgColumnarReadRowByNumber caches a decoded row group keyed by # (storage_id, group_number). That is only sound because such a pair is never # re-served with different bytes, which rests on two allocator properties rather # than on any locking: # # group numbers come from the monotonic metapage counter reserved in -# ColumnarReserveRowNumbers (meta->reservedStripeId += 1), never from the +# PgColumnarReserveRowNumbers (meta->reservedStripeId += 1), never from the # catalog maximum, so retiring a group does not free its number for reuse; # # a rewrite writes into freshly allocated storage rather than over the old. diff --git a/test/native_vacuum_race.sh b/test/native_vacuum_race.sh index 632722d..d6b752e 100644 --- a/test/native_vacuum_race.sh +++ b/test/native_vacuum_race.sh @@ -2,12 +2,12 @@ # # Regression for #295: compaction must not drop rows committed by another # transaction while it waited for (or before it took) its read snapshot. -# columnar_compact_relation used the caller's pre-lock statement snapshot, so a +# pgcolumnar_compact_relation used the caller's pre-lock statement snapshot, so a # row group committed after that snapshot was invisible to the rewrite and was # destroyed by the relfilenode swap. The fix takes a fresh snapshot after the # lock. Here session B pins a REPEATABLE READ snapshot, session A commits 100 # rows, then B runs the maintenance op: all 150 rows must survive. -# Covers vacuum() and vacuum_sorted() (columnar_compact_relation) and cluster() +# Covers vacuum() and vacuum_sorted() (pgcolumnar_compact_relation) and cluster() # (its Z-order twin). Written fresh for pgColumnar. # # Usage: test/native_vacuum_race.sh [PG_CONFIG] diff --git a/test/parallel_vector_agg.sh b/test/parallel_vector_agg.sh index 14cecd5..b70c977 100644 --- a/test/parallel_vector_agg.sh +++ b/test/parallel_vector_agg.sh @@ -218,7 +218,7 @@ check "grouped few-groups-many-workers: parallel == serial (#349)" "$GW_VEC" "$G # require the parallel answer to match the serial one. # # What this does NOT establish, measured rather than assumed: it does not prove -# the leader-side flush in ColumnarInitializeDSMGroupAggScan. Removing that flush +# the leader-side flush in PgColumnarInitializeDSMGroupAggScan. Removing that flush # -- and the ungrouped one this is modelled on -- leaves both H2 checks green, # with in-transaction INSERT and DELETE and a confirmed parallel plan, because # the write and delete buffers are already flushed at the command boundary before diff --git a/test/pbt/columnar.h b/test/pbt/columnar.h index acc3e60..2e4599d 100644 --- a/test/pbt/columnar.h +++ b/test/pbt/columnar.h @@ -24,30 +24,30 @@ #define PG_UINT32_MAX 0xFFFFFFFFU #endif -typedef struct ColumnarBlockReader +typedef struct PgColumnarBlockReader { const char *raw; uint64 valueCount; int width; uint64 pos; -} ColumnarBlockReader; +} PgColumnarBlockReader; -extern int ColumnarEncodeChunk(const char *raw, uint32 rawLen, +extern int PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen); -extern char *ColumnarDecodeChunk(const char *enc, uint32 encLen, +extern char *PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx); -extern bool ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut); -extern const char *ColumnarEncodingName(int encodingType); -extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +extern const char *PgColumnarEncodingName(int encodingType); +extern void PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width); -extern bool ColumnarBlockNextRun(ColumnarBlockReader *br, +extern bool PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen); #endif /* PGCOLUMNAR_PBT_COLUMNAR_H */ diff --git a/test/pbt/test_encoding.c b/test/pbt/test_encoding.c index 25a3765..42f6e14 100644 --- a/test/pbt/test_encoding.c +++ b/test/pbt/test_encoding.c @@ -3,7 +3,7 @@ * standalone against the test/pbt PostgreSQL shim. * * The governing property is round-trip: for any raw value stream, - * ColumnarDecodeChunk(ColumnarEncodeChunk(raw)) reproduces the exact bytes. It + * PgColumnarDecodeChunk(PgColumnarEncodeChunk(raw)) reproduces the exact bytes. It * is exercised over randomized data shaped to hit each encoding (constant, * alternating, monotonic, clustered, low-cardinality, runs, random) across all * fixed widths, floats (gorilla), and varlena (dict), plus explicit boundary @@ -64,8 +64,8 @@ check_fixed(int w, Oid typid, uint32 n, const char *raw) att.attbyval = true; att.atttypid = typid; - code = ColumnarEncodeChunk(raw, rawLen, &att, n, NULL, 0, &enc, &encLen); - dec = ColumnarDecodeChunk(enc, encLen, code, &att, n, rawLen, NULL, 0, NULL); + code = PgColumnarEncodeChunk(raw, rawLen, &att, n, NULL, 0, &enc, &encLen); + dec = PgColumnarDecodeChunk(enc, encLen, code, &att, n, rawLen, NULL, 0, NULL); checks++; if (rawLen > 0 && memcmp(dec, raw, rawLen) != 0) @@ -73,7 +73,7 @@ check_fixed(int w, Oid typid, uint32 n, const char *raw) failures++; fprintf(stderr, "FAIL fixed w=%d typid=%u n=%u code=%s(%d) encLen=%u\n", - w, typid, n, ColumnarEncodingName(code), code, encLen); + w, typid, n, PgColumnarEncodingName(code), code, encLen); } } @@ -272,20 +272,20 @@ gen_varlena(uint32 n, int shape) { char *tbl = NULL; uint32 tblLen = 0; - bool haveTbl = ColumnarFsstBuildChunkTable(s.data, (uint32) s.len, + bool haveTbl = PgColumnarFsstBuildChunkTable(s.data, (uint32) s.len, &att, &tbl, &tblLen); - code = ColumnarEncodeChunk(s.data, (uint32) s.len, &att, n, + code = PgColumnarEncodeChunk(s.data, (uint32) s.len, &att, n, haveTbl ? tbl : NULL, haveTbl ? tblLen : 0, &enc, &encLen); - dec = ColumnarDecodeChunk(enc, encLen, code, &att, n, (uint32) s.len, + dec = PgColumnarDecodeChunk(enc, encLen, code, &att, n, (uint32) s.len, haveTbl ? tbl : NULL, haveTbl ? tblLen : 0, NULL); checks++; if (s.len > 0 && memcmp(dec, s.data, s.len) != 0) { failures++; fprintf(stderr, "FAIL varlena n=%u code=%s rawLen=%d\n", - n, ColumnarEncodingName(code), s.len); + n, PgColumnarEncodingName(code), s.len); } if (haveTbl) free(tbl); diff --git a/test/pushdown_report.sh b/test/pushdown_report.sh index 2c2e854..026f11a 100755 --- a/test/pushdown_report.sh +++ b/test/pushdown_report.sh @@ -3,8 +3,8 @@ # pgColumnar: EXPLAIN must report the pushdown the scan performs, not the # pushdown the planner offered it (#191). # -# columnar_enable_qual_pushdown gates columnar_build_predicates in -# ColumnarBeginRead, so with the setting off the reader builds no predicates and +# pgcolumnar_enable_qual_pushdown gates pgcolumnar_build_predicates in +# PgColumnarBeginRead, so with the setting off the reader builds no predicates and # skips no chunk groups. cstate->nScanKeys is the planner's count and does not # move, so EXPLAIN reported the same "Columnar Pushed-Down Filters: 1" either # way -- telling someone who had just turned the setting off to test a theory diff --git a/test/replication.sh b/test/replication.sh index 0e9951a..df8aa45 100755 --- a/test/replication.sh +++ b/test/replication.sh @@ -446,7 +446,7 @@ psql_run "SELECT pgcolumnar.compact('r');" >/dev/null 2>&1 sync_or_fail "sync 6" check "compact replays identically" "$(hash_standby r)" "$(hash_primary r)" -# ColumnarTruncateMainFork is the one direct XLogInsert in the tree +# PgColumnarTruncateMainFork is the one direct XLogInsert in the tree # (RM_SMGR_ID / XLOG_SMGR_TRUNCATE). wal_envelope.sh asserts the envelope around # it by reading the source; nothing has ever executed the record, let alone # replayed it. diff --git a/test/rewrite_group_scan.sh b/test/rewrite_group_scan.sh index 96d849b..00c665f 100755 --- a/test/rewrite_group_scan.sh +++ b/test/rewrite_group_scan.sh @@ -2,7 +2,7 @@ # # pgColumnar: a rewrite must read each group once, not once per row (#196). # -# rewrite_one_group used ColumnarReadRowByNumber for every row. That call +# rewrite_one_group used PgColumnarReadRowByNumber for every row. That call # decodes the whole group to return one value and depends on the fetch cache to # make the next call cheap -- and the cache drops any group whose decoded form # exceeds COLUMNAR_FETCH_CACHE_MAX_BYTES, after every fetch. So a group over the diff --git a/test/row_triggers.sh b/test/row_triggers.sh index 3388c75..15f16e5 100755 --- a/test/row_triggers.sh +++ b/test/row_triggers.sh @@ -100,7 +100,7 @@ both "a multi-row insert fires once per row with the right values" 2 # disk before any trigger runs # 2. the first trigger's fetch therefore succeeds, and its body runs # 3. the body is itself a statement, and pgColumnar's ExecutorEnd hook calls -# ColumnarFlushAllPendingWrites when it ends -- flushing the outer INSERT's +# PgColumnarFlushAllPendingWrites when it ends -- flushing the outer INSERT's # remaining buffered rows # 4. every later fetch then finds its row on disk # diff --git a/test/run_san.sh b/test/run_san.sh index 50ea33c..0cfcaa6 100644 --- a/test/run_san.sh +++ b/test/run_san.sh @@ -16,7 +16,7 @@ # # Prove it works the way #224 asks: undo the fix it guards and confirm this gate # goes red while the ordinary matrix stays green. Undo it in its TRUE pre-#225 -# form -- delete the ColumnarVarSizeAnyUnaligned helper and restore VARSIZE_ANY at +# form -- delete the PgColumnarVarSizeAnyUnaligned helper and restore VARSIZE_ANY at # the three call sites (columnar_encoding.c, columnar_reader.c x2). That reports # the misalignment (about 20 of the 23 suites, at columnar_encoding.c:1135). # Do NOT instead rewrite the helper's body to the cast while keeping the inline diff --git a/test/server_file_privilege.sh b/test/server_file_privilege.sh index bcb0e8c..7f72d6a 100644 --- a/test/server_file_privilege.sh +++ b/test/server_file_privilege.sh @@ -37,7 +37,12 @@ set -uo pipefail export PGC_EXTRA_CONF=$'max_prepared_transactions=4\nmax_worker_processes=8' pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" -SQLFILE="$(dirname "${BASH_SOURCE[0]}")/../pgcolumnar--1.0-dev.sql" +# Derived from the control file, not hardcoded. This broke once when the install script +# was renamed for a version bump, and a hardcoded name would break again at the next one. +_root="$(dirname "${BASH_SOURCE[0]}")/.." +_ver=$(grep -oE "default_version = '[^']+'" "$_root/pgcolumnar.control" | sed "s/.*'\(.*\)'/\1/") +SQLFILE="$_root/pgcolumnar--$_ver.sql" +[ -f "$SQLFILE" ] || { echo "FAIL install script $SQLFILE not found"; exit 1; } nope="/tmp/pgc_sfp_does_not_exist.parquet" out="$PGC_WORKDIR/sfp_out" outdir="$PGC_WORKDIR/sfp_dir" diff --git a/test/ungrouped_vector_agg.sh b/test/ungrouped_vector_agg.sh index 0d26e79..96ce5c0 100644 --- a/test/ungrouped_vector_agg.sh +++ b/test/ungrouped_vector_agg.sh @@ -6,7 +6,7 @@ # zone map cannot answer -- one with a WHERE filter, or sum/avg over # int8/float/numeric -- to a single-pass scan-fold node instead of the row-wise # core Agg. This suite proves the new path returns byte-for-byte what core Agg -# returns (the fold reuses columnar_apply_one in scan order, so floats match +# returns (the fold reuses pgcolumnar_apply_one in scan order, so floats match # exactly), across types, nulls, filters, empty and all-null inputs; and it # ASSERTS THE PREMISE that the new path actually runs with the GUC on and does # not with it off, so the A/B is never vacuous. diff --git a/test/unique_conc.sh b/test/unique_conc.sh index 077510e..40e0489 100755 --- a/test/unique_conc.sh +++ b/test/unique_conc.sh @@ -7,7 +7,7 @@ # synthetic TID) is written immediately. So while transaction T1's inserting # statement is still in flight (row buffered, not flushed), a second transaction # T2 inserting the SAME key finds T1's index entry but cannot resolve the row: -# columnar_index_fetch_tuple returns false for a row still in T1's private write +# pgcolumnar_index_fetch_tuple returns false for a row still in T1's private write # buffer. The btree dirty-snapshot uniqueness check then treats the entry as dead # and T2 inserts a duplicate -- two live rows with the same unique key. # diff --git a/test/wal_envelope.sh b/test/wal_envelope.sh index 8bd82ca..ebc8bdd 100755 --- a/test/wal_envelope.sh +++ b/test/wal_envelope.sh @@ -59,12 +59,12 @@ check "every other WAL emitter is a core full-page-image helper" \ # ---- 2. the envelope around the one direct record --------------------------- FN="$SRC/columnar_storage.c" -# line range of ColumnarTruncateMainFork: its header to the next top-level function -start="$(grep -n '^ColumnarTruncateMainFork(' "$FN" | cut -d: -f1)" +# line range of PgColumnarTruncateMainFork: its header to the next top-level function +start="$(grep -n '^PgColumnarTruncateMainFork(' "$FN" | cut -d: -f1)" end="$(awk -v s="$start" 'NR > s && /^[A-Za-z_][A-Za-z0-9_]*\(/ {print NR; exit}' "$FN")" [ -z "$end" ] && end="$(wc -l < "$FN")" -check "ColumnarTruncateMainFork was found" "$([ -n "$start" ] && echo yes || echo no)" "yes" +check "PgColumnarTruncateMainFork was found" "$([ -n "$start" ] && echo yes || echo no)" "yes" # Line number of the first line inside the function that CONTAINS the literal # string, or empty.