From 7420b838794c8222588f6a2d9f1471457f1d6ac2 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 19:42:18 +0000 Subject: [PATCH 01/10] Namespace the exported C symbols under pgcolumnar (#382) Two extensions that both call themselves columnar can define the same symbol. We share four with citus_columnar today: columnar_handler, columnar_relation_storageid, and the pg_finfo_ twin of each. columnar_handler is the table access method handler, which is the worst entry on that list to share a name for. The exposure is larger on PostgreSQL 15 than the symbol count suggests. PGXS adds -fvisibility=hidden from 16 onward and does not on 15, so 15 exports the internal API as well: major exported columnar_* Columnar* fb_* 15 257 52 145 24 16 63 27 0 0 17 63 27 0 0 18 63 27 0 0 Comparing our 15 exports against the full citus_columnar symbol table, which is the overlap if neither side hides anything, finds twelve of ours rather than four. Four of the additions are GUC backing variables: columnar_stripe_row_limit, columnar_chunk_group_row_limit, columnar_compression and columnar_compression_level. Citus uses those exact names for the same settings. A duplicate definition there does not crash. It binds one library's setting to the other's storage, silently. So this renames the identifiers, not just the four that collide today: columnar_ -> pgcolumnar_ Columnar -> PgColumnar fb_ -> pgc_fb_ fb_init, fb_start, fb_end and fb_grow are generic enough to collide with anything, and they export on 15. Nothing a user types or reads changes. The SQL names were already namespaced, so columnar_vacuum only ever appeared as the link name: CREATE FUNCTION pgcolumnar.vacuum(tablename regclass, ...) AS 'MODULE_PATHNAME', 'columnar_vacuum'; Only that second argument moves. The GUC strings were already pgcolumnar.*, and only the C variables behind them were not. Deliberately unchanged, because both are user visible and belong in their own change: the CustomScan node name, which makes EXPLAIN print a line identical to Citus's, and the SQL name pgcolumnar.columnar_handler. Source file names are unchanged too, so #include targets and the Makefile object list still read columnar_*. The rename skips string and character literals. Three kinds of literal in src/ contain these tokens and none of them should move: the #include targets, the CustomScan node name, and the EXPLAIN keys such as "Columnar Chunk Groups Read". Verified by rebuilding and reading nm -D rather than by reading the diff. On 15 and 18 the build is warning free and no symbol beginning columnar_, Columnar or fb_ is exported. What remains outside our namespace is _PG_init and Pg_magic_func, which every extension defines. Co-Authored-By: Claude Opus 5 (1M context) --- pgcolumnar--1.0-dev.sql | 50 +-- src/columnar.h | 416 +++++++++---------- src/columnar_arrow.c | 240 +++++------ src/columnar_bloom.c | 14 +- src/columnar_compat.h | 44 +- src/columnar_compression.c | 14 +- src/columnar_customscan.c | 328 +++++++-------- src/columnar_delete_vector.c | 82 ++-- src/columnar_encoding.c | 62 +-- src/columnar_flatbuffers.c | 146 +++---- src/columnar_flatbuffers.h | 56 +-- src/columnar_index.c | 28 +- src/columnar_metadata.c | 272 ++++++------- src/columnar_parallel_copy.c | 34 +- src/columnar_parallel_export.c | 30 +- src/columnar_parquet.c | 148 +++---- src/columnar_parquet_codec.c | 6 +- src/columnar_parquet_codec.h | 4 +- src/columnar_parquet_format.h | 6 +- src/columnar_parquet_reader.c | 160 ++++---- src/columnar_projection.c | 108 ++--- src/columnar_reader.c | 372 ++++++++--------- src/columnar_storage.c | 116 +++--- src/columnar_tableam.c | 708 ++++++++++++++++----------------- src/columnar_thrift.c | 74 ++-- src/columnar_thrift.h | 32 +- src/columnar_unique.c | 50 +-- src/columnar_vacuum.c | 408 +++++++++---------- src/columnar_vector.c | 670 +++++++++++++++---------------- src/columnar_visibilitymap.c | 66 +-- src/columnar_write_state.c | 396 +++++++++--------- 31 files changed, 2570 insertions(+), 2570 deletions(-) diff --git a/pgcolumnar--1.0-dev.sql b/pgcolumnar--1.0-dev.sql index 39da81c..decad2a 100644 --- a/pgcolumnar--1.0-dev.sql +++ b/pgcolumnar--1.0-dev.sql @@ -277,7 +277,7 @@ CREATE INDEX free_space_fit CREATE FUNCTION pgcolumnar.columnar_handler(internal) RETURNS table_am_handler LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_handler'; + AS 'MODULE_PATHNAME', 'pgcolumnar_handler'; CREATE ACCESS METHOD pgcolumnar TYPE TABLE @@ -492,7 +492,7 @@ COMMENT ON FUNCTION pgcolumnar.reset_options(regclass, bool, bool, bool, bool, b CREATE FUNCTION pgcolumnar.get_storage_id(rel regclass) RETURNS bigint LANGUAGE C STABLE STRICT - AS 'MODULE_PATHNAME', 'columnar_relation_storageid'; + AS 'MODULE_PATHNAME', 'pgcolumnar_relation_storageid'; COMMENT ON FUNCTION pgcolumnar.get_storage_id(regclass) IS 'storage id linking a columnar table to its metadata rows'; @@ -504,7 +504,7 @@ CREATE FUNCTION pgcolumnar.add_projection( sort_key text[] DEFAULT '{}') RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_add_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_add_projection'; COMMENT ON FUNCTION pgcolumnar.add_projection(regclass, text, text[], text[]) IS 'declare a physical projection: a named column subset sorted on sort_key (gap 26)'; @@ -512,7 +512,7 @@ COMMENT ON FUNCTION pgcolumnar.add_projection(regclass, text, text[], text[]) CREATE FUNCTION pgcolumnar.drop_projection(rel regclass, name text) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_drop_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_drop_projection'; COMMENT ON FUNCTION pgcolumnar.drop_projection(regclass, text) IS 'drop a declared projection and free its storage (gap 26)'; @@ -572,7 +572,7 @@ COMMENT ON FUNCTION pgcolumnar.rebuild_projections(regclass) CREATE FUNCTION pgcolumnar.read_projection(rel regclass, name text) RETURNS SETOF text LANGUAGE C STABLE - AS 'MODULE_PATHNAME', 'columnar_read_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_read_projection'; COMMENT ON FUNCTION pgcolumnar.read_projection(regclass, text) IS 'read a projection''s stored columns (live rows), joined by | -- verification/debug (gap 26)'; @@ -580,7 +580,7 @@ COMMENT ON FUNCTION pgcolumnar.read_projection(regclass, text) CREATE FUNCTION pgcolumnar.reconstruct_via_projection(rel regclass, name text) RETURNS SETOF text LANGUAGE C STABLE - AS 'MODULE_PATHNAME', 'columnar_reconstruct_via_projection'; + AS 'MODULE_PATHNAME', 'pgcolumnar_reconstruct_via_projection'; COMMENT ON FUNCTION pgcolumnar.reconstruct_via_projection(regclass, text) IS 'read all live rows via a projection, reconstructing non-covered columns from the base by row number (gap 26)'; @@ -707,7 +707,7 @@ COMMENT ON FUNCTION pgcolumnar.sort_status(regclass) CREATE FUNCTION pgcolumnar.vacuum(tablename regclass, stripe_count int DEFAULT 0) RETURNS void LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_vacuum'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum'; COMMENT ON FUNCTION pgcolumnar.vacuum(regclass, int) IS 'compact a columnar table by combining stripes and reclaiming deleted rows'; @@ -717,7 +717,7 @@ CREATE FUNCTION pgcolumnar.vacuum_sorted( VARIADIC sort_columns name[]) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vacuum_sorted'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass, name[]) IS 'compact a columnar table, storing rows sorted ascending (NULLS LAST) on the given columns. With no columns, applies the table''s declared sort_by key from set_options (#288), like a bare CLUSTER re-applying a remembered index; errors if none is declared. Supports any btree-orderable column including text (unlike the numeric-only Z-order cluster()). One-shot: not auto-maintained.'; @@ -732,7 +732,7 @@ COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass, name[]) CREATE FUNCTION pgcolumnar.vacuum_sorted(tablename regclass) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vacuum_sorted'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; COMMENT ON FUNCTION pgcolumnar.vacuum_sorted(regclass) IS 'apply the table''s declared sort_by key from set_options (#288); errors if none is declared. Equivalent to a bare CLUSTER re-applying a remembered index.'; @@ -742,7 +742,7 @@ CREATE FUNCTION pgcolumnar.cluster( VARIADIC columns name[]) RETURNS void LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_cluster'; + AS 'MODULE_PATHNAME', 'pgcolumnar_cluster'; COMMENT ON FUNCTION pgcolumnar.cluster(regclass, name[]) IS 'eager reorg: rewrite a columnar table with rows ordered by the Z-order space-filling curve over the given columns. Holds AccessExclusiveLock like CLUSTER/VACUUM FULL; the online incremental path is Phase F3'; @@ -750,7 +750,7 @@ COMMENT ON FUNCTION pgcolumnar.cluster(regclass, name[]) CREATE FUNCTION pgcolumnar.compact(tablename regclass) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_compact'; + AS 'MODULE_PATHNAME', 'pgcolumnar_compact'; COMMENT ON FUNCTION pgcolumnar.compact(regclass) IS 'lazy online compaction: retire row groups that are fully deleted, dropping their metadata so scans skip them. Holds only ShareUpdateExclusiveLock (concurrent reads and writes). Returns the number of groups retired (Phase F3a)'; @@ -758,7 +758,7 @@ COMMENT ON FUNCTION pgcolumnar.compact(regclass) CREATE FUNCTION pgcolumnar.truncate(tablename regclass) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_truncate'; + AS 'MODULE_PATHNAME', 'pgcolumnar_truncate'; COMMENT ON FUNCTION pgcolumnar.truncate(regclass) IS 'physical end-truncation: return trailing reclaimed blocks to the OS. Best-effort -- takes AccessExclusiveLock conditionally for the brief physical step and returns 0 without waiting if the table is busy. Only removes space freed before the oldest-xmin horizon. Gated by pgcolumnar.enable_end_truncation. Returns the number of blocks truncated (Phase F)'; @@ -769,7 +769,7 @@ CREATE FUNCTION pgcolumnar.compact_rewrite( max_groups int DEFAULT 0) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_compact_rewrite'; + AS 'MODULE_PATHNAME', 'pgcolumnar_compact_rewrite'; COMMENT ON FUNCTION pgcolumnar.compact_rewrite(regclass, float8, int) IS 'lazy online space reclaim: rewrite partially-deleted row groups (deleted fraction >= min_deleted_fraction) to drop their dead rows, under ShareUpdateExclusiveLock (concurrent reads and writes). Returns the number of groups rewritten (Phase F3b)'; @@ -779,7 +779,7 @@ CREATE FUNCTION pgcolumnar.recluster( VARIADIC columns name[]) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_recluster'; + AS 'MODULE_PATHNAME', 'pgcolumnar_recluster'; COMMENT ON FUNCTION pgcolumnar.recluster(regclass, name[]) IS 'lazy online reclustering: re-establish global Z-order clustering over the given columns under ShareUpdateExclusiveLock (concurrent reads and writes), unlike the eager cluster() which holds AccessExclusiveLock. Returns the number of groups reclustered (Phase F3c)'; @@ -787,7 +787,7 @@ COMMENT ON FUNCTION pgcolumnar.recluster(regclass, name[]) CREATE FUNCTION pgcolumnar.export_arrow(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_export_arrow'; + AS 'MODULE_PATHNAME', 'pgcolumnar_export_arrow'; COMMENT ON FUNCTION pgcolumnar.export_arrow(regclass, text) IS 'export a columnar table to an Arrow IPC stream file; returns rows written'; @@ -795,7 +795,7 @@ COMMENT ON FUNCTION pgcolumnar.export_arrow(regclass, text) CREATE FUNCTION pgcolumnar.export_parquet(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_export_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_export_parquet'; COMMENT ON FUNCTION pgcolumnar.export_parquet(regclass, text) IS 'export a columnar table to a Parquet file; returns rows written'; @@ -804,7 +804,7 @@ CREATE FUNCTION pgcolumnar.parallel_export_parquet(target regclass, path text, workers int DEFAULT NULL) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_parallel_export_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_export_parquet'; COMMENT ON FUNCTION pgcolumnar.parallel_export_parquet(regclass, text, int) IS 'parallel Parquet export using read-only background workers into a directory readable by pgcolumnar.read_parquet: a single columnar table split by row-group ranges, or a partitioned columnar table one file per partition; returns rows written (#300)'; @@ -812,7 +812,7 @@ COMMENT ON FUNCTION pgcolumnar.parallel_export_parquet(regclass, text, int) CREATE FUNCTION pgcolumnar.import_arrow(rel regclass, path text) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_import_arrow'; + AS 'MODULE_PATHNAME', 'pgcolumnar_import_arrow'; COMMENT ON FUNCTION pgcolumnar.import_arrow(regclass, text) IS 'insert rows from an Arrow IPC stream file into a columnar table; returns rows inserted'; @@ -820,7 +820,7 @@ COMMENT ON FUNCTION pgcolumnar.import_arrow(regclass, text) CREATE FUNCTION pgcolumnar.import_parquet(rel regclass, path text) RETURNS bigint LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_import_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_import_parquet'; COMMENT ON FUNCTION pgcolumnar.import_parquet(regclass, text) IS 'insert rows from a Parquet file, directory, or glob into a table; returns rows inserted (gap 27)'; @@ -828,7 +828,7 @@ COMMENT ON FUNCTION pgcolumnar.import_parquet(regclass, text) CREATE FUNCTION pgcolumnar.parquet_schema(path text) RETURNS TABLE(column_name text, data_type text, nullable boolean) LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_parquet_schema'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_schema'; COMMENT ON FUNCTION pgcolumnar.parquet_schema(text) IS 'report the leaf columns of a Parquet file and the PostgreSQL type each maps to; for a directory or glob, of its first file (Phase G scan core)'; @@ -836,7 +836,7 @@ COMMENT ON FUNCTION pgcolumnar.parquet_schema(text) CREATE FUNCTION pgcolumnar.read_parquet(path text) RETURNS SETOF record LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_read_parquet'; + AS 'MODULE_PATHNAME', 'pgcolumnar_read_parquet'; COMMENT ON FUNCTION pgcolumnar.read_parquet(text) IS 'read a Parquet file, directory, or glob in place as a set of rows; requires a column definition list covering every leaf column, e.g. SELECT * FROM pgcolumnar.read_parquet(path) AS t(id int, name text) (Phase G)'; @@ -872,7 +872,7 @@ COMMENT ON FOREIGN DATA WRAPPER pgcolumnar_parquet CREATE FUNCTION pgcolumnar.vm_selftest(rel regclass, blk int) RETURNS boolean LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vm_selftest'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_selftest'; COMMENT ON FUNCTION pgcolumnar.vm_selftest(regclass, int) IS 'gap 28 phase-1 self-test: set a VM-fork all-visible bit and read it back'; @@ -880,7 +880,7 @@ COMMENT ON FUNCTION pgcolumnar.vm_selftest(regclass, int) CREATE FUNCTION pgcolumnar.vm_is_visible(rel regclass, blk int) RETURNS boolean LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_vm_is_visible'; + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_is_visible'; COMMENT ON FUNCTION pgcolumnar.vm_is_visible(regclass, int) IS 'gap 28: is the synthetic block marked all-visible in the VM fork?'; @@ -927,7 +927,7 @@ COMMENT ON FUNCTION pgcolumnar.vacuum_full(name, real, int) CREATE FUNCTION pgcolumnar.file_split_offsets(path text, workers int) RETURNS bigint[] LANGUAGE C STRICT - AS 'MODULE_PATHNAME', 'columnar_file_split_offsets'; + AS 'MODULE_PATHNAME', 'pgcolumnar_file_split_offsets'; COMMENT ON FUNCTION pgcolumnar.file_split_offsets(text, int) IS 'byte offsets that split a COPY text-format file into N record-aligned ranges (#300)'; @@ -956,7 +956,7 @@ CREATE FUNCTION pgcolumnar.parallel_copy(target regclass, filename text, workers int DEFAULT NULL) RETURNS bigint LANGUAGE C - AS 'MODULE_PATHNAME', 'columnar_parallel_copy'; + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_copy'; COMMENT ON FUNCTION pgcolumnar.parallel_copy(regclass, text, int) IS 'atomic parallel bulk load of a COPY text file into a columnar table using background workers: a single columnar table (any row order), or a RANGE-partitioned columnar table sorted by the partition key with one distinct partition set per worker (#300)'; diff --git a/src/columnar.h b/src/columnar.h index 7c52fb2..6741e6c 100644 --- a/src/columnar.h +++ b/src/columnar.h @@ -84,7 +84,7 @@ /* * Round a logical byte length up to a whole number of pages. Every reservation - * (ColumnarReserveOffset) starts on a page boundary and the next one starts on + * (PgColumnarReserveOffset) starts on a page boundary and the next one starts on * the next page boundary, so a group's on-disk footprint is its data length * rounded up to a page. Physical reclaim keeps free ranges page-aligned in both * offset and length by working in these footprints. @@ -114,7 +114,7 @@ /* * Value-stream encoding codes (I1, format 2.1). An encoding is a reversible * transform of the raw value-stream bytes, applied before block compression on - * write and reversed after decompression on read (columnar_encoding.c). + * write and reversed after decompression on read (pgcolumnar_encoding.c). */ #define COLUMNAR_ENCODING_NONE 0 #define COLUMNAR_ENCODING_RLE 1 /* run-length of a fixed-width value */ @@ -146,7 +146,7 @@ #define COLUMNAR_ENCODE_EFFORT_FULL 0 #define COLUMNAR_ENCODE_EFFORT_FAST 1 -typedef struct ColumnarOptions +typedef struct PgColumnarOptions { bool chunkGroupRowLimitSet; int chunkGroupRowLimit; @@ -158,47 +158,47 @@ typedef struct ColumnarOptions int compressionLevel; bool encodeEffortSet; int encodeEffort; /* one of COLUMNAR_ENCODE_EFFORT_* */ -} ColumnarOptions; +} PgColumnarOptions; /* GUC-backed instance defaults (spec 8.3) */ -extern int columnar_stripe_row_limit; -extern int columnar_chunk_group_row_limit; -extern int columnar_encoding_sample_rows; -extern int columnar_compression; /* one of COLUMNAR_COMPRESSION_* */ -extern int columnar_compression_level; /* zstd level */ -extern int columnar_fsst_min_gain_percent; /* min compressed FSST win to keep it (#155) */ -extern bool columnar_enable_qual_pushdown; -extern bool columnar_enable_column_projection; -extern bool columnar_enable_custom_scan; -extern bool columnar_enable_bloom_filter; /* bloom equality skipping (I7) */ +extern int pgcolumnar_stripe_row_limit; +extern int pgcolumnar_chunk_group_row_limit; +extern int pgcolumnar_encoding_sample_rows; +extern int pgcolumnar_compression; /* one of COLUMNAR_COMPRESSION_* */ +extern int pgcolumnar_compression_level; /* zstd level */ +extern int pgcolumnar_fsst_min_gain_percent; /* min compressed FSST win to keep it (#155) */ +extern bool pgcolumnar_enable_qual_pushdown; +extern bool pgcolumnar_enable_column_projection; +extern bool pgcolumnar_enable_custom_scan; +extern bool pgcolumnar_enable_bloom_filter; /* bloom equality skipping (I7) */ /* Phase 6 GUCs (spec 8.3) */ -extern bool columnar_enable_vectorization; /* vectorized aggregate path */ -extern bool columnar_enable_group_vectorization; /* GROUP BY vectorized agg (#289) */ -extern bool columnar_enable_ungrouped_vector_agg; /* filtered/extended ungrouped agg (#289) */ -extern bool columnar_enable_parallel_vector_agg; /* parallel-aware ungrouped batch fold (#289 phase 5/6) */ -extern int columnar_groupagg_max_groups; /* plan-time group-count cap (#289) */ -extern bool columnar_enable_read_stream; /* stream/prefetch block reads (PG17+) */ -extern bool columnar_enable_index_only_scan; /* allow index-only scans (gap 28) */ -extern bool columnar_bulk_parallel_writer; /* internal: parallel_copy loader skips the storage-row creation lock (#300) */ -extern bool columnar_enable_projection_scan; /* scan a covering projection (gap 26) */ -extern bool columnar_enable_index_fetch_penalty; /* price a columnar index scan's per-row fetch (#355) */ +extern bool pgcolumnar_enable_vectorization; /* vectorized aggregate path */ +extern bool pgcolumnar_enable_group_vectorization; /* GROUP BY vectorized agg (#289) */ +extern bool pgcolumnar_enable_ungrouped_vector_agg; /* filtered/extended ungrouped agg (#289) */ +extern bool pgcolumnar_enable_parallel_vector_agg; /* parallel-aware ungrouped batch fold (#289 phase 5/6) */ +extern int pgcolumnar_groupagg_max_groups; /* plan-time group-count cap (#289) */ +extern bool pgcolumnar_enable_read_stream; /* stream/prefetch block reads (PG17+) */ +extern bool pgcolumnar_enable_index_only_scan; /* allow index-only scans (gap 28) */ +extern bool pgcolumnar_bulk_parallel_writer; /* internal: parallel_copy loader skips the storage-row creation lock (#300) */ +extern bool pgcolumnar_enable_projection_scan; /* scan a covering projection (gap 26) */ +extern bool pgcolumnar_enable_index_fetch_penalty; /* price a columnar index scan's per-row fetch (#355) */ /* - * Statement-scoped by-row-number fetch cache cap (columnar_reader.c). Named here + * Statement-scoped by-row-number fetch cache cap (pgcolumnar_reader.c). Named here * so the index-fetch cost model (#355) can tell when a stripe is too wide to be * retained across fetches and must be treated as re-decoded per row. */ #define COLUMNAR_FETCH_CACHE_MAX_BYTES (32 * 1024 * 1024) /* issue #5: concurrent unique-key insert serialization */ -extern bool columnar_enable_unique_lock; /* serialize same-key inserters */ -extern int columnar_unique_lock_buckets; /* advisory-lock buckets per index */ +extern bool pgcolumnar_enable_unique_lock; /* serialize same-key inserters */ +extern int pgcolumnar_unique_lock_buckets; /* advisory-lock buckets per index */ /* ------------------------------------------------------------------------- * Metapage (spec 3) * ------------------------------------------------------------------------- */ -typedef struct ColumnarMetapage +typedef struct PgColumnarMetapage { uint32 versionMajor; uint32 versionMinor; @@ -207,7 +207,7 @@ typedef struct ColumnarMetapage uint64 reservedRowNumber; uint64 reservedOffset; bool unloggedReset; -} ColumnarMetapage; +} PgColumnarMetapage; @@ -251,7 +251,7 @@ typedef struct NativeColumnChunkMetadata * One pgcolumnar.zone_map row (native spec 7.1, Phase D5): a Small Materialized * Aggregate for one vector of a column chunk (vectorIndex 0-based) or for the * whole column chunk (vectorIndex -1). minimum and maximum are the column's - * value serialized with ColumnarEncodeValue (NULL when the type has no btree + * value serialized with PgColumnarEncodeValue (NULL when the type has no btree * ordering); sum is a numeric Datum (D5a leaves it unset, hasSum false; the * zone-map-only aggregate that consumes it lands in D5b). value_count and * null_count are always present. @@ -263,7 +263,7 @@ typedef struct NativeZoneMapMetadata int columnIndex; int vectorIndex; /* 0-based vector; -1 for the whole chunk */ bool hasMinMax; - const char *minimum; /* ColumnarEncodeValue bytes, when hasMinMax */ + const char *minimum; /* PgColumnarEncodeValue bytes, when hasMinMax */ uint32 minimumLen; const char *maximum; uint32 maximumLen; @@ -276,7 +276,7 @@ typedef struct NativeZoneMapMetadata /* * One pgcolumnar.bloom row (native spec 7.2, Phase D5b): a per-column-chunk bloom * filter over the chunk's hashable values, for equality skipping on unsorted - * columns. filter is the ColumnarBloomBuild byte image. + * columns. filter is the PgColumnarBloomBuild byte image. */ typedef struct NativeBloomMetadata { @@ -309,7 +309,7 @@ typedef struct DeleteVectorMetadata * projectionId 0 is the implicit base projection. attnums are 1-based; sortKey * attnums are a subset of columns. */ -typedef struct ColumnarProjection +typedef struct PgColumnarProjection { uint64 storageId; /* the table's base storage id */ int projectionId; /* 0 = base, 1..N additional */ @@ -319,51 +319,51 @@ typedef struct ColumnarProjection int sortKeyLen; int16 *columns; /* stored attnums, columnsLen entries */ int columnsLen; -} ColumnarProjection; +} PgColumnarProjection; /* ------------------------------------------------------------------------- - * storage layer (columnar_storage.c) + * storage layer (pgcolumnar_storage.c) * ------------------------------------------------------------------------- */ struct SMgrRelationData; -extern void ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, +extern void PgColumnarWriteNewMetapage(const RelFileLocator *newrlocator, struct SMgrRelationData *srel, char persistence, uint64 storageId); -extern void ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta); -extern uint64 ColumnarStorageId(Relation rel); -extern void ColumnarEnsureStorageRow(Relation rel); /* pre-create storage row (#300 parallel_copy) */ -extern void ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, +extern void PgColumnarReadMetapage(Relation rel, PgColumnarMetapage *meta); +extern uint64 PgColumnarStorageId(Relation rel); +extern void PgColumnarEnsureStorageRow(Relation rel); /* pre-create storage row (#300 parallel_copy) */ +extern void PgColumnarReserveRowNumbers(Relation rel, uint64 rowCount, uint64 *stripeId, uint64 *firstRowNumber); -extern void ColumnarReserveOffset(Relation rel, uint64 dataLength, +extern void PgColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset); -extern void ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes); -extern void ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, +extern void PgColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes); +extern void PgColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor); -extern void ColumnarSetReservedOffset(Relation rel, uint64 newOffset); -extern void ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks); -extern void ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, +extern void PgColumnarSetReservedOffset(Relation rel, uint64 newOffset); +extern void PgColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks); +extern void PgColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, char *data, uint64 length); -extern void ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, +extern void PgColumnarReadLogicalData(Relation rel, uint64 logicalOffset, char *dest, uint64 length); -extern void ColumnarResetMetapage(Relation rel); +extern void PgColumnarResetMetapage(Relation rel); /* row number <-> item pointer (spec 6) */ -extern void ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid); -extern uint64 ColumnarItemPointerToRowNumber(ItemPointer tid); +extern void PgColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid); +extern uint64 PgColumnarItemPointerToRowNumber(ItemPointer tid); /* ------------------------------------------------------------------------- - * visibility map for index-only scans (columnar_visibilitymap.c, gap 28) + * visibility map for index-only scans (pgcolumnar_visibilitymap.c, gap 28) * ------------------------------------------------------------------------- */ -extern void ColumnarVMSetVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMClearVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMClearForRow(Relation rel, uint64 rowNumber); -extern bool ColumnarVMIsVisible(Relation rel, BlockNumber blk); -extern void ColumnarVMSetVisibleForRelation(Relation rel); -extern void ColumnarDiscardFetchCache(void); +extern void PgColumnarVMSetVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMClearVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMClearForRow(Relation rel, uint64 rowNumber); +extern bool PgColumnarVMIsVisible(Relation rel, BlockNumber blk); +extern void PgColumnarVMSetVisibleForRelation(Relation rel); +extern void PgColumnarDiscardFetchCache(void); /* index maintenance for callers that insert rows without an executor (#153) */ -typedef struct ColumnarIndexInsertState +typedef struct PgColumnarIndexInsertState { EState *estate; TupleTableSlot *slot; @@ -384,55 +384,55 @@ typedef struct ColumnarIndexInsertState Relation *rels; IndexInfo **infos; ExprState **predicates; /* partial-index predicate, or NULL */ -} ColumnarIndexInsertState; +} PgColumnarIndexInsertState; /* * enforceConstraints is fixed for the life of the state rather than passed per * row, because it selects which of the two routes above is built: an importer * enforces for every row it inserts and a rewrite enforces for none. */ -extern ColumnarIndexInsertState *ColumnarIndexInsertBegin(Relation rel, +extern PgColumnarIndexInsertState *PgColumnarIndexInsertBegin(Relation rel, bool enforceConstraints); -extern void ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, +extern void PgColumnarIndexInsertRow(PgColumnarIndexInsertState *st, Relation rel, Datum *values, bool *isnull, uint64 rowNumber); -extern void ColumnarIndexInsertEnd(ColumnarIndexInsertState *st); -extern bool ColumnarRelationHasIndexes(Relation rel); +extern void PgColumnarIndexInsertEnd(PgColumnarIndexInsertState *st); +extern bool PgColumnarRelationHasIndexes(Relation rel); /* a contiguous run of all-visible row numbers (gap 28 phase 3) */ -typedef struct ColumnarRowRange +typedef struct PgColumnarRowRange { uint64 firstRowNumber; uint64 rowCount; -} ColumnarRowRange; +} PgColumnarRowRange; /* row groups every one of whose rows is deleted as-of oldestXmin. Returns a * List of palloc'd uint64 group numbers. */ -extern List *ColumnarComputeFullyDeletedGroups(uint64 storageId, +extern List *PgColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin); -extern void ColumnarRetireGroup(uint64 storageId, uint64 groupNumber); -extern int64 ColumnarRetireFullyDeletedGroups(Relation rel); -extern void ColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber); -extern bool ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, +extern void PgColumnarRetireGroup(uint64 storageId, uint64 groupNumber); +extern int64 PgColumnarRetireFullyDeletedGroups(Relation rel); +extern void PgColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber); +extern bool PgColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, TransactionId oldestXmin, uint64 *fileOffset); -extern bool ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, +extern bool PgColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, TransactionId oldestXmin); -extern void ColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd); -extern void ColumnarReconcileFreeList(Relation dataRel); +extern void PgColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd); +extern void PgColumnarReconcileFreeList(Relation dataRel); /* all-visible chunk-group row ranges: stripe committed past the horizon and no - * deletes (committed or in-progress). Returns a List of ColumnarRowRange *. */ -extern List *ColumnarComputeAllVisibleGroups(uint64 storageId, + * deletes (committed or in-progress). Returns a List of PgColumnarRowRange *. */ +extern List *PgColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin); /* physical reclaim: split freed ranges on allocate and coalesce on free (GUC) */ -extern bool columnar_reclaim_coalesce; +extern bool pgcolumnar_reclaim_coalesce; /* physical end-truncation opt-in (GUC) */ -extern bool columnar_enable_end_truncation; +extern bool pgcolumnar_enable_end_truncation; /* error unless the current user owns the relation (maintenance/DDL gate) */ -extern void ColumnarRequireTableOwner(Relation rel); +extern void PgColumnarRequireTableOwner(Relation rel); /* * Assert-only invariant: a storage's live row-group footprints and its @@ -440,64 +440,64 @@ extern void ColumnarRequireTableOwner(Relation rel); * called only in assert builds (the version matrix builds with asserts). */ #ifdef USE_ASSERT_CHECKING -extern void ColumnarCheckFreeSpaceNoOverlap(uint64 storageId); -#define COLUMNAR_ASSERT_NO_OVERLAP(sid) ColumnarCheckFreeSpaceNoOverlap(sid) +extern void PgColumnarCheckFreeSpaceNoOverlap(uint64 storageId); +#define COLUMNAR_ASSERT_NO_OVERLAP(sid) PgColumnarCheckFreeSpaceNoOverlap(sid) #else #define COLUMNAR_ASSERT_NO_OVERLAP(sid) ((void) 0) #endif /* ------------------------------------------------------------------------- - * metadata layer (columnar_metadata.c) + * metadata layer (pgcolumnar_metadata.c) * ------------------------------------------------------------------------- */ -extern uint64 ColumnarNextStorageId(void); -extern void ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s); -extern void ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, +extern uint64 PgColumnarNextStorageId(void); +extern void PgColumnarInsertNativeStorageRow(const NativeStorageMetadata *s); +extern void PgColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup); -extern void ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName); -extern void ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg); -extern void ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc); -extern void ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z); -extern void ColumnarInsertBloomRow(const NativeBloomMetadata *b); -extern List *ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot); -extern List *ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, +extern void PgColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName); +extern void PgColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg); +extern void PgColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc); +extern void PgColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z); +extern void PgColumnarInsertBloomRow(const NativeBloomMetadata *b); +extern List *PgColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot); +extern List *PgColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern List *ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, +extern List *PgColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot); -extern NativeBloomMetadata *ColumnarReadBloomForColumn(uint64 storageId, +extern NativeBloomMetadata *PgColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, int columnIndex, Snapshot snapshot); -extern void ColumnarDeleteMetadata(uint64 storageId); +extern void PgColumnarDeleteMetadata(uint64 storageId); /* per-table options catalog (spec 7.4) */ -extern bool ColumnarReadOptions(Oid relid, ColumnarOptions *opts); -extern void ColumnarDeleteOptions(Oid relid); +extern bool PgColumnarReadOptions(Oid relid, PgColumnarOptions *opts); +extern void PgColumnarDeleteOptions(Oid relid); /* declared physical sort key (#288); List of pstrdup'd column names, NIL if * none is declared. Names (not attnums) so the value survives dump/restore. */ -extern List *ColumnarReadSortBy(Oid relid); +extern List *PgColumnarReadSortBy(Oid relid); -/* projection catalog (gap 26, format 2.2). List entries are ColumnarProjection* +/* projection catalog (gap 26, format 2.2). List entries are PgColumnarProjection* * palloc'd in the current context, ordered by projection_id. */ -extern List *ColumnarListProjections(uint64 storageId); -extern void ColumnarInsertProjectionRow(const ColumnarProjection *proj); +extern List *PgColumnarListProjections(uint64 storageId); +extern void PgColumnarInsertProjectionRow(const PgColumnarProjection *proj); /* The dumpable declaration behind a projection, keyed by regclass and stored as * column names so a dump and restore can carry it (#266). */ -extern void ColumnarRecordProjectionDeclaration(Oid relid, const char *name, +extern void PgColumnarRecordProjectionDeclaration(Oid relid, const char *name, ArrayType *columns, ArrayType *sortKey); -extern void ColumnarDeleteProjectionDeclaration(Oid relid, const char *name); +extern void PgColumnarDeleteProjectionDeclaration(Oid relid, const char *name); /* Every declaration for a relation, for the drop hook: a dropped table must not * leave rows behind whose regclass no longer resolves (#304). */ -extern void ColumnarDeleteProjectionDeclarationsForRel(Oid relid); -extern void ColumnarDeleteProjectionRow(uint64 storageId, int projectionId); +extern void PgColumnarDeleteProjectionDeclarationsForRel(Oid relid); +extern void PgColumnarDeleteProjectionRow(uint64 storageId, int projectionId); /* whether a relation uses the columnar table access method */ -extern bool ColumnarIsColumnarRelation(Oid relid); +extern bool PgColumnarIsColumnarRelation(Oid relid); /* * A snapshot suitable for reading the columnar metadata catalog during a scan @@ -509,113 +509,113 @@ extern bool ColumnarIsColumnarRelation(Oid relid); * MVCC snapshot. The result is palloc'd in the current context and shares the * base snapshot's arrays, so the base must outlive it. */ -extern Snapshot ColumnarCatalogSnapshot(Snapshot base); +extern Snapshot PgColumnarCatalogSnapshot(Snapshot base); /* delete_vector catalog access (spec 7.5) */ -extern List *ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, +extern List *PgColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot); -extern bool ColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot); -extern void ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm); +extern bool PgColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot); +extern void PgColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm); /* ------------------------------------------------------------------------- - * writer (columnar_write_state.c) + * writer (pgcolumnar_write_state.c) * ------------------------------------------------------------------------- */ -typedef struct ColumnarWriteState ColumnarWriteState; +typedef struct PgColumnarWriteState PgColumnarWriteState; -extern ColumnarWriteState *ColumnarGetWriteState(Relation rel); -extern int ColumnarWriteStateStripeCount(ColumnarWriteState *ws); -extern uint64 *ColumnarWriteStateStripeIds(ColumnarWriteState *ws, int *n); -extern uint64 *ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n); -extern uint64 ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, +extern PgColumnarWriteState *PgColumnarGetWriteState(Relation rel); +extern int PgColumnarWriteStateStripeCount(PgColumnarWriteState *ws); +extern uint64 *PgColumnarWriteStateStripeIds(PgColumnarWriteState *ws, int *n); +extern uint64 *PgColumnarWriteStateProjStripeIds(PgColumnarWriteState *ws, int *n); +extern uint64 PgColumnarWriteRow(PgColumnarWriteState *writeState, Relation rel, Datum *values, bool *nulls); -extern void ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, +extern void PgColumnarProjectionFanoutRow(Relation rel, PgColumnarWriteState *baseWs, uint64 rowNumber, Datum *values, bool *nulls); -extern void ColumnarBackfillProjection(Relation rel, - const ColumnarProjection *proj); -extern bool ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, +extern void PgColumnarBackfillProjection(Relation rel, + const PgColumnarProjection *proj); +extern bool PgColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, Datum *values, bool *nulls); -extern void ColumnarFlushWriteStateForRelation(Oid relid); -extern void ColumnarForgetWriteStateForRelation(Oid relid); -extern void ColumnarFlushAllPendingWrites(void); -extern void ColumnarDiscardAllPendingWrites(void); -extern void ColumnarWriteStateDiscardSubXact(SubTransactionId subid); -extern void ColumnarWriteStatePromoteSubXact(SubTransactionId subid, +extern void PgColumnarFlushWriteStateForRelation(Oid relid); +extern void PgColumnarForgetWriteStateForRelation(Oid relid); +extern void PgColumnarFlushAllPendingWrites(void); +extern void PgColumnarDiscardAllPendingWrites(void); +extern void PgColumnarWriteStateDiscardSubXact(SubTransactionId subid); +extern void PgColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent); /* ------------------------------------------------------------------------- - * delete vector / delete tracking (columnar_delete_vector.c, spec 7.5, 9) + * delete vector / delete tracking (pgcolumnar_delete_vector.c, spec 7.5, 9) * ------------------------------------------------------------------------- */ -extern void ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber); -extern bool ColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber); -extern void ColumnarFlushDeleteVectorForRelation(Relation rel); -extern void ColumnarFlushAllDeleteVectors(void); -extern void ColumnarDiscardAllDeleteVectors(void); -extern void ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid); -extern void ColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, +extern void PgColumnarMarkRowDeleted(Relation rel, uint64 rowNumber); +extern bool PgColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber); +extern void PgColumnarFlushDeleteVectorForRelation(Relation rel); +extern void PgColumnarFlushAllDeleteVectors(void); +extern void PgColumnarDiscardAllDeleteVectors(void); +extern void PgColumnarDeleteVectorDiscardSubXact(SubTransactionId subid); +extern void PgColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent); /* ------------------------------------------------------------------------- - * reader (columnar_reader.c) + * reader (pgcolumnar_reader.c) * ------------------------------------------------------------------------- */ -typedef struct ColumnarReadState ColumnarReadState; +typedef struct PgColumnarReadState PgColumnarReadState; -extern ColumnarReadState *ColumnarBeginRead(Relation rel, Snapshot snapshot, +extern PgColumnarReadState *PgColumnarBeginRead(Relation rel, Snapshot snapshot, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys); -/* like ColumnarBeginRead but reads an explicit storage id with an explicit +/* like PgColumnarBeginRead but reads an explicit storage id with an explicit * tuple descriptor -- used to read a projection's storage (gap 26) */ -extern ColumnarReadState *ColumnarBeginReadWithStorage(Relation rel, +extern PgColumnarReadState *PgColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, uint64 storageId, TupleDesc tupdesc, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys); -extern bool ColumnarReadNextRow(ColumnarReadState *readState, +extern bool PgColumnarReadNextRow(PgColumnarReadState *readState, Datum *values, bool *nulls, uint64 *rowNumber); -extern void ColumnarRescanRead(ColumnarReadState *readState); -extern void ColumnarEndRead(ColumnarReadState *readState); +extern void PgColumnarRescanRead(PgColumnarReadState *readState); +extern void PgColumnarEndRead(PgColumnarReadState *readState); /* * Batch-fold accessors (#289): expose the current loaded group's decoded buffer * so an ungrouped aggregate can fold it column-at-a-time instead of one Datum - * tuple per row. See the block comment in columnar_reader.c for the contract. + * tuple per row. See the block comment in pgcolumnar_reader.c for the contract. */ -extern bool ColumnarReadFoldNextGroup(ColumnarReadState *readState); -extern void ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, +extern bool PgColumnarReadFoldNextGroup(PgColumnarReadState *readState); +extern void PgColumnarReadFoldGroupInfo(PgColumnarReadState *readState, uint64 *nrows, const char **deleteMask, uint32 *deleteMaskLen, const bool **skipVec, const uint32 **vecStart, int *vectorCount); -extern bool ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, +extern bool PgColumnarReadFoldColumn(PgColumnarReadState *readState, int attidx, const char **validity, const char **packed, int16 *attlen, const uint32 **vecRawLen); /* * Restrict a scan to a set of row groups (issue #149). Groups outside the set * are skipped without their bytes being read. Must be called before the first - * ColumnarReadNextRow; ngroups == 0 makes the scan return no rows. + * PgColumnarReadNextRow; ngroups == 0 makes the scan return no rows. */ -extern void ColumnarReadRestrictToGroups(ColumnarReadState *readState, +extern void PgColumnarReadRestrictToGroups(PgColumnarReadState *readState, const uint64 *groupNumbers, int ngroups); /* Parquet export helpers, shared by the serial and parallel exporters - * (src/columnar_parquet.c). */ -extern int64 ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, + * (src/pgcolumnar_parquet.c). */ +extern int64 PgColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, const uint64 *restrictGroups, int nRestrictGroups); -extern void ColumnarParquetCheckExportable(Relation rel); +extern void PgColumnarParquetCheckExportable(Relation rel); /* * Parallel scan (gap 23): point the read state at a shared atomic that hands out * stripe indices, so several workers scanning the same relation each claim * distinct stripes. Set by the custom scan's DSM init callbacks. */ -extern void ColumnarReadSetParallelCounter(ColumnarReadState *readState, +extern void PgColumnarReadSetParallelCounter(PgColumnarReadState *readState, pg_atomic_uint32 *counter); /* @@ -623,26 +623,26 @@ extern void ColumnarReadSetParallelCounter(ColumnarReadState *readState, * scan's EXPLAIN output to show how many chunk groups the min/max skip lists * removed. total = read + skipped over the groups the scan has reached. */ -extern void ColumnarReadStats(ColumnarReadState *readState, +extern void PgColumnarReadStats(PgColumnarReadState *readState, uint64 *groupsRead, uint64 *groupsSkipped, uint64 *groupsTotal); -extern uint64 ColumnarVectorsSkipped(ColumnarReadState *readState); +extern uint64 PgColumnarVectorsSkipped(PgColumnarReadState *readState); /* cached base-liveness for a projection scan (gap 26): build once per scan, * probe per row with a binary search instead of a per-row catalog scan */ -typedef struct ColumnarLivenessCache ColumnarLivenessCache; -extern ColumnarLivenessCache *ColumnarBuildLivenessCache(Relation rel, +typedef struct PgColumnarLivenessCache PgColumnarLivenessCache; +extern PgColumnarLivenessCache *PgColumnarBuildLivenessCache(Relation rel, Snapshot snapshot); -extern bool ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, +extern bool PgColumnarLivenessCacheIsLive(PgColumnarLivenessCache *cache, uint64 rowNumber); -extern void ColumnarFreeLivenessCache(ColumnarLivenessCache *cache); +extern void PgColumnarFreeLivenessCache(PgColumnarLivenessCache *cache); /* * Fetch a single row by its 1-based row number (spec 6), for the table AM's * fetch-by-tid callback used by UPDATE. Fills values/nulls (by-reference values * are allocated in the current memory context) and returns true when the row * exists and is not marked deleted in the delete vector. */ -extern bool ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, +extern bool PgColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls); /* @@ -651,93 +651,93 @@ extern bool ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, * this deliberately has no "NULL means all" convention, because a Bitmapset * cannot distinguish empty from NULL, so a caller whose computed set came out * empty would silently get the opposite of what it asked for. For every column - * call ColumnarReadRowByNumber, which takes no set. + * call PgColumnarReadRowByNumber, which takes no set. * * Decoding every column regardless makes a wide table exceed the fetch cache's * size cap, so the entry is dropped after every fetch and the group is decoded * again for the next row (issue #157). */ -extern bool ColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, +extern bool PgColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, Bitmapset *needed); /* Is the row visible? Decodes nothing. */ -extern bool ColumnarRowIsLive(Relation rel, Snapshot snapshot, +extern bool PgColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber); /* ------------------------------------------------------------------------- - * Decoded chunk group (columnar_vector.c aggregate path) + * Decoded chunk group (pgcolumnar_vector.c aggregate path) * - * A ColumnarVector is one decoded chunk group: for each projected column, the + * A PgColumnarVector is one decoded chunk group: for each projected column, the * whole group's values and null flags as flat arrays, plus the per-row deleted * flag resolved from the delete vector. The vectorized aggregate builds a selection * vector over it; the scan itself is the scalar per-row - * reader (ColumnarReadNextRow). + * reader (PgColumnarReadNextRow). * ------------------------------------------------------------------------- */ -typedef struct ColumnarVector +typedef struct PgColumnarVector { uint64 nrows; /* rows in this chunk group */ uint64 firstRowNumber; /* row number of local row 0 */ Datum **values; /* [natts]; values[c] is Datum[nrows] or NULL */ bool **isnull; /* [natts]; isnull[c] is bool[nrows] or NULL */ bool *deleted; /* [nrows]; true when row-mask-deleted */ -} ColumnarVector; +} PgColumnarVector; /* value stream encode/decode shared by writer and reader */ -extern void ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, +extern void PgColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value); -extern Datum ColumnarDecodeValue(Form_pg_attribute att, char **cursor, +extern Datum PgColumnarDecodeValue(Form_pg_attribute att, char **cursor, MemoryContext targetContext); /* ------------------------------------------------------------------------- - * lightweight value-stream encodings (columnar_encoding.c, I1) + * lightweight value-stream encodings (pgcolumnar_encoding.c, I1) * ------------------------------------------------------------------------- */ -extern int ColumnarEncodeChunk(const char *raw, uint32 rawLen, +extern int PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen); -extern char *ColumnarDecodeChunk(const char *enc, uint32 encLen, +extern char *PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx); -extern const char *ColumnarEncodingName(int encodingType); +extern const char *PgColumnarEncodingName(int encodingType); /* * Build one FSST symbol table for a whole column chunk from a sample of its * concatenated varlena value streams (E3b). Returns true and sets *tableOut / * *tableLenOut (palloc'd, serialized as [uint8 nSym][ nSym x (uint8 len, bytes)]) * when a table was built; false for non-varlena columns or when no useful table - * exists. The table is passed back into ColumnarEncodeChunk / ColumnarDecodeChunk + * exists. The table is passed back into PgColumnarEncodeChunk / PgColumnarDecodeChunk * as fsstTable so the per-vector build cost is paid once per chunk. */ -extern bool ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut); /* Cheap distinct-count pre-check: true when dictionary encoding wins outright, so * the costly FSST table build can be skipped with byte-identical output (#155). */ -extern bool ColumnarFsstDictWins(const char *corpus, uint32 corpusLen); +extern bool PgColumnarFsstDictWins(const char *corpus, uint32 corpusLen); /* * True when encoding the chunk with the table just built is still a win after * the block compressor runs over the result, judged on the same sample the - * table was trained on. The per-vector test inside ColumnarEncodeChunk compares + * table was trained on. The per-vector test inside PgColumnarEncodeChunk compares * uncompressed lengths, which is the wrong objective when a codec is configured: * FSST codes are smaller than repetitive text but much less compressible, so * FSST can win every vector and still enlarge the chunk. Callers pass NULL for * fsstTable when this returns false. */ -extern bool ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, const char *table, uint32 tableLen, int compressionType, int compressionLevel); /* ------------------------------------------------------------------------- - * per-chunk bloom filters (columnar_bloom.c, I7) + * per-chunk bloom filters (pgcolumnar_bloom.c, I7) * ------------------------------------------------------------------------- */ -extern bool ColumnarBloomBuild(const uint32 *hashes, uint32 n, +extern bool PgColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen); -extern bool ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); +extern bool PgColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); /* * True when a column of the given collation can carry a bloom filter (I7/gap 25): @@ -745,10 +745,10 @@ extern bool ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash); * values are byte-identical and hash consistently between build and probe. * Nondeterministic collations return false and are left unbloomed. */ -extern bool ColumnarCollationIsDeterministic(Oid collid); +extern bool PgColumnarCollationIsDeterministic(Oid collid); /* ------------------------------------------------------------------------- - * compression-block run iterator (columnar_encoding.c, I2) + * compression-block run iterator (pgcolumnar_encoding.c, I2) * * Exposes a column chunk's (non-null) values as a sequence of (value, run * length) pairs so operators run once per run instead of once per row (I3 @@ -756,15 +756,15 @@ extern bool ColumnarCollationIsDeterministic(Oid collid); * adjacent equal fixed-width values, so a repetitive or run-length-encoded * column yields long runs. Fixed-width columns only. * ------------------------------------------------------------------------- */ -typedef struct ColumnarBlockReader +typedef struct PgColumnarBlockReader { const char *raw; /* raw value stream (packed fixed-width values) */ uint64 valueCount; /* number of values in the stream */ int width; /* bytes per value (attlen) */ uint64 pos; /* next value index */ -} ColumnarBlockReader; +} PgColumnarBlockReader; -extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +extern void PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width); /* @@ -772,65 +772,65 @@ extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, * while the underlying stream is), *runLen is how many consecutive values equal * it. Returns false at end of stream. */ -extern bool ColumnarBlockNextRun(ColumnarBlockReader *br, +extern bool PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen); /* ------------------------------------------------------------------------- - * compression (columnar_compression.c, spec 5) + * compression (pgcolumnar_compression.c, spec 5) * ------------------------------------------------------------------------- */ -extern bool ColumnarCodecAvailable(int compressionType); -extern void ColumnarCompressValueStream(const char *raw, uint32 rawLen, +extern bool PgColumnarCodecAvailable(int compressionType); +extern void PgColumnarCompressValueStream(const char *raw, uint32 rawLen, int requestedType, int level, char **outData, uint32 *outLen, int *usedType, int *usedLevel); -extern char *ColumnarDecompressValueStream(const char *comp, uint32 compLen, +extern char *PgColumnarDecompressValueStream(const char *comp, uint32 compLen, int compressionType, uint32 rawLen, MemoryContext targetContext); /* ------------------------------------------------------------------------- - * concurrent unique-key insert serialization (columnar_unique.c, issue #5) + * concurrent unique-key insert serialization (pgcolumnar_unique.c, issue #5) * * Before an inserted row is handed to the executor's index maintenance, the - * table AM insert paths call ColumnarLockUniqueKeys to take a transaction- + * table AM insert paths call PgColumnarLockUniqueKeys to take a transaction- * scoped advisory lock per applicable unique index key, so a concurrent * inserter of an equal key serializes behind this transaction until it commits * (and has therefore flushed its row), at which point the ordinary btree - * uniqueness check catches the duplicate. See columnar_unique.c. + * uniqueness check catches the duplicate. See pgcolumnar_unique.c. * ------------------------------------------------------------------------- */ -extern void ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot); -extern void ColumnarUniqueInit(void); +extern void PgColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot); +extern void PgColumnarUniqueInit(void); /* ------------------------------------------------------------------------- - * planner integration (columnar_customscan.c, spec 8.3, 9) + * planner integration (pgcolumnar_customscan.c, spec 8.3, 9) * ------------------------------------------------------------------------- */ -extern void ColumnarCustomScanInit(void); +extern void PgColumnarCustomScanInit(void); /* * The single registered CustomScanMethods, shared by the base custom scan and * the vectorized aggregate. The create-state callback dispatches on scanrelid: * a scanrelid==0 upper node is the vectorized aggregate. */ -extern const CustomScanMethods columnar_scan_methods; -extern Node *ColumnarCreateAggScanState(CustomScan *cscan); -extern Node *ColumnarCreateGroupAggScanState(CustomScan *cscan); +extern const CustomScanMethods pgcolumnar_scan_methods; +extern Node *PgColumnarCreateAggScanState(CustomScan *cscan); +extern Node *PgColumnarCreateGroupAggScanState(CustomScan *cscan); /* * Build the chunk-group skip scan keys from a plan's restriction clauses. * Shared by the base custom scan and the vectorized aggregate (spec 9). Clauses * that are not simple "column op const" comparisons are ignored. */ -extern ScanKey ColumnarBuildScanKeys(List *qual, Index scanrelid, +extern ScanKey PgColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, int *nkeys); /* ------------------------------------------------------------------------- - * vectorized aggregation and filtering (columnar_vector.c, spec 9) + * vectorized aggregation and filtering (pgcolumnar_vector.c, spec 9) * ------------------------------------------------------------------------- */ -extern void ColumnarVectorInit(void); +extern void PgColumnarVectorInit(void); /* * The vectorized predicate machinery that used to be declared here is private to - * columnar_vector.c. ColumnarVecRowPasses and ColumnarVecSelect were exported + * pgcolumnar_vector.c. PgColumnarVecRowPasses and PgColumnarVecSelect were exported * with no call site anywhere in the tree and are gone; see issue #200. What * remains of it is a convertibility probe the planner uses, and it has no * business outside that file. @@ -869,7 +869,7 @@ extern void ColumnarVectorInit(void); * duplicating. */ static inline uint32 -ColumnarVarSizeAnyUnaligned(const char *p) +PgColumnarVarSizeAnyUnaligned(const char *p) { uint32 hdr; diff --git a/src/columnar_arrow.c b/src/columnar_arrow.c index 81ebcdb..4ff8bcd 100644 --- a/src/columnar_arrow.c +++ b/src/columnar_arrow.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_arrow.c + * pgcolumnar_arrow.c * Arrow IPC stream export for pgColumnar (gap 27, piece 1). * * pgcolumnar.export_arrow(rel regclass, path text) writes a columnar table @@ -52,8 +52,8 @@ #include "utils/typcache.h" #include "utils/uuid.h" -PG_FUNCTION_INFO_V1(columnar_export_arrow); -PG_FUNCTION_INFO_V1(columnar_import_arrow); +PG_FUNCTION_INFO_V1(pgcolumnar_export_arrow); +PG_FUNCTION_INFO_V1(pgcolumnar_import_arrow); /* one RecordBatch per this many rows */ #define ARROW_BATCH_ROWS 16384 @@ -158,7 +158,7 @@ numeric_to_int128(Datum numd, int scale, __int128 *out) /* ---- Arrow Type table for one column; returns tag via *typetag ---- */ static uint32 -fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) +pgc_fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) { switch (kind) { @@ -168,80 +168,80 @@ fb_arrow_type(FBB *b, ArrowKind kind, int precision, int scale, uint8 *typetag) { int32 bits = (kind == A_INT16) ? 16 : (kind == A_INT32) ? 32 : 64; - fb_start(b, 2); - fb_add_i32(b, 0, bits, 0); /* bitWidth */ - fb_add_bool(b, 1, true, false); /* is_signed */ + pgc_fb_start(b, 2); + pgc_fb_add_i32(b, 0, bits, 0); /* bitWidth */ + pgc_fb_add_bool(b, 1, true, false); /* is_signed */ *typetag = ARROW_TYPE_Int; - return fb_end(b); + return pgc_fb_end(b); } case A_FLOAT32: case A_FLOAT64: - fb_start(b, 1); - fb_add_i16(b, 0, (kind == A_FLOAT32) ? 1 : 2, 0); /* SINGLE/DOUBLE */ + pgc_fb_start(b, 1); + pgc_fb_add_i16(b, 0, (kind == A_FLOAT32) ? 1 : 2, 0); /* SINGLE/DOUBLE */ *typetag = ARROW_TYPE_FloatingPoint; - return fb_end(b); + return pgc_fb_end(b); case A_BOOL: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Bool; - return fb_end(b); + return pgc_fb_end(b); case A_UTF8: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Utf8; - return fb_end(b); + return pgc_fb_end(b); case A_BINARY: - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Binary; - return fb_end(b); + return pgc_fb_end(b); case A_DATE32: /* Date { unit: DateUnit = MILLISECOND (1) }; want DAY (0) */ - fb_start(b, 1); - fb_add_i16(b, 0, 0, 1); + pgc_fb_start(b, 1); + pgc_fb_add_i16(b, 0, 0, 1); *typetag = ARROW_TYPE_Date; - return fb_end(b); + return pgc_fb_end(b); case A_TIME64: /* Time { unit: TimeUnit = MILLISECOND (1); bitWidth: int = 32 } */ - fb_start(b, 2); - fb_add_i16(b, 0, 2, 1); /* MICROSECOND */ - fb_add_i32(b, 1, 64, 32); + pgc_fb_start(b, 2); + pgc_fb_add_i16(b, 0, 2, 1); /* MICROSECOND */ + pgc_fb_add_i32(b, 1, 64, 32); *typetag = ARROW_TYPE_Time; - return fb_end(b); + return pgc_fb_end(b); case A_TIMESTAMP: case A_TIMESTAMPTZ: { uint32 tzOff = 0; if (kind == A_TIMESTAMPTZ) - tzOff = fb_create_string(b, "UTC"); + tzOff = pgc_fb_create_string(b, "UTC"); /* Timestamp { unit: TimeUnit = SECOND (0); timezone: string } */ - fb_start(b, 2); - fb_add_i16(b, 0, 2, 0); /* MICROSECOND */ - fb_add_offset(b, 1, tzOff); + pgc_fb_start(b, 2); + pgc_fb_add_i16(b, 0, 2, 0); /* MICROSECOND */ + pgc_fb_add_offset(b, 1, tzOff); *typetag = ARROW_TYPE_Timestamp; - return fb_end(b); + return pgc_fb_end(b); } case A_UUID: /* FixedSizeBinary { byteWidth: int } */ - fb_start(b, 1); - fb_add_i32(b, 0, 16, 0); + pgc_fb_start(b, 1); + pgc_fb_add_i32(b, 0, 16, 0); *typetag = ARROW_TYPE_FixedSizeBinary; - return fb_end(b); + return pgc_fb_end(b); case A_DECIMAL128: /* Decimal { precision: int; scale: int; bitWidth: int = 128 } */ - fb_start(b, 3); - fb_add_i32(b, 0, precision, 0); - fb_add_i32(b, 1, scale, 0); + pgc_fb_start(b, 3); + pgc_fb_add_i32(b, 0, precision, 0); + pgc_fb_add_i32(b, 1, scale, 0); *typetag = ARROW_TYPE_Decimal; - return fb_end(b); + return pgc_fb_end(b); case A_LIST: /* List {} -- the element type is carried in the Field's children */ - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_List; - return fb_end(b); + return pgc_fb_end(b); case A_STRUCT: /* Struct_ {} -- the field types are the Field's children */ - fb_start(b, 0); + pgc_fb_start(b, 0); *typetag = ARROW_TYPE_Struct; - return fb_end(b); + return pgc_fb_end(b); } *typetag = 0; return 0; /* unreachable */ @@ -809,7 +809,7 @@ arrow_emit_buffers(StringInfo body, ArrowCol *c, static uint32 arrow_build_field(FBB *b, ArrowCol *c) { - uint32 nameOff = fb_create_string(b, c->name ? c->name : ""); + uint32 nameOff = pgc_fb_create_string(b, c->name ? c->name : ""); uint8 typetag; uint32 typeOff; uint32 childrenVec = 0; @@ -821,22 +821,22 @@ arrow_build_field(FBB *b, ArrowCol *c) childOff = palloc(sizeof(uint32) * c->nchildren); for (i = 0; i < c->nchildren; i++) childOff[i] = arrow_build_field(b, &c->children[i]); - fb_start_vector(b, 4, c->nchildren, 4); + pgc_fb_start_vector(b, 4, c->nchildren, 4); for (i = c->nchildren - 1; i >= 0; i--) - fb_push_uoffset(b, childOff[i]); - childrenVec = fb_end_vector(b, c->nchildren); + pgc_fb_push_uoffset(b, childOff[i]); + childrenVec = pgc_fb_end_vector(b, c->nchildren); } - typeOff = fb_arrow_type(b, c->kind, c->precision, c->scale, &typetag); + typeOff = pgc_fb_arrow_type(b, c->kind, c->precision, c->scale, &typetag); - fb_start(b, 7); - fb_add_offset(b, 0, nameOff); /* name */ - fb_add_bool(b, 1, true, false); /* nullable */ - fb_add_u8(b, 2, typetag, 0); /* type_type */ - fb_add_offset(b, 3, typeOff); /* type */ + pgc_fb_start(b, 7); + pgc_fb_add_offset(b, 0, nameOff); /* name */ + pgc_fb_add_bool(b, 1, true, false); /* nullable */ + pgc_fb_add_u8(b, 2, typetag, 0); /* type_type */ + pgc_fb_add_offset(b, 3, typeOff); /* type */ if (c->nchildren > 0) - fb_add_offset(b, 5, childrenVec); /* children (Field slot 5) */ - return fb_end(b); + pgc_fb_add_offset(b, 5, childrenVec); /* children (Field slot 5) */ + return pgc_fb_end(b); } /* build one RecordBatch (metadata + body) and write it */ @@ -883,41 +883,41 @@ write_record_batch(FILE *f, ArrowCol *cols, int ncols, int64 nrows) } /* ---- RecordBatch metadata flatbuffer ---- */ - fb_init(&b); + pgc_fb_init(&b); /* nodes vector: [FieldNode{length,null_count}] structs, 16B/8-align */ - fb_start_vector(&b, 16, nnodes, 8); + pgc_fb_start_vector(&b, 16, nnodes, 8); for (i = nnodes - 1; i >= 0; i--) { - fb_prep(&b, 8, 0); - fb_place(&b, &nodeNull[i], 8); /* null_count (higher) */ - fb_place(&b, &nodeLen[i], 8); /* length (lower) */ + pgc_fb_prep(&b, 8, 0); + pgc_fb_place(&b, &nodeNull[i], 8); /* null_count (higher) */ + pgc_fb_place(&b, &nodeLen[i], 8); /* length (lower) */ } - nodesVec = fb_end_vector(&b, nnodes); + nodesVec = pgc_fb_end_vector(&b, nnodes); /* buffers vector: [Buffer{offset,length}] structs */ - fb_start_vector(&b, 16, nbuf, 8); + pgc_fb_start_vector(&b, 16, nbuf, 8); for (i = nbuf - 1; i >= 0; i--) { - fb_prep(&b, 8, 0); - fb_place(&b, &bufLen[i], 8); /* length (higher) */ - fb_place(&b, &bufOff[i], 8); /* offset (lower) */ + pgc_fb_prep(&b, 8, 0); + pgc_fb_place(&b, &bufLen[i], 8); /* length (higher) */ + pgc_fb_place(&b, &bufOff[i], 8); /* offset (lower) */ } - bufsVec = fb_end_vector(&b, nbuf); - - fb_start(&b, 4); - fb_add_i64(&b, 0, nrows, 0); - fb_add_offset(&b, 1, nodesVec); - fb_add_offset(&b, 2, bufsVec); - rbOff = fb_end(&b); - - fb_start(&b, 5); - fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); - fb_add_u8(&b, 1, ARROW_MSG_RecordBatch, 0); - fb_add_offset(&b, 2, rbOff); - fb_add_i64(&b, 3, body.len, 0); /* bodyLength */ - msgOff = fb_end(&b); - fb_finish(&b, msgOff); + bufsVec = pgc_fb_end_vector(&b, nbuf); + + pgc_fb_start(&b, 4); + pgc_fb_add_i64(&b, 0, nrows, 0); + pgc_fb_add_offset(&b, 1, nodesVec); + pgc_fb_add_offset(&b, 2, bufsVec); + rbOff = pgc_fb_end(&b); + + pgc_fb_start(&b, 5); + pgc_fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); + pgc_fb_add_u8(&b, 1, ARROW_MSG_RecordBatch, 0); + pgc_fb_add_offset(&b, 2, rbOff); + pgc_fb_add_i64(&b, 3, body.len, 0); /* bodyLength */ + msgOff = pgc_fb_end(&b); + pgc_fb_finish(&b, msgOff); /* ---- write encapsulated message ---- */ metaLen = b.tail; @@ -940,13 +940,13 @@ write_record_batch(FILE *f, ArrowCol *cols, int ncols, int64 nrows) } /* - * columnar_export_arrow + * pgcolumnar_export_arrow * SQL: pgcolumnar.export_arrow(rel regclass, path text) -> bigint. * Write a columnar table to an Arrow IPC stream file; returns the number * of rows written. */ Datum -columnar_export_arrow(PG_FUNCTION_ARGS) +pgcolumnar_export_arrow(PG_FUNCTION_ARGS) { Oid relid; text *pathText; @@ -956,7 +956,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) int ncols; ArrowCol *cols; Snapshot snapshot; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *values; bool *nulls; uint64 rowNumber; @@ -987,7 +987,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) path = text_to_cstring(pathText); rel = table_open(relid, AccessShareLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessShareLock); ereport(ERROR, @@ -1045,25 +1045,25 @@ columnar_export_arrow(PG_FUNCTION_ARGS) /* ---- Schema message ---- */ fieldOff = palloc(sizeof(uint32) * ncols); - fb_init(&b); + pgc_fb_init(&b); for (i = 0; i < ncols; i++) fieldOff[i] = arrow_build_field(&b, &cols[i]); - fb_start_vector(&b, 4, ncols, 4); + pgc_fb_start_vector(&b, 4, ncols, 4); for (i = ncols - 1; i >= 0; i--) - fb_push_uoffset(&b, fieldOff[i]); - vec = fb_end_vector(&b, ncols); + pgc_fb_push_uoffset(&b, fieldOff[i]); + vec = pgc_fb_end_vector(&b, ncols); - fb_start(&b, 4); + pgc_fb_start(&b, 4); /* endianness Little=0 is the default, so omit slot 0 */ - fb_add_offset(&b, 1, vec); /* fields */ - schemaOff = fb_end(&b); + pgc_fb_add_offset(&b, 1, vec); /* fields */ + schemaOff = pgc_fb_end(&b); - fb_start(&b, 5); - fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); - fb_add_u8(&b, 1, ARROW_MSG_Schema, 0); - fb_add_offset(&b, 2, schemaOff); - msgOff = fb_end(&b); - fb_finish(&b, msgOff); + pgc_fb_start(&b, 5); + pgc_fb_add_i16(&b, 0, ARROW_METADATA_V5, 0); + pgc_fb_add_u8(&b, 1, ARROW_MSG_Schema, 0); + pgc_fb_add_offset(&b, 2, schemaOff); + msgOff = pgc_fb_end(&b); + pgc_fb_finish(&b, msgOff); { uint32 cont = 0xFFFFFFFF; @@ -1089,9 +1089,9 @@ columnar_export_arrow(PG_FUNCTION_ARGS) nulls = palloc(sizeof(bool) * ncols); snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(readState, values, nulls, &rowNumber)) { CHECK_FOR_INTERRUPTS(); for (i = 0; i < ncols; i++) @@ -1110,7 +1110,7 @@ columnar_export_arrow(PG_FUNCTION_ARGS) batchRows = 0; } } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); if (batchRows > 0) { @@ -1205,7 +1205,7 @@ fbr_i64(const uint8 *b, uint32 len, uint32 pos) /* absolute position of field `i` of the table at `tab`, or 0 if absent */ static uint32 -fb_field(const uint8 *b, uint32 len, uint32 tab, int i) +pgc_fb_field(const uint8 *b, uint32 len, uint32 tab, int i) { int32 soff = fbr_i32(b, len, tab); int64 vt = (int64) tab - soff; @@ -1226,7 +1226,7 @@ fb_field(const uint8 *b, uint32 len, uint32 tab, int i) /* follow the uoffset stored at `pos` to the object it points at */ static uint32 -fb_indirect(const uint8 *b, uint32 len, uint32 pos) +pgc_fb_indirect(const uint8 *b, uint32 len, uint32 pos) { return pos + fbr_u32(b, len, pos); } @@ -1726,13 +1726,13 @@ imp_check_bounds(ImpNode *n, const uint8 *body, const int64 *bufOff, } /* - * columnar_import_arrow + * pgcolumnar_import_arrow * SQL: pgcolumnar.import_arrow(rel regclass, path text) -> bigint. * Insert the rows of an Arrow IPC stream file into a columnar table; * returns the number of rows inserted. */ Datum -columnar_import_arrow(PG_FUNCTION_ARGS) +pgcolumnar_import_arrow(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1743,7 +1743,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) int totalBuffers = 0; FILE *f; TupleTableSlot *slot; - ColumnarIndexInsertState *indexes; + PgColumnarIndexInsertState *indexes; CommandId cid; MemoryContext rowCtx; int64 total = 0; @@ -1763,7 +1763,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) path = text_to_cstring(PG_GETARG_TEXT_PP(1)); rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1812,8 +1812,8 @@ columnar_import_arrow(PG_FUNCTION_ARGS) slot = table_slot_create(rel, NULL); cid = GetCurrentCommandId(true); - indexes = ColumnarRelationHasIndexes(rel) - ? ColumnarIndexInsertBegin(rel, true) : NULL; + indexes = PgColumnarRelationHasIndexes(rel) + ? PgColumnarIndexInsertBegin(rel, true) : NULL; /* * Per-row scratch context. Reconstructing a nested value (array/composite) @@ -1855,12 +1855,12 @@ columnar_import_arrow(PG_FUNCTION_ARGS) if (fread(meta, 1, metaLen, f) != metaLen) IMPORT_CORRUPT("truncated metadata"); - msg = fb_indirect(meta, metaLen, 0); - pos = fb_field(meta, metaLen, msg, 1); /* header_type (u8) */ + msg = pgc_fb_indirect(meta, metaLen, 0); + pos = pgc_fb_field(meta, metaLen, msg, 1); /* header_type (u8) */ headerType = pos ? fbr_u8(meta, metaLen, pos) : 0; - pos = fb_field(meta, metaLen, msg, 2); /* header (offset) */ - hdr = pos ? fb_indirect(meta, metaLen, pos) : 0; - pos = fb_field(meta, metaLen, msg, 3); /* bodyLength (i64) */ + pos = pgc_fb_field(meta, metaLen, msg, 2); /* header (offset) */ + hdr = pos ? pgc_fb_indirect(meta, metaLen, pos) : 0; + pos = pgc_fb_field(meta, metaLen, msg, 3); /* bodyLength (i64) */ bodyLength = pos ? fbr_i64(meta, metaLen, pos) : 0; if (bodyLength < 0) IMPORT_CORRUPT("negative body length"); @@ -1874,9 +1874,9 @@ columnar_import_arrow(PG_FUNCTION_ARGS) if (headerType == ARROW_MSG_Schema) { - uint32 fieldsVecPos = hdr ? fb_field(meta, metaLen, hdr, 1) : 0; + uint32 fieldsVecPos = hdr ? pgc_fb_field(meta, metaLen, hdr, 1) : 0; uint32 fieldsVec = fieldsVecPos ? - fb_indirect(meta, metaLen, fieldsVecPos) : 0; + pgc_fb_indirect(meta, metaLen, fieldsVecPos) : 0; uint32 nfields = fieldsVec ? fbr_u32(meta, metaLen, fieldsVec) : 0; if ((int) nfields != ncols) @@ -1898,16 +1898,16 @@ columnar_import_arrow(PG_FUNCTION_ARGS) IMPORT_CORRUPT("RecordBatch before Schema"); if (!hdr) IMPORT_CORRUPT("missing RecordBatch header"); - if (fb_field(meta, metaLen, hdr, 3) != 0) + if (pgc_fb_field(meta, metaLen, hdr, 3) != 0) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("columnar.import_arrow does not support compressed Arrow bodies"))); - pos = fb_field(meta, metaLen, hdr, 0); /* length */ + pos = pgc_fb_field(meta, metaLen, hdr, 0); /* length */ nrows = pos ? fbr_i64(meta, metaLen, pos) : 0; - nodesVecPos = fb_field(meta, metaLen, hdr, 1); - buffersVecPos = fb_field(meta, metaLen, hdr, 2); - buffersVec = buffersVecPos ? fb_indirect(meta, metaLen, buffersVecPos) : 0; + nodesVecPos = pgc_fb_field(meta, metaLen, hdr, 1); + buffersVecPos = pgc_fb_field(meta, metaLen, hdr, 2); + buffersVec = buffersVecPos ? pgc_fb_indirect(meta, metaLen, buffersVecPos) : 0; nbuffers = buffersVec ? fbr_u32(meta, metaLen, buffersVec) : 0; (void) nodesVecPos; @@ -1968,9 +1968,9 @@ columnar_import_arrow(PG_FUNCTION_ARGS) * (issue #153). tts_tid carries the assigned row number. */ if (indexes != NULL) - ColumnarIndexInsertRow(indexes, rel, slot->tts_values, + PgColumnarIndexInsertRow(indexes, rel, slot->tts_values, slot->tts_isnull, - ColumnarItemPointerToRowNumber(&slot->tts_tid)); + PgColumnarItemPointerToRowNumber(&slot->tts_tid)); MemoryContextSwitchTo(oldCtx); MemoryContextReset(rowCtx); total++; @@ -1993,7 +1993,7 @@ columnar_import_arrow(PG_FUNCTION_ARGS) FreeFile(f); MemoryContextDelete(rowCtx); if (indexes != NULL) - ColumnarIndexInsertEnd(indexes); + PgColumnarIndexInsertEnd(indexes); ExecDropSingleTupleTableSlot(slot); /* diff --git a/src/columnar_bloom.c b/src/columnar_bloom.c index 462e4c3..fd78b02 100644 --- a/src/columnar_bloom.c +++ b/src/columnar_bloom.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_bloom.c + * pgcolumnar_bloom.c * Per-chunk bloom filters for equality chunk-group skipping (I7). * * min/max skip lists (spec 7.2) prune equality predicates only when the probed @@ -28,14 +28,14 @@ #include "utils/syscache.h" /* - * ColumnarCollationIsDeterministic + * PgColumnarCollationIsDeterministic * Whether a bloom filter is safe for this collation: InvalidOid (a * non-collatable type) and deterministic collations qualify; a * nondeterministic collation does not, since equal values need not be * byte-identical and would hash inconsistently. */ bool -ColumnarCollationIsDeterministic(Oid collid) +PgColumnarCollationIsDeterministic(Oid collid) { HeapTuple tp; bool result = true; @@ -77,12 +77,12 @@ bloom_hashes(uint32 h, uint32 *h1, uint32 *h2) } /* - * ColumnarBloomBuild + * PgColumnarBloomBuild * Build a filter over n precomputed value hashes. Returns false (no filter) * when n is too small for a filter to be worthwhile. */ bool -ColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) +PgColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) { uint32 nbits; uint32 nbytes; @@ -142,13 +142,13 @@ ColumnarBloomBuild(const uint32 *hashes, uint32 n, char **out, uint32 *outLen) } /* - * ColumnarBloomProbe + * PgColumnarBloomProbe * Return true when the hash may be present (all k bits set), false when it * is definitely absent. A malformed/empty filter conservatively returns * true (never skips wrongly). */ bool -ColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash) +PgColumnarBloomProbe(const char *bloom, uint32 bloomLen, uint32 hash) { uint32 nbits; uint8 k; diff --git a/src/columnar_compat.h b/src/columnar_compat.h index 3db88a6..e7c871c 100644 --- a/src/columnar_compat.h +++ b/src/columnar_compat.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_compat.h + * pgcolumnar_compat.h * PostgreSQL major-version compatibility shims for pgColumnar. * * pgColumnar keeps a single source tree that builds on PostgreSQL 13 through @@ -72,10 +72,10 @@ #include "utils/lsyscache.h" #if PG_VERSION_NUM >= 180000 -#define ColumnarOpInterpretation OpIndexInterpretation -#define ColumnarGetOpInterpretation(opno) get_op_index_interpretation(opno) +#define PgColumnarOpInterpretation OpIndexInterpretation +#define PgColumnarGetOpInterpretation(opno) get_op_index_interpretation(opno) static inline int -ColumnarOpInterpStrategy(const OpIndexInterpretation *o) +PgColumnarOpInterpStrategy(const OpIndexInterpretation *o) { switch (o->cmptype) { @@ -94,10 +94,10 @@ ColumnarOpInterpStrategy(const OpIndexInterpretation *o) } } #else -#define ColumnarOpInterpretation OpBtreeInterpretation -#define ColumnarGetOpInterpretation(opno) get_op_btree_interpretation(opno) +#define PgColumnarOpInterpretation OpBtreeInterpretation +#define PgColumnarGetOpInterpretation(opno) get_op_btree_interpretation(opno) static inline int -ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) +PgColumnarOpInterpStrategy(const OpBtreeInterpretation *o) { return o->strategy; } @@ -122,10 +122,10 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * only (locator, persistence). Heap passes true, and so do we. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 150000 -#define ColumnarRelationCreateStorage(loc, persistence) \ +#define PgColumnarRelationCreateStorage(loc, persistence) \ RelationCreateStorage((loc), (persistence)) #else -#define ColumnarRelationCreateStorage(loc, persistence) \ +#define PgColumnarRelationCreateStorage(loc, persistence) \ RelationCreateStorage((loc), (persistence), true) #endif @@ -134,7 +134,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * index_delete_tuples(Relation, TM_IndexDeleteOp *) in PG14+. In PG13 the slot * was compute_xid_horizon_for_tuples(Relation, ItemPointerData *, int). The two * take different arguments, so the callback itself is compiled per major (see - * columnar_tableam.c); this macro only selects the struct field name. + * pgcolumnar_tableam.c); this macro only selects the struct field name. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 140000 #define COLUMNAR_AM_INDEX_DELETE_FIELD compute_xid_horizon_for_tuples @@ -156,15 +156,15 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) /* ------------------------------------------------------------------------- * tuple_update() reports which indexes to maintain through a pointer whose * target changed from bool (PG13-15) to the TU_UpdateIndexes enum (PG16+). We - * want "maintain all indexes", which is true / TU_All. ColumnarUpdateIndexes is + * want "maintain all indexes", which is true / TU_All. PgColumnarUpdateIndexes is * a macro (not a typedef) so the enum name is not referenced until the .c file * has already included access/tableam.h, which defines it. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 160000 -#define ColumnarUpdateIndexes bool +#define PgColumnarUpdateIndexes bool #define COLUMNAR_TU_ALL true #else -#define ColumnarUpdateIndexes TU_UpdateIndexes +#define PgColumnarUpdateIndexes TU_UpdateIndexes #define COLUMNAR_TU_ALL TU_All #endif @@ -198,13 +198,13 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) Relation rel, ItemPointer otid, TupleTableSlot *slot, CommandId cid, \ uint32 options, Snapshot snapshot, Snapshot crosscheck, bool wait, \ TM_FailureData *tmfd, LockTupleMode *lockmode, \ - ColumnarUpdateIndexes *update_indexes + PgColumnarUpdateIndexes *update_indexes #else #define COLUMNAR_TUPLE_UPDATE_ARGS \ Relation rel, ItemPointer otid, TupleTableSlot *slot, CommandId cid, \ Snapshot snapshot, Snapshot crosscheck, bool wait, \ TM_FailureData *tmfd, LockTupleMode *lockmode, \ - ColumnarUpdateIndexes *update_indexes + PgColumnarUpdateIndexes *update_indexes #endif /* ------------------------------------------------------------------------- @@ -252,7 +252,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * scan_analyze_next_block() took (BlockNumber, BufferAccessStrategy) through * PG16 and (ReadStream *) from PG17 (the read-stream ANALYZE rework). Pre-17 * has no ReadStream type. The callback is compiled per major in - * columnar_tableam.c; this macro supplies its parameter list. + * pgcolumnar_tableam.c; this macro supplies its parameter list. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 170000 #define COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS \ @@ -267,7 +267,7 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * argument in PG18. An access method supplying its own slot operations has to * match the struct exactly, so the wrapper is declared and forwarded through * these two macros. Keep the boundary here in step with the callback in - * columnar_tableam.c; when the analyze block callback's guard drifted from the + * pgcolumnar_tableam.c; when the analyze block callback's guard drifted from the * macro that supplies its parameters, PG17 stopped compiling. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM >= 180000 @@ -294,9 +294,9 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * snapshot. Callers include "storage/procarray.h". * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM >= 140000 -#define ColumnarOldestXmin(rel) GetOldestNonRemovableTransactionId(rel) +#define PgColumnarOldestXmin(rel) GetOldestNonRemovableTransactionId(rel) #else -#define ColumnarOldestXmin(rel) GetOldestXmin((rel), PROCARRAY_FLAGS_VACUUM) +#define PgColumnarOldestXmin(rel) GetOldestXmin((rel), PROCARRAY_FLAGS_VACUUM) #endif /* ------------------------------------------------------------------------- @@ -306,9 +306,9 @@ ColumnarOpInterpStrategy(const OpBtreeInterpretation *o) * smgrextend require on those majors. * ------------------------------------------------------------------------- */ #if PG_VERSION_NUM < 160000 -#define ColumnarAllocPage() ((Page) palloc0(BLCKSZ)) +#define PgColumnarAllocPage() ((Page) palloc0(BLCKSZ)) #else -#define ColumnarAllocPage() \ +#define PgColumnarAllocPage() \ ((Page) palloc_aligned(BLCKSZ, PG_IO_ALIGN_SIZE, MCXT_ALLOC_ZERO)) #endif @@ -380,7 +380,7 @@ extern void _PG_init(void); #include "catalog/index.h" static inline void -ColumnarReindexRelation(Oid relid, int flags) +PgColumnarReindexRelation(Oid relid, int flags) { #if PG_VERSION_NUM < 140000 reindex_relation(relid, flags, 0); diff --git a/src/columnar_compression.c b/src/columnar_compression.c index 084ab29..004adfb 100644 --- a/src/columnar_compression.c +++ b/src/columnar_compression.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_compression.c + * pgcolumnar_compression.c * Value-stream compression codecs for pgColumnar (spec 5): none, pglz, * lz4, and zstd. Each chunk's value stream is compressed independently. * If a codec does not shrink the data, or the codec is not built into @@ -29,12 +29,12 @@ #endif /* - * ColumnarCodecAvailable + * PgColumnarCodecAvailable * Whether a compression type can be produced by this binary. none and * pglz are always available; lz4 and zstd depend on the build. */ bool -ColumnarCodecAvailable(int compressionType) +PgColumnarCodecAvailable(int compressionType) { switch (compressionType) { @@ -114,7 +114,7 @@ try_zstd(const char *raw, uint32 rawLen, char *dest, size_t destCap, int level) #endif /* - * ColumnarCompressValueStream + * PgColumnarCompressValueStream * Compress rawLen bytes at raw using the requested codec at the given * level. On success at shrinking the data, returns a palloc'd buffer in * *outData with its length in *outLen and the codec actually used in @@ -125,7 +125,7 @@ try_zstd(const char *raw, uint32 rawLen, char *dest, size_t destCap, int level) * The output is always allocated in the current memory context. */ void -ColumnarCompressValueStream(const char *raw, uint32 rawLen, +PgColumnarCompressValueStream(const char *raw, uint32 rawLen, int requestedType, int level, char **outData, uint32 *outLen, int *usedType, int *usedLevel) @@ -227,14 +227,14 @@ ColumnarCompressValueStream(const char *raw, uint32 rawLen, } /* - * ColumnarDecompressValueStream + * PgColumnarDecompressValueStream * Decompress compLen bytes at comp of the given codec into a fresh buffer * of rawLen bytes allocated in targetContext, and return it. For type * none the bytes are copied. Errors out if the codec is not built in or * the decoded length does not match rawLen. */ char * -ColumnarDecompressValueStream(const char *comp, uint32 compLen, +PgColumnarDecompressValueStream(const char *comp, uint32 compLen, int compressionType, uint32 rawLen, MemoryContext targetContext) { diff --git a/src/columnar_customscan.c b/src/columnar_customscan.c index 4414fb2..6572757 100644 --- a/src/columnar_customscan.c +++ b/src/columnar_customscan.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_customscan.c + * pgcolumnar_customscan.c * Planner and executor integration for pgColumnar (spec 8.3, 9). * * A set_rel_pathlist_hook replaces the sequential-scan path of a columnar @@ -60,19 +60,19 @@ #include "utils/typcache.h" /* GUC: use the columnar custom scan path (spec 8.3) */ -bool columnar_enable_custom_scan = true; +bool pgcolumnar_enable_custom_scan = true; /* GUC: let the planner scan a covering projection instead of the base (gap 26) */ -bool columnar_enable_projection_scan = true; +bool pgcolumnar_enable_projection_scan = true; /* GUC: price a columnar index scan's per-row heap fetch (#355) */ -bool columnar_enable_index_fetch_penalty = true; +bool pgcolumnar_enable_index_fetch_penalty = true; static set_rel_pathlist_hook_type prev_set_rel_pathlist_hook = NULL; /* our executor-time scan state embeds CustomScanState as its first field */ -typedef struct ColumnarCustomScanState +typedef struct PgColumnarCustomScanState { CustomScanState css; - ColumnarReadState *readState; + PgColumnarReadState *readState; Bitmapset *projectedColumns; /* 0-based; NULL means all columns */ ScanKey scanKeys; int nScanKeys; @@ -97,63 +97,63 @@ typedef struct ColumnarCustomScanState int *projColMap; /* base attno-1 -> index into projValues, or -1 */ Datum *projValues; /* scratch, length K+1 (index 0 = rownumber) */ bool *projNulls; - ColumnarLivenessCache *livenessCache; /* cached base liveness for the scan */ -} ColumnarCustomScanState; + PgColumnarLivenessCache *livenessCache; /* cached base liveness for the scan */ +} PgColumnarCustomScanState; /* path -> plan */ -static Plan *ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, +static Plan *PgColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans); /* plan -> scan state (dispatches base vs vectorized-aggregate) */ -static Node *ColumnarCreateScanState(CustomScan *cscan); -static Node *ColumnarCreateBaseScanState(CustomScan *cscan); +static Node *PgColumnarCreateScanState(CustomScan *cscan); +static Node *PgColumnarCreateBaseScanState(CustomScan *cscan); /* executor callbacks */ -static void ColumnarBeginCustomScan(CustomScanState *node, EState *estate, +static void PgColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags); -static TupleTableSlot *ColumnarExecCustomScan(CustomScanState *node); -static void ColumnarEndCustomScan(CustomScanState *node); -static void ColumnarReScanCustomScan(CustomScanState *node); -static Size ColumnarEstimateDSMCustomScan(CustomScanState *node, +static TupleTableSlot *PgColumnarExecCustomScan(CustomScanState *node); +static void PgColumnarEndCustomScan(CustomScanState *node); +static void PgColumnarReScanCustomScan(CustomScanState *node); +static Size PgColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt); -static void ColumnarInitializeDSMCustomScan(CustomScanState *node, +static void PgColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate); -static void ColumnarReInitializeDSMCustomScan(CustomScanState *node, +static void PgColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate); -static void ColumnarInitializeWorkerCustomScan(CustomScanState *node, +static void PgColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, void *coordinate); -static void ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, +static void PgColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainState *es); /* ExecScan helpers */ -static TupleTableSlot *ColumnarScanNext(ScanState *ss); -static bool ColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot); +static TupleTableSlot *PgColumnarScanNext(ScanState *ss); +static bool PgColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot); -static const CustomPathMethods columnar_path_methods = { +static const CustomPathMethods pgcolumnar_path_methods = { .CustomName = "ColumnarScan", - .PlanCustomPath = ColumnarPlanCustomPath, + .PlanCustomPath = PgColumnarPlanCustomPath, .ReparameterizeCustomPathByChild = NULL, }; -const CustomScanMethods columnar_scan_methods = { +const CustomScanMethods pgcolumnar_scan_methods = { .CustomName = "ColumnarScan", - .CreateCustomScanState = ColumnarCreateScanState, + .CreateCustomScanState = PgColumnarCreateScanState, }; -static const CustomExecMethods columnar_exec_methods = { +static const CustomExecMethods pgcolumnar_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginCustomScan, - .ExecCustomScan = ColumnarExecCustomScan, - .EndCustomScan = ColumnarEndCustomScan, - .ReScanCustomScan = ColumnarReScanCustomScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMCustomScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMCustomScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMCustomScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerCustomScan, - .ExplainCustomScan = ColumnarExplainCustomScan, + .BeginCustomScan = PgColumnarBeginCustomScan, + .ExecCustomScan = PgColumnarExecCustomScan, + .EndCustomScan = PgColumnarEndCustomScan, + .ReScanCustomScan = PgColumnarReScanCustomScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMCustomScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMCustomScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMCustomScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerCustomScan, + .ExplainCustomScan = PgColumnarExplainCustomScan, }; /* ------------------------------------------------------------------------- @@ -161,7 +161,7 @@ static const CustomExecMethods columnar_exec_methods = { * ------------------------------------------------------------------------- */ /* - * columnar_projected_columns + * pgcolumnar_projected_columns * Build the 0-based set of columns the plan actually references, from the * Vars in its target list and its restriction clauses. Returns NULL when * a whole-row or system column is requested, meaning "all columns", so the @@ -170,7 +170,7 @@ static const CustomExecMethods columnar_exec_methods = { * (spec 9). */ static Bitmapset * -columnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) +pgcolumnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) { Bitmapset *needed = NULL; Bitmapset *projected = NULL; @@ -211,12 +211,12 @@ columnar_projected_columns(CustomScan *cscan, int natts, int *nProjected) } /* - * columnar_commute_strategy + * pgcolumnar_commute_strategy * The btree comparison strategy for "value op column" given the strategy * for "column op value", used when the constant is on the left. */ static StrategyNumber -columnar_commute_strategy(StrategyNumber s) +pgcolumnar_commute_strategy(StrategyNumber s) { switch (s) { @@ -236,7 +236,7 @@ columnar_commute_strategy(StrategyNumber s) } /* - * columnar_clause_to_scankey + * pgcolumnar_clause_to_scankey * Translate a single restriction clause of the form "column op const" * (or "const op column") into a scan key for chunk-group skipping, when * op is a btree comparison operator in the column type's default btree @@ -245,7 +245,7 @@ columnar_commute_strategy(StrategyNumber s) * executor still applies them as a filter, so results are unaffected. */ static bool -columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, +pgcolumnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, ScanKey key) { OpExpr *op; @@ -294,7 +294,7 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, /* * The stored per-chunk min/max are ordered under the column's own - * collation (that is what the writer used, columnar_write_state.c), and the + * collation (that is what the writer used, pgcolumnar_write_state.c), and the * reader evaluates the skip under that same collation. Only push a * predicate whose comparison uses that collation; otherwise a differently * collated comparison (for example an explicit COLLATE in the query) could @@ -315,12 +315,12 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, if (strat == InvalidStrategy) return false; if (!varOnLeft) - strat = columnar_commute_strategy(strat); + strat = pgcolumnar_commute_strategy(strat); if (strat == InvalidStrategy) return false; /* - * Fill the scan key directly. The reader (columnar_build_predicates) uses + * Fill the scan key directly. The reader (pgcolumnar_build_predicates) uses * sk_attno, sk_strategy, sk_subtype and sk_argument, and looks up the * column type's own comparison proc; it never calls sk_func, so we leave * that zeroed rather than build one for a possibly-cross-type operator. @@ -337,13 +337,13 @@ columnar_clause_to_scankey(Node *clause, Index scanrelid, TupleDesc tupdesc, } /* - * ColumnarBuildScanKeys + * PgColumnarBuildScanKeys * Build the scan-key array for chunk-group skipping from a plan's * restriction clauses (spec 9). Clauses that are not simple comparisons - * are skipped. Shared with the vectorized aggregate (columnar_vector.c). + * are skipped. Shared with the vectorized aggregate (pgcolumnar_vector.c). */ ScanKey -ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, +PgColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, int *nkeys) { ScanKey keys; @@ -357,7 +357,7 @@ ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, keys = (ScanKey) palloc0(sizeof(ScanKeyData) * list_length(qual)); foreach(lc, qual) { - if (columnar_clause_to_scankey((Node *) lfirst(lc), scanrelid, tupdesc, + if (pgcolumnar_clause_to_scankey((Node *) lfirst(lc), scanrelid, tupdesc, &keys[n])) n++; } @@ -367,14 +367,14 @@ ColumnarBuildScanKeys(List *qual, Index scanrelid, TupleDesc tupdesc, } /* - * ColumnarPlanCustomPath + * PgColumnarPlanCustomPath * Convert the CustomPath to a CustomScan plan node. The restriction * clauses become the scan's qual so the executor re-applies them (this is * what makes chunk-group skipping safe); custom_scan_tlist stays NIL so * the scan tuple is the full base-relation rowtype. */ static Plan * -ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, +PgColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans) { @@ -389,13 +389,13 @@ ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, /* carry the chosen projection name (gap 26), if any, into the plan */ cscan->custom_private = best_path->custom_private; cscan->custom_scan_tlist = NIL; - cscan->methods = &columnar_scan_methods; + cscan->methods = &pgcolumnar_scan_methods; return &cscan->scan.plan; } /* - * columnar_choose_projection + * pgcolumnar_choose_projection * Pick a projection that serves this scan better than the base: it must * cover every referenced column (so no base reconstruction is needed at * scan time) and its leading sort column must appear in a restriction @@ -404,7 +404,7 @@ ColumnarPlanCustomPath(PlannerInfo *root, RelOptInfo *rel, * reference disqualifies a projection scan. */ static char * -columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) +pgcolumnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) { uint64 storageId; Relation r; @@ -417,16 +417,16 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) int x; bool haveAdditional = false; - if (!columnar_enable_projection_scan) + if (!pgcolumnar_enable_projection_scan) return NULL; r = table_open(relid, AccessShareLock); - storageId = ColumnarStorageId(r); + storageId = PgColumnarStorageId(r); table_close(r, AccessShareLock); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) - if (((ColumnarProjection *) lfirst(lc))->projectionId > 0) + if (((PgColumnarProjection *) lfirst(lc))->projectionId > 0) haveAdditional = true; if (!haveAdditional) return NULL; @@ -447,7 +447,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); bool covers = true; bool skips; int y; @@ -494,7 +494,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) } /* - * columnar_index_correlation + * pgcolumnar_index_correlation * |correlation| of the index's leading key against heap (row-number) order, * read from pg_statistic exactly as btcostestimate does (selfuncs.c). A value * near 1 means rows an ordered index scan visits are already clustered into a @@ -507,7 +507,7 @@ columnar_choose_projection(PlannerInfo *root, RelOptInfo *rel, Oid relid) * key, no ANALYZE, no correlation slot. */ static double -columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) +pgcolumnar_index_correlation(IndexOptInfo *index, Oid heapRelid) { AttrNumber attno; Oid sortop; @@ -549,14 +549,14 @@ columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) } /* - * columnar_scan_decode_shape + * pgcolumnar_scan_decode_shape * How much a by-row-number fetch of this rel actually decodes: the number of * columns and their summed width. * * Not the columns the scan emits. The deferred index-fetch slot decodes the * attribute *prefix* 0..max-referenced, because slot_getsomeattrs asks for a - * prefix and cannot ask for a set (columnar_tableam.c, - * columnar_slot_decode_upto). A query referencing only a late column therefore + * prefix and cannot ask for a set (pgcolumnar_tableam.c, + * pgcolumnar_slot_decode_upto). A query referencing only a late column therefore * decodes every column before it, and sizing this from reltarget -- the * emitted columns -- understates the decode by the ratio of the prefix to the * projection (issue #363). @@ -571,7 +571,7 @@ columnar_index_correlation(IndexOptInfo *index, Oid heapRelid) * unreferenced columns in the prefix are not in reltarget at all. */ static void -columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, +pgcolumnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, int *nprefix, double *prefixWidth) { Bitmapset *attrs = NULL; @@ -636,11 +636,11 @@ columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, } /* - * columnar_index_fetch_penalty + * pgcolumnar_index_fetch_penalty * extra cost to add to a heap-fetching columnar index scan for the row-group * decodes its per-row fetches force. Core's cost_index prices a heap fetch as * a page or two; a columnar fetch decodes the whole row group the row lives in - * (columnar_reader.c, ColumnarReadRowByNumber), which is why the planner picks + * (pgcolumnar_reader.c, PgColumnarReadRowByNumber), which is why the planner picks * an index scan for an unclustered ORDER BY and then runs for minutes (#355). * * A statement-scoped cache (issue #143) means a group is decoded once per scan, not @@ -663,10 +663,10 @@ columnar_scan_decode_shape(RelOptInfo *rel, Index rti, Oid relid, * decode of R rows across the columns the scan needs. */ static Cost -columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, +pgcolumnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, int nproj, double decodedWidth, bool tid_ordered) { - double R = (double) columnar_stripe_row_limit; + double R = (double) pgcolumnar_stripe_row_limit; double N = (rel->tuples > 0) ? rel->tuples : rows; double n_groups, pages_per_stripe, @@ -728,13 +728,13 @@ columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, /* * How far above one full scan a fetching index path may be priced (issue #376). - * See columnar_penalize_index_fetches for why this is a bound rather than a + * See pgcolumnar_penalize_index_fetches for why this is a bound rather than a * better model, and for the two measurements that fix the window it sits in. */ #define COLUMNAR_INDEX_FETCH_PENALTY_MAX_SCANS 20.0 /* - * columnar_full_scan_cost + * pgcolumnar_full_scan_cost * What one full scan of this relation costs. * * The seqscan's own number when core still has one, and the same work priced @@ -744,7 +744,7 @@ columnar_index_fetch_penalty(RelOptInfo *rel, double rows, double rho, * penalty below. */ static Cost -columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) +pgcolumnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) { QualCost qcost; double ntuples; @@ -761,7 +761,7 @@ columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) } /* - * columnar_path_order_cmp + * pgcolumnar_path_order_cmp * Order two paths the way add_path keeps rel->pathlist ordered. * * PG18 sorts by disabled_nodes and then total_cost; before that there is no @@ -770,7 +770,7 @@ columnar_full_scan_cost(RelOptInfo *rel, Path *seqpath) * wrong key, so it is spelled out per version rather than assumed. */ static int -columnar_path_order_cmp(const ListCell *a, const ListCell *b) +pgcolumnar_path_order_cmp(const ListCell *a, const ListCell *b) { const Path *pa = (const Path *) lfirst(a); const Path *pb = (const Path *) lfirst(b); @@ -787,7 +787,7 @@ columnar_path_order_cmp(const ListCell *a, const ListCell *b) } /* - * columnar_penalize_index_fetches + * pgcolumnar_penalize_index_fetches * Price the per-row heap fetch of the surviving index and bitmap paths * (#355), and restore the ordering add_path expects. * @@ -818,7 +818,7 @@ columnar_path_order_cmp(const ListCell *a, const ListCell *b) * planner models by fractioning (total - startup). */ static void -columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, +pgcolumnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, Cost fullScanCost) { int nproj; @@ -827,10 +827,10 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, bool mutated = false; ListCell *lc; - if (!columnar_enable_index_fetch_penalty) + if (!pgcolumnar_enable_index_fetch_penalty) return; - columnar_scan_decode_shape(rel, rti, relid, &nproj, &decodedWidth); + pgcolumnar_scan_decode_shape(rel, rti, relid, &nproj, &decodedWidth); /* * The penalty is bounded by a multiple of one full scan (issue #376). @@ -875,15 +875,15 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, if (p->pathtype == T_IndexScan) { IndexPath *ip = castNode(IndexPath, p); - double rho = columnar_index_correlation(ip->indexinfo, relid); + double rho = pgcolumnar_index_correlation(ip->indexinfo, relid); - add = columnar_index_fetch_penalty(rel, p->rows, rho, nproj, + add = pgcolumnar_index_fetch_penalty(rel, p->rows, rho, nproj, decodedWidth, false); } else if (p->pathtype == T_BitmapHeapScan) { /* a bitmap heap scan fetches in TID (row-number) order */ - add = columnar_index_fetch_penalty(rel, p->rows, 1.0, nproj, + add = pgcolumnar_index_fetch_penalty(rel, p->rows, 1.0, nproj, decodedWidth, true); } /* T_IndexOnlyScan and the custom scans do no heap fetch */ @@ -905,11 +905,11 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, } if (mutated) - list_sort(rel->pathlist, columnar_path_order_cmp); + list_sort(rel->pathlist, pgcolumnar_path_order_cmp); } /* - * ColumnarSetRelPathlist + * PgColumnarSetRelPathlist * set_rel_pathlist_hook: for a columnar base relation, replace the * sequential-scan path with the columnar custom scan and drop parallel * paths. Index and bitmap paths are left in place, so ordinary index @@ -918,7 +918,7 @@ columnar_penalize_index_fetches(RelOptInfo *rel, Index rti, Oid relid, * steers the planner to an index scan exactly as before. */ static void -ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, +PgColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, RangeTblEntry *rte) { CustomPath *cpath; @@ -931,20 +931,20 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, if (prev_set_rel_pathlist_hook) prev_set_rel_pathlist_hook(root, rel, rti, rte); - if (!columnar_enable_custom_scan) + if (!pgcolumnar_enable_custom_scan) return; if (rte->rtekind != RTE_RELATION || rte->relkind != RELKIND_RELATION) return; if (rel->reloptkind != RELOPT_BASEREL) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; /* - * The custom scan is the scalar per-row path (ColumnarReadNextRow), so + * The custom scan is the scalar per-row path (PgColumnarReadNextRow), so * pushed-down predicates drive zone-map row-group and per-vector skipping * (native spec 7.1). Ungrouped aggregates are answered from the zone maps by - * the separate aggregate path (columnar_vector.c). + * the separate aggregate path (pgcolumnar_vector.c). */ /* find a non-parameterized seqscan path to inherit its costs from */ @@ -975,8 +975,8 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * doing this after the add_path calls meant the columnar path was judged * against index costs that had not yet been penalized, and freed. */ - columnar_penalize_index_fetches(rel, rti, rte->relid, - columnar_full_scan_cost(rel, seqpath)); + pgcolumnar_penalize_index_fetches(rel, rti, rte->relid, + pgcolumnar_full_scan_cost(rel, seqpath)); cpath = makeNode(CustomPath); cpath->path.pathtype = T_CustomScan; @@ -1011,7 +1011,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, */ cpath->path.startup_cost = (seqpath != NULL) ? seqpath->startup_cost : rel->baserestrictcost.startup; - cpath->path.total_cost = columnar_full_scan_cost(rel, seqpath); + cpath->path.total_cost = pgcolumnar_full_scan_cost(rel, seqpath); cpath->path.pathkeys = NIL; cpath->flags = 0; cpath->custom_paths = NIL; @@ -1025,7 +1025,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, */ cpath->custom_restrictinfo = rel->baserestrictinfo; #endif - cpath->methods = &columnar_path_methods; + cpath->methods = &pgcolumnar_path_methods; /* * Keep the costs before offering the path. add_path FREES a path it judges @@ -1049,7 +1049,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * result is correct whichever path wins (the executor re-applies the qual). */ { - char *projName = columnar_choose_projection(root, rel, rte->relid); + char *projName = pgcolumnar_choose_projection(root, rel, rte->relid); if (projName != NULL) { @@ -1079,7 +1079,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, #if PG_VERSION_NUM >= 170000 ppath->custom_restrictinfo = rel->baserestrictinfo; #endif - ppath->methods = &columnar_path_methods; + ppath->methods = &pgcolumnar_path_methods; add_path(rel, &ppath->path); } @@ -1135,7 +1135,7 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, #if PG_VERSION_NUM >= 170000 ppath->custom_restrictinfo = rel->baserestrictinfo; #endif - ppath->methods = &columnar_path_methods; + ppath->methods = &pgcolumnar_path_methods; add_partial_path(rel, &ppath->path); } } @@ -1146,13 +1146,13 @@ ColumnarSetRelPathlist(PlannerInfo *root, RelOptInfo *rel, Index rti, * ------------------------------------------------------------------------- */ /* - * ColumnarCreateScanState + * PgColumnarCreateScanState * Shared create-state callback for the one registered CustomScanMethods. A * scanrelid==0 plan is the vectorized aggregate upper node; anything else * is a base-relation columnar scan. */ static Node * -ColumnarCreateScanState(CustomScan *cscan) +PgColumnarCreateScanState(CustomScan *cscan) { if (cscan->scan.scanrelid == 0) { @@ -1163,28 +1163,28 @@ ColumnarCreateScanState(CustomScan *cscan) * path carries length 3. */ if (list_length(cscan->custom_private) == 5) - return ColumnarCreateGroupAggScanState(cscan); - return ColumnarCreateAggScanState(cscan); + return PgColumnarCreateGroupAggScanState(cscan); + return PgColumnarCreateAggScanState(cscan); } - return ColumnarCreateBaseScanState(cscan); + return PgColumnarCreateBaseScanState(cscan); } static Node * -ColumnarCreateBaseScanState(CustomScan *cscan) +PgColumnarCreateBaseScanState(CustomScan *cscan) { - ColumnarCustomScanState *cstate = - (ColumnarCustomScanState *) palloc0(sizeof(ColumnarCustomScanState)); + PgColumnarCustomScanState *cstate = + (PgColumnarCustomScanState *) palloc0(sizeof(PgColumnarCustomScanState)); cstate->css.ss.ps.type = T_CustomScanState; - cstate->css.methods = &columnar_exec_methods; + cstate->css.methods = &pgcolumnar_exec_methods; return (Node *) cstate; } /* the projection name the planner chose, or NULL for a base scan (gap 26) */ static char * -columnar_chosen_projection(CustomScan *cscan) +pgcolumnar_chosen_projection(CustomScan *cscan) { if (cscan->custom_private == NIL) return NULL; @@ -1192,7 +1192,7 @@ columnar_chosen_projection(CustomScan *cscan) } /* - * columnar_setup_projection_scan + * pgcolumnar_setup_projection_scan * Open a read on the named projection's storage with a synthetic * [rownumber, cols...] descriptor, build the base<->projection column map, * and translate the pushed-down scan keys to the projection's attnums so the @@ -1202,13 +1202,13 @@ columnar_chosen_projection(CustomScan *cscan) * since planning. */ static void -columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, +pgcolumnar_setup_projection_scan(PgColumnarCustomScanState *cstate, Relation rel, Snapshot snapshot, const char *projName) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); TupleDesc tableDesc = RelationGetDescr(rel); - List *projs = ColumnarListProjections(storageId); - ColumnarProjection *proj = NULL; + List *projs = PgColumnarListProjections(storageId); + PgColumnarProjection *proj = NULL; ListCell *lc; TupleDesc projTupdesc; ScanKey projKeys = NULL; @@ -1217,7 +1217,7 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projectionId > 0 && strcmp(p->name, projName) == 0) { @@ -1227,7 +1227,7 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, } if (proj == NULL) { - cstate->readState = ColumnarBeginRead(rel, snapshot, NULL, + cstate->readState = PgColumnarBeginRead(rel, snapshot, NULL, cstate->projectedColumns, cstate->nScanKeys, cstate->scanKeys); return; @@ -1268,20 +1268,20 @@ columnar_setup_projection_scan(ColumnarCustomScanState *cstate, Relation rel, } } - cstate->readState = ColumnarBeginReadWithStorage(rel, snapshot, + cstate->readState = PgColumnarBeginReadWithStorage(rel, snapshot, proj->projStorageId, projTupdesc, NULL, NULL, nProjKeys, projKeys); /* cache base liveness once so the per-row deletion test is a binary search, * not a per-row catalog scan */ - cstate->livenessCache = ColumnarBuildLivenessCache(rel, snapshot); + cstate->livenessCache = PgColumnarBuildLivenessCache(rel, snapshot); cstate->projScan = true; } static void -ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; CustomScan *cscan = (CustomScan *) node->ss.ps.plan; Relation rel = node->ss.ss_currentRelation; TupleDesc tupdesc = RelationGetDescr(rel); @@ -1294,13 +1294,13 @@ ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) * touches the catalog and data pages, is skipped when we will not execute. */ cstate->projectedColumns = - columnar_projected_columns(cscan, tupdesc->natts, &cstate->nProjected); + pgcolumnar_projected_columns(cscan, tupdesc->natts, &cstate->nProjected); cstate->scanKeys = - ColumnarBuildScanKeys(cscan->scan.plan.qual, cscan->scan.scanrelid, + PgColumnarBuildScanKeys(cscan->scan.plan.qual, cscan->scan.scanrelid, tupdesc, &cstate->nScanKeys); /* the projection the planner chose (gap 26); reported by EXPLAIN */ - cstate->projName = columnar_chosen_projection(cscan); + cstate->projName = pgcolumnar_chosen_projection(cscan); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) return; @@ -1310,36 +1310,36 @@ ColumnarBeginCustomScan(CustomScanState *node, EState *estate, int eflags) * reader's catalog snapshot sees them (read-your-writes, spec 9), matching * the table AM's own scan_begin. */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); if (cstate->projName != NULL) { /* gap 26: read a covering projection instead of the base. */ - columnar_setup_projection_scan(cstate, rel, estate->es_snapshot, + pgcolumnar_setup_projection_scan(cstate, rel, estate->es_snapshot, cstate->projName); if (!cstate->projScan) cstate->projName = NULL; /* projection vanished; base fallback */ } else { - cstate->readState = ColumnarBeginRead(rel, estate->es_snapshot, NULL, + cstate->readState = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, cstate->projectedColumns, cstate->nScanKeys, cstate->scanKeys); } } /* - * ColumnarScanNext + * PgColumnarScanNext * ExecScan access method: fetch the next columnar row into the scan slot. * The row's synthetic item pointer (spec 6) is stored on the slot so an * UPDATE/DELETE above the scan can identify the row by its ctid. */ static TupleTableSlot * -columnar_projection_scan_next(ScanState *ss) +pgcolumnar_projection_scan_next(ScanState *ss) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) ss; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) ss; TupleTableSlot *slot = ss->ss_ScanTupleSlot; Relation rel = ss->ss_currentRelation; int natts = cstate->nTotalColumns; @@ -1350,14 +1350,14 @@ columnar_projection_scan_next(ScanState *ss) uint64 baseRow; int c; - if (!ColumnarReadNextRow(cstate->readState, cstate->projValues, + if (!PgColumnarReadNextRow(cstate->readState, cstate->projValues, cstate->projNulls, &projRowNum)) return NULL; /* deletes/visibility come from the base (gap 26): the projection stores * no delete vector, so filter by the stored base row number via the cache */ baseRow = (uint64) DatumGetInt64(cstate->projValues[0]); - if (!ColumnarLivenessCacheIsLive(cstate->livenessCache, baseRow)) + if (!PgColumnarLivenessCacheIsLive(cstate->livenessCache, baseRow)) continue; ExecClearTuple(slot); @@ -1377,60 +1377,60 @@ columnar_projection_scan_next(ScanState *ss) } } ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(baseRow, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(baseRow, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return slot; } } static TupleTableSlot * -ColumnarScanNext(ScanState *ss) +PgColumnarScanNext(ScanState *ss) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) ss; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) ss; TupleTableSlot *slot = ss->ss_ScanTupleSlot; uint64 rowNumber; if (cstate->projScan) - return columnar_projection_scan_next(ss); + return pgcolumnar_projection_scan_next(ss); ExecClearTuple(slot); - if (!ColumnarReadNextRow(cstate->readState, slot->tts_values, + if (!PgColumnarReadNextRow(cstate->readState, slot->tts_values, slot->tts_isnull, &rowNumber)) return NULL; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(ss->ss_currentRelation); return slot; } static bool -ColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot) +PgColumnarScanRecheck(ScanState *ss, TupleTableSlot *slot) { return true; } static TupleTableSlot * -ColumnarExecCustomScan(CustomScanState *node) +PgColumnarExecCustomScan(CustomScanState *node) { return ExecScan(&node->ss, - (ExecScanAccessMtd) ColumnarScanNext, - (ExecScanRecheckMtd) ColumnarScanRecheck); + (ExecScanAccessMtd) PgColumnarScanNext, + (ExecScanRecheckMtd) PgColumnarScanRecheck); } static void -ColumnarReScanCustomScan(CustomScanState *node) +PgColumnarReScanCustomScan(CustomScanState *node) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->readState != NULL) { - ColumnarRescanRead(cstate->readState); + PgColumnarRescanRead(cstate->readState); /* a parallel rescan keeps sharing the same stripe counter */ if (cstate->parallelCounter != NULL) - ColumnarReadSetParallelCounter(cstate->readState, + PgColumnarReadSetParallelCounter(cstate->readState, cstate->parallelCounter); } @@ -1445,26 +1445,26 @@ ColumnarReScanCustomScan(CustomScanState *node) * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMCustomScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); cstate->parallelCounter = counter; if (cstate->readState != NULL) - ColumnarReadSetParallelCounter(cstate->readState, counter); + PgColumnarReadSetParallelCounter(cstate->readState, counter); } static void -ColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -1473,39 +1473,39 @@ ColumnarReInitializeDSMCustomScan(CustomScanState *node, ParallelContext *pcxt, } static void -ColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerCustomScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; cstate->parallelCounter = counter; if (cstate->readState != NULL) - ColumnarReadSetParallelCounter(cstate->readState, counter); + PgColumnarReadSetParallelCounter(cstate->readState, counter); } static void -ColumnarEndCustomScan(CustomScanState *node) +PgColumnarEndCustomScan(CustomScanState *node) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->readState != NULL) { - ColumnarEndRead(cstate->readState); + PgColumnarEndRead(cstate->readState); cstate->readState = NULL; } if (cstate->livenessCache != NULL) { - ColumnarFreeLivenessCache(cstate->livenessCache); + PgColumnarFreeLivenessCache(cstate->livenessCache); cstate->livenessCache = NULL; } } static void -ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, +PgColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarCustomScanState *cstate = (ColumnarCustomScanState *) node; + PgColumnarCustomScanState *cstate = (PgColumnarCustomScanState *) node; if (cstate->projName != NULL) ExplainPropertyText("Columnar Projection", cstate->projName, es); @@ -1518,8 +1518,8 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * Report what the scan pushes down, not what the planner handed it. * * cstate->nScanKeys is the count the planner produced, and it is the same - * whether or not pushdown is enabled. But columnar_enable_qual_pushdown - * gates columnar_build_predicates in ColumnarBeginRead, so with the setting + * whether or not pushdown is enabled. But pgcolumnar_enable_qual_pushdown + * gates pgcolumnar_build_predicates in PgColumnarBeginRead, so with the setting * off the reader builds no predicates and skips no chunk groups: nothing is * pushed down in any sense the scan acts on. Someone turning the setting off * to test a theory, and checking EXPLAIN to confirm it took effect, was told @@ -1530,7 +1530,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * so this one line meant something different from all of its neighbours. */ ExplainPropertyInteger("Columnar Pushed-Down Filters", NULL, - columnar_enable_qual_pushdown ? cstate->nScanKeys : 0, + pgcolumnar_enable_qual_pushdown ? cstate->nScanKeys : 0, es); if (cstate->readState != NULL) @@ -1539,7 +1539,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, uint64 groupsSkipped = 0; uint64 groupsTotal = 0; - ColumnarReadStats(cstate->readState, &groupsRead, &groupsSkipped, + PgColumnarReadStats(cstate->readState, &groupsRead, &groupsSkipped, &groupsTotal); ExplainPropertyInteger("Columnar Chunk Groups Total", NULL, @@ -1549,7 +1549,7 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, ExplainPropertyInteger("Columnar Chunk Groups Removed by Filter", NULL, (int64) groupsSkipped, es); ExplainPropertyInteger("Columnar Vectors Skipped", NULL, - (int64) ColumnarVectorsSkipped(cstate->readState), es); + (int64) PgColumnarVectorsSkipped(cstate->readState), es); } } @@ -1558,10 +1558,10 @@ ColumnarExplainCustomScan(CustomScanState *node, List *ancestors, * ------------------------------------------------------------------------- */ void -ColumnarCustomScanInit(void) +PgColumnarCustomScanInit(void) { - RegisterCustomScanMethods(&columnar_scan_methods); + RegisterCustomScanMethods(&pgcolumnar_scan_methods); prev_set_rel_pathlist_hook = set_rel_pathlist_hook; - set_rel_pathlist_hook = ColumnarSetRelPathlist; + set_rel_pathlist_hook = PgColumnarSetRelPathlist; } diff --git a/src/columnar_delete_vector.c b/src/columnar_delete_vector.c index 5637442..792288d 100644 --- a/src/columnar_delete_vector.c +++ b/src/columnar_delete_vector.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_delete_vector.c + * pgcolumnar_delete_vector.c * Delete and update marking for pgColumnar (spec 7.5, 9). Deletes do not * rewrite stripes; instead a bit is set in the columnar.delete_vector entry for * the affected chunk group. Update is delete-plus-insert, so it also uses @@ -57,8 +57,8 @@ typedef struct DeleteVectorBuffer DeleteVectorChunkBuffer *lastChunk; } DeleteVectorBuffer; -static MemoryContext ColumnarDeleteVectorContext = NULL; -static List *ColumnarDeleteVectorBuffers = NIL; +static MemoryContext PgColumnarDeleteVectorContext = NULL; +static List *PgColumnarDeleteVectorBuffers = NIL; static DeleteVectorBuffer *delete_vector_get_buffer(Relation rel, uint64 storageId); static NativeRowGroupMetadata *delete_vector_find_row_group(DeleteVectorBuffer *buf, @@ -74,7 +74,7 @@ static void delete_vector_flush_buffer(DeleteVectorBuffer *buf); * delete_vector_chunk_cmp * Total order over chunk-group buffers by (stripeId, chunkId, * startRowNumber). Flushing in this order makes every transaction acquire - * the per-chunk-group locks (columnar_metadata.c) in the same global + * the per-chunk-group locks (pgcolumnar_metadata.c) in the same global * order, so two concurrent deleters cannot form an AB-BA deadlock cycle. */ static int @@ -105,19 +105,19 @@ delete_vector_get_buffer(Relation rel, uint64 storageId) MemoryContext oldContext; DeleteVectorBuffer *buf; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { buf = (DeleteVectorBuffer *) lfirst(lc); if (buf->storageId == storageId && buf->subid == subid) return buf; } - if (ColumnarDeleteVectorContext == NULL) - ColumnarDeleteVectorContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarDeleteVectorContext == NULL) + PgColumnarDeleteVectorContext = AllocSetContextCreate(TopTransactionContext, "columnar delete vector", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); buf = palloc0(sizeof(DeleteVectorBuffer)); buf->relid = RelationGetRelid(rel); buf->storageId = storageId; @@ -126,7 +126,7 @@ delete_vector_get_buffer(Relation rel, uint64 storageId) buf->rowGroupCache = NIL; buf->lastChunk = NULL; buf->lastGroup = NULL; - ColumnarDeleteVectorBuffers = lappend(ColumnarDeleteVectorBuffers, buf); + PgColumnarDeleteVectorBuffers = lappend(PgColumnarDeleteVectorBuffers, buf); MemoryContextSwitchTo(oldContext); return buf; @@ -166,10 +166,10 @@ delete_vector_find_row_group(DeleteVectorBuffer *buf, uint64 rowNumber) if (attempt == 0) { MemoryContext oldContext = - MemoryContextSwitchTo(ColumnarDeleteVectorContext); - Snapshot snap = ColumnarCatalogSnapshot(GetActiveSnapshot()); + MemoryContextSwitchTo(PgColumnarDeleteVectorContext); + Snapshot snap = PgColumnarCatalogSnapshot(GetActiveSnapshot()); - buf->rowGroupCache = ColumnarReadRowGroupList(buf->storageId, snap); + buf->rowGroupCache = PgColumnarReadRowGroupList(buf->storageId, snap); buf->lastGroup = NULL; /* points into the list just replaced */ MemoryContextSwitchTo(oldContext); } @@ -206,7 +206,7 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, } } - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); chunk = palloc0(sizeof(DeleteVectorChunkBuffer)); chunk->stripeId = stripeId; chunk->chunkId = chunkId; @@ -223,7 +223,7 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, } /* - * ColumnarMarkRowDeleted + * PgColumnarMarkRowDeleted * Record that the row with the given 1-based row number is deleted, by * setting its bit in the in-memory delete buffer for its chunk group. * The mark targets the whole enclosing row group as one bitmap (chunk id @@ -231,9 +231,9 @@ delete_vector_get_chunk(DeleteVectorBuffer *buf, uint64 stripeId, int chunkId, * delete_vector_find_row_group using its firstRowNumber and rowCount. */ void -ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) +PgColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); DeleteVectorBuffer *buf = delete_vector_get_buffer(rel, storageId); uint64 startRowNumber; uint64 endRowNumber; @@ -263,26 +263,26 @@ ColumnarMarkRowDeleted(Relation rel, uint64 rowNumber) * index-only scan never skips the fetch for a block with a dead row (gap 28). * A no-op unless a prior vacuum had marked the block visible. */ - ColumnarVMClearForRow(rel, rowNumber); + PgColumnarVMClearForRow(rel, rowNumber); } /* - * ColumnarDeleteVectorBufferedDeleted + * PgColumnarDeleteVectorBufferedDeleted * True when the row is marked deleted in an in-memory row-mask buffer that * has not yet been flushed to the catalog. The unique/primary-key check runs * as part of an insert (or the insert half of an update) and fetches a - * conflicting row through ColumnarReadRowByNumber before the delete of the + * conflicting row through PgColumnarReadRowByNumber before the delete of the * old row is flushed; consulting the buffer here lets a same-key UPDATE (the * old row is buffered-deleted) proceed. Checks every buffer for the relation, * across subtransactions. */ bool -ColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber) +PgColumnarDeleteVectorBufferedDeleted(Relation rel, uint64 rowNumber) { Oid relid = RelationGetRelid(rel); ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); ListCell *cc; @@ -350,7 +350,7 @@ delete_vector_flush_buffer(DeleteVectorBuffer *buf) rm.bitmap = chunk->mask; rm.bitmapLen = chunk->maskLen; - ColumnarUpsertDeleteVector(buf->storageId, &rm); + PgColumnarUpsertDeleteVector(buf->storageId, &rm); } if (pushedSnapshot) @@ -363,17 +363,17 @@ delete_vector_flush_buffer(DeleteVectorBuffer *buf) } /* - * ColumnarFlushDeleteVectorForRelation + * PgColumnarFlushDeleteVectorForRelation * Flush pending delete marks for one relation. Called at scan start so a * delete made earlier in this transaction is visible to a later scan. */ void -ColumnarFlushDeleteVectorForRelation(Relation rel) +PgColumnarFlushDeleteVectorForRelation(Relation rel) { Oid relid = RelationGetRelid(rel); ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); @@ -383,47 +383,47 @@ ColumnarFlushDeleteVectorForRelation(Relation rel) } /* - * ColumnarFlushAllDeleteVectors + * PgColumnarFlushAllDeleteVectors * Flush every pending delete buffer. Called at transaction pre-commit. */ void -ColumnarFlushAllDeleteVectors(void) +PgColumnarFlushAllDeleteVectors(void) { ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) delete_vector_flush_buffer((DeleteVectorBuffer *) lfirst(lc)); } /* - * ColumnarDiscardAllDeleteVectors + * PgColumnarDiscardAllDeleteVectors * Forget all pending delete buffers (transaction end). */ void -ColumnarDiscardAllDeleteVectors(void) +PgColumnarDiscardAllDeleteVectors(void) { - ColumnarDeleteVectorBuffers = NIL; - ColumnarDeleteVectorContext = NULL; + PgColumnarDeleteVectorBuffers = NIL; + PgColumnarDeleteVectorContext = NULL; } /* - * ColumnarDeleteVectorDiscardSubXact + * PgColumnarDeleteVectorDiscardSubXact * Drop delete buffers made in an aborting subtransaction. The catalog * rows they would have produced were never written (or, if a scan flushed * them, are made invisible by the subtransaction abort itself). */ void -ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) +PgColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarDeleteVectorBuffers == NIL) + if (PgColumnarDeleteVectorBuffers == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarDeleteVectorContext); - foreach(lc, ColumnarDeleteVectorBuffers) + oldContext = MemoryContextSwitchTo(PgColumnarDeleteVectorContext); + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); @@ -432,20 +432,20 @@ ColumnarDeleteVectorDiscardSubXact(SubTransactionId subid) } MemoryContextSwitchTo(oldContext); - ColumnarDeleteVectorBuffers = kept; + PgColumnarDeleteVectorBuffers = kept; } /* - * ColumnarDeleteVectorPromoteSubXact + * PgColumnarDeleteVectorPromoteSubXact * On subtransaction commit, reassign its delete buffers to the parent so * they survive until the parent resolves. */ void -ColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent) +PgColumnarDeleteVectorPromoteSubXact(SubTransactionId subid, SubTransactionId parent) { ListCell *lc; - foreach(lc, ColumnarDeleteVectorBuffers) + foreach(lc, PgColumnarDeleteVectorBuffers) { DeleteVectorBuffer *buf = (DeleteVectorBuffer *) lfirst(lc); diff --git a/src/columnar_encoding.c b/src/columnar_encoding.c index ca43719..490c0f1 100644 --- a/src/columnar_encoding.c +++ b/src/columnar_encoding.c @@ -1,16 +1,16 @@ /*------------------------------------------------------------------------- * - * columnar_encoding.c + * pgcolumnar_encoding.c * Lightweight, type-aware value-stream encodings (I1) and the * compression-block abstraction they implement (I2). * * The encoding layer sits between the raw serialized value stream that the - * writer builds (a packed sequence of ColumnarEncodeValue outputs, spec 4) and - * the general-purpose block codec (columnar_compression.c, spec 5). An encoding + * writer builds (a packed sequence of PgColumnarEncodeValue outputs, spec 4) and + * the general-purpose block codec (pgcolumnar_compression.c, spec 5). An encoding * is a reversible transform of the raw value-stream BYTES: encode(raw) -> a * smaller encoded buffer, and decode(encoded) -> the byte-identical raw stream. * Because decode reconstructs the exact raw stream, every downstream consumer - * (per-value ColumnarDecodeValue, the vectorized group decoder, the min/max + * (per-value PgColumnarDecodeValue, the vectorized group decoder, the min/max * skip list) is unchanged; only the flush and load paths gain one step. * * The techniques come from the public column-store literature (see @@ -1162,7 +1162,7 @@ decode_alp(const char *enc, uint32 encLen, int w, uint32 n, uint32 rawLen, #define DICT_MAX_DISTINCT 1024 -static inline uint64 columnar_fnv1a(const char *p, uint32 n); +static inline uint64 pgcolumnar_fnv1a(const char *p, uint32 n); static bool encode_dict(const char *raw, uint32 rawLen, Form_pg_attribute att, uint32 n, @@ -1197,8 +1197,8 @@ encode_dict(const char *raw, uint32 rawLen, Form_pg_attribute att, uint32 n, { const char *vp = raw + pos; uint32 vlen = (w > 0) ? (uint32) w - : ColumnarVarSizeAnyUnaligned(vp); - uint64 h = columnar_fnv1a(vp, vlen); + : PgColumnarVarSizeAnyUnaligned(vp); + uint64 h = pgcolumnar_fnv1a(vp, vlen); uint32 s = (uint32) (h & (DICT_HASH_SLOTS - 1)); int code = -1; @@ -1708,7 +1708,7 @@ fsst_deserialize_table(const char *p, uint32 len, FsstTable *t) * probe below. Our own trivial implementation of the public FNV-1a formula, used * only to bucket values while counting distinct ones. */ static inline uint64 -columnar_fnv1a(const char *p, uint32 n) +pgcolumnar_fnv1a(const char *p, uint32 n) { uint64 h = UINT64CONST(1469598103934665603); uint32 i; @@ -1722,7 +1722,7 @@ columnar_fnv1a(const char *p, uint32 n) } /* - * ColumnarFsstDictWins + * PgColumnarFsstDictWins * Cheap pre-check for the FSST table build (issue #155): would dictionary * encoding win outright, making the costly FSST symbol-table build wasted * work? FSST is only attempted per vector when a cheaper encoding has not @@ -1737,7 +1737,7 @@ columnar_fnv1a(const char *p, uint32 n) * This is our own heuristic; FSST is the public scheme (VLDB 2020). */ bool -ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) +PgColumnarFsstDictWins(const char *corpus, uint32 corpusLen) { /* Open-addressing set of value hashes, capacity a power of two comfortably * above the distinct cap so the load factor at the early-exit stays low. */ @@ -1750,14 +1750,14 @@ ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) memset(used, 0, sizeof(used)); while (pos < corpusLen) { - uint32 vlen = ColumnarVarSizeAnyUnaligned(corpus + pos); + uint32 vlen = PgColumnarVarSizeAnyUnaligned(corpus + pos); uint64 h; uint32 s; if (vlen == 0 || pos + vlen > corpusLen) return false; /* malformed run: let the caller build normally */ - h = columnar_fnv1a(corpus + pos, vlen); + h = pgcolumnar_fnv1a(corpus + pos, vlen); s = (uint32) (h & (FSST_CARD_SLOTS - 1)); while (used[s] && slot[s] != h) s = (s + 1) & (FSST_CARD_SLOTS - 1); @@ -1774,13 +1774,13 @@ ColumnarFsstDictWins(const char *corpus, uint32 corpusLen) } /* - * ColumnarFsstBuildChunkTable + * PgColumnarFsstBuildChunkTable * Build one FSST symbol table for a whole column chunk (E3b). The expensive * iterative table build runs once here; the serialized table is handed back * to be stored once per chunk and reused by every FSST vector. */ bool -ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut) { @@ -1809,7 +1809,7 @@ ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, * the raw length: the shared table's bytes are amortized across the chunk and * not charged here, so FSST wins per vector whenever the codes are smaller. * - * That test is only meaningful once ColumnarFsstHelpsCompressed has decided the + * That test is only meaningful once PgColumnarFsstHelpsCompressed has decided the * chunk should use FSST at all; it is what charges the table and what compares * against the compressed size actually written. See the comment on it below. */ @@ -1873,7 +1873,7 @@ encode_fsst_shared(const char *raw, uint32 rawLen, const char *table, * Would FSST still pay once the block compressor has had its turn? * * The per-vector win test above compares encoded lengths, but the bytes that - * reach disk are compressed afterwards (columnar_write_state.c compresses the + * reach disk are compressed afterwards (pgcolumnar_write_state.c compresses the * whole encoded stream). Those are different objectives, and for some shapes * they disagree sharply: FSST turns highly repetitive text into a stream of * high-entropy codes, which is smaller than the text but far less compressible @@ -1894,7 +1894,7 @@ encode_fsst_shared(const char *raw, uint32 rawLen, const char *table, * the per-vector encode is then skipped for every vector in the chunk. */ bool -ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, +PgColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, const char *table, uint32 tableLen, int compressionType, int compressionLevel) { @@ -1918,10 +1918,10 @@ ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, if (!encode_fsst_shared(corpus, corpusLen, table, tableLen, &codes, &codesLen)) return false; - ColumnarCompressValueStream(corpus, corpusLen, compressionType, + PgColumnarCompressValueStream(corpus, corpusLen, compressionType, compressionLevel, &plainComp, &plainCompLen, &usedType, &usedLevel); - ColumnarCompressValueStream(codes, codesLen, compressionType, + PgColumnarCompressValueStream(codes, codesLen, compressionType, compressionLevel, &codesComp, &codesCompLen, &usedType, &usedLevel); @@ -1935,7 +1935,7 @@ ColumnarFsstHelpsCompressed(const char *corpus, uint32 corpusLen, * cannot underflow when plainCompLen is small. */ helps = (((uint64) codesCompLen + tableLen) * 100 - < (uint64) plainCompLen * (uint64) (100 - columnar_fsst_min_gain_percent)); + < (uint64) plainCompLen * (uint64) (100 - pgcolumnar_fsst_min_gain_percent)); pfree(codes); if (plainComp) @@ -2026,7 +2026,7 @@ decode_fsst_shared(const char *enc, uint32 encLen, const char *table, * ------------------------------------------------------------------------- */ void -ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width) { br->raw = raw; @@ -2036,7 +2036,7 @@ ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, } bool -ColumnarBlockNextRun(ColumnarBlockReader *br, const char **valBytes, +PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen) { const char *v; @@ -2058,7 +2058,7 @@ ColumnarBlockNextRun(ColumnarBlockReader *br, const char **valBytes, } const char * -ColumnarEncodingName(int encodingType) +PgColumnarEncodingName(int encodingType) { switch (encodingType) { @@ -2161,7 +2161,7 @@ build_sample(const char *raw, int w, uint32 n, uint32 want, uint32 *sampleN) } /* - * ColumnarEncodeChunk + * PgColumnarEncodeChunk * Choose and apply the best lightweight encoding for one chunk's raw value * stream. Returns the encoding code used and sets out and outLen to the * encoded buffer. When no encoding beats the raw size, returns NONE and @@ -2183,7 +2183,7 @@ build_sample(const char *raw, int w, uint32 n, uint32 want, uint32 *sampleN) * candidate. */ int -ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, +PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen) { @@ -2211,11 +2211,11 @@ ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, * cheap next to the three it replaces. With sampling off, or a chunk too * small to sample, every candidate is applied as before. */ - if (w > 0 && columnar_encoding_sample_rows >= ENCODE_SAMPLE_MIN) + if (w > 0 && pgcolumnar_encoding_sample_rows >= ENCODE_SAMPLE_MIN) { uint32 sampleN = 0; char *sample = build_sample(raw, w, n, - (uint32) columnar_encoding_sample_rows, + (uint32) pgcolumnar_encoding_sample_rows, &sampleN); if (sample != NULL) @@ -2428,12 +2428,12 @@ ColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, } /* - * ColumnarDecodeChunk + * PgColumnarDecodeChunk * Reverse an encoding, reconstructing the byte-identical raw value stream * in cx. For NONE, returns the input pointer unchanged (no copy). */ char * -ColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, +PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx) @@ -2622,10 +2622,10 @@ selftest_rand(uint64 *state) return z ^ (z >> 31); } -PG_FUNCTION_INFO_V1(columnar_debug_encoding_selftest); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_encoding_selftest); Datum -columnar_debug_encoding_selftest(PG_FUNCTION_ARGS) +pgcolumnar_debug_encoding_selftest(PG_FUNCTION_ARGS) { ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; TupleDesc retdesc; diff --git a/src/columnar_flatbuffers.c b/src/columnar_flatbuffers.c index 858d9a1..b7c5510 100644 --- a/src/columnar_flatbuffers.c +++ b/src/columnar_flatbuffers.c @@ -1,7 +1,7 @@ /*------------------------------------------------------------------------- * - * columnar_flatbuffers.c - * A minimal FlatBuffers builder. See columnar_flatbuffers.h. + * pgcolumnar_flatbuffers.c + * A minimal FlatBuffers builder. See pgcolumnar_flatbuffers.h. * * Written fresh for pgColumnar from the public FlatBuffers format description. * @@ -20,7 +20,7 @@ * ------------------------------------------------------------------------- */ void -fb_init(FBB *b) +pgc_fb_init(FBB *b) { b->cap = 256; b->buf = palloc(b->cap); @@ -31,7 +31,7 @@ fb_init(FBB *b) } void -fb_grow(FBB *b, uint32 need) +pgc_fb_grow(FBB *b, uint32 need) { uint64 want; uint32 newcap; @@ -55,25 +55,25 @@ fb_grow(FBB *b, uint32 need) /* prepend n raw bytes (already in final order) */ void -fb_place(FBB *b, const void *src, uint32 n) +pgc_fb_place(FBB *b, const void *src, uint32 n) { - fb_grow(b, n); + pgc_fb_grow(b, n); b->tail += n; memcpy(b->buf + b->cap - b->tail, src, n); } void -fb_pad(FBB *b, uint32 n) +pgc_fb_pad(FBB *b, uint32 n) { if (n == 0) return; - fb_grow(b, n); + pgc_fb_grow(b, n); b->tail += n; memset(b->buf + b->cap - b->tail, 0, n); } uint32 -fb_offset(FBB *b) +pgc_fb_offset(FBB *b) { return b->tail; } @@ -81,71 +81,71 @@ fb_offset(FBB *b) /* align so that, after `additional` more bytes plus a `size`-aligned scalar are * written, the scalar lands aligned (relative to the eventually-aligned end). */ void -fb_prep(FBB *b, uint32 size, uint32 additional) +pgc_fb_prep(FBB *b, uint32 size, uint32 additional) { uint32 alignsize; if (size > b->minalign) b->minalign = size; alignsize = ((~(b->tail + additional)) + 1) & (size - 1); - fb_pad(b, alignsize); + pgc_fb_pad(b, alignsize); } void -fb_push_u8(FBB *b, uint8 v) +pgc_fb_push_u8(FBB *b, uint8 v) { - fb_prep(b, 1, 0); - fb_place(b, &v, 1); + pgc_fb_prep(b, 1, 0); + pgc_fb_place(b, &v, 1); } void -fb_push_i16(FBB *b, int16 v) +pgc_fb_push_i16(FBB *b, int16 v) { - fb_prep(b, 2, 0); - fb_place(b, &v, 2); + pgc_fb_prep(b, 2, 0); + pgc_fb_place(b, &v, 2); } void -fb_push_i32(FBB *b, int32 v) +pgc_fb_push_i32(FBB *b, int32 v) { - fb_prep(b, 4, 0); - fb_place(b, &v, 4); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &v, 4); } void -fb_push_i64(FBB *b, int64 v) +pgc_fb_push_i64(FBB *b, int64 v) { - fb_prep(b, 8, 0); - fb_place(b, &v, 8); + pgc_fb_prep(b, 8, 0); + pgc_fb_place(b, &v, 8); } /* prepend a uoffset that references object at `off` (bytes-from-end) */ void -fb_push_uoffset(FBB *b, uint32 off) +pgc_fb_push_uoffset(FBB *b, uint32 off) { uint32 v; - fb_prep(b, 4, 0); - v = (fb_offset(b) + 4) - off; - fb_place(b, &v, 4); + pgc_fb_prep(b, 4, 0); + v = (pgc_fb_offset(b) + 4) - off; + pgc_fb_place(b, &v, 4); } /* ---- vectors ---- */ void -fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align) +pgc_fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align) { - fb_prep(b, 4, elemSize * count); /* length prefix */ - fb_prep(b, align, elemSize * count); /* element alignment */ + pgc_fb_prep(b, 4, elemSize * count); /* length prefix */ + pgc_fb_prep(b, align, elemSize * count); /* element alignment */ } uint32 -fb_end_vector(FBB *b, uint32 count) +pgc_fb_end_vector(FBB *b, uint32 count) { - fb_prep(b, 4, 0); - fb_place(b, &count, 4); /* length prefix precedes the elements */ - return fb_offset(b); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &count, 4); /* length prefix precedes the elements */ + return pgc_fb_offset(b); } /* ---- tables ---- */ void -fb_start(FBB *b, int nslots) +pgc_fb_start(FBB *b, int nslots) { int i; @@ -153,66 +153,66 @@ fb_start(FBB *b, int nslots) b->nslots = nslots; for (i = 0; i < nslots; i++) b->vslot[i] = 0; - b->objectEnd = fb_offset(b); + b->objectEnd = pgc_fb_offset(b); } void -fb_slot(FBB *b, int i) +pgc_fb_slot(FBB *b, int i) { - b->vslot[i] = fb_offset(b); + b->vslot[i] = pgc_fb_offset(b); } void -fb_add_i16(FBB *b, int i, int16 val, int16 def) +pgc_fb_add_i16(FBB *b, int i, int16 val, int16 def) { if (val == def) return; - fb_push_i16(b, val); - fb_slot(b, i); + pgc_fb_push_i16(b, val); + pgc_fb_slot(b, i); } void -fb_add_i32(FBB *b, int i, int32 val, int32 def) +pgc_fb_add_i32(FBB *b, int i, int32 val, int32 def) { if (val == def) return; - fb_push_i32(b, val); - fb_slot(b, i); + pgc_fb_push_i32(b, val); + pgc_fb_slot(b, i); } void -fb_add_i64(FBB *b, int i, int64 val, int64 def) +pgc_fb_add_i64(FBB *b, int i, int64 val, int64 def) { if (val == def) return; - fb_push_i64(b, val); - fb_slot(b, i); + pgc_fb_push_i64(b, val); + pgc_fb_slot(b, i); } void -fb_add_bool(FBB *b, int i, bool val, bool def) +pgc_fb_add_bool(FBB *b, int i, bool val, bool def) { if (val == def) return; - fb_push_u8(b, val ? 1 : 0); - fb_slot(b, i); + pgc_fb_push_u8(b, val ? 1 : 0); + pgc_fb_slot(b, i); } void -fb_add_u8(FBB *b, int i, uint8 val, uint8 def) +pgc_fb_add_u8(FBB *b, int i, uint8 val, uint8 def) { if (val == def) return; - fb_push_u8(b, val); - fb_slot(b, i); + pgc_fb_push_u8(b, val); + pgc_fb_slot(b, i); } void -fb_add_offset(FBB *b, int i, uint32 off) +pgc_fb_add_offset(FBB *b, int i, uint32 off) { if (off == 0) return; - fb_push_uoffset(b, off); - fb_slot(b, i); + pgc_fb_push_uoffset(b, off); + pgc_fb_slot(b, i); } uint32 -fb_end(FBB *b) +pgc_fb_end(FBB *b) { uint32 objectOffset; uint32 vtOffset; @@ -223,50 +223,50 @@ fb_end(FBB *b) int32 zero = 0; /* soffset placeholder = table location */ - fb_prep(b, 4, 0); - fb_place(b, &zero, 4); - objectOffset = fb_offset(b); + pgc_fb_prep(b, 4, 0); + pgc_fb_place(b, &zero, 4); + objectOffset = pgc_fb_offset(b); /* vtable: field voffsets (high slot first), then objsize, then vtsize */ for (i = b->nslots - 1; i >= 0; i--) { int16 voff = b->vslot[i] ? (int16) (objectOffset - b->vslot[i]) : 0; - fb_place(b, &voff, 2); + pgc_fb_place(b, &voff, 2); } objsize = (int16) (objectOffset - b->objectEnd); - fb_place(b, &objsize, 2); + pgc_fb_place(b, &objsize, 2); vtsize = (int16) ((b->nslots + 2) * 2); - fb_place(b, &vtsize, 2); + pgc_fb_place(b, &vtsize, 2); - vtOffset = fb_offset(b); + vtOffset = pgc_fb_offset(b); soff = (int32) (vtOffset - objectOffset); memcpy(b->buf + b->cap - objectOffset, &soff, 4); return objectOffset; } void -fb_finish(FBB *b, uint32 root) +pgc_fb_finish(FBB *b, uint32 root) { - fb_prep(b, b->minalign, 4); - fb_push_uoffset(b, root); + pgc_fb_prep(b, b->minalign, 4); + pgc_fb_push_uoffset(b, root); } uint32 -fb_create_string(FBB *b, const char *s) +pgc_fb_create_string(FBB *b, const char *s) { uint32 n = (uint32) strlen(s); uint8 zero = 0; - fb_prep(b, 4, n + 1); - fb_place(b, &zero, 1); /* null terminator */ - fb_place(b, s, n); /* characters (s[0] ends lowest) */ + pgc_fb_prep(b, 4, n + 1); + pgc_fb_place(b, &zero, 1); /* null terminator */ + pgc_fb_place(b, s, n); /* characters (s[0] ends lowest) */ { uint32 len = n; - fb_place(b, &len, 4); /* length prefix (already aligned) */ + pgc_fb_place(b, &len, 4); /* length prefix (already aligned) */ } - return fb_offset(b); + return pgc_fb_offset(b); } /* ---- bounds-checked little-endian FlatBuffers reader ---- */ diff --git a/src/columnar_flatbuffers.h b/src/columnar_flatbuffers.h index 39577ee..13fbe33 100644 --- a/src/columnar_flatbuffers.h +++ b/src/columnar_flatbuffers.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_flatbuffers.h + * pgcolumnar_flatbuffers.h * A minimal FlatBuffers builder. * * FlatBuffers is the serialization format Arrow uses for its IPC metadata, and @@ -14,13 +14,13 @@ * captured right after the object is written. * * The short fb_ names are this module's namespace and are kept deliberately, - * unlike the ColumnarThrift* naming next door: the call sites are dense - * (fb_add_i32(b, 0, bits, 0) appears in runs of a dozen), and lengthening them + * unlike the PgColumnarThrift* naming next door: the call sites are dense + * (pgc_fb_add_i32(b, 0, bits, 0) appears in runs of a dozen), and lengthening them * would cost more in readability there than the consistency buys. * * What stays with Arrow, and why: * - * fb_arrow_type maps a column type onto Arrow's Type union. That is Arrow + * pgc_fb_arrow_type maps a column type onto Arrow's Type union. That is Arrow * semantics expressed through this builder, not part of it. * fbr_* the read-side scalar accessors raise Arrow's own * "malformed Arrow IPC file" on a short buffer. Moving them @@ -51,29 +51,29 @@ typedef struct FBB uint32 vslot[16]; } FBB; -extern void fb_init(FBB *b); -extern void fb_grow(FBB *b, uint32 need); -extern void fb_place(FBB *b, const void *src, uint32 n); -extern void fb_pad(FBB *b, uint32 n); -extern uint32 fb_offset(FBB *b); -extern void fb_prep(FBB *b, uint32 size, uint32 additional); -extern void fb_push_u8(FBB *b, uint8 v); -extern void fb_push_i16(FBB *b, int16 v); -extern void fb_push_i32(FBB *b, int32 v); -extern void fb_push_i64(FBB *b, int64 v); -extern void fb_push_uoffset(FBB *b, uint32 off); -extern void fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align); -extern uint32 fb_end_vector(FBB *b, uint32 count); -extern void fb_start(FBB *b, int nslots); -extern void fb_slot(FBB *b, int i); -extern void fb_add_i16(FBB *b, int i, int16 val, int16 def); -extern void fb_add_i32(FBB *b, int i, int32 val, int32 def); -extern void fb_add_i64(FBB *b, int i, int64 val, int64 def); -extern void fb_add_bool(FBB *b, int i, bool val, bool def); -extern void fb_add_u8(FBB *b, int i, uint8 val, uint8 def); -extern void fb_add_offset(FBB *b, int i, uint32 off); -extern uint32 fb_end(FBB *b); -extern void fb_finish(FBB *b, uint32 root); -extern uint32 fb_create_string(FBB *b, const char *s); +extern void pgc_fb_init(FBB *b); +extern void pgc_fb_grow(FBB *b, uint32 need); +extern void pgc_fb_place(FBB *b, const void *src, uint32 n); +extern void pgc_fb_pad(FBB *b, uint32 n); +extern uint32 pgc_fb_offset(FBB *b); +extern void pgc_fb_prep(FBB *b, uint32 size, uint32 additional); +extern void pgc_fb_push_u8(FBB *b, uint8 v); +extern void pgc_fb_push_i16(FBB *b, int16 v); +extern void pgc_fb_push_i32(FBB *b, int32 v); +extern void pgc_fb_push_i64(FBB *b, int64 v); +extern void pgc_fb_push_uoffset(FBB *b, uint32 off); +extern void pgc_fb_start_vector(FBB *b, uint32 elemSize, uint32 count, uint32 align); +extern uint32 pgc_fb_end_vector(FBB *b, uint32 count); +extern void pgc_fb_start(FBB *b, int nslots); +extern void pgc_fb_slot(FBB *b, int i); +extern void pgc_fb_add_i16(FBB *b, int i, int16 val, int16 def); +extern void pgc_fb_add_i32(FBB *b, int i, int32 val, int32 def); +extern void pgc_fb_add_i64(FBB *b, int i, int64 val, int64 def); +extern void pgc_fb_add_bool(FBB *b, int i, bool val, bool def); +extern void pgc_fb_add_u8(FBB *b, int i, uint8 val, uint8 def); +extern void pgc_fb_add_offset(FBB *b, int i, uint32 off); +extern uint32 pgc_fb_end(FBB *b); +extern void pgc_fb_finish(FBB *b, uint32 root); +extern uint32 pgc_fb_create_string(FBB *b, const char *s); #endif /* PGCOLUMNAR_FLATBUFFERS_H */ diff --git a/src/columnar_index.c b/src/columnar_index.c index 7f95db6..e46d18f 100644 --- a/src/columnar_index.c +++ b/src/columnar_index.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_index.c + * pgcolumnar_index.c * Index maintenance for callers that insert rows without an executor. * * PostgreSQL puts index maintenance in the executor, not in the table access @@ -10,7 +10,7 @@ * the indexes silently stop describing the table and an index scan returns rows * that are not there and misses rows that are. * - * Two callers are in that position: the online rewrite in columnar_vacuum.c, + * Two callers are in that position: the online rewrite in pgcolumnar_vacuum.c, * which moves live rows into fresh groups, and the Arrow and Parquet importers, * which load rows from a file. The importers did not maintain indexes at all * (issue #153): an index scan over an imported table returned nothing, and a @@ -72,7 +72,7 @@ #include "nodes/parsenodes.h" /* - * ColumnarIndexInsertBegin + * PgColumnarIndexInsertBegin * Prepare to maintain the relation's indexes for a run of inserted rows. * * enforceConstraints selects the route: an importer gets the executor's index @@ -81,10 +81,10 @@ * insertion, and -- when enforcing -- must keep holding it until commit, since * a deferred constraint queued here is fired after the caller has returned. */ -ColumnarIndexInsertState * -ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) +PgColumnarIndexInsertState * +PgColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) { - ColumnarIndexInsertState *st = palloc0(sizeof(ColumnarIndexInsertState)); + PgColumnarIndexInsertState *st = palloc0(sizeof(PgColumnarIndexInsertState)); List *oids; int cap; ListCell *lc; @@ -154,7 +154,7 @@ ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) * double insert. * * Not reachable through CREATE INDEX CONCURRENTLY today, because - * columnar_index_validate_scan is unsupported and the build fails + * pgcolumnar_index_validate_scan is unsupported and the build fails * before the index becomes valid. It is reachable as debris: a failed * concurrent build leaves the index ready but invalid until someone * drops or reindexes it, and in that state an ordinary INSERT @@ -178,12 +178,12 @@ ColumnarIndexInsertBegin(Relation rel, bool enforceConstraints) } /* - * ColumnarIndexInsertRow + * PgColumnarIndexInsertRow * Put one row into every open index, under the row number's synthetic item * pointer. */ void -ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, +PgColumnarIndexInsertRow(PgColumnarIndexInsertState *st, Relation rel, Datum *values, bool *isnull, uint64 rowNumber) { int natts = RelationGetDescr(rel)->natts; @@ -194,7 +194,7 @@ ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, if (!st->enforcing && st->n == 0) return; - ColumnarRowNumberToItemPointer(rowNumber, &tid); + PgColumnarRowNumberToItemPointer(rowNumber, &tid); ExecClearTuple(st->slot); memcpy(st->slot->tts_values, values, natts * sizeof(Datum)); @@ -246,12 +246,12 @@ ColumnarIndexInsertRow(ColumnarIndexInsertState *st, Relation rel, } /* - * ColumnarIndexInsertEnd + * PgColumnarIndexInsertEnd * Close the indexes and free the state. The locks are held until the end of * the transaction, as index_close with RowExclusiveLock leaves them. */ void -ColumnarIndexInsertEnd(ColumnarIndexInsertState *st) +PgColumnarIndexInsertEnd(PgColumnarIndexInsertState *st) { int i; @@ -279,12 +279,12 @@ ColumnarIndexInsertEnd(ColumnarIndexInsertState *st) } /* - * ColumnarRelationHasIndexes + * PgColumnarRelationHasIndexes * Does this relation have any index at all? Used by the importers to skip * the machinery entirely on the common bulk-load-into-a-bare-table case. */ bool -ColumnarRelationHasIndexes(Relation rel) +PgColumnarRelationHasIndexes(Relation rel) { return RelationGetIndexList(rel) != NIL; } diff --git a/src/columnar_metadata.c b/src/columnar_metadata.c index f23c73a..84f22c0 100644 --- a/src/columnar_metadata.c +++ b/src/columnar_metadata.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_metadata.c + * pgcolumnar_metadata.c * Access to the "columnar" metadata catalog tables and the storage-id * sequence (spec 7). Most metadata are ordinary heap tables keyed by * storage id (the options and projection_declaration tables are keyed by @@ -118,17 +118,17 @@ #define Anum_delete_vector_deleted_count 4 #define Natts_delete_vector 4 -static Oid columnar_schema_oid(void); +static Oid pgcolumnar_schema_oid(void); static Relation open_columnar_table(const char *name, LOCKMODE lockmode); -static Oid columnar_index_oid(const char *name); +static Oid pgcolumnar_index_oid(const char *name); /* - * columnar_schema_oid + * pgcolumnar_schema_oid * OID of the "columnar" schema. It always exists once the extension is * installed. */ static Oid -columnar_schema_oid(void) +pgcolumnar_schema_oid(void) { return get_namespace_oid(COLUMNAR_SCHEMA_NAME, false); } @@ -136,7 +136,7 @@ columnar_schema_oid(void) static Relation open_columnar_table(const char *name, LOCKMODE lockmode) { - Oid nspOid = columnar_schema_oid(); + Oid nspOid = pgcolumnar_schema_oid(); Oid relOid = get_relname_relid(name, nspOid); if (!OidIsValid(relOid)) @@ -149,26 +149,26 @@ open_columnar_table(const char *name, LOCKMODE lockmode) } /* - * columnar_index_oid + * pgcolumnar_index_oid * The OID of one of the metadata indexes, by name. Returns InvalidOid when * it cannot be resolved, which callers pass through to systable_beginscan * as indexOK = false so the lookup degrades to a heap scan rather than * failing. */ static Oid -columnar_index_oid(const char *name) +pgcolumnar_index_oid(const char *name) { - return get_relname_relid(name, columnar_schema_oid()); + return get_relname_relid(name, pgcolumnar_schema_oid()); } /* - * ColumnarNextStorageId + * PgColumnarNextStorageId * Draw the next value from pgcolumnar.storageid_seq (spec 3, 7.6). */ uint64 -ColumnarNextStorageId(void) +PgColumnarNextStorageId(void) { - Oid nspOid = columnar_schema_oid(); + Oid nspOid = pgcolumnar_schema_oid(); Oid seqOid = get_relname_relid("storageid_seq", nspOid); int64 value; @@ -182,7 +182,7 @@ ColumnarNextStorageId(void) } /* - * ColumnarCatalogSnapshot + * PgColumnarCatalogSnapshot * Return a snapshot for reading the columnar metadata catalog that also * sees this transaction's own writes made in the current command (spec 9). * curcid only affects visibility of the current transaction's own tuples, @@ -190,7 +190,7 @@ ColumnarNextStorageId(void) * other transactions. */ Snapshot -ColumnarCatalogSnapshot(Snapshot base) +PgColumnarCatalogSnapshot(Snapshot base) { Snapshot copy; CommandId now; @@ -223,7 +223,7 @@ ColumnarCatalogSnapshot(Snapshot base) /* - * ColumnarComputeAllVisibleGroups + * PgColumnarComputeAllVisibleGroups * Return the chunk groups that are all-visible to every snapshot (gap 28 * phase 3): the covering stripe's insert xid is frozen or precedes * `oldestXmin` (so every current/future snapshot sees the insert), and the @@ -231,10 +231,10 @@ ColumnarCatalogSnapshot(Snapshot base) * under a dirty snapshot so a group being modified concurrently is excluded; * combined with clear-on-write (which removes a bit for any later * delete/insert), this keeps a set bit from ever covering a modified row. - * Returns a List of ColumnarRowRange * (one per all-visible group). + * Returns a List of PgColumnarRowRange * (one per all-visible group). */ List * -ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) +PgColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) { Relation grel = open_columnar_table("row_group", AccessShareLock); TupleDesc gtd = RelationGetDescr(grel); @@ -272,7 +272,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) List *rmList; ListCell *lc; bool hasDelete = false; - ColumnarRowRange *r; + PgColumnarRowRange *r; if (TransactionIdIsNormal(xmin) && !TransactionIdPrecedes(xmin, oldestXmin)) @@ -287,7 +287,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) if (rowCount == 0) continue; - rmList = ColumnarReadDeleteVectorList(storageId, groupNumber, &dirty); + rmList = PgColumnarReadDeleteVectorList(storageId, groupNumber, &dirty); foreach(lc, rmList) { if (((DeleteVectorMetadata *) lfirst(lc))->deletedCount > 0) @@ -299,7 +299,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) if (hasDelete) continue; - r = palloc(sizeof(ColumnarRowRange)); + r = palloc(sizeof(PgColumnarRowRange)); r->firstRowNumber = firstRow; r->rowCount = rowCount; ranges = lappend(ranges, r); @@ -312,7 +312,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) } /* - * ColumnarComputeFullyDeletedGroups + * PgColumnarComputeFullyDeletedGroups * Return the group numbers of row groups every one of whose rows is deleted * as-of oldestXmin -- i.e. the group's catalog row and every delete on it * committed before oldestXmin, so every live snapshot agrees the group is @@ -323,7 +323,7 @@ ColumnarComputeAllVisibleGroups(uint64 storageId, TransactionId oldestXmin) * catalog version via heap MVCC. Returns a List of palloc'd uint64. */ List * -ColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin) +PgColumnarComputeFullyDeletedGroups(uint64 storageId, TransactionId oldestXmin) { Relation grel = open_columnar_table("row_group", AccessShareLock); TupleDesc gtd = RelationGetDescr(grel); @@ -461,7 +461,7 @@ read_row_group_range(uint64 storageId, uint64 groupNumber, } /* physical reclaim: split freed ranges on allocate and coalesce on free */ -bool columnar_reclaim_coalesce = true; +bool pgcolumnar_reclaim_coalesce = true; /* insert one free_space row (offset, length page-aligned, freed at freedXid) */ static void @@ -508,7 +508,7 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) uint64 origEnd = off + len; TransactionId freedXid = GetCurrentTransactionId(); - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) { Snapshot snap = RegisterSnapshot(GetLatestSnapshot()); ScanKeyData key[1]; @@ -570,13 +570,13 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) } insert_free_space_row(rel, td, storageId, off, len, freedXid); - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) CommandCounterIncrement(); table_close(rel, RowExclusiveLock); } /* - * ColumnarRetireGroup + * PgColumnarRetireGroup * Drop all catalog rows for one row group (row_group, column_chunk, * zone_map, bloom, delete_vector) in the current transaction (Phase F3a). The * storage row is left intact. Heap MVCC on these deletes keeps the group @@ -586,7 +586,7 @@ record_free_space(uint64 storageId, uint64 fileOffset, uint64 byteLength) * The caller must have verified the group is fully deleted as-of oldestXmin. */ void -ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) +PgColumnarRetireGroup(uint64 storageId, uint64 groupNumber) { uint64 fileOffset = 0; uint64 byteLength = 0; @@ -609,7 +609,7 @@ ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) } /* - * ColumnarAllocateFreeSpace + * PgColumnarAllocateFreeSpace * Try to satisfy a data reservation of dataLength bytes from a previously * freed range (Phase F physical reclaim). Returns true and sets *fileOffset * to a page-aligned freed range that is large enough AND whose freeing @@ -619,7 +619,7 @@ ColumnarRetireGroup(uint64 storageId, uint64 groupNumber) * so no two callers race for the same row. */ bool -ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, +PgColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, TransactionId oldestXmin, uint64 *fileOffset) { Relation rel = open_columnar_table("free_space", RowExclusiveLock); @@ -677,7 +677,7 @@ ColumnarAllocateFreeSpace(uint64 storageId, uint64 dataLength, * same oldest-xmin gate still applies). Without this the tail of an * oversized freed range would leak until its group is re-freed. */ - if (columnar_reclaim_coalesce) + if (pgcolumnar_reclaim_coalesce) { uint64 allocLen = COLUMNAR_PAGE_ROUND_UP(dataLength); @@ -721,7 +721,7 @@ reclaim_range_cmp(const void *a, const void *b) } /* - * ColumnarCheckFreeSpaceNoOverlap + * PgColumnarCheckFreeSpaceNoOverlap * Assert-only invariant check for physical reclaim: a storage's live * row-group footprints (data rounded up to whole pages) and its free_space * ranges must not overlap. An overlap would mean a reused block was handed @@ -729,7 +729,7 @@ reclaim_range_cmp(const void *a, const void *b) * the end of the online maintenance operations in assert builds. */ void -ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) +PgColumnarCheckFreeSpaceNoOverlap(uint64 storageId) { Relation rg; Relation fs; @@ -814,7 +814,7 @@ ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) #endif /* USE_ASSERT_CHECKING */ /* - * ColumnarTrailingFreeSpaceSafe + * PgColumnarTrailingFreeSpaceSafe * Physical end-truncation guard: return false if any free_space row for this * storage at or above liveEnd was freed at a transaction the oldest-xmin * horizon has NOT passed. Such a row is a recently retired group whose bytes @@ -824,7 +824,7 @@ ColumnarCheckFreeSpaceNoOverlap(uint64 storageId) * the tail safe to drop. */ bool -ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, +PgColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, TransactionId oldestXmin) { Relation rel = open_columnar_table("free_space", AccessShareLock); @@ -862,14 +862,14 @@ ColumnarTrailingFreeSpaceSafe(uint64 storageId, uint64 liveEnd, } /* - * ColumnarDeleteFreeSpaceAtOrAbove + * PgColumnarDeleteFreeSpaceAtOrAbove * Physical end-truncation: drop every free_space row for this storage whose * offset is at or above liveEnd. Those ranges are being physically removed * from the file, so they must no longer appear as reusable space. The caller * holds AccessExclusiveLock and has verified the tail is safe. */ void -ColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd) +PgColumnarDeleteFreeSpaceAtOrAbove(uint64 storageId, uint64 liveEnd) { Relation rel = open_columnar_table("free_space", RowExclusiveLock); TupleDesc td = RelationGetDescr(rel); @@ -930,7 +930,7 @@ static void collect_footprints(uint64 storageId, Snapshot snap, FootRange **foots, int *nf, int *capf) { - List *rgs = ColumnarReadRowGroupList(storageId, snap); + List *rgs = PgColumnarReadRowGroupList(storageId, snap); ListCell *lc; foreach(lc, rgs) @@ -973,7 +973,7 @@ range_overlaps_footprint(uint64 start, uint64 end, FootRange *foots, int nf) } /* - * ColumnarReconcileFreeList + * PgColumnarReconcileFreeList * Delete any free_space row (for the base or any projection storage of this * relation) that overlaps a LIVE row-group footprint as-of the latest * committed state. @@ -988,15 +988,15 @@ range_overlaps_footprint(uint64 start, uint64 end, FootRange *foots, int nf) * with the trailing free rows restored; a later insert then places a live * group over one of those ranges. Running this at the start of every reuse * (compact_rewrite, recluster), under ShareUpdateExclusiveLock and before any - * ColumnarAllocateFreeSpace, drops such an entry before it can be handed out + * PgColumnarAllocateFreeSpace, drops such an entry before it can be handed out * on top of a live group. Inserts never reuse, so no stale entry is consumed * between the crash and the next reuse. */ void -ColumnarReconcileFreeList(Relation dataRel) +PgColumnarReconcileFreeList(Relation dataRel) { - uint64 base = ColumnarStorageId(dataRel); - List *projs = ColumnarListProjections(base); + uint64 base = PgColumnarStorageId(dataRel); + List *projs = PgColumnarListProjections(base); ListCell *lc; Snapshot snap; FootRange *foots; @@ -1012,7 +1012,7 @@ ColumnarReconcileFreeList(Relation dataRel) /* live footprints and the storage id set (base + distinct projections). Store * storage ids as heap uint64s, not stuffed into pointers, so an id > 2^32 is - * safe on 32-bit builds (matches columnar_end_truncation_storages). */ + * safe on 32-bit builds (matches pgcolumnar_end_truncation_storages). */ collect_footprints(base, snap, &foots, &nf, &capf); { uint64 *s = palloc(sizeof(uint64)); @@ -1022,7 +1022,7 @@ ColumnarReconcileFreeList(Relation dataRel) } foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != base) { @@ -1083,23 +1083,23 @@ ColumnarReconcileFreeList(Relation dataRel) } /* - * ColumnarRetireFullyDeletedGroups + * PgColumnarRetireFullyDeletedGroups * Online compaction (Phase F3a, lazy path): retire every row group that is * fully deleted as-of oldestXmin. Safe under ShareUpdateExclusiveLock, * concurrent with readers and writers. Returns the number of groups retired. */ int64 -ColumnarRetireFullyDeletedGroups(Relation rel) +PgColumnarRetireFullyDeletedGroups(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *groups = ColumnarComputeFullyDeletedGroups(storageId, oldestXmin); + uint64 storageId = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *groups = PgColumnarComputeFullyDeletedGroups(storageId, oldestXmin); ListCell *lc; int64 retired = 0; foreach(lc, groups) { - ColumnarRetireGroup(storageId, *(uint64 *) lfirst(lc)); + PgColumnarRetireGroup(storageId, *(uint64 *) lfirst(lc)); retired++; } return retired; @@ -1108,12 +1108,12 @@ ColumnarRetireFullyDeletedGroups(Relation rel) /* - * ColumnarReadDeleteVectorList + * PgColumnarReadDeleteVectorList * Read all delete_vector rows for a stripe (spec 7.5). Returns a list of * DeleteVectorMetadata* allocated in the current memory context. */ List * -ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot) +PgColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapshot) { Relation rel = open_columnar_table("delete_vector", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1163,14 +1163,14 @@ ColumnarReadDeleteVectorList(uint64 storageId, uint64 stripeId, Snapshot snapsho } /* - * ColumnarStorageHasDeleteVector + * PgColumnarStorageHasDeleteVector * True when the storage has any delete_vector row, i.e. at least one delete has * been recorded. Used to decide whether the native zone-map-only aggregate * is valid (it is only correct when no rows are deleted) or must fall back to * a delete-applying scan (D6b). */ bool -ColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot) +PgColumnarStorageHasDeleteVector(uint64 storageId, Snapshot snapshot) { Relation rel = open_columnar_table("delete_vector", AccessShareLock); ScanKeyData key[1]; @@ -1218,7 +1218,7 @@ delete_vector_chunk_lock_key(uint64 storageId, uint64 stripeId, int chunkId) /* * delete_vector_lock_chunk_group * Take a transaction-scoped exclusive lock covering one chunk group's - * delete_vector tuple, so that the read-modify-write in ColumnarUpsertDeleteVector + * delete_vector tuple, so that the read-modify-write in PgColumnarUpsertDeleteVector * serializes against any concurrent deleter or updater touching the SAME * chunk group, while deletes to different chunk groups still proceed * concurrently. The lock is held until this transaction ends (commit or @@ -1247,11 +1247,11 @@ delete_vector_lock_chunk_group(uint64 storageId, uint64 stripeId, int chunkId) } /* - * ColumnarUpsertDeleteVector + * PgColumnarUpsertDeleteVector * Insert or replace the delete_vector row for one chunk group, identified by * (storage_id, group_number). If a row already exists it is replaced with the * merged bitmap carried in rm; otherwise a fresh row is inserted. Used at - * flush of the in-memory delete buffer (columnar_delete_vector.c), at most + * flush of the in-memory delete buffer (pgcolumnar_delete_vector.c), at most * once per chunk group per flush, so a single heap tuple is never updated * twice in the same command. * @@ -1287,20 +1287,20 @@ row_group_exists(uint64 storageId, uint64 groupNumber, Snapshot snapshot) } /* - * ColumnarLockChunkGroup + * PgColumnarLockChunkGroup * Take the transaction-scoped advisory lock for one chunk group (the whole - * row group, chunk id 0), the same lock ColumnarUpsertDeleteVector takes. Used by + * row group, chunk id 0), the same lock PgColumnarUpsertDeleteVector takes. Used by * online compaction (Phase F3b) so a rewrite of a group serializes with * concurrent deletes to that group. */ void -ColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber) +PgColumnarLockChunkGroup(uint64 storageId, uint64 groupNumber) { delete_vector_lock_chunk_group(storageId, groupNumber, 0); } void -ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) +PgColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) { Relation rel; TupleDesc tupdesc; @@ -1332,13 +1332,13 @@ ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) bool exists; /* - * Latest committed state, with curcid advanced (ColumnarCatalogSnapshot) + * Latest committed state, with curcid advanced (PgColumnarCatalogSnapshot) * so a group this same transaction just flushed (pending writes flush * before delete vectors at pre-commit) is visible and does not look retired. * A group retired by a concurrent COMMITTED rewrite is gone here. */ PushActiveSnapshot(GetLatestSnapshot()); - latest = ColumnarCatalogSnapshot(GetActiveSnapshot()); + latest = PgColumnarCatalogSnapshot(GetActiveSnapshot()); exists = row_group_exists(storageId, rm->groupNumber, latest); PopActiveSnapshot(); if (!exists) @@ -1415,7 +1415,7 @@ ColumnarUpsertDeleteVector(uint64 storageId, DeleteVectorMetadata *rm) } /* - * ColumnarDeleteMetadata + * PgColumnarDeleteMetadata * Remove every metadata row for a storage id. Used when a columnar * table is dropped or truncated. */ @@ -1440,7 +1440,7 @@ delete_rows_by_storage_id(const char *tableName, AttrNumber storageAttno, } void -ColumnarDeleteMetadata(uint64 storageId) +PgColumnarDeleteMetadata(uint64 storageId) { delete_rows_by_storage_id("delete_vector", Anum_delete_vector_storage_id, storageId); /* native format catalog (PGCN v1); no-op rows for 2.2-line tables */ @@ -1455,20 +1455,20 @@ ColumnarDeleteMetadata(uint64 storageId) /* * Opt-in, default off. Set for its own session by a pgcolumnar.parallel_copy * loader (backing the pgcolumnar.bulk_parallel_writer GUC, registered in - * _PG_init) so ColumnarInsertNativeStorageRow can skip the storage-row creation + * _PG_init) so PgColumnarInsertNativeStorageRow can skip the storage-row creation * advisory lock when the row already exists committed. Ordinary writes never set * it, so the default write path is unchanged. */ -bool columnar_bulk_parallel_writer = false; +bool pgcolumnar_bulk_parallel_writer = false; /* - * ColumnarInsertNativeStorageRow, ColumnarInsertRowGroupRow, - * ColumnarInsertColumnChunkRow + * PgColumnarInsertNativeStorageRow, PgColumnarInsertRowGroupRow, + * PgColumnarInsertColumnChunkRow * Record the native-format catalog rows (PGCN v1, native spec 11). Called * by the native writer's flush. The 2.2-line writer does not use these. */ void -ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) +PgColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) { Relation rel = open_columnar_table("storage", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1496,7 +1496,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) */ /* * Bulk-parallel fast path (opt-in, default off). A pgcolumnar.parallel_copy - * loader sets columnar_bulk_parallel_writer for its own session; when the + * loader sets pgcolumnar_bulk_parallel_writer for its own session; when the * storage row already exists in the latest committed state we skip the advisory * lock entirely and return. That lock exists ONLY to serialize the first-writer * creation race -- once the row is committed there is nothing to wait for -- and @@ -1507,10 +1507,10 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * table's storage concurrently and 2PC-safely. Every ordinary write leaves the * flag false and takes the unchanged path below. */ - if (columnar_bulk_parallel_writer) + if (pgcolumnar_bulk_parallel_writer) { PushActiveSnapshot(GetLatestSnapshot()); - snapshot = ColumnarCatalogSnapshot(GetActiveSnapshot()); + snapshot = PgColumnarCatalogSnapshot(GetActiveSnapshot()); ScanKeyInit(&key[0], Anum_native_storage_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) s->storageId)); scan = systable_beginscan(rel, InvalidOid, false, snapshot, 1, key); @@ -1533,7 +1533,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) /* * Re-check under the lock against a fresh snapshot so the loser of a * cross-transaction race sees the winner's just-committed row (GetLatestSnapshot), - * with curcid advanced (ColumnarCatalogSnapshot) so a second flush in this same + * with curcid advanced (PgColumnarCatalogSnapshot) so a second flush in this same * transaction still sees the row this transaction already inserted. The latest * snapshot must be pushed active before it drives a heap visibility check: * PostgreSQL 18 asserts a scan snapshot is registered or active @@ -1541,7 +1541,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * count. Pop it once the existence scan is done. */ PushActiveSnapshot(GetLatestSnapshot()); - snapshot = ColumnarCatalogSnapshot(GetActiveSnapshot()); + snapshot = PgColumnarCatalogSnapshot(GetActiveSnapshot()); ScanKeyInit(&key[0], Anum_native_storage_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) s->storageId)); scan = systable_beginscan(rel, InvalidOid, false, snapshot, 1, key); @@ -1562,7 +1562,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) values[Anum_native_storage_row_group_limit - 1] = Int32GetDatum(s->rowGroupLimit); /* * A new storage starts unordered. An ordering rewrite sets this afterwards - * (ColumnarSetSortedThrough); an unsorted one leaves it NULL, which is what + * (PgColumnarSetSortedThrough); an unsorted one leaves it NULL, which is what * makes a rewrite reset the sort state with no invalidation step. */ nulls[Anum_native_storage_sorted_through - 1] = true; @@ -1575,7 +1575,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) } /* - * ColumnarSetSortedThrough + * PgColumnarSetSortedThrough * Record the row group number the last ordering rewrite ended at, so a * reader can tell how much of the layout is still ordered (issue #301). * @@ -1594,7 +1594,7 @@ ColumnarInsertNativeStorageRow(const NativeStorageMetadata *s) * groups and reports no decay. */ void -ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) +PgColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) { Relation rel; TupleDesc tupdesc; @@ -1645,10 +1645,10 @@ ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) } /* - * ColumnarCheckNativeFormatVersion + * PgColumnarCheckNativeFormatVersion * Reject a native data format version this build does not understand, before * any bytes are decoded (issue #240). The physical metapage version is checked - * separately when the metapage is read (ColumnarReadMetapage); this is the + * separately when the metapage is read (PgColumnarReadMetapage); this is the * independent data-format stamp (pgcolumnar.storage.format_version), so a * future PGCN version that changes the encoding while keeping the metapage * layout is caught here rather than silently misread. @@ -1659,7 +1659,7 @@ ColumnarSetSortedExtent(uint64 storageId, int64 firstGroup, int64 lastGroup) * nothing is left open across the error. */ void -ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) +PgColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) { Relation rel = open_columnar_table("storage", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1701,7 +1701,7 @@ ColumnarCheckNativeFormatVersion(uint64 storageId, const char *relName) } void -ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) +PgColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) { Relation rel = open_columnar_table("row_group", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1726,7 +1726,7 @@ ColumnarInsertRowGroupRow(const NativeRowGroupMetadata *rg) } void -ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) +PgColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) { Relation rel = open_columnar_table("column_chunk", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1757,13 +1757,13 @@ ColumnarInsertColumnChunkRow(const NativeColumnChunkMetadata *cc) } /* - * ColumnarInsertZoneMapRow + * PgColumnarInsertZoneMapRow * Record one native zone-map row (Small Materialized Aggregate) for a vector * or for a whole column chunk (native spec 7.1, Phase D5). Called by the * native writer's flush. */ void -ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) +PgColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) { Relation rel = open_columnar_table("zone_map", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1813,11 +1813,11 @@ ColumnarInsertZoneMapRow(const NativeZoneMapMetadata *z) } /* - * ColumnarInsertBloomRow + * PgColumnarInsertBloomRow * Record one per-column-chunk bloom filter (native spec 7.2, Phase D5b). */ void -ColumnarInsertBloomRow(const NativeBloomMetadata *b) +PgColumnarInsertBloomRow(const NativeBloomMetadata *b) { Relation rel = open_columnar_table("bloom", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1844,13 +1844,13 @@ ColumnarInsertBloomRow(const NativeBloomMetadata *b) } /* - * ColumnarReadBloomList + * PgColumnarReadBloomList * The per-column-chunk bloom filters of one row group (native spec 7.2, * Phase D5b). The caller indexes the result by column_index; the filter * bytes are copied into the current memory context. */ List * -ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("bloom", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1864,7 +1864,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_bloom_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("bloom_pkey"); + idxOid = pgcolumnar_index_oid("bloom_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -1895,7 +1895,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) } /* - * ColumnarReadBloomForColumn + * PgColumnarReadBloomForColumn * One column's bloom filter for one row group, or NULL when it has none * (issue #314). * @@ -1907,7 +1907,7 @@ ColumnarReadBloomList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) * column reads most of what it fetches for nothing. */ NativeBloomMetadata * -ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, +PgColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, int columnIndex, Snapshot snapshot) { Relation rel = open_columnar_table("bloom", AccessShareLock); @@ -1924,7 +1924,7 @@ ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); ScanKeyInit(&key[2], Anum_bloom_column_index, BTEqualStrategyNumber, F_INT2EQ, Int16GetDatum((int16) columnIndex)); - idxOid = columnar_index_oid("bloom_pkey"); + idxOid = pgcolumnar_index_oid("bloom_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 3, key); tuple = systable_getnext(scan); @@ -1954,14 +1954,14 @@ ColumnarReadBloomForColumn(uint64 storageId, uint64 groupNumber, } /* - * ColumnarReadZoneMapVectors + * PgColumnarReadZoneMapVectors * The per-vector zone maps (vector_index >= 0) of one row group, for * per-vector skipping (native spec 7.1, Phase D5b). Only min/max and the * vector/column indices are needed by the caller. The min/max bytes are * copied into the current memory context. */ List * -ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("zone_map", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1975,7 +1975,7 @@ ColumnarReadZoneMapVectors(uint64 storageId, uint64 groupNumber, Snapshot snapsh F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_zone_map_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("zone_map_pkey"); + idxOid = pgcolumnar_index_oid("zone_map_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2044,11 +2044,11 @@ row_group_cmp(const ListCell *a, const ListCell *b) } /* - * ColumnarReadRowGroupList + * PgColumnarReadRowGroupList * The native row groups of a storage, ordered by group number. */ List * -ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) +PgColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) { Relation rel = open_columnar_table("row_group", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2056,7 +2056,7 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) SysScanDesc scan; HeapTuple tuple; List *result = NIL; - Oid rgIdx = columnar_index_oid("row_group_pkey"); + Oid rgIdx = pgcolumnar_index_oid("row_group_pkey"); ScanKeyInit(&key[0], Anum_row_group_storage_id, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) storageId)); @@ -2065,12 +2065,12 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) * rather than a heap scan of every storage's row groups in the database. * * This read is on the unique-check path: _bt_check_unique() reaches it - * through columnar_index_fetch_tuple() under its own on-stack SnapshotDirty + * through pgcolumnar_index_fetch_tuple() under its own on-stack SnapshotDirty * and reads xmin/xmax back out afterwards. A bare index scan here once * spun the check forever (test/unique_conc.sh scenario 7): when this storage * has nothing flushed the scan matches no rows, HeapTupleSatisfiesDirty() * never runs, and the uninitialised out-fields are read as a phantom xact to - * wait on. columnar_index_fetch_tuple() now resets those fields before the + * wait on. pgcolumnar_index_fetch_tuple() now resets those fields before the * fetch, the way HeapTupleSatisfiesDirty() does for a heap row, so the index * scan is safe: an in-progress group still sets xmin here and the check waits * on the real inserter. @@ -2102,13 +2102,13 @@ ColumnarReadRowGroupList(uint64 storageId, Snapshot snapshot) } /* - * ColumnarReadColumnChunkList + * PgColumnarReadColumnChunkList * The native column chunks of one row group. The caller indexes the result * by column_index; the encoding descriptor bytes are copied into the * current memory context. */ List * -ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("column_chunk", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2122,7 +2122,7 @@ ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snaps F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_column_chunk_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("column_chunk_pkey"); + idxOid = pgcolumnar_index_oid("column_chunk_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2162,14 +2162,14 @@ ColumnarReadColumnChunkList(uint64 storageId, uint64 groupNumber, Snapshot snaps } /* - * ColumnarReadZoneMapList + * PgColumnarReadZoneMapList * The whole-chunk zone maps (vector_index -1) of one row group, for group * skipping (native spec 7.1, Phase D5b). The caller indexes the result by * column_index; the minimum/maximum bytes are copied into the current memory * context. Per-vector rows (vector_index >= 0) are skipped by this reader. */ List * -ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) +PgColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) { Relation rel = open_columnar_table("zone_map", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2183,7 +2183,7 @@ ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) F_INT8EQ, Int64GetDatum((int64) storageId)); ScanKeyInit(&key[1], Anum_zone_map_group_number, BTEqualStrategyNumber, F_INT8EQ, Int64GetDatum((int64) groupNumber)); - idxOid = columnar_index_oid("zone_map_pkey"); + idxOid = pgcolumnar_index_oid("zone_map_pkey"); scan = systable_beginscan(rel, idxOid, OidIsValid(idxOid), snapshot, 2, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) @@ -2250,12 +2250,12 @@ ColumnarReadZoneMapList(uint64 storageId, uint64 groupNumber, Snapshot snapshot) * ------------------------------------------------------------------------- */ /* - * columnar_compression_from_name + * pgcolumnar_compression_from_name * Map a compression codec name to its code (spec 5). Returns -1 for an * unrecognized name so the caller can fall back to the instance default. */ static int -columnar_compression_from_name(const char *name) +pgcolumnar_compression_from_name(const char *name) { if (strcmp(name, "none") == 0) return COLUMNAR_COMPRESSION_NONE; @@ -2269,7 +2269,7 @@ columnar_compression_from_name(const char *name) } /* - * ColumnarReadOptions + * PgColumnarReadOptions * Load the per-table options row for a relation (spec 7.4) into *opts, * setting a per-field "set" flag for each column that is present (not * SQL NULL). Returns true when a row exists. The catalog is read with a @@ -2277,7 +2277,7 @@ columnar_compression_from_name(const char *name) * take effect for subsequent writes (spec 9). */ bool -ColumnarReadOptions(Oid relid, ColumnarOptions *opts) +PgColumnarReadOptions(Oid relid, PgColumnarOptions *opts) { Relation rel = open_columnar_table("options", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2288,10 +2288,10 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) Snapshot snapshot; bool found = false; - memset(opts, 0, sizeof(ColumnarOptions)); + memset(opts, 0, sizeof(PgColumnarOptions)); base = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - snapshot = ColumnarCatalogSnapshot(base); + snapshot = PgColumnarCatalogSnapshot(base); ScanKeyInit(&key[0], Anum_options_regclass, BTEqualStrategyNumber, F_OIDEQ, ObjectIdGetDatum(relid)); @@ -2328,7 +2328,7 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) d = heap_getattr(tuple, Anum_options_compression, tupdesc, &isnull); if (!isnull) { - int code = columnar_compression_from_name(NameStr(*DatumGetName(d))); + int code = pgcolumnar_compression_from_name(NameStr(*DatumGetName(d))); if (code >= 0) { @@ -2369,18 +2369,18 @@ ColumnarReadOptions(Oid relid, ColumnarOptions *opts) /* - * ColumnarReadSortBy + * PgColumnarReadSortBy * Load the declared sort_by column names for a relation (#288) from * pgcolumnar.options. Returns a List of pstrdup'd column-name strings in * the caller's memory context, or NIL when no sort key is declared (no * options row, or sort_by is SQL NULL, or the array is empty). Stored as * column NAMES, not attnums, so it survives dump/restore; the caller * resolves the names to attnums and validates them each apply. Read with - * the same command-id-advanced snapshot as ColumnarReadOptions so a + * the same command-id-advanced snapshot as PgColumnarReadOptions so a * sort_by set earlier in this transaction is visible. */ List * -ColumnarReadSortBy(Oid relid) +PgColumnarReadSortBy(Oid relid) { Relation rel = open_columnar_table("options", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2392,7 +2392,7 @@ ColumnarReadSortBy(Oid relid) List *names = NIL; base = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - snapshot = ColumnarCatalogSnapshot(base); + snapshot = PgColumnarCatalogSnapshot(base); ScanKeyInit(&key[0], Anum_options_regclass, BTEqualStrategyNumber, F_OIDEQ, ObjectIdGetDatum(relid)); @@ -2431,13 +2431,13 @@ ColumnarReadSortBy(Oid relid) /* - * ColumnarDeleteOptions + * PgColumnarDeleteOptions * Remove a relation's per-table options row, called when the table is * dropped. The options table is keyed by regclass (relation oid), not by - * storage id, so it is cleaned up separately from ColumnarDeleteMetadata. + * storage id, so it is cleaned up separately from PgColumnarDeleteMetadata. */ void -ColumnarDeleteOptions(Oid relid) +PgColumnarDeleteOptions(Oid relid) { Relation rel = open_columnar_table("options", RowExclusiveLock); ScanKeyData key[1]; @@ -2456,12 +2456,12 @@ ColumnarDeleteOptions(Oid relid) } /* - * ColumnarIsColumnarRelation + * PgColumnarIsColumnarRelation * Whether a relation uses the columnar table access method. The access * method oid is resolved once and cached. */ bool -ColumnarIsColumnarRelation(Oid relid) +PgColumnarIsColumnarRelation(Oid relid) { static Oid columnarAmOid = InvalidOid; @@ -2519,8 +2519,8 @@ int16_array_from_datum(Datum d, int *len) static int projection_cmp(const ListCell *a, const ListCell *b) { - const ColumnarProjection *pa = (const ColumnarProjection *) lfirst(a); - const ColumnarProjection *pb = (const ColumnarProjection *) lfirst(b); + const PgColumnarProjection *pa = (const PgColumnarProjection *) lfirst(a); + const PgColumnarProjection *pb = (const PgColumnarProjection *) lfirst(b); if (pa->projectionId < pb->projectionId) return -1; @@ -2530,7 +2530,7 @@ projection_cmp(const ListCell *a, const ListCell *b) } void -ColumnarInsertProjectionRow(const ColumnarProjection *proj) +PgColumnarInsertProjectionRow(const PgColumnarProjection *proj) { Relation rel = open_columnar_table("projection", RowExclusiveLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2559,7 +2559,7 @@ ColumnarInsertProjectionRow(const ColumnarProjection *proj) } /* - * ColumnarRecordProjectionDeclaration + * PgColumnarRecordProjectionDeclaration * Record the intent behind a projection: which relation, which name, and * which columns by NAME rather than by attnum (#266). * @@ -2573,7 +2573,7 @@ ColumnarInsertProjectionRow(const ColumnarProjection *proj) * not leave two declarations behind. */ void -ColumnarRecordProjectionDeclaration(Oid relid, const char *name, +PgColumnarRecordProjectionDeclaration(Oid relid, const char *name, ArrayType *columns, ArrayType *sortKey) { Relation rel; @@ -2582,7 +2582,7 @@ ColumnarRecordProjectionDeclaration(Oid relid, const char *name, bool nulls[Natts_projection_declaration]; HeapTuple tuple; - ColumnarDeleteProjectionDeclaration(relid, name); + PgColumnarDeleteProjectionDeclaration(relid, name); rel = open_columnar_table("projection_declaration", RowExclusiveLock); tupdesc = RelationGetDescr(rel); @@ -2603,12 +2603,12 @@ ColumnarRecordProjectionDeclaration(Oid relid, const char *name, } /* - * ColumnarDeleteProjectionDeclaration + * PgColumnarDeleteProjectionDeclaration * Forget the declaration for one projection. Called when it is dropped, and * before recording a replacement. */ void -ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) +PgColumnarDeleteProjectionDeclaration(Oid relid, const char *name) { Relation rel = open_columnar_table("projection_declaration", RowExclusiveLock); @@ -2637,7 +2637,7 @@ ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) } /* - * ColumnarDeleteProjectionDeclarationsForRel + * PgColumnarDeleteProjectionDeclarationsForRel * Forget every declaration for a relation. Called when the relation is * dropped (#304). * @@ -2649,7 +2649,7 @@ ColumnarDeleteProjectionDeclaration(Oid relid, const char *name) * the same hook; this catalog needs the same treatment. */ void -ColumnarDeleteProjectionDeclarationsForRel(Oid relid) +PgColumnarDeleteProjectionDeclarationsForRel(Oid relid) { Relation rel = open_columnar_table("projection_declaration", RowExclusiveLock); @@ -2670,7 +2670,7 @@ ColumnarDeleteProjectionDeclarationsForRel(Oid relid) } List * -ColumnarListProjections(uint64 storageId) +PgColumnarListProjections(uint64 storageId) { Relation rel = open_columnar_table("projection", AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); @@ -2687,7 +2687,7 @@ ColumnarListProjections(uint64 storageId) scan = systable_beginscan(rel, InvalidOid, false, NULL, 1, key); while (HeapTupleIsValid(tuple = systable_getnext(scan))) { - ColumnarProjection *p = palloc0(sizeof(ColumnarProjection)); + PgColumnarProjection *p = palloc0(sizeof(PgColumnarProjection)); bool isnull; Datum d; @@ -2713,7 +2713,7 @@ ColumnarListProjections(uint64 storageId) } void -ColumnarDeleteProjectionRow(uint64 storageId, int projectionId) +PgColumnarDeleteProjectionRow(uint64 storageId, int projectionId) { Relation rel = open_columnar_table("projection", RowExclusiveLock); ScanKeyData key[2]; diff --git a/src/columnar_parallel_copy.c b/src/columnar_parallel_copy.c index 2cf794f..e8bd676 100644 --- a/src/columnar_parallel_copy.c +++ b/src/columnar_parallel_copy.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parallel_copy.c + * pgcolumnar_parallel_copy.c * Parallel bulk ingest for pgColumnar (#300). * * Splits a load across background workers, each running core COPY over a @@ -22,8 +22,8 @@ * - the single (non-partitioned) columnar table shape: N loaders write ONE shared * storage concurrently. Here parallelism does not come from distinct storage; * it comes from the coordinator pre-creating and committing the storage row so - * the loaders skip its creation lock, each writing via columnar_bulk_parallel_writer. - * - a standalone byte splitter (columnar_file_split_offsets) exposed to SQL: N+1 + * the loaders skip its creation lock, each writing via pgcolumnar_bulk_parallel_writer. + * - a standalone byte splitter (pgcolumnar_file_split_offsets) exposed to SQL: N+1 * line-aligned offsets, a diagnostic the parallel load itself no longer calls. * Text format only for now, numeric/date-time partition keys only (their text form * is escape-free); CSV and other key types are later phases (see the plan). @@ -223,10 +223,10 @@ pcopy_naive_offsets(const char *path, int workers) return offs; } -PG_FUNCTION_INFO_V1(columnar_file_split_offsets); +PG_FUNCTION_INFO_V1(pgcolumnar_file_split_offsets); /* - * columnar_file_split_offsets(path text, workers int) -> bigint[] + * pgcolumnar_file_split_offsets(path text, workers int) -> bigint[] * * Returns workers+1 ascending byte offsets [0 .. filesize] that split the file * into `workers` line-aligned ranges. off[0] is always 0 and off[workers] is @@ -241,7 +241,7 @@ PG_FUNCTION_INFO_V1(columnar_file_split_offsets); * PCOPY_MAX_WORKERS. */ Datum -columnar_file_split_offsets(PG_FUNCTION_ARGS) +pgcolumnar_file_split_offsets(PG_FUNCTION_ARGS) { char *path; int32 workers; @@ -363,7 +363,7 @@ typedef struct PcopyHeader int nworkers; bool single_table; /* target is one columnar table (not partitioned): * coordinator pre-creates the storage row and loaders - * set columnar_bulk_parallel_writer */ + * set pgcolumnar_bulk_parallel_writer */ char filename[MAXPGPATH]; /* coordinator -> function result channel */ pg_atomic_uint32 coord_state; /* PcopyCoordState */ @@ -777,7 +777,7 @@ pgcolumnar_parallel_copy_worker(Datum main_arg) * never contends, so the flag stays off there. */ if (hdr->single_table) - columnar_bulk_parallel_writer = true; + pgcolumnar_bulk_parallel_writer = true; PG_TRY(); { @@ -1017,7 +1017,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) * Single-table load: pre-create and COMMIT the storage catalog row before any * loader starts, in the coordinator's own top-level session (the SQL function * cannot commit). With the row committed, each loader -- which sets - * columnar_bulk_parallel_writer -- sees it and skips the storage-row creation + * pgcolumnar_bulk_parallel_writer -- sees it and skips the storage-row creation * lock, so N loaders write the one storage concurrently and 2PC-safely. The * coordinator itself leaves the flag off, so this uses the normal create path. */ @@ -1028,7 +1028,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) StartTransactionCommand(); /* * StartTransactionCommand does not push an active snapshot, but - * ColumnarEnsureStorageRow reads pgcolumnar.options/storage via + * PgColumnarEnsureStorageRow reads pgcolumnar.options/storage via * systable scans, and those visibility checks require a registered or * active snapshot. A normal backend has one from the executor; this * bgworker does not, so push one explicitly. Without it the scan runs @@ -1038,7 +1038,7 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) */ PushActiveSnapshot(GetTransactionSnapshot()); rel = table_open(hdr->relid, RowExclusiveLock); - ColumnarEnsureStorageRow(rel); + PgColumnarEnsureStorageRow(rel); table_close(rel, NoLock); PopActiveSnapshot(); CommitTransactionCommand(); @@ -1213,17 +1213,17 @@ pgcolumnar_parallel_copy_coordinator(Datum main_arg) proc_exit(0); } -PG_FUNCTION_INFO_V1(columnar_parallel_copy); +PG_FUNCTION_INFO_V1(pgcolumnar_parallel_copy); /* - * columnar_parallel_copy(target regclass, filename text, workers int) + * pgcolumnar_parallel_copy(target regclass, filename text, workers int) * -> rows loaded. * * Atomic bulk load: launches the coordinator bgworker (which spawns the loaders, * 2-phase-commits them all or rolls them all back) and returns its total. */ Datum -columnar_parallel_copy(PG_FUNCTION_ARGS) +pgcolumnar_parallel_copy(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1272,7 +1272,7 @@ columnar_parallel_copy(PG_FUNCTION_ARGS) * partition-aligned byte ranges (this may lower `workers` to the partition * count), which requires the file sorted by the partition key. * - a single columnar table: the loaders write the one storage concurrently - * via columnar_bulk_parallel_writer (see below), so a naive record-aligned + * via pgcolumnar_bulk_parallel_writer (see below), so a naive record-aligned * byte split is enough and the file needs no ordering. * Any other target (e.g. a heap, or a partitioned table with non-columnar * partitions) is rejected. A naive split of one non-partitioned columnar table @@ -1308,13 +1308,13 @@ columnar_parallel_copy(PG_FUNCTION_ARGS) offs = pcopy_partition_aligned_offsets(target, path, &workers); single_table = false; } - else if (ColumnarIsColumnarRelation(relid)) + else if (PgColumnarIsColumnarRelation(relid)) { /* * A single columnar table: workers write the ONE storage concurrently * (distinct stripe/row-number reservations; the coordinator pre-creates * the storage row and the loaders skip its creation lock via - * columnar_bulk_parallel_writer). Any record-aligned byte split is + * pgcolumnar_bulk_parallel_writer). Any record-aligned byte split is * correct -- no partition key, so no sorted-input requirement. */ offs = pcopy_naive_offsets(path, workers); diff --git a/src/columnar_parallel_export.c b/src/columnar_parallel_export.c index f9ad60a..c92ad5a 100644 --- a/src/columnar_parallel_export.c +++ b/src/columnar_parallel_export.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parallel_export.c + * pgcolumnar_parallel_export.c * pgcolumnar.parallel_export_parquet: parallel Parquet export. * * N read-only background workers each write a disjoint slice of the source to @@ -22,7 +22,7 @@ * the dispatcher ships the oid-sorted leaf list so workers do not re-derive * it from the live catalog (which a concurrent ATTACH could desynchronise); * - a single columnar table: split by row-group index ranges; each worker - * restricts its read to its slice via ColumnarReadRestrictToGroups. + * restricts its read to its slice via PgColumnarReadRestrictToGroups. * * Cleanroom: public PostgreSQL APIs and this project's own code only. * @@ -334,8 +334,8 @@ pgcolumnar_parallel_export_worker(Datum main_arg) if (hdr->single_table) { Relation rel = table_open(hdr->relid, AccessShareLock); - List *groups = ColumnarReadRowGroupList(hdr->storageId, - ColumnarCatalogSnapshot(snap)); + List *groups = PgColumnarReadRowGroupList(hdr->storageId, + PgColumnarCatalogSnapshot(snap)); int ntake = me->endIdx - me->startIdx; /* always non-NULL, so an empty slice restricts to nothing (not all) */ uint64 *gnos = palloc(sizeof(uint64) * Max(ntake, 1)); @@ -353,7 +353,7 @@ pgcolumnar_parallel_export_worker(Datum main_arg) } idx++; } - rows = ColumnarWriteParquetFile(rel, snap, me->filepath, gnos, k); + rows = PgColumnarWriteParquetFile(rel, snap, me->filepath, gnos, k); table_close(rel, AccessShareLock); } else @@ -371,7 +371,7 @@ pgcolumnar_parallel_export_worker(Datum main_arg) char fp[MAXPGPATH]; snprintf(fp, sizeof(fp), "%s/part-%04d.parquet", hdr->dirpath, i); - rows += ColumnarWriteParquetFile(part, snap, fp, NULL, 0); + rows += PgColumnarWriteParquetFile(part, snap, fp, NULL, 0); table_close(part, AccessShareLock); MemoryContextSwitchTo(old); MemoryContextDelete(pctx); @@ -405,13 +405,13 @@ pgcolumnar_parallel_export_worker(Datum main_arg) } /* - * columnar_parallel_export_parquet + * pgcolumnar_parallel_export_parquet * SQL: pgcolumnar.parallel_export_parquet(target regclass, path text, * workers int DEFAULT NULL) -> bigint. */ -PG_FUNCTION_INFO_V1(columnar_parallel_export_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_parallel_export_parquet); Datum -columnar_parallel_export_parquet(PG_FUNCTION_ARGS) +pgcolumnar_parallel_export_parquet(PG_FUNCTION_ARGS) { Oid relid; char *dir; @@ -497,7 +497,7 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) { Relation part = table_open(leafoids[i], AccessShareLock); - if (!ColumnarIsColumnarRelation(leafoids[i])) + if (!PgColumnarIsColumnarRelation(leafoids[i])) { char nm[NAMEDATALEN]; @@ -507,7 +507,7 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("partition \"%s\" is not a columnar table", nm))); } - ColumnarParquetCheckExportable(part); + PgColumnarParquetCheckExportable(part); table_close(part, AccessShareLock); } single_table = false; @@ -515,13 +515,13 @@ columnar_parallel_export_parquet(PG_FUNCTION_ARGS) if (workers > npart) workers = npart; } - else if (ColumnarIsColumnarRelation(relid)) + else if (PgColumnarIsColumnarRelation(relid)) { List *groups; - ColumnarParquetCheckExportable(rel); - storageId = ColumnarStorageId(rel); - groups = ColumnarReadRowGroupList(storageId, ColumnarCatalogSnapshot(snap)); + PgColumnarParquetCheckExportable(rel); + storageId = PgColumnarStorageId(rel); + groups = PgColumnarReadRowGroupList(storageId, PgColumnarCatalogSnapshot(snap)); ntasks = list_length(groups); single_table = true; if (workers > Max(ntasks, 1)) diff --git a/src/columnar_parquet.c b/src/columnar_parquet.c index 663b524..db9fc60 100644 --- a/src/columnar_parquet.c +++ b/src/columnar_parquet.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet.c + * pgcolumnar_parquet.c * Parquet file export for pgColumnar (gap 27, piece 2). * * pgcolumnar.export_parquet(rel regclass, path text) writes a columnar table @@ -47,7 +47,7 @@ #include "utils/typcache.h" #include "utils/uuid.h" -PG_FUNCTION_INFO_V1(columnar_export_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_export_parquet); #define PARQUET_ROWGROUP_ROWS 65536 @@ -531,7 +531,7 @@ build_rle_levels(StringInfo out, const uint8 *levels, int64 n, int bit_width) int32 len; initStringInfo(&h); - ColumnarThriftPutVarint(&h, (uint64) ((ngroups << 1) | 1)); /* one bit-packed run */ + PgColumnarThriftPutVarint(&h, (uint64) ((ngroups << 1) | 1)); /* one bit-packed run */ for (i = 0; i < n; i++) { uint32 v = levels[i]; @@ -594,17 +594,17 @@ write_page_header(StringInfo out, int64 nrows, int32 body_size) int16 dlast = 0; /* PageHeader */ - ColumnarThriftPutI32Field(out, &last, 1, 0); /* type = DATA_PAGE */ - ColumnarThriftPutI32Field(out, &last, 2, body_size); /* uncompressed_page_size */ - ColumnarThriftPutI32Field(out, &last, 3, body_size); /* compressed_page_size */ + PgColumnarThriftPutI32Field(out, &last, 1, 0); /* type = DATA_PAGE */ + PgColumnarThriftPutI32Field(out, &last, 2, body_size); /* uncompressed_page_size */ + PgColumnarThriftPutI32Field(out, &last, 3, body_size); /* compressed_page_size */ /* field 5: data_page_header (struct) */ - ColumnarThriftPutField(out, &last, 5, TC_STRUCT); - ColumnarThriftPutI32Field(out, &dlast, 1, (int32) nrows); /* num_values */ - ColumnarThriftPutI32Field(out, &dlast, 2, PQ_ENC_PLAIN); /* encoding */ - ColumnarThriftPutI32Field(out, &dlast, 3, PQ_ENC_RLE); /* def level encoding */ - ColumnarThriftPutI32Field(out, &dlast, 4, PQ_ENC_RLE); /* rep level encoding */ - ColumnarThriftPutStop(out); /* end data_page_header */ - ColumnarThriftPutStop(out); /* end PageHeader */ + PgColumnarThriftPutField(out, &last, 5, TC_STRUCT); + PgColumnarThriftPutI32Field(out, &dlast, 1, (int32) nrows); /* num_values */ + PgColumnarThriftPutI32Field(out, &dlast, 2, PQ_ENC_PLAIN); /* encoding */ + PgColumnarThriftPutI32Field(out, &dlast, 3, PQ_ENC_RLE); /* def level encoding */ + PgColumnarThriftPutI32Field(out, &dlast, 4, PQ_ENC_RLE); /* rep level encoding */ + PgColumnarThriftPutStop(out); /* end data_page_header */ + PgColumnarThriftPutStop(out); /* end PageHeader */ } /* ---- FileMetaData footer ---- */ @@ -613,9 +613,9 @@ write_schema_element_root(StringInfo b, int ncols) { int16 last = 0; - ColumnarThriftPutStringField(b, &last, 4, "schema", 6); /* name */ - ColumnarThriftPutI32Field(b, &last, 5, ncols); /* num_children */ - ColumnarThriftPutStop(b); + PgColumnarThriftPutStringField(b, &last, 4, "schema", 6); /* name */ + PgColumnarThriftPutI32Field(b, &last, 5, ncols); /* num_children */ + PgColumnarThriftPutStop(b); } /* one leaf SchemaElement (a primitive) */ @@ -624,19 +624,19 @@ write_schema_leaf(StringInfo b, const char *name, PqLeaf *leaf, int repetition) { int16 last = 0; - ColumnarThriftPutI32Field(b, &last, 1, leaf->ptype); /* type */ + PgColumnarThriftPutI32Field(b, &last, 1, leaf->ptype); /* type */ if (leaf->ptype == PQ_FIXED_LEN_BYTE_ARRAY) - ColumnarThriftPutI32Field(b, &last, 2, leaf->typeLength); - ColumnarThriftPutI32Field(b, &last, 3, repetition); - ColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); + PgColumnarThriftPutI32Field(b, &last, 2, leaf->typeLength); + PgColumnarThriftPutI32Field(b, &last, 3, repetition); + PgColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); if (leaf->convType >= 0) - ColumnarThriftPutI32Field(b, &last, 6, leaf->convType); + PgColumnarThriftPutI32Field(b, &last, 6, leaf->convType); if (leaf->convType == PQ_CT_DECIMAL) { - ColumnarThriftPutI32Field(b, &last, 7, leaf->scale); - ColumnarThriftPutI32Field(b, &last, 8, leaf->precision); + PgColumnarThriftPutI32Field(b, &last, 7, leaf->scale); + PgColumnarThriftPutI32Field(b, &last, 8, leaf->precision); } - ColumnarThriftPutStop(b); + PgColumnarThriftPutStop(b); } /* one group SchemaElement (no physical type; has num_children) */ @@ -646,12 +646,12 @@ write_schema_group(StringInfo b, const char *name, int repetition, { int16 last = 0; - ColumnarThriftPutI32Field(b, &last, 3, repetition); - ColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); - ColumnarThriftPutI32Field(b, &last, 5, num_children); + PgColumnarThriftPutI32Field(b, &last, 3, repetition); + PgColumnarThriftPutStringField(b, &last, 4, name, (int) strlen(name)); + PgColumnarThriftPutI32Field(b, &last, 5, num_children); if (convType >= 0) - ColumnarThriftPutI32Field(b, &last, 6, convType); - ColumnarThriftPutStop(b); + PgColumnarThriftPutI32Field(b, &last, 6, convType); + PgColumnarThriftPutStop(b); } /* number of SchemaElements a top column contributes (excluding the root) */ @@ -711,27 +711,27 @@ write_column_chunk(StringInfo b, PqLeaf *c, PqColMeta *m) int16 mlast = 0; int p; - ColumnarThriftPutI64Field(b, &last, 2, m->dataPageOffset); /* file_offset */ - ColumnarThriftPutField(b, &last, 3, TC_STRUCT); /* meta_data */ - ColumnarThriftPutI32Field(b, &mlast, 1, c->ptype); /* type */ - ColumnarThriftPutField(b, &mlast, 2, TC_LIST); /* encodings [PLAIN, RLE] */ - ColumnarThriftPutListHeader(b, 2, TC_I32); - ColumnarThriftPutZigzag32(b, PQ_ENC_PLAIN); - ColumnarThriftPutZigzag32(b, PQ_ENC_RLE); - ColumnarThriftPutField(b, &mlast, 3, TC_LIST); /* path_in_schema */ - ColumnarThriftPutListHeader(b, c->pathlen, TC_BINARY); + PgColumnarThriftPutI64Field(b, &last, 2, m->dataPageOffset); /* file_offset */ + PgColumnarThriftPutField(b, &last, 3, TC_STRUCT); /* meta_data */ + PgColumnarThriftPutI32Field(b, &mlast, 1, c->ptype); /* type */ + PgColumnarThriftPutField(b, &mlast, 2, TC_LIST); /* encodings [PLAIN, RLE] */ + PgColumnarThriftPutListHeader(b, 2, TC_I32); + PgColumnarThriftPutZigzag32(b, PQ_ENC_PLAIN); + PgColumnarThriftPutZigzag32(b, PQ_ENC_RLE); + PgColumnarThriftPutField(b, &mlast, 3, TC_LIST); /* path_in_schema */ + PgColumnarThriftPutListHeader(b, c->pathlen, TC_BINARY); for (p = 0; p < c->pathlen; p++) { - ColumnarThriftPutVarint(b, (uint64) strlen(c->path[p])); + PgColumnarThriftPutVarint(b, (uint64) strlen(c->path[p])); appendBinaryStringInfo(b, c->path[p], strlen(c->path[p])); } - ColumnarThriftPutI32Field(b, &mlast, 4, 0); /* codec = UNCOMPRESSED */ - ColumnarThriftPutI64Field(b, &mlast, 5, m->numValues); /* num_values */ - ColumnarThriftPutI64Field(b, &mlast, 6, m->totalSize); /* total_uncompressed_size */ - ColumnarThriftPutI64Field(b, &mlast, 7, m->totalSize); /* total_compressed_size */ - ColumnarThriftPutI64Field(b, &mlast, 9, m->dataPageOffset); /* data_page_offset */ - ColumnarThriftPutStop(b); /* end ColumnMetaData */ - ColumnarThriftPutStop(b); /* end ColumnChunk */ + PgColumnarThriftPutI32Field(b, &mlast, 4, 0); /* codec = UNCOMPRESSED */ + PgColumnarThriftPutI64Field(b, &mlast, 5, m->numValues); /* num_values */ + PgColumnarThriftPutI64Field(b, &mlast, 6, m->totalSize); /* total_uncompressed_size */ + PgColumnarThriftPutI64Field(b, &mlast, 7, m->totalSize); /* total_compressed_size */ + PgColumnarThriftPutI64Field(b, &mlast, 9, m->dataPageOffset); /* data_page_offset */ + PgColumnarThriftPutStop(b); /* end ColumnMetaData */ + PgColumnarThriftPutStop(b); /* end ColumnChunk */ } static void @@ -740,13 +740,13 @@ write_row_group(StringInfo b, PqLeaf *leaves, int nleaves, PqRowGroup *rg) int16 last = 0; int i; - ColumnarThriftPutField(b, &last, 1, TC_LIST); /* columns */ - ColumnarThriftPutListHeader(b, nleaves, TC_STRUCT); + PgColumnarThriftPutField(b, &last, 1, TC_LIST); /* columns */ + PgColumnarThriftPutListHeader(b, nleaves, TC_STRUCT); for (i = 0; i < nleaves; i++) write_column_chunk(b, &leaves[i], &rg->cols[i]); - ColumnarThriftPutI64Field(b, &last, 2, rg->totalByteSize); - ColumnarThriftPutI64Field(b, &last, 3, rg->numRows); - ColumnarThriftPutStop(b); + PgColumnarThriftPutI64Field(b, &last, 2, rg->totalByteSize); + PgColumnarThriftPutI64Field(b, &last, 3, rg->numRows); + PgColumnarThriftPutStop(b); } /* initialize a scalar leaf for a given type; *ok=false if unsupported */ @@ -886,13 +886,13 @@ build_top_column(TopColumn *tc, const char *name, Oid typid, int32 typmod, } /* - * ColumnarParquetCheckExportable + * PgColumnarParquetCheckExportable * Ereport if rel cannot be exported to Parquet (a dropped column, or a * column type the writer does not support). Lets the parallel exporter fail * fast in the dispatcher, before it opens files or spawns workers. */ void -ColumnarParquetCheckExportable(Relation rel) +PgColumnarParquetCheckExportable(Relation rel) { TupleDesc tupdesc = RelationGetDescr(rel); int ntop = tupdesc->natts; @@ -933,16 +933,16 @@ ColumnarParquetCheckExportable(Relation rel) } /* - * ColumnarWriteParquetFile + * PgColumnarWriteParquetFile * Write rel's live rows to a Parquet file at filepath, under snapshot. * When restrictGroups is non-NULL, only those row groups are written (the * parallel exporter gives each worker a disjoint group slice). Returns rows * written. The caller owns rel (kept open) and the snapshot; on error this * ereports and the resource owner releases the lock. Shared by the serial - * columnar_export_parquet and the parallel exporter. + * pgcolumnar_export_parquet and the parallel exporter. */ int64 -ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, +PgColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, const uint64 *restrictGroups, int nRestrictGroups) { TupleDesc tupdesc; @@ -951,7 +951,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, PqLeaf *leaves; int nleaves = 0; int totalLeaves = 0; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *values; bool *nulls; uint64 rowNumber; @@ -1010,7 +1010,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, values = palloc(sizeof(Datum) * ntop); nulls = palloc(sizeof(bool) * ntop); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); /* * NULL restrictGroups means no restriction (the whole table). A non-NULL * list restricts to exactly those groups, so an empty list (nRestrictGroups @@ -1019,11 +1019,11 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, * earlier version did -- the entire table. */ if (restrictGroups != NULL) - ColumnarReadRestrictToGroups(readState, restrictGroups, nRestrictGroups); + PgColumnarReadRestrictToGroups(readState, restrictGroups, nRestrictGroups); for (;;) { - bool got = ColumnarReadNextRow(readState, values, nulls, &rowNumber); + bool got = PgColumnarReadNextRow(readState, values, nulls, &rowNumber); if (got) { @@ -1081,7 +1081,7 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, if (!got) break; } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); /* ---- FileMetaData footer ---- */ { @@ -1094,21 +1094,21 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, nschema += schema_count_for_top(&tops[i]); initStringInfo(&fmd); - ColumnarThriftPutI32Field(&fmd, &last, 1, 1); /* version */ + PgColumnarThriftPutI32Field(&fmd, &last, 1, 1); /* version */ /* schema list (2): root + the (possibly nested) elements per column */ - ColumnarThriftPutField(&fmd, &last, 2, TC_LIST); - ColumnarThriftPutListHeader(&fmd, nschema, TC_STRUCT); + PgColumnarThriftPutField(&fmd, &last, 2, TC_LIST); + PgColumnarThriftPutListHeader(&fmd, nschema, TC_STRUCT); write_schema_element_root(&fmd, ntop); for (i = 0; i < ntop; i++) write_top_schema(&fmd, &tops[i], leaves); - ColumnarThriftPutI64Field(&fmd, &last, 3, total); /* num_rows */ + PgColumnarThriftPutI64Field(&fmd, &last, 3, total); /* num_rows */ /* row_groups list (4) */ - ColumnarThriftPutField(&fmd, &last, 4, TC_LIST); - ColumnarThriftPutListHeader(&fmd, nrgs, TC_STRUCT); + PgColumnarThriftPutField(&fmd, &last, 4, TC_LIST); + PgColumnarThriftPutListHeader(&fmd, nrgs, TC_STRUCT); for (i = 0; i < nrgs; i++) write_row_group(&fmd, leaves, nleaves, &rgs[i]); - ColumnarThriftPutStringField(&fmd, &last, 6, "pgColumnar", 10); /* created_by */ - ColumnarThriftPutStop(&fmd); + PgColumnarThriftPutStringField(&fmd, &last, 6, "pgColumnar", 10); /* created_by */ + PgColumnarThriftPutStop(&fmd); fwrite(fmd.data, 1, fmd.len, f); footerLen = fmd.len; @@ -1126,12 +1126,12 @@ ColumnarWriteParquetFile(Relation rel, Snapshot snapshot, const char *filepath, } /* - * columnar_export_parquet + * pgcolumnar_export_parquet * SQL: pgcolumnar.export_parquet(rel regclass, path text) -> bigint. - * Thin wrapper over ColumnarWriteParquetFile for the whole table. + * Thin wrapper over PgColumnarWriteParquetFile for the whole table. */ Datum -columnar_export_parquet(PG_FUNCTION_ARGS) +pgcolumnar_export_parquet(PG_FUNCTION_ARGS) { Oid relid; char *path; @@ -1152,7 +1152,7 @@ columnar_export_parquet(PG_FUNCTION_ARGS) path = text_to_cstring(PG_GETARG_TEXT_PP(1)); rel = table_open(relid, AccessShareLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessShareLock); ereport(ERROR, @@ -1162,7 +1162,7 @@ columnar_export_parquet(PG_FUNCTION_ARGS) } snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - total = ColumnarWriteParquetFile(rel, snapshot, path, NULL, 0); + total = PgColumnarWriteParquetFile(rel, snapshot, path, NULL, 0); table_close(rel, AccessShareLock); PG_RETURN_INT64(total); diff --git a/src/columnar_parquet_codec.c b/src/columnar_parquet_codec.c index 661bc95..9c1046b 100644 --- a/src/columnar_parquet_codec.c +++ b/src/columnar_parquet_codec.c @@ -1,7 +1,7 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_codec.c - * Parquet data-page decompression. See columnar_parquet_codec.h. + * pgcolumnar_parquet_codec.c + * Parquet data-page decompression. See pgcolumnar_parquet_codec.h. * * Written fresh for pgColumnar from the public Apache Parquet specification and * the public Snappy format description. @@ -143,7 +143,7 @@ snappy_raw_uncompress(const uint8 *in, size_t inlen, StringInfo out) * driven to allocate or inflate more than the header declares. */ bool -ColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, +PgColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, StringInfo scratch, const uint8 **out, size_t *outlen) { if (srclen > MaxAllocSize || usize > MaxAllocSize) diff --git a/src/columnar_parquet_codec.h b/src/columnar_parquet_codec.h index a7cd30b..f692e35 100644 --- a/src/columnar_parquet_codec.h +++ b/src/columnar_parquet_codec.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_codec.h + * pgcolumnar_parquet_codec.h * Parquet data-page decompression. * * One entry point that turns a compressed page into bytes, dispatching on the @@ -30,7 +30,7 @@ * false on malformed input, on a codec this build cannot decode, and on any * length that disagrees with the page header. */ -extern bool ColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, +extern bool PgColumnarParquetDecompress(int codec, const uint8 *src, size_t srclen, size_t usize, StringInfo scratch, const uint8 **out, size_t *outlen); diff --git a/src/columnar_parquet_format.h b/src/columnar_parquet_format.h index ec96729..fd2a4c7 100644 --- a/src/columnar_parquet_format.h +++ b/src/columnar_parquet_format.h @@ -1,13 +1,13 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_format.h + * pgcolumnar_parquet_format.h * Parquet and Thrift compact-protocol constants, shared by the reader and * the writer. * * These are values defined by the file format, not by this implementation, so * there is exactly one correct value for each and both directions must agree on - * it. They were previously declared twice, once in columnar_parquet.c and once - * in columnar_parquet_reader.c, with 21 names in common. The values did agree, + * it. They were previously declared twice, once in pgcolumnar_parquet.c and once + * in pgcolumnar_parquet_reader.c, with 21 names in common. The values did agree, * but nothing made them: a writer and a reader that disagree on a type code * produce a file that is silently wrong rather than one that fails to parse, * which is the worst shape a defect can take here. diff --git a/src/columnar_parquet_reader.c b/src/columnar_parquet_reader.c index 0051a69..ba56caf 100644 --- a/src/columnar_parquet_reader.c +++ b/src/columnar_parquet_reader.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_parquet_reader.c + * pgcolumnar_parquet_reader.c * Parquet file import: pgcolumnar.import_parquet(rel regclass, path text). * * A self-contained Parquet reader with no libparquet/libarrow dependency. It @@ -83,9 +83,9 @@ #include #endif -PG_FUNCTION_INFO_V1(columnar_import_parquet); -PG_FUNCTION_INFO_V1(columnar_parquet_schema); -PG_FUNCTION_INFO_V1(columnar_read_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_import_parquet); +PG_FUNCTION_INFO_V1(pgcolumnar_parquet_schema); +PG_FUNCTION_INFO_V1(pgcolumnar_read_parquet); PG_FUNCTION_INFO_V1(pgcolumnar_parquet_fdw_handler); PG_FUNCTION_INFO_V1(pgcolumnar_parquet_fdw_validator); @@ -197,7 +197,7 @@ parse_statistics(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) @@ -205,7 +205,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 1: /* max (deprecated): fallback only */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p && !ch->has_max) { @@ -218,7 +218,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 2: /* min (deprecated): fallback only */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p && !ch->has_min) { @@ -231,7 +231,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 5: /* max_value (preferred) */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) { @@ -244,7 +244,7 @@ parse_statistics(TCReader *r, PqChunk *ch) case 6: /* min_value (preferred) */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) { @@ -255,7 +255,7 @@ parse_statistics(TCReader *r, PqChunk *ch) break; } default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -284,34 +284,34 @@ parse_column_meta(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 4: /* codec */ - ch->codec = (int) ColumnarThriftZigzag(r); + ch->codec = (int) PgColumnarThriftZigzag(r); break; case 5: /* num_values */ - ch->num_values = ColumnarThriftZigzag(r); + ch->num_values = PgColumnarThriftZigzag(r); break; case 7: /* total_compressed_size */ - ch->total_compressed_size = ColumnarThriftZigzag(r); + ch->total_compressed_size = PgColumnarThriftZigzag(r); break; case 9: /* data_page_offset */ - ch->data_page_offset = ColumnarThriftZigzag(r); + ch->data_page_offset = PgColumnarThriftZigzag(r); break; case 11: /* dictionary_page_offset */ - ch->dict_page_offset = ColumnarThriftZigzag(r); + ch->dict_page_offset = PgColumnarThriftZigzag(r); break; case 12: /* statistics */ if (ft == TC_STRUCT) parse_statistics(r, ch); else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -328,13 +328,13 @@ parse_column_chunk(TCReader *r, PqChunk *ch) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if (fid == 3 && ft == TC_STRUCT) parse_column_meta(r, ch); else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } } @@ -347,7 +347,7 @@ parse_column_chunk(TCReader *r, PqChunk *ch) * This hand-walks nested Thrift unions and assumes the compact-protocol field * ordering the spec prescribes. A writer that reorders or partially populates the * union could desync the cursor -- but a desync fails the surrounding footer - * parse (ColumnarThriftField / bounds checks catch it), so the blast radius is "file + * parse (PgColumnarThriftField / bounds checks catch it), so the blast radius is "file * rejected", never a wrong value silently returned from a good decode. */ static void @@ -360,7 +360,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if ((fid == PQ_LT_TIME || fid == PQ_LT_TIMESTAMP) && ft == TC_STRUCT) @@ -373,7 +373,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int ift, ifid; - ColumnarThriftField(r, &ift, &ifid, &innerLast); + PgColumnarThriftField(r, &ift, &ifid, &innerLast); if (ift == TC_STOP || r->error) break; if (ifid == 2 && ift == TC_STRUCT) /* unit: union TimeUnit */ @@ -382,7 +382,7 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) int uft, ufid; - ColumnarThriftField(r, &uft, &ufid, &uLast); + PgColumnarThriftField(r, &uft, &ufid, &uLast); if (uft != TC_STOP && !r->error) { switch (ufid) @@ -398,17 +398,17 @@ parse_logical_type(TCReader *r, PqSchemaCol *sc) break; } sc->is_timestamp = isTs; - ColumnarThriftSkip(r, uft); /* the unit member is an empty struct */ + PgColumnarThriftSkip(r, uft); /* the unit member is an empty struct */ /* consume the union's STOP */ - ColumnarThriftField(r, &uft, &ufid, &uLast); + PgColumnarThriftField(r, &uft, &ufid, &uLast); } } else - ColumnarThriftSkip(r, ift); /* isAdjustedToUTC and anything later */ + PgColumnarThriftSkip(r, ift); /* isAdjustedToUTC and anything later */ } } else - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } } @@ -434,47 +434,47 @@ parse_schema_element(TCReader *r, PqSchemaCol *sc) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 1: /* type */ - sc->phys_type = (int) ColumnarThriftZigzag(r); + sc->phys_type = (int) PgColumnarThriftZigzag(r); break; case 2: /* type_length */ - sc->type_length = (int) ColumnarThriftZigzag(r); + sc->type_length = (int) PgColumnarThriftZigzag(r); break; case 3: /* repetition_type */ - sc->repetition = (int) ColumnarThriftZigzag(r); + sc->repetition = (int) PgColumnarThriftZigzag(r); break; case 4: /* name */ { uint32 n; - const uint8 *p = ColumnarThriftBytes(r, &n); + const uint8 *p = PgColumnarThriftBytes(r, &n); if (p) sc->name = pnstrdup((const char *) p, n); break; } case 5: /* num_children */ - num_children = (int) ColumnarThriftZigzag(r); + num_children = (int) PgColumnarThriftZigzag(r); sc->num_children = num_children; break; case 6: /* converted_type */ - sc->converted_type = (int) ColumnarThriftZigzag(r); + sc->converted_type = (int) PgColumnarThriftZigzag(r); break; case 7: /* scale (DECIMAL) */ - sc->scale = (int) ColumnarThriftZigzag(r); + sc->scale = (int) PgColumnarThriftZigzag(r); break; case 8: /* precision (DECIMAL) */ - sc->precision = (int) ColumnarThriftZigzag(r); + sc->precision = (int) PgColumnarThriftZigzag(r); break; case 10: /* logicalType */ parse_logical_type(r, sc); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -525,7 +525,7 @@ walk_schema(PqFile *pf, int *cursor, int def, int rep, * num_children comes straight from the footer, so a schema that chains * group inside group descends here as deep as the file says. This is a * second unbounded recursion, reached only after the footer parses, so the - * guard on ColumnarThriftSkip does not cover it; without one, a crafted + * guard on PgColumnarThriftSkip does not cover it; without one, a crafted * schema SIGSEGVs the backend and restarts the cluster. */ check_stack_depth(); @@ -574,14 +574,14 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) int ft, fid; - ColumnarThriftField(&r, &ft, &fid, &lastId); + PgColumnarThriftField(&r, &ft, &fid, &lastId); if (ft == TC_STOP || r.error) break; if (fid == 2 && ft == TC_LIST) /* schema: list */ { int etype; - uint32 n = ColumnarThriftListHeader(&r, &etype); + uint32 n = PgColumnarThriftListHeader(&r, &etype); uint32 i; PqSchemaCol *tmp = palloc0(sizeof(PqSchemaCol) * Max(n, 1)); @@ -593,7 +593,7 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) else if (fid == 4 && ft == TC_LIST) /* row_groups */ { int etype; - uint32 n = ColumnarThriftListHeader(&r, &etype); + uint32 n = PgColumnarThriftListHeader(&r, &etype); uint32 i; pf->nrowgroups = n; @@ -610,13 +610,13 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) int rft, rfid; - ColumnarThriftField(&r, &rft, &rfid, &rgLast); + PgColumnarThriftField(&r, &rft, &rfid, &rgLast); if (rft == TC_STOP || r.error) break; if (rfid == 1 && rft == TC_LIST) /* columns */ { int cet; - uint32 cn = ColumnarThriftListHeader(&r, &cet); + uint32 cn = PgColumnarThriftListHeader(&r, &cet); uint32 ci; rg->chunks = palloc0(sizeof(PqChunk) * Max(cn, 1)); @@ -625,14 +625,14 @@ parse_file_metadata(const uint8 *buf, size_t len, PqFile *pf) parse_column_chunk(&r, &rg->chunks[ci]); } else if (rfid == 3) /* num_rows */ - rg->num_rows = ColumnarThriftZigzag(&r); + rg->num_rows = PgColumnarThriftZigzag(&r); else - ColumnarThriftSkip(&r, rft); + PgColumnarThriftSkip(&r, rft); } } } else - ColumnarThriftSkip(&r, ft); + PgColumnarThriftSkip(&r, ft); } if (r.error || pf->nelems < 1) @@ -1285,7 +1285,7 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; if (!v2) @@ -1293,13 +1293,13 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) switch (fid) { case 1: - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); break; case 2: - h->encoding = (int) ColumnarThriftZigzag(r); + h->encoding = (int) PgColumnarThriftZigzag(r); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1308,22 +1308,22 @@ parse_data_page_header(TCReader *r, PqPageHeader *h, bool v2) switch (fid) { case 1: - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); break; case 4: - h->encoding = (int) ColumnarThriftZigzag(r); + h->encoding = (int) PgColumnarThriftZigzag(r); break; case 5: - h->def_levels_len = (int) ColumnarThriftZigzag(r); + h->def_levels_len = (int) PgColumnarThriftZigzag(r); break; case 6: - h->rep_levels_len = (int) ColumnarThriftZigzag(r); + h->rep_levels_len = (int) PgColumnarThriftZigzag(r); break; case 7: h->is_compressed = (ft == TC_BOOL_TRUE); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1343,19 +1343,19 @@ parse_page_header(TCReader *r, PqPageHeader *h) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; switch (fid) { case 1: - h->type = (int) ColumnarThriftZigzag(r); + h->type = (int) PgColumnarThriftZigzag(r); break; case 2: - h->uncompressed_size = (int) ColumnarThriftZigzag(r); + h->uncompressed_size = (int) PgColumnarThriftZigzag(r); break; case 3: - h->compressed_size = (int) ColumnarThriftZigzag(r); + h->compressed_size = (int) PgColumnarThriftZigzag(r); break; case 5: /* DataPageHeader (v1) */ parse_data_page_header(r, h, false); @@ -1369,13 +1369,13 @@ parse_page_header(TCReader *r, PqPageHeader *h) int dft, dfid; - ColumnarThriftField(r, &dft, &dfid, &dl); + PgColumnarThriftField(r, &dft, &dfid, &dl); if (dft == TC_STOP || r->error) break; if (dfid == 1) - h->num_values = (int) ColumnarThriftZigzag(r); + h->num_values = (int) PgColumnarThriftZigzag(r); else - ColumnarThriftSkip(r, dft); + PgColumnarThriftSkip(r, dft); } break; } @@ -1383,7 +1383,7 @@ parse_page_header(TCReader *r, PqPageHeader *h) parse_data_page_header(r, h, true); break; default: - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); break; } } @@ -1687,7 +1687,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, const uint8 *end; int i; - if (!ColumnarParquetDecompress(ch->codec, praw, h.compressed_size, + if (!PgColumnarParquetDecompress(ch->codec, praw, h.compressed_size, h.uncompressed_size, &dec, &db, &dblen)) return false; dictCount = h.num_values; @@ -1755,7 +1755,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, size_t vusize = (h.uncompressed_size > levLen) ? (size_t) (h.uncompressed_size - levLen) : 0; - if (!ColumnarParquetDecompress(ch->codec, vraw, vrawlen, vusize, + if (!PgColumnarParquetDecompress(ch->codec, vraw, vrawlen, vusize, &dec, &valbuf, &vallen)) return false; } @@ -1771,7 +1771,7 @@ decode_leaf_entries(PqSource *src, PqChunk *ch, size_t pblen; size_t off = 0; - if (!ColumnarParquetDecompress(ch->codec, praw, h.compressed_size, + if (!PgColumnarParquetDecompress(ch->codec, praw, h.compressed_size, h.uncompressed_size, &dec, &pb, &pblen)) return false; /* v1: repetition levels first, then definition levels, both @@ -2017,7 +2017,7 @@ pq_want_phys_for(Oid typid, const PqSchemaCol *sc) /* * Infer the PostgreSQL column type a Parquet leaf should map to. This is the - * inverse of the exporter's parquet_kind_for_type (columnar_parquet.c): it reads + * inverse of the exporter's parquet_kind_for_type (pgcolumnar_parquet.c): it reads * the physical type plus the ConvertedType annotation, so a round-tripped file * reports the source column types. It is tolerant of files other writers produce * (both the millis and micros time/timestamp variants, INT_8/INT_32 widths) so @@ -2561,7 +2561,7 @@ typedef struct PqInsertSinkArg { Relation rel; CommandId cid; - ColumnarIndexInsertState *indexes; /* NULL when the table has none */ + PgColumnarIndexInsertState *indexes; /* NULL when the table has none */ } PqInsertSinkArg; static void @@ -2579,9 +2579,9 @@ pq_insert_sink(TupleTableSlot *slot, void *arg) * assigned. */ if (a->indexes != NULL) - ColumnarIndexInsertRow(a->indexes, a->rel, slot->tts_values, + PgColumnarIndexInsertRow(a->indexes, a->rel, slot->tts_values, slot->tts_isnull, - ColumnarItemPointerToRowNumber(&slot->tts_tid)); + PgColumnarItemPointerToRowNumber(&slot->tts_tid)); } static void @@ -2885,7 +2885,7 @@ pq_read_file_into(const char *path, TupleDesc tupdesc, TupleTableSlot *slot, * pgcolumnar.import_parquet(rel regclass, path text) -> bigint */ Datum -columnar_import_parquet(PG_FUNCTION_ARGS) +pgcolumnar_import_parquet(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); char *path = text_to_cstring(PG_GETARG_TEXT_PP(1)); @@ -2912,8 +2912,8 @@ columnar_import_parquet(PG_FUNCTION_ARGS) sinkarg.rel = rel; sinkarg.cid = GetCurrentCommandId(true); - sinkarg.indexes = ColumnarRelationHasIndexes(rel) - ? ColumnarIndexInsertBegin(rel, true) : NULL; + sinkarg.indexes = PgColumnarRelationHasIndexes(rel) + ? PgColumnarIndexInsertBegin(rel, true) : NULL; /* insert every resolved file's rows; the per-file decode is bounded by * fileCtx. Each file is bound against the target's descriptor, so a directory * whose files disagree with the table errors rather than importing garbage. */ @@ -2932,7 +2932,7 @@ columnar_import_parquet(PG_FUNCTION_ARGS) MemoryContextDelete(fileCtx); if (sinkarg.indexes != NULL) - ColumnarIndexInsertEnd(sinkarg.indexes); + PgColumnarIndexInsertEnd(sinkarg.indexes); ExecDropSingleTupleTableSlot(slot); /* @@ -2964,7 +2964,7 @@ columnar_import_parquet(PG_FUNCTION_ARGS) * materialize-mode SRF. */ Datum -columnar_read_parquet(PG_FUNCTION_ARGS) +pgcolumnar_read_parquet(PG_FUNCTION_ARGS) { char *path = text_to_cstring(PG_GETARG_TEXT_PP(0)); ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; @@ -3040,7 +3040,7 @@ columnar_read_parquet(PG_FUNCTION_ARGS) * files are reported as their flattened leaf columns. */ Datum -columnar_parquet_schema(PG_FUNCTION_ARGS) +pgcolumnar_parquet_schema(PG_FUNCTION_ARGS) { char *path = text_to_cstring(PG_GETARG_TEXT_PP(0)); ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo; @@ -3808,11 +3808,11 @@ pqfdw_compute_skip(ForeignScanState *node, PqFile *pf, * btree family will do: a given operator means the same comparison * in every family that lists it. */ - interp = ColumnarGetOpInterpretation(op->opno); + interp = PgColumnarGetOpInterpretation(op->opno); foreach(ic, interp) { - ColumnarOpInterpretation *o = (ColumnarOpInterpretation *) lfirst(ic); - int s = ColumnarOpInterpStrategy(o); + PgColumnarOpInterpretation *o = (PgColumnarOpInterpretation *) lfirst(ic); + int s = PgColumnarOpInterpStrategy(o); if (s >= BTLessStrategyNumber && s <= BTGreaterStrategyNumber) { diff --git a/src/columnar_projection.c b/src/columnar_projection.c index b60f838..dd22cea 100644 --- a/src/columnar_projection.c +++ b/src/columnar_projection.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_projection.c + * pgcolumnar_projection.c * DDL for multiple physical projections (gap 26, format 2.2). * * A projection is a named, ordered subset of a table's columns stored as its @@ -8,7 +8,7 @@ * identity space (the C-Store model; see design/gaps/26-*). The catalog * (pgcolumnar.projection) and the add/drop DDL are provided here: declaring a * projection allocates its storage id, records the catalog row, and back-fills - * the projection's storage from existing rows (ColumnarBackfillProjection). Read + * the projection's storage from existing rows (PgColumnarBackfillProjection). Read * paths (read_projection, reconstruct_via_projection) are also provided. * * projection_id 0 is the implicit base projection (all live columns, insert @@ -34,9 +34,9 @@ #include "utils/snapmgr.h" #include "utils/tuplestore.h" -PG_FUNCTION_INFO_V1(columnar_add_projection); -PG_FUNCTION_INFO_V1(columnar_drop_projection); -PG_FUNCTION_INFO_V1(columnar_read_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_add_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_drop_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_read_projection); /* * Collect the live (non-dropped) attribute numbers of a relation, in attnum @@ -118,11 +118,11 @@ static void record_base_projection(Relation rel, uint64 storageId, List *existing) { ListCell *lc; - ColumnarProjection base; + PgColumnarProjection base; foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projectionId == 0) return; @@ -136,7 +136,7 @@ record_base_projection(Relation rel, uint64 storageId, List *existing) base.sortKey = NULL; base.sortKeyLen = 0; base.columns = live_attnums(rel, &base.columnsLen); - ColumnarInsertProjectionRow(&base); + PgColumnarInsertProjectionRow(&base); } /* @@ -144,7 +144,7 @@ record_base_projection(Relation rel, uint64 storageId, List *existing) * Declare a projection: a named column subset sorted on sort_key. */ Datum -columnar_add_projection(PG_FUNCTION_ARGS) +pgcolumnar_add_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -154,7 +154,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) uint64 storageId; List *existing; ListCell *lc; - ColumnarProjection proj; + PgColumnarProjection proj; int nextId = 1; int i, j; @@ -169,7 +169,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) colsArr = PG_GETARG_ARRAYTYPE_P(2); sortArr = PG_ARGISNULL(3) ? NULL : PG_GETARG_ARRAYTYPE_P(3); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", @@ -191,13 +191,13 @@ columnar_add_projection(PG_FUNCTION_ARGS) * unaffected. (A CONCURRENTLY variant is future work.) */ rel = table_open(relid, ShareLock); - ColumnarRequireTableOwner(rel); - storageId = ColumnarStorageId(rel); + PgColumnarRequireTableOwner(rel); + storageId = PgColumnarStorageId(rel); - existing = ColumnarListProjections(storageId); + existing = PgColumnarListProjections(storageId); record_base_projection(rel, storageId, existing); /* re-read so the base row is included when picking the next id / name check */ - existing = ColumnarListProjections(storageId); + existing = PgColumnarListProjections(storageId); memset(&proj, 0, sizeof(proj)); proj.storageId = storageId; @@ -232,7 +232,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) /* name must be unique for this table; next id is max + 1 */ foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) ereport(ERROR, @@ -244,11 +244,11 @@ columnar_add_projection(PG_FUNCTION_ARGS) } proj.projectionId = nextId; - proj.projStorageId = ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&proj); + proj.projStorageId = PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&proj); /* populate the projection from the table's existing rows (gap 26 back-fill) */ - ColumnarBackfillProjection(rel, &proj); + PgColumnarBackfillProjection(rel, &proj); /* * Record the declaration behind it, by relation and column name, so a dump @@ -256,7 +256,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) * (#266). Written here rather than in the SQL binding so that a projection * cannot come into existence without one. */ - ColumnarRecordProjectionDeclaration(relid, projname, colsArr, + PgColumnarRecordProjectionDeclaration(relid, projname, colsArr, sortArr ? sortArr : construct_empty_array(TEXTOID)); @@ -269,7 +269,7 @@ columnar_add_projection(PG_FUNCTION_ARGS) * Drop a declared projection. The base projection cannot be dropped. */ Datum -columnar_drop_projection(PG_FUNCTION_ARGS) +pgcolumnar_drop_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -288,20 +288,20 @@ columnar_drop_projection(PG_FUNCTION_ARGS) relid = PG_GETARG_OID(0); projname = text_to_cstring(PG_GETARG_TEXT_PP(1)); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, ShareUpdateExclusiveLock); - ColumnarRequireTableOwner(rel); - storageId = ColumnarStorageId(rel); - existing = ColumnarListProjections(storageId); + PgColumnarRequireTableOwner(rel); + storageId = PgColumnarStorageId(rel); + existing = PgColumnarListProjections(storageId); foreach(lc, existing) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -331,11 +331,11 @@ columnar_drop_projection(PG_FUNCTION_ARGS) * deleted. */ if (targetStorageId != storageId) - ColumnarDeleteMetadata(targetStorageId); - ColumnarDeleteProjectionRow(storageId, targetId); + PgColumnarDeleteMetadata(targetStorageId); + PgColumnarDeleteProjectionRow(storageId, targetId); /* and forget the declaration, so a later rebuild does not resurrect it (#266) */ - ColumnarDeleteProjectionDeclaration(relid, projname); + PgColumnarDeleteProjectionDeclaration(relid, projname); table_close(rel, ShareUpdateExclusiveLock); PG_RETURN_VOID(); @@ -351,7 +351,7 @@ columnar_drop_projection(PG_FUNCTION_ARGS) * flushed first so rows written earlier in this transaction are visible. */ Datum -columnar_read_projection(PG_FUNCTION_ARGS) +pgcolumnar_read_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -360,7 +360,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) uint64 storageId; List *projs; ListCell *lc; - ColumnarProjection *proj = NULL; + PgColumnarProjection *proj = NULL; TupleDesc projTupdesc; TupleDesc retdesc; Tuplestorestate *tupstore; @@ -369,7 +369,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) int ncols; int i; Snapshot snap; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *rvals; bool *rnulls; uint64 projRowNum; @@ -387,20 +387,20 @@ columnar_read_projection(PG_FUNCTION_ARGS) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("set-valued function called in context that cannot accept a set"))); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, AccessShareLock); /* persist pending base + projection writes so this read sees them */ - ColumnarFlushWriteStateForRelation(relid); - storageId = ColumnarStorageId(rel); + PgColumnarFlushWriteStateForRelation(relid); + storageId = PgColumnarStorageId(rel); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -448,10 +448,10 @@ columnar_read_projection(PG_FUNCTION_ARGS) snap = GetActiveSnapshot(); rvals = palloc(sizeof(Datum) * (ncols + 1)); rnulls = palloc(sizeof(bool) * (ncols + 1)); - readState = ColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, + readState = PgColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, projTupdesc, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) + while (PgColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) { uint64 baseRow = (uint64) DatumGetInt64(rvals[0]); StringInfoData buf; @@ -464,7 +464,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) * base row to answer that decoded every column and threw all of it away * (issue #157). */ - if (!ColumnarRowIsLive(rel, snap, baseRow)) + if (!PgColumnarRowIsLive(rel, snap, baseRow)) continue; initStringInfo(&buf); @@ -484,7 +484,7 @@ columnar_read_projection(PG_FUNCTION_ARGS) pfree(buf.data); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); table_close(rel, AccessShareLock); return (Datum) 0; @@ -500,9 +500,9 @@ columnar_read_projection(PG_FUNCTION_ARGS) * does not cover every referenced column. All live table columns are * rendered by their output functions and joined by '|'. */ -PG_FUNCTION_INFO_V1(columnar_reconstruct_via_projection); +PG_FUNCTION_INFO_V1(pgcolumnar_reconstruct_via_projection); Datum -columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) +pgcolumnar_reconstruct_via_projection(PG_FUNCTION_ARGS) { Oid relid; char *projname; @@ -512,7 +512,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) uint64 storageId; List *projs; ListCell *lc; - ColumnarProjection *proj = NULL; + PgColumnarProjection *proj = NULL; TupleDesc projTupdesc; TupleDesc retdesc; Tuplestorestate *tupstore; @@ -523,7 +523,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) int tnatts; int i; Snapshot snap; - ColumnarReadState *readState; + PgColumnarReadState *readState; Datum *rvals; bool *rnulls; Datum *basevals; @@ -544,21 +544,21 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("set-valued function called in context that cannot accept a set"))); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) ereport(ERROR, (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("\"%s\" is not a columnar table", get_rel_name(relid)))); rel = table_open(relid, AccessShareLock); - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); tableDesc = RelationGetDescr(rel); tnatts = tableDesc->natts; - storageId = ColumnarStorageId(rel); + storageId = PgColumnarStorageId(rel); - projs = ColumnarListProjections(storageId); + projs = PgColumnarListProjections(storageId); foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (strcmp(p->name, projname) == 0) { @@ -637,10 +637,10 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) uncovered = bms_add_member(uncovered, i); } - readState = ColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, + readState = PgColumnarBeginReadWithStorage(rel, snap, proj->projStorageId, projTupdesc, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) + while (PgColumnarReadNextRow(readState, rvals, rnulls, &projRowNum)) { uint64 baseRow = (uint64) DatumGetInt64(rvals[0]); StringInfoData buf; @@ -649,7 +649,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) bool first = true; /* fetch the base row: liveness, and only the columns not covered */ - if (!ColumnarReadRowByNumberCols(rel, snap, baseRow, basevals, + if (!PgColumnarReadRowByNumberCols(rel, snap, baseRow, basevals, basenulls, uncovered)) continue; @@ -688,7 +688,7 @@ columnar_reconstruct_via_projection(PG_FUNCTION_ARGS) pfree(buf.data); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); table_close(rel, AccessShareLock); return (Datum) 0; diff --git a/src/columnar_reader.c b/src/columnar_reader.c index 87c9dc6..4a1ebe2 100644 --- a/src/columnar_reader.c +++ b/src/columnar_reader.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_reader.c + * pgcolumnar_reader.c * The columnar reader: a sequential scan that reads all columns of all * stripes and reconstructs rows (spec 4, 6). Also holds the value-stream * codec shared with the writer. @@ -49,7 +49,7 @@ typedef struct SkipPredicate Oid hashCollation; } SkipPredicate; -struct ColumnarReadState +struct PgColumnarReadState { Relation rel; Snapshot snapshot; @@ -161,13 +161,13 @@ struct ColumnarReadState MemoryContext skipContext; /* scratch for skip-list evaluation */ }; -static void columnar_build_predicates(ColumnarReadState *readState, +static void pgcolumnar_build_predicates(PgColumnarReadState *readState, int nkeys, ScanKey keys); -static int64 columnar_next_group_index(ColumnarReadState *readState); +static int64 pgcolumnar_next_group_index(PgColumnarReadState *readState); /* qsort comparator for the row group restriction set */ static int -columnar_uint64_cmp(const void *a, const void *b) +pgcolumnar_uint64_cmp(const void *a, const void *b) { uint64 x = *(const uint64 *) a; uint64 y = *(const uint64 *) b; @@ -176,13 +176,13 @@ columnar_uint64_cmp(const void *a, const void *b) } /* - * columnar_group_is_restricted_in + * pgcolumnar_group_is_restricted_in * Is this group number in the read state's restriction set? Binary search - * over the sorted array set by ColumnarReadRestrictToGroups. Only called + * over the sorted array set by PgColumnarReadRestrictToGroups. Only called * when restrictGroups is non-NULL. */ static bool -columnar_group_is_restricted_in(ColumnarReadState *rs, uint64 groupNumber) +pgcolumnar_group_is_restricted_in(PgColumnarReadState *rs, uint64 groupNumber) { int lo = 0; int hi = rs->numRestrictGroups - 1; @@ -206,13 +206,13 @@ columnar_group_is_restricted_in(ColumnarReadState *rs, uint64 groupNumber) * ------------------------------------------------------------------------- */ /* - * ColumnarEncodeValue + * PgColumnarEncodeValue * Append a non-null value to a column's value stream. Fixed-length * values are stored as their raw bytes; varlena values are detoasted * and stored with a full 4-byte header so the reader can size them. */ void -ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) +PgColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) { if (att->attbyval) { @@ -245,13 +245,13 @@ ColumnarEncodeValue(StringInfo buf, Form_pg_attribute att, Datum value) } /* - * ColumnarDecodeValue + * PgColumnarDecodeValue * Read one value from a column's value stream, advancing *cursor. * By-reference values are copied into targetContext so they outlive the * stripe buffer's next reset. */ Datum -ColumnarDecodeValue(Form_pg_attribute att, char **cursor, +PgColumnarDecodeValue(Form_pg_attribute att, char **cursor, MemoryContext targetContext) { char *p = *cursor; @@ -272,7 +272,7 @@ ColumnarDecodeValue(Form_pg_attribute att, char **cursor, } else { - Size len = ColumnarVarSizeAnyUnaligned(p); + Size len = PgColumnarVarSizeAnyUnaligned(p); char *copy = MemoryContextAlloc(targetContext, len); memcpy(copy, p, len); @@ -287,23 +287,23 @@ ColumnarDecodeValue(Form_pg_attribute att, char **cursor, * sequential scan * ------------------------------------------------------------------------- */ -ColumnarReadState * -ColumnarBeginRead(Relation rel, Snapshot snapshot, +PgColumnarReadState * +PgColumnarBeginRead(Relation rel, Snapshot snapshot, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys) { - return ColumnarBeginReadWithStorage(rel, snapshot, ColumnarStorageId(rel), + return PgColumnarBeginReadWithStorage(rel, snapshot, PgColumnarStorageId(rel), RelationGetDescr(rel), parallelScan, projectedColumns, nkeys, keys); } -ColumnarReadState * -ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, +PgColumnarReadState * +PgColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, uint64 storageId, TupleDesc tupdesc, ParallelTableScanDesc parallelScan, Bitmapset *projectedColumns, int nkeys, ScanKey keys) { - ColumnarReadState *readState; + PgColumnarReadState *readState; MemoryContext readContext; MemoryContext oldContext; @@ -312,10 +312,10 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(readContext); - readState = palloc0(sizeof(ColumnarReadState)); + readState = palloc0(sizeof(PgColumnarReadState)); readState->rel = rel; readState->snapshot = snapshot; - readState->metaSnapshot = ColumnarCatalogSnapshot(snapshot); + readState->metaSnapshot = PgColumnarCatalogSnapshot(snapshot); readState->tupdesc = tupdesc; readState->natts = readState->tupdesc->natts; readState->storageId = storageId; @@ -323,10 +323,10 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* * Reject a native data format version this build does not understand before * any bytes are decoded (#240). The metapage version was already checked when - * ColumnarStorageId read the metapage; this is the independent data-format + * PgColumnarStorageId read the metapage; this is the independent data-format * stamp, catching a future encoding change that keeps the metapage layout. */ - ColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); + PgColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); /* * Resolve each column's missing value once, for stripes that predate an @@ -348,13 +348,13 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* * Flatten the projection (#338). A NULL bitmap means "all columns" -- that is - * how columnar_projected_columns reports a whole-row Var, any system column, + * how pgcolumnar_projected_columns reports a whole-row Var, any system column, * and a query referencing no column at all (count(*)), and it is what every * caller that does not compute a projection passes. */ readState->colWanted = palloc(sizeof(bool) * readState->natts); readState->allColumnsWanted = (projectedColumns == NULL || - !columnar_enable_column_projection); + !pgcolumnar_enable_column_projection); { int pc; @@ -380,8 +380,8 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, "columnar read skip", ALLOCSET_DEFAULT_SIZES); - if (columnar_enable_qual_pushdown) - columnar_build_predicates(readState, nkeys, keys); + if (pgcolumnar_enable_qual_pushdown) + pgcolumnar_build_predicates(readState, nkeys, keys); MemoryContextSwitchTo(oldContext); return readState; @@ -389,14 +389,14 @@ ColumnarBeginReadWithStorage(Relation rel, Snapshot snapshot, /* - * columnar_build_predicates + * pgcolumnar_build_predicates * Translate the scan's ScanKeys into skip predicates for chunk-group * filtering (spec 9). Only simple, same-type btree comparison keys on an * orderable column are used; anything else is ignored, so skipping stays * conservative. Runs in readContext. */ static void -columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) +pgcolumnar_build_predicates(PgColumnarReadState *readState, int nkeys, ScanKey keys) { int i; int n = 0; @@ -448,13 +448,13 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) * For an equality predicate on a hashable column with a safe collation, * enable the bloom-filter probe (I7, gap 25), matching how the filter was * built. The scan key already matches the column collation (a - * differently collated predicate is not pushed; see ColumnarBuildScanKeys), + * differently collated predicate is not pushed; see PgColumnarBuildScanKeys), * so hashing the constant under the column collation is consistent. */ readState->predicates[n].hasHash = false; if (key->sk_strategy == BTEqualStrategyNumber && OidIsValid(tce->hash_proc_finfo.fn_oid) && - ColumnarCollationIsDeterministic(att->attcollation)) + PgColumnarCollationIsDeterministic(att->attcollation)) { readState->predicates[n].hasHash = true; fmgr_info_copy(&readState->predicates[n].hashFn, @@ -468,7 +468,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) } /* - * columnar_group_can_match + * pgcolumnar_group_can_match * Decide whether a chunk group could contain a row satisfying every * pushed-down predicate, using the stored per-chunk min/max (spec 9). A * return of false means the group can be skipped. Missing min/max, or a @@ -476,7 +476,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_setup_group + * pgcolumnar_setup_group * Position on a chunk group: decompress each projected column's value * stream into the group context and point the column cursors at the * decompressed bytes and the (uncompressed) exists bytes. Non-projected @@ -484,7 +484,7 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_position_group + * pgcolumnar_position_group * Advance from the current groupIndex to the next chunk group that could * match the pushed-down predicates, skipping groups whose min/max rule * them out (spec 9). Returns true when positioned on a readable group, @@ -492,19 +492,19 @@ columnar_build_predicates(ColumnarReadState *readState, int nkeys, ScanKey keys) */ /* - * columnar_load_stripe + * pgcolumnar_load_stripe * Read a stripe's metadata and data into memory and position at its * first chunk group. */ /* - * columnar_read_start + * pgcolumnar_read_start * Lazily load the stripe list on the first fetch. For a parallel scan a * single worker claims the whole scan and the others see it exhausted, * which is a correct (if not parallel-accelerated) behaviour. */ static void -columnar_read_start(ColumnarReadState *readState) +pgcolumnar_read_start(PgColumnarReadState *readState) { if (readState->started) return; @@ -526,7 +526,7 @@ columnar_read_start(ColumnarReadState *readState) MemoryContext oldContext = MemoryContextSwitchTo(readState->readContext); readState->rowGroupList = - ColumnarReadRowGroupList(readState->storageId, + PgColumnarReadRowGroupList(readState->storageId, readState->metaSnapshot); readState->rowGroupIndex = 0; MemoryContextSwitchTo(oldContext); @@ -534,18 +534,18 @@ columnar_read_start(ColumnarReadState *readState) } /* - * columnar_native_decode_chunk + * pgcolumnar_native_decode_chunk * Reconstruct a native column chunk's raw present-value stream (D4) from its * encoding descriptor. The on-disk values region is the per-1024-value-vector * encoded streams concatenated, optionally block-compressed as a whole. This - * reverses the block codec, then decodes each vector with ColumnarDecodeChunk + * reverses the block codec, then decodes each vector with PgColumnarDecodeChunk * into one raw buffer byte-identical to what the writer buffered, so the * per-row producer walks it exactly as it walks the D2b baseline. Allocated * in the group context. The descriptor lengths are cross-checked so a corrupt * chunk cannot drive a decoder past its buffers. */ static char * -columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, +pgcolumnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, char *values, uint32 valuesLen, const char *desc, uint32 descLen, int blockCodec, uint32 **outVecRawLen, int *outVecCount) @@ -622,7 +622,7 @@ columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, /* reverse the block codec to recover the concatenated encoded region */ if (blockCodec != COLUMNAR_COMPRESSION_NONE) - encRegion = ColumnarDecompressValueStream(values, valuesLen, blockCodec, + encRegion = PgColumnarDecompressValueStream(values, valuesLen, blockCodec, (uint32) encTotal, decodeScratch); else @@ -657,7 +657,7 @@ columnar_native_decode_chunk(MemoryContext cx, Form_pg_attribute att, vecRawLen[v] = rawLen; if (rawLen > 0) { - char *rawVec = ColumnarDecodeChunk(encCursor, encLen, encType, + char *rawVec = PgColumnarDecodeChunk(encCursor, encLen, encType, att, valueCount, rawLen, sharedTable, sharedTableLen, decodeScratch); @@ -700,9 +700,9 @@ native_zone_excludes(SkipPredicate *pred, Form_pg_attribute att, return false; cur = (char *) z->minimum; - minv = ColumnarDecodeValue(att, &cur, cx); + minv = PgColumnarDecodeValue(att, &cur, cx); cur = (char *) z->maximum; - maxv = ColumnarDecodeValue(att, &cur, cx); + maxv = PgColumnarDecodeValue(att, &cur, cx); switch (pred->strategy) { @@ -734,16 +734,16 @@ native_zone_excludes(SkipPredicate *pred, Form_pg_attribute att, } /* - * columnar_native_group_can_match + * pgcolumnar_native_group_can_match * Decide whether a native row group could hold a row satisfying every * pushed-down predicate, using its whole-chunk zone maps (native spec 7.1, * Phase D5b). Returns false when the group can be skipped. Mirrors the 2.2 - * columnar_group_can_match, reading min/max from pgcolumnar.zone_map instead + * pgcolumnar_group_can_match, reading min/max from pgcolumnar.zone_map instead * of the 2.2 chunk catalog. A missing or non-orderable zone map is treated * conservatively as "may match". Runs in rs->skipContext (caller-reset). */ static bool -columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) +pgcolumnar_native_group_can_match(PgColumnarReadState *rs, uint64 groupNumber) { List *zones; NativeZoneMapMetadata **byCol; @@ -755,7 +755,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) if (rs->numPredicates == 0) return true; - zones = ColumnarReadZoneMapList(rs->storageId, groupNumber, rs->metaSnapshot); + zones = PgColumnarReadZoneMapList(rs->storageId, groupNumber, rs->metaSnapshot); byCol = palloc0(sizeof(NativeZoneMapMetadata *) * rs->natts); foreach(lc, zones) { @@ -779,7 +779,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) * columns that min/max cannot. */ if (pred->strategy == BTEqualStrategyNumber && - columnar_enable_bloom_filter && pred->hasHash) + pgcolumnar_enable_bloom_filter && pred->hasHash) { NativeBloomMetadata *b; @@ -811,7 +811,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) if (!bloomLookedUp[pred->attidx]) { byColBloom[pred->attidx] = - ColumnarReadBloomForColumn(rs->storageId, groupNumber, + PgColumnarReadBloomForColumn(rs->storageId, groupNumber, pred->attidx, rs->metaSnapshot); bloomLookedUp[pred->attidx] = true; } @@ -823,7 +823,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) FunctionCall1Coll(&pred->hashFn, pred->hashCollation, pred->compareValue)); - if (!ColumnarBloomProbe(b->filter, b->filterLen, h)) + if (!PgColumnarBloomProbe(b->filter, b->filterLen, h)) return false; } } @@ -833,7 +833,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) } /* - * columnar_native_build_skipvec + * pgcolumnar_native_build_skipvec * Build the per-vector skip flags for a loaded row group (native spec 7.1, * Phase D5b): vector v is skipped when any predicate's per-vector zone map * proves no row in it can match. Also fills rs->nativeVecStart with the @@ -844,7 +844,7 @@ columnar_native_group_can_match(ColumnarReadState *rs, uint64 groupNumber) * min/max in rs->skipContext. */ static void -columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vecCount) +pgcolumnar_native_build_skipvec(PgColumnarReadState *rs, uint64 groupNumber, int vecCount) { List *zones; NativeZoneMapMetadata ***byColVec; @@ -863,7 +863,7 @@ columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vec if (rs->numPredicates == 0 || vecCount <= 0) return; - zones = ColumnarReadZoneMapVectors(rs->storageId, groupNumber, rs->metaSnapshot); + zones = PgColumnarReadZoneMapVectors(rs->storageId, groupNumber, rs->metaSnapshot); if (zones == NIL) return; /* legacy: no per-vector zone maps */ @@ -921,21 +921,21 @@ columnar_native_build_skipvec(ColumnarReadState *rs, uint64 groupNumber, int vec } /* a half-open span of the row group's bytes, used to build coalesced reads */ -typedef struct ColumnarByteRange +typedef struct PgColumnarByteRange { uint64 start; uint64 end; -} ColumnarByteRange; +} PgColumnarByteRange; /* - * columnar_byte_range_cmp + * pgcolumnar_byte_range_cmp * Order byte ranges by start offset, so adjacent ones can be coalesced. */ static int -columnar_byte_range_cmp(const void *a, const void *b) +pgcolumnar_byte_range_cmp(const void *a, const void *b) { - uint64 sa = ((const ColumnarByteRange *) a)->start; - uint64 sb = ((const ColumnarByteRange *) b)->start; + uint64 sa = ((const PgColumnarByteRange *) a)->start; + uint64 sb = ((const PgColumnarByteRange *) b)->start; if (sa < sb) return -1; @@ -945,21 +945,21 @@ columnar_byte_range_cmp(const void *a, const void *b) } /* - * columnar_native_read_projected + * pgcolumnar_native_read_projected * Read only the byte ranges the projected columns occupy (#338), rather * than the whole row group. * * Ranges that touch or overlap in the file are coalesced, so a projection * covering neighbouring columns costs one read rather than one per column. - * Chunks are written column-major (columnar_write_state.c), so in practice + * Chunks are written column-major (pgcolumnar_write_state.c), so in practice * a projection is a small number of runs. Everything lands at its natural * offset inside the full-size group buffer, leaving the rest untouched. */ static void -columnar_native_read_projected(ColumnarReadState *rs, +pgcolumnar_native_read_projected(PgColumnarReadState *rs, NativeRowGroupMetadata *rg, List *chunks) { - ColumnarByteRange *ranges; + PgColumnarByteRange *ranges; uint64 groupEnd = rg->fileOffset + rg->byteLength; uint64 minStart = groupEnd; uint64 maxEnd = rg->fileOffset; @@ -971,8 +971,8 @@ columnar_native_read_projected(ColumnarReadState *rs, if (chunks == NIL) return; - ranges = (ColumnarByteRange *) - palloc(sizeof(ColumnarByteRange) * list_length(chunks)); + ranges = (PgColumnarByteRange *) + palloc(sizeof(PgColumnarByteRange) * list_length(chunks)); foreach(lc, chunks) { @@ -1052,7 +1052,7 @@ columnar_native_read_projected(ColumnarReadState *rs, if (n == 0) return; - qsort(ranges, n, sizeof(ColumnarByteRange), columnar_byte_range_cmp); + qsort(ranges, n, sizeof(PgColumnarByteRange), pgcolumnar_byte_range_cmp); for (i = 0; i < n;) { @@ -1067,7 +1067,7 @@ columnar_native_read_projected(ColumnarReadState *rs, j++; } - ColumnarReadLogicalData(rs->rel, start, + PgColumnarReadLogicalData(rs->rel, start, rs->nativeBuffer + (start - rg->fileOffset), end - start); i = j; @@ -1077,7 +1077,7 @@ columnar_native_read_projected(ColumnarReadState *rs, } /* - * columnar_native_load_group + * pgcolumnar_native_load_group * Load the next native row group (PGCN v1, Phase D3): read the bytes of the * projected columns into the group context and set each such column's * validity-bitmap pointer and values cursor. Row groups the zone maps prove @@ -1089,7 +1089,7 @@ columnar_native_read_projected(ColumnarReadState *rs, * left unmaterialised and the row loop emits NULL for them. */ static bool -columnar_native_load_group(ColumnarReadState *rs) +pgcolumnar_native_load_group(PgColumnarReadState *rs) { MemoryContext oldContext; NativeRowGroupMetadata *rg; @@ -1102,7 +1102,7 @@ columnar_native_load_group(ColumnarReadState *rs) /* * Claim the next row group and advance past any the zone maps rule out * (native spec 7.1). Under a parallel custom scan each worker claims distinct - * groups from the shared counter (columnar_next_group_index), so a group is + * groups from the shared counter (pgcolumnar_next_group_index), so a group is * read by exactly one backend; serially it walks rowGroupIndex. Without the * counter every worker read every group and a parallel scan returned each row * once per participating backend (D6e). @@ -1110,7 +1110,7 @@ columnar_native_load_group(ColumnarReadState *rs) rg = NULL; for (;;) { - int64 gi = columnar_next_group_index(rs); + int64 gi = pgcolumnar_next_group_index(rs); bool match = true; if (gi < 0) @@ -1118,14 +1118,14 @@ columnar_native_load_group(ColumnarReadState *rs) rg = (NativeRowGroupMetadata *) list_nth(rs->rowGroupList, (int) gi); if (rs->restrictGroups != NULL && - !columnar_group_is_restricted_in(rs, rg->groupNumber)) + !pgcolumnar_group_is_restricted_in(rs, rg->groupNumber)) match = false; else if (rs->numPredicates > 0) { MemoryContext old = MemoryContextSwitchTo(rs->skipContext); MemoryContextReset(rs->skipContext); - match = columnar_native_group_can_match(rs, rg->groupNumber); + match = pgcolumnar_native_group_can_match(rs, rg->groupNumber); MemoryContextSwitchTo(old); } if (match) @@ -1145,7 +1145,7 @@ columnar_native_load_group(ColumnarReadState *rs) * per-column byte ranges the projected read needs. It is a catalog read and * touches none of the group's data pages. */ - chunks = ColumnarReadColumnChunkList(rs->storageId, rg->groupNumber, + chunks = PgColumnarReadColumnChunkList(rs->storageId, rg->groupNumber, rs->metaSnapshot); /* @@ -1159,10 +1159,10 @@ columnar_native_load_group(ColumnarReadState *rs) if (rg->byteLength > 0) { if (rs->allColumnsWanted) - ColumnarReadLogicalData(rs->rel, rg->fileOffset, rs->nativeBuffer, + PgColumnarReadLogicalData(rs->rel, rg->fileOffset, rs->nativeBuffer, rg->byteLength); else - columnar_native_read_projected(rs, rg, chunks); + pgcolumnar_native_read_projected(rs, rg, chunks); } rs->nativeValidity = palloc0(sizeof(char *) * rs->natts); @@ -1211,7 +1211,7 @@ columnar_native_load_group(ColumnarReadState *rs) /* D4: reconstruct the raw present-value stream from the descriptor */ rs->nativeValueCursor[cc->columnIndex] = - columnar_native_decode_chunk(rs->groupContext, att, base + validityBytes, + pgcolumnar_native_decode_chunk(rs->groupContext, att, base + validityBytes, (uint32) (cc->pageLength - validityBytes), cc->encodingDescriptor, cc->encodingDescriptorLen, @@ -1228,7 +1228,7 @@ columnar_native_load_group(ColumnarReadState *rs) * vector boundaries line up); a legacy baseline chunk disables it. */ if (allDescriptor) - columnar_native_build_skipvec(rs, rg->groupNumber, maxVecCount); + pgcolumnar_native_build_skipvec(rs, rg->groupNumber, maxVecCount); else { rs->nativeSkipVec = NULL; @@ -1240,12 +1240,12 @@ columnar_native_load_group(ColumnarReadState *rs) /* * Native delete visibility (D6b): combine this group's row-mask rows (keyed * by group number, one bit per row-in-group) into a single delete mask that - * columnar_native_next_row consults to skip deleted rows. + * pgcolumnar_native_next_row consults to skip deleted rows. */ rs->nativeDeleteMask = NULL; rs->nativeDeleteMaskLen = 0; { - List *maskList = ColumnarReadDeleteVectorList(rs->storageId, + List *maskList = PgColumnarReadDeleteVectorList(rs->storageId, rg->groupNumber, rs->metaSnapshot); ListCell *mlc; @@ -1276,7 +1276,7 @@ columnar_native_load_group(ColumnarReadState *rs) } /* - * columnar_native_skip_current_vector + * pgcolumnar_native_skip_current_vector * Per-vector skipping (native spec 7.1, D5b): when rowInGroup sits at the * start of a vector the zone maps rule out, step each column's value cursor * past that vector's decoded bytes and jump rowInGroup to the next vector, @@ -1284,7 +1284,7 @@ columnar_native_load_group(ColumnarReadState *rs) * caller re-checks bounds), false when the current row must be emitted. */ static bool -columnar_native_skip_current_vector(ColumnarReadState *rs) +pgcolumnar_native_skip_current_vector(PgColumnarReadState *rs) { int v = rs->nativeCurVec; int V = rs->nativeVectorCount; @@ -1309,14 +1309,14 @@ columnar_native_skip_current_vector(ColumnarReadState *rs) } /* - * columnar_native_next_row + * pgcolumnar_native_next_row * Native-format sequential row production (Phase D3). Decodes one row from * the current row group, reconstructing each column from its validity bit * and, when present, the next value on its cursor. Vectors the zone maps rule * out are stepped over without decoding (Phase D5b). */ static bool -columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, +pgcolumnar_native_next_row(PgColumnarReadState *rs, Datum *values, bool *nulls, uint64 *rowNumber) { MemoryContext oldContext; @@ -1341,7 +1341,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, if (rs->nativeGroup == NULL || rs->rowInGroup >= rs->groupRowCount) { - if (!columnar_native_load_group(rs)) + if (!pgcolumnar_native_load_group(rs)) { rs->exhausted = true; return false; @@ -1349,7 +1349,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, } if (rs->nativeSkipVec != NULL && - columnar_native_skip_current_vector(rs)) + pgcolumnar_native_skip_current_vector(rs)) continue; /* stepped past a ruled-out vector; re-check */ /* @@ -1394,7 +1394,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, { /* * Fast path (#289): inline the attbyval decode. This is exactly - * what ColumnarDecodeValue does for a by-value type -- one + * what PgColumnarDecodeValue does for a by-value type -- one * fetch_att and advance attlen -- but skips the out-of-line call * and its own attbyval branch, which is the per-row decode * dispatch #289 profiled as hot. It works for both baseline and @@ -1411,7 +1411,7 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, rs->nativeValueCursor[c] = p + att->attlen; } else - values[c] = ColumnarDecodeValue(att, + values[c] = PgColumnarDecodeValue(att, &rs->nativeValueCursor[c], rs->rowContext); nulls[c] = false; @@ -1441,37 +1441,37 @@ columnar_native_next_row(ColumnarReadState *rs, Datum *values, bool *nulls, } bool -ColumnarReadNextRow(ColumnarReadState *readState, Datum *values, bool *nulls, +PgColumnarReadNextRow(PgColumnarReadState *readState, Datum *values, bool *nulls, uint64 *rowNumber) { - columnar_read_start(readState); - return columnar_native_next_row(readState, values, nulls, rowNumber); + pgcolumnar_read_start(readState); + return pgcolumnar_native_next_row(readState, values, nulls, rowNumber); } /* ------------------------------------------------------------------------- * Batch-fold accessors (#289) * * These expose the current loaded group's decoded buffer so an ungrouped - * aggregate can fold it column-at-a-time, without columnar_native_next_row + * aggregate can fold it column-at-a-time, without pgcolumnar_native_next_row * producing one Datum tuple per row. Correctness is the caller's: it must walk * each column's validity bitmap to map a row to its packed value (nulls have no * slot), honor the delete mask, and step the per-column present index past a * ruled-out vector. Only fixed-width by-value columns can be read this way; the - * caller checks that from the tuple descriptor before using ColumnarReadFoldColumn. + * caller checks that from the tuple descriptor before using PgColumnarReadFoldColumn. * ------------------------------------------------------------------------- */ /* * Advance to the next row group to fold, loading it (and honoring restrict, - * parallel and zone-map group skipping via columnar_native_load_group). Returns + * parallel and zone-map group skipping via pgcolumnar_native_load_group). Returns * false at end of scan; on true, the accessors below describe the loaded group. */ bool -ColumnarReadFoldNextGroup(ColumnarReadState *readState) +PgColumnarReadFoldNextGroup(PgColumnarReadState *readState) { - columnar_read_start(readState); + pgcolumnar_read_start(readState); if (readState->exhausted) return false; - if (!columnar_native_load_group(readState)) + if (!pgcolumnar_native_load_group(readState)) { readState->exhausted = true; return false; @@ -1487,7 +1487,7 @@ ColumnarReadFoldNextGroup(ColumnarReadState *readState) * the vector count. */ void -ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, +PgColumnarReadFoldGroupInfo(PgColumnarReadState *readState, uint64 *nrows, const char **deleteMask, uint32 *deleteMaskLen, const bool **skipVec, const uint32 **vecStart, int *vectorCount) @@ -1509,7 +1509,7 @@ ColumnarReadFoldGroupInfo(ColumnarReadState *readState, uint64 *nrows, * COLUMN); the caller then folds it from the missing value or falls back. */ bool -ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, +PgColumnarReadFoldColumn(PgColumnarReadState *readState, int attidx, const char **validity, const char **packed, int16 *attlen, const uint32 **vecRawLen) { @@ -1526,14 +1526,14 @@ ColumnarReadFoldColumn(ColumnarReadState *readState, int attidx, } /* - * columnar_next_group_index + * pgcolumnar_next_group_index * The next native row group to scan, or -1 when none remain. The native - * counterpart of columnar_next_stripe_index: a parallel custom scan claims + * counterpart of pgcolumnar_next_stripe_index: a parallel custom scan claims * it from the shared atomic so each worker reads distinct row groups (gap * 23, D6e); a serial scan walks rowGroupIndex. */ static int64 -columnar_next_group_index(ColumnarReadState *readState) +pgcolumnar_next_group_index(PgColumnarReadState *readState) { int ngroups = list_length(readState->rowGroupList); uint32 gi; @@ -1547,26 +1547,26 @@ columnar_next_group_index(ColumnarReadState *readState) } void -ColumnarReadSetParallelCounter(ColumnarReadState *readState, +PgColumnarReadSetParallelCounter(PgColumnarReadState *readState, pg_atomic_uint32 *counter) { readState->parallelCounter = counter; } /* - * ColumnarReadRestrictToGroups + * PgColumnarReadRestrictToGroups * Restrict this scan to the given row group numbers (issue #149). Groups * outside the set are skipped in the claim loop, so their bytes are never * read and their column chunks never decoded. The array is copied into the * read state's own context and sorted there, so the caller may free its own. * - * Must be called before the first ColumnarReadNextRow. Passing ngroups == 0 + * Must be called before the first PgColumnarReadNextRow. Passing ngroups == 0 * makes the scan return no rows, which is the honest reading of "restrict to * nothing" and is what the aggregate path relies on when every group is * clean. */ void -ColumnarReadRestrictToGroups(ColumnarReadState *readState, +PgColumnarReadRestrictToGroups(PgColumnarReadState *readState, const uint64 *groupNumbers, int ngroups) { MemoryContext oldContext; @@ -1582,7 +1582,7 @@ ColumnarReadRestrictToGroups(ColumnarReadState *readState, memcpy(readState->restrictGroups, groupNumbers, sizeof(uint64) * ngroups); qsort(readState->restrictGroups, ngroups, sizeof(uint64), - columnar_uint64_cmp); + pgcolumnar_uint64_cmp); } MemoryContextSwitchTo(oldContext); } @@ -1592,7 +1592,7 @@ ColumnarReadRestrictToGroups(ColumnarReadState *readState, * row number for deletion/visibility. The cache reads the base row-group list * and delete vectors once (at the scan's snapshot) into memory, then answers each * test with a binary search over row groups plus a bitmap probe. Consistent - * with the scan's fixed snapshot, the same way ColumnarBeginRead reads those + * with the scan's fixed snapshot, the same way PgColumnarBeginRead reads those * lists once at begin. * ------------------------------------------------------------------------- */ typedef struct LiveStripeEntry @@ -1605,7 +1605,7 @@ typedef struct LiveStripeEntry uint32 *maskLens; /* [chunkGroupCount] */ } LiveStripeEntry; -struct ColumnarLivenessCache +struct PgColumnarLivenessCache { LiveStripeEntry *stripes; /* sorted ascending by firstRowNumber */ int nstripes; @@ -1625,17 +1625,17 @@ livestripe_cmp(const void *a, const void *b) return 0; } -ColumnarLivenessCache * -ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) +PgColumnarLivenessCache * +PgColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) { - uint64 storageId = ColumnarStorageId(rel); - Snapshot metaSnapshot = ColumnarCatalogSnapshot(snapshot); + uint64 storageId = PgColumnarStorageId(rel); + Snapshot metaSnapshot = PgColumnarCatalogSnapshot(snapshot); MemoryContext ctx = AllocSetContextCreate(CurrentMemoryContext, "columnar liveness cache", ALLOCSET_DEFAULT_SIZES); MemoryContext oldContext = MemoryContextSwitchTo(ctx); - List *rgList = ColumnarReadRowGroupList(storageId, metaSnapshot); - ColumnarLivenessCache *cache = palloc0(sizeof(ColumnarLivenessCache)); + List *rgList = PgColumnarReadRowGroupList(storageId, metaSnapshot); + PgColumnarLivenessCache *cache = palloc0(sizeof(PgColumnarLivenessCache)); ListCell *lc; int i = 0; @@ -1643,7 +1643,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) * Each native row group is one liveness entry with a single whole-group * delete mask (the delete vector is keyed by group number, chunk id 0). Modeling * it as chunkGroupCount 1 with chunkRowCount == rowCount makes the shared - * ColumnarLivenessCacheIsLive map every row to chunk 0. + * PgColumnarLivenessCacheIsLive map every row to chunk 0. */ cache->ctx = ctx; cache->nstripes = list_length(rgList); @@ -1664,7 +1664,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) e->masks = palloc0(sizeof(char *) * 1); e->maskLens = palloc0(sizeof(uint32) * 1); - rml = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); + rml = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); foreach(mc, rml) { DeleteVectorMetadata *rm = (DeleteVectorMetadata *) lfirst(mc); @@ -1691,7 +1691,7 @@ ColumnarBuildLivenessCache(Relation rel, Snapshot snapshot) } bool -ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, uint64 rowNumber) +PgColumnarLivenessCacheIsLive(PgColumnarLivenessCache *cache, uint64 rowNumber) { int lo = 0; int hi = cache->nstripes - 1; @@ -1724,14 +1724,14 @@ ColumnarLivenessCacheIsLive(ColumnarLivenessCache *cache, uint64 rowNumber) } void -ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) +PgColumnarFreeLivenessCache(PgColumnarLivenessCache *cache) { if (cache != NULL) MemoryContextDelete(cache->ctx); } /* - * ColumnarReadRowByNumber + * PgColumnarReadRowByNumber * Fetch a single row addressed by its row number (spec 6). Used by the * table AM's fetch-by-tid callback (UPDATE re-fetches the old row). Reads * only the one chunk group that holds the row and decodes each column up @@ -1742,7 +1742,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) /* ------------------------------------------------------------------------- * Statement-scoped decoded row-group cache (issue #143). * - * ColumnarReadRowByNumber() read and decoded a whole row group to return one + * PgColumnarReadRowByNumber() read and decoded a whole row group to return one * row, so fetching N rows out of one group cost N times the group: measured at * 878 ms for 5,000 rows, 4,452 ms for 10,000 and 19,211 ms for 20,000, all in a * single group. Every index scan, bitmap scan, and index-driven UPDATE or DELETE @@ -1773,7 +1773,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) * decoded column values are held here. * * Entries live in contexts under TopTransactionContext, so an abort or commit - * frees them without a hook; ColumnarDiscardFetchCache() clears the descriptors + * frees them without a hook; PgColumnarDiscardFetchCache() clears the descriptors * to match. * ------------------------------------------------------------------------- */ @@ -1804,7 +1804,7 @@ ColumnarFreeLivenessCache(ColumnarLivenessCache *cache) * crossing the cap costs the overflow fraction rather than everything. */ -typedef struct ColumnarFetchGroup +typedef struct PgColumnarFetchGroup { MemoryContext cx; /* holds every pointer below; NULL when free */ uint64 storageId; @@ -1855,18 +1855,18 @@ typedef struct ColumnarFetchGroup MemoryContext *colCx; /* [natts] */ bool *overflow; /* [natts] */ uint64 lastUsed; -} ColumnarFetchGroup; +} PgColumnarFetchGroup; -static ColumnarFetchGroup columnarFetchCache[COLUMNAR_FETCH_CACHE_ENTRIES]; +static PgColumnarFetchGroup columnarFetchCache[COLUMNAR_FETCH_CACHE_ENTRIES]; static uint64 columnarFetchClock = 0; /* * Memoized native-format-version validation for the by-row-number fetch path - * (#240). columnar_fetch_row runs once per row, so checking the storage's + * (#240). pgcolumnar_fetch_row runs once per row, so checking the storage's * format_version against a catalog row on every call would be a per-row systable * scan on a hot path. The value is immutable for a storage, so one check per * (command, storageId) is enough; this single slot is cleared in - * ColumnarDiscardFetchCache, the same command/transaction boundary the fetch + * PgColumnarDiscardFetchCache, the same command/transaction boundary the fetch * cache resets on. */ static uint64 columnarFetchFmtOkStorageId = 0; @@ -1877,7 +1877,7 @@ static bool columnarFetchFmtOk = false; #define COLUMNAR_RANK_BLOCK_BYTES (COLUMNAR_RANK_BLOCK_ROWS / 8) /* - * columnar_build_rank_prefix + * pgcolumnar_build_rank_prefix * Cumulative count of set validity bits at each 64-row boundary, so the * number of present values before an arbitrary row can be had without * walking to it. Entry b counts the bits below row b * 64; the array has one @@ -1887,7 +1887,7 @@ static bool columnarFetchFmtOk = false; * chunk does not hold. */ static uint32 * -columnar_build_rank_prefix(const char *vbits, uint64 rowCount) +pgcolumnar_build_rank_prefix(const char *vbits, uint64 rowCount) { uint64 nblocks = (rowCount + COLUMNAR_RANK_BLOCK_ROWS - 1) / COLUMNAR_RANK_BLOCK_ROWS; @@ -1921,13 +1921,13 @@ columnar_build_rank_prefix(const char *vbits, uint64 rowCount) } /* - * columnar_rank_before + * pgcolumnar_rank_before * How many values are present in this column before row `row` of the group. * The block prefix plus at most eight byte lookups, in place of a loop over * every earlier row. */ static inline uint64 -columnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) +pgcolumnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) { uint64 blk = row / COLUMNAR_RANK_BLOCK_ROWS; uint64 rank = prefix[blk]; @@ -1944,14 +1944,14 @@ columnar_rank_before(const char *vbits, const uint32 *prefix, uint64 row) } /* - * columnar_build_val_offsets + * pgcolumnar_build_val_offsets * Byte offset of every present value in a decoded varying-length stream. * One pass over the values, paid once per column per cached group, in place * of a partial pass on every fetch. Fixed-length columns never call this: * their k-th value is at k * attlen and needs no table. */ static uint32 * -columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) +pgcolumnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) { uint32 *offsets = (uint32 *) palloc(sizeof(uint32) * (nvalues + 1)); char *cursor = rawBuf; @@ -1960,7 +1960,7 @@ columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) for (k = 0; k < nvalues; k++) { offsets[k] = (uint32) (cursor - rawBuf); - cursor += ColumnarVarSizeAnyUnaligned(cursor); + cursor += PgColumnarVarSizeAnyUnaligned(cursor); /* * A chunk holds as many values as chunk_group_row_limit allows, which is @@ -1977,7 +1977,7 @@ columnar_build_val_offsets(Form_pg_attribute att, char *rawBuf, uint32 nvalues) /* drop one entry and everything it holds */ static void -columnar_fetch_entry_reset(ColumnarFetchGroup *e) +pgcolumnar_fetch_entry_reset(PgColumnarFetchGroup *e) { if (e->cx != NULL) MemoryContextDelete(e->cx); @@ -1985,13 +1985,13 @@ columnar_fetch_entry_reset(ColumnarFetchGroup *e) } /* - * ColumnarDiscardFetchCache + * PgColumnarDiscardFetchCache * Forget every cached group. The contexts hang off TopTransactionContext * and are already gone by the time this runs at transaction end, so this * only clears the descriptors that pointed at them. */ void -ColumnarDiscardFetchCache(void) +PgColumnarDiscardFetchCache(void) { memset(columnarFetchCache, 0, sizeof(columnarFetchCache)); columnarFetchClock = 0; @@ -2003,18 +2003,18 @@ ColumnarDiscardFetchCache(void) * Returns NULL when nothing should be cached, in which case the caller decodes * into its own scratch context exactly as before. */ -static ColumnarFetchGroup * -columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) +static PgColumnarFetchGroup * +pgcolumnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) { CommandId cid = GetCurrentCommandId(false); - ColumnarFetchGroup *victim = NULL; + PgColumnarFetchGroup *victim = NULL; int i; *hit = false; for (i = 0; i < COLUMNAR_FETCH_CACHE_ENTRIES; i++) { - ColumnarFetchGroup *e = &columnarFetchCache[i]; + PgColumnarFetchGroup *e = &columnarFetchCache[i]; if (e->cx == NULL) { @@ -2025,7 +2025,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) /* an entry from an earlier command can never be used again */ if (e->cid != cid) { - columnar_fetch_entry_reset(e); + pgcolumnar_fetch_entry_reset(e); if (victim == NULL) victim = e; continue; @@ -2049,7 +2049,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) oldest = columnarFetchCache[i].lastUsed; victim = &columnarFetchCache[i]; } - columnar_fetch_entry_reset(victim); + pgcolumnar_fetch_entry_reset(victim); } victim->cx = AllocSetContextCreate(TopTransactionContext, @@ -2063,7 +2063,7 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) } /* - * columnar_fetch_row + * pgcolumnar_fetch_row * Shared worker behind the three fetch entry points below. * * Which columns to decode is said two ways, and deliberately not one. @@ -2080,11 +2080,11 @@ columnar_fetch_group_slot(uint64 storageId, uint64 groupNumber, bool *hit) * without touching the group's bytes at all. */ static bool -columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, +pgcolumnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, bool allColumns, Bitmapset *needed, bool wantValues) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); TupleDesc tupdesc = RelationGetDescr(rel); int natts = tupdesc->natts; MemoryContext target = CurrentMemoryContext; @@ -2093,7 +2093,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, Snapshot metaSnapshot; List *rgList; NativeRowGroupMetadata *rg = NULL; - ColumnarFetchGroup *entry; + PgColumnarFetchGroup *entry; bool hit; int validityBytes; uint64 rowInGrp; @@ -2101,9 +2101,9 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, int c; /* - * columnar_fetch_row is called once per item pointer by the executor -- per + * pgcolumnar_fetch_row is called once per item pointer by the executor -- per * row on an index or bitmap scan, and per duplicate by _bt_check_unique() - * while it holds the index page (see columnar_metadata.c). None of those + * while it holds the index page (see pgcolumnar_metadata.c). None of those * callers checks for interrupts between fetches, and each fetch reads the * row-group list out of the catalog, so a statement that fetches many rows * spends its whole time in here. Without a check the loop is uncancellable @@ -2116,7 +2116,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, /* * Reject a native format_version this build does not understand before * decoding any bytes (#240). The scan-open paths check this in - * ColumnarBeginReadWithStorage / ColumnarBeginAggScan, but the by-row-number + * PgColumnarBeginReadWithStorage / PgColumnarBeginAggScan, but the by-row-number * fetch path -- an index scan returning a decoded column, UPDATE re-fetching * the old row, and the vacuum row reader -- reaches neither. Only an actual * decode is guarded (wantValues); a visibility-only probe decodes nothing and @@ -2126,7 +2126,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, if (wantValues && !(columnarFetchFmtOk && columnarFetchFmtOkStorageId == storageId)) { - ColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); + PgColumnarCheckNativeFormatVersion(storageId, RelationGetRelationName(rel)); columnarFetchFmtOkStorageId = storageId; columnarFetchFmtOk = true; } @@ -2135,7 +2135,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, ALLOCSET_SMALL_SIZES); oldContext = MemoryContextSwitchTo(tmp); - metaSnapshot = ColumnarCatalogSnapshot(snapshot); + metaSnapshot = PgColumnarCatalogSnapshot(snapshot); /* * Native (PGCN v1) fetch-by-row-number: find the row group covering the row @@ -2146,7 +2146,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * The row-group list is read per fetch and deliberately not cached: a group * flushed earlier in this same statement has to become visible here. */ - rgList = ColumnarReadRowGroupList(storageId, metaSnapshot); + rgList = PgColumnarReadRowGroupList(storageId, metaSnapshot); foreach(nlc, rgList) { NativeRowGroupMetadata *g = (NativeRowGroupMetadata *) lfirst(nlc); @@ -2183,11 +2183,11 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, } else { - List *maskList = ColumnarReadDeleteVectorList(storageId, + List *maskList = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, metaSnapshot); ListCell *mlc; - bool deleted = ColumnarDeleteVectorBufferedDeleted(rel, rowNumber); + bool deleted = PgColumnarDeleteVectorBufferedDeleted(rel, rowNumber); foreach(mlc, maskList) { @@ -2223,7 +2223,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * statement-scoped cache above. A miss fills the entry; a hit skips the read * and the decode entirely. */ - entry = columnar_fetch_group_slot(storageId, rg->groupNumber, &hit); + entry = pgcolumnar_fetch_group_slot(storageId, rg->groupNumber, &hit); /* * The geometry the entry was filled with has to match the group just read @@ -2241,8 +2241,8 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, entry->fileOffset != rg->fileOffset || entry->natts != natts)) { - columnar_fetch_entry_reset(entry); - entry = columnar_fetch_group_slot(storageId, rg->groupNumber, &hit); + pgcolumnar_fetch_entry_reset(entry); + entry = pgcolumnar_fetch_group_slot(storageId, rg->groupNumber, &hit); Assert(!hit); } @@ -2265,10 +2265,10 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, MemoryContextSwitchTo(tmp); if (rg->byteLength > 0) - ColumnarReadLogicalData(rel, rg->fileOffset, entry->groupBuffer, + PgColumnarReadLogicalData(rel, rg->fileOffset, entry->groupBuffer, rg->byteLength); - nchunks = ColumnarReadColumnChunkList(storageId, rg->groupNumber, + nchunks = PgColumnarReadColumnChunkList(storageId, rg->groupNumber, metaSnapshot); foreach(nlc, nchunks) { @@ -2384,7 +2384,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, rawBuf = base + validityBytes; else rawBuf = - columnar_native_decode_chunk(decCx, att, + pgcolumnar_native_decode_chunk(decCx, att, base + validityBytes, (uint32) (cc->pageLength - validityBytes), cc->encodingDescriptor, @@ -2409,7 +2409,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, { MemoryContext idxOld = MemoryContextSwitchTo(entry->cx); - entry->rankPrefix[c] = columnar_build_rank_prefix(vbits, + entry->rankPrefix[c] = pgcolumnar_build_rank_prefix(vbits, entry->rowCount); if (att->attlen < 0) { @@ -2418,7 +2418,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, COLUMNAR_RANK_BLOCK_ROWS; entry->valOffset[c] = - columnar_build_val_offsets(att, rawBuf, + pgcolumnar_build_val_offsets(att, rawBuf, entry->rankPrefix[c][nblocks]); } MemoryContextSwitchTo(idxOld); @@ -2439,14 +2439,14 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * how far into the group the row is, which is what made a cache hit * proportional to the row's position before. */ - present = columnar_rank_before(vbits, entry->rankPrefix[c], rowInGrp); + present = pgcolumnar_rank_before(vbits, entry->rankPrefix[c], rowInGrp); if (att->attlen > 0) cursor = rawBuf + present * (uint64) att->attlen; else cursor = rawBuf + entry->valOffset[c][present]; - values[c] = ColumnarDecodeValue(att, &cursor, target); + values[c] = PgColumnarDecodeValue(att, &cursor, target); nulls[c] = false; /* @@ -2464,7 +2464,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * * It is safe because nothing handed back points into the column: the * value returned was copied into the caller's context by the - * ColumnarDecodeValue call immediately above, and the position indexes + * PgColumnarDecodeValue call immediately above, and the position indexes * live in entry->cx rather than in the column's own context. */ if (justDecoded && entry->colCx[c] != NULL && @@ -2509,7 +2509,7 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, * of keeping them. */ if (rg->byteLength > COLUMNAR_FETCH_CACHE_MAX_BYTES) - columnar_fetch_entry_reset(entry); + pgcolumnar_fetch_entry_reset(entry); MemoryContextSwitchTo(oldContext); MemoryContextDelete(tmp); @@ -2517,24 +2517,24 @@ columnar_fetch_row(Relation rel, Snapshot snapshot, uint64 rowNumber, } /* - * ColumnarReadRowByNumber + * PgColumnarReadRowByNumber * Reconstruct every column of the row addressed by a row number. False when * the row is not visible. */ bool -ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, +PgColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls) { - return columnar_fetch_row(rel, snapshot, rowNumber, values, nulls, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, values, nulls, true, NULL, true); } /* - * ColumnarReadRowByNumberCols + * PgColumnarReadRowByNumberCols * Decode exactly the columns in `needed`; every other column reads as null. * An empty or NULL set therefore decodes nothing, which is what it says * rather than a silent "everything" -- for every column, call - * ColumnarReadRowByNumber, which takes no set and cannot be misread. + * PgColumnarReadRowByNumber, which takes no set and cannot be misread. * * Decoding every column whatever the caller wanted is not merely wasted * work on a wide table. The decoded bytes are measured against the fetch @@ -2543,30 +2543,30 @@ ColumnarReadRowByNumber(Relation rel, Snapshot snapshot, uint64 rowNumber, * remove (issue #157). */ bool -ColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, +PgColumnarReadRowByNumberCols(Relation rel, Snapshot snapshot, uint64 rowNumber, Datum *values, bool *nulls, Bitmapset *needed) { - return columnar_fetch_row(rel, snapshot, rowNumber, values, nulls, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, values, nulls, false, needed, true); } /* - * ColumnarRowIsLive + * PgColumnarRowIsLive * Is the row visible? Decodes nothing. * - * columnar_index_delete_tuples asks exactly this, once per candidate index + * pgcolumnar_index_delete_tuples asks exactly this, once per candidate index * tuple on a path nbtree drives during deletion, and answered it by * reconstructing every column and freeing the result unread. */ bool -ColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber) +PgColumnarRowIsLive(Relation rel, Snapshot snapshot, uint64 rowNumber) { - return columnar_fetch_row(rel, snapshot, rowNumber, NULL, NULL, + return pgcolumnar_fetch_row(rel, snapshot, rowNumber, NULL, NULL, false, NULL, false); } void -ColumnarRescanRead(ColumnarReadState *readState) +PgColumnarRescanRead(PgColumnarReadState *readState) { MemoryContextReset(readState->stripeContext); readState->started = false; @@ -2589,18 +2589,18 @@ ColumnarRescanRead(ColumnarReadState *readState) } void -ColumnarEndRead(ColumnarReadState *readState) +PgColumnarEndRead(PgColumnarReadState *readState) { MemoryContextDelete(readState->readContext); } /* - * ColumnarReadStats + * PgColumnarReadStats * Report how many chunk groups the scan has read versus skipped by the * min/max skip lists (spec 9). Used by the custom scan's EXPLAIN output. */ void -ColumnarReadStats(ColumnarReadState *readState, uint64 *groupsRead, +PgColumnarReadStats(PgColumnarReadState *readState, uint64 *groupsRead, uint64 *groupsSkipped, uint64 *groupsTotal) { *groupsRead = readState->groupsRead; @@ -2609,12 +2609,12 @@ ColumnarReadStats(ColumnarReadState *readState, uint64 *groupsRead, } /* - * ColumnarVectorsSkipped + * PgColumnarVectorsSkipped * How many 1024-value vectors the native scan skipped within read row groups * via per-vector zone maps (native spec 7.1, D5b). Used by EXPLAIN. */ uint64 -ColumnarVectorsSkipped(ColumnarReadState *readState) +PgColumnarVectorsSkipped(PgColumnarReadState *readState) { return readState->vectorsSkipped; } diff --git a/src/columnar_storage.c b/src/columnar_storage.c index 0da5afe..2bce800 100644 --- a/src/columnar_storage.c +++ b/src/columnar_storage.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_storage.c + * pgcolumnar_storage.c * Physical storage layer for pgColumnar: metapage, the logical-to- * physical byte mapping, and the append-only reservation model. * @@ -35,30 +35,30 @@ /* the metapage struct lives right after the page header on block 0 */ #define COLUMNAR_METAPAGE_BLOCKNO 0 #define COLUMNAR_EMPTY_BLOCKNO 1 -#define ColumnarMetapagePointer(page) ((ColumnarMetapage *) PageGetContents(page)) +#define PgColumnarMetapagePointer(page) ((PgColumnarMetapage *) PageGetContents(page)) /* - * Stream/prefetch the block reads in ColumnarReadLogicalData via the read + * Stream/prefetch the block reads in PgColumnarReadLogicalData via the read * stream API (PostgreSQL 17+), which lets the PostgreSQL 18 asynchronous I/O * subsystem read ahead. On by default; off falls back to synchronous ReadBuffer. * On PostgreSQL 16 and earlier the streaming path is compiled out and this GUC * has no effect. */ -bool columnar_enable_read_stream = true; +bool pgcolumnar_enable_read_stream = true; #if PG_VERSION_NUM >= 170000 /* contiguous ascending block range for the read stream callback */ -typedef struct ColumnarBlockRange +typedef struct PgColumnarBlockRange { BlockNumber next; BlockNumber last; -} ColumnarBlockRange; +} PgColumnarBlockRange; static BlockNumber -columnar_read_stream_next(ReadStream *stream, void *private_data, +pgcolumnar_read_stream_next(ReadStream *stream, void *private_data, void *per_buffer_data) { - ColumnarBlockRange *range = (ColumnarBlockRange *) private_data; + PgColumnarBlockRange *range = (PgColumnarBlockRange *) private_data; if (range->next > range->last) return InvalidBlockNumber; @@ -67,7 +67,7 @@ columnar_read_stream_next(ReadStream *stream, void *private_data, #endif /* - * ColumnarWriteNewMetapage + * PgColumnarWriteNewMetapage * Initialize a freshly created relation's storage: block 0 holds the * metapage with the initial reserved values from spec 3, block 1 is * reserved and left empty. Written with a WAL full-page image and an @@ -75,17 +75,17 @@ columnar_read_stream_next(ReadStream *stream, void *private_data, * buffers here. */ void -ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, +PgColumnarWriteNewMetapage(const RelFileLocator *newrlocator, SMgrRelation srel, char persistence, uint64 storageId) { - Page page = ColumnarAllocPage(); - ColumnarMetapage *meta; + Page page = PgColumnarAllocPage(); + PgColumnarMetapage *meta; bool needsWAL = (persistence == RELPERSISTENCE_PERMANENT); /* block 0: metapage */ PageInit(page, BLCKSZ, 0); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->versionMajor = COLUMNAR_VERSION_MAJOR; meta->versionMinor = COLUMNAR_VERSION_MINOR; meta->storageId = storageId; @@ -94,7 +94,7 @@ ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, meta->reservedOffset = COLUMNAR_FIRST_LOGICAL_OFFSET; meta->unloggedReset = false; ((PageHeader) page)->pd_lower = - ((char *) meta - (char *) page) + sizeof(ColumnarMetapage); + ((char *) meta - (char *) page) + sizeof(PgColumnarMetapage); if (needsWAL) log_newpage(&COLUMNAR_SMGR_LOCATOR(srel), MAIN_FORKNUM, @@ -116,11 +116,11 @@ ColumnarWriteNewMetapage(const RelFileLocator *newrlocator, } /* - * ColumnarReadMetapage + * PgColumnarReadMetapage * Read the metapage of an existing relation into *meta. */ void -ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) +PgColumnarReadMetapage(Relation rel, PgColumnarMetapage *meta) { Buffer buffer; Page page; @@ -128,7 +128,7 @@ ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_SHARE); page = BufferGetPage(buffer); - memcpy(meta, ColumnarMetapagePointer(page), sizeof(ColumnarMetapage)); + memcpy(meta, PgColumnarMetapagePointer(page), sizeof(PgColumnarMetapage)); UnlockReleaseBuffer(buffer); if (meta->versionMajor != COLUMNAR_VERSION_MAJOR) @@ -139,16 +139,16 @@ ColumnarReadMetapage(Relation rel, ColumnarMetapage *meta) } uint64 -ColumnarStorageId(Relation rel) +PgColumnarStorageId(Relation rel) { - ColumnarMetapage meta; + PgColumnarMetapage meta; - ColumnarReadMetapage(rel, &meta); + PgColumnarReadMetapage(rel, &meta); return meta.storageId; } /* - * ColumnarReserveRowNumbers + * PgColumnarReserveRowNumbers * Reserve a stripe id and a contiguous run of "rowCount" row numbers by * advancing the metapage's reservedStripeId and reservedRowNumber marks * (spec 2.2, 6). Returns the reserved stripe id and the first row number @@ -159,24 +159,24 @@ ColumnarStorageId(Relation rel) * at insert time. That is what lets an index carry a correct TID for a * freshly inserted row (spec 6, 9). The byte offset is reserved * separately, at flush, once the stripe's size is known - * (ColumnarReserveOffset). Row numbers not used by a short stripe are + * (PgColumnarReserveOffset). Row numbers not used by a short stripe are * simply left as a gap; row numbers need only be unique and stable. * * Serialized by the exclusive lock on the metapage buffer; no relation * extension lock is needed here because no data pages are extended. */ void -ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, +PgColumnarReserveRowNumbers(Relation rel, uint64 rowCount, uint64 *stripeId, uint64 *firstRowNumber) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); *stripeId = meta->reservedStripeId; *firstRowNumber = meta->reservedRowNumber; @@ -198,28 +198,28 @@ ColumnarReserveRowNumbers(Relation rel, uint64 rowCount, } /* - * ColumnarReserveOffset + * PgColumnarReserveOffset * Reserve a page-aligned logical byte range of "dataLength" bytes for a * stripe's data and return its file offset (spec 2.1, 2.2). New * reservations start on a fresh page. * * The caller must already hold the relation extension lock and must - * write the reserved data immediately (via ColumnarWriteLogicalData) + * write the reserved data immediately (via PgColumnarWriteLogicalData) * before releasing it, so that reservation is serialized and the P_NEW * extends match the reserved blocks. */ void -ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) +PgColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; uint64 alignedOffset; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); /* align the start of this reservation up to a page boundary */ alignedOffset = ((meta->reservedOffset + COLUMNAR_BYTES_PER_PAGE - 1) / @@ -242,7 +242,7 @@ ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) } /* - * ColumnarAdvanceReservedOffset + * PgColumnarAdvanceReservedOffset * Increase the metapage highwater by addBytes without writing any data, * leaving a gap between the physical EOF and the new highwater. This is a * test hook for the gap-tolerant write path (it deliberately produces the @@ -250,16 +250,16 @@ ColumnarReserveOffset(Relation rel, uint64 dataLength, uint64 *fileOffset) * Increase-only, so it can never make the highwater overlap live data. */ void -ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) +PgColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->reservedOffset += addBytes; @@ -277,24 +277,24 @@ ColumnarAdvanceReservedOffset(Relation rel, uint64 addBytes) } /* - * ColumnarDebugSetMetapageVersion + * PgColumnarDebugSetMetapageVersion * Test-only: overwrite the metapage's stored physical format version. Reads - * go through ColumnarReadMetapage, which rejects a version it does not + * go through PgColumnarReadMetapage, which rejects a version it does not * understand; this lets a test plant a bad version and confirm that guard * fires cleanly. Reachable only via the SQL binding the format suite * creates, never from the shipped catalog. */ void -ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor) +PgColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versionMinor) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->versionMajor = versionMajor; meta->versionMinor = versionMinor; @@ -313,7 +313,7 @@ ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versio } /* - * ColumnarSetReservedOffset + * PgColumnarSetReservedOffset * Lower the metapage highwater to newOffset (physical end-truncation). The * caller has computed newOffset as the end of all live data and holds an * exclusive lock on the relation, so no reservation can race. Future @@ -321,16 +321,16 @@ ColumnarDebugSetMetapageVersion(Relation rel, uint32 versionMajor, uint32 versio * the (now truncated) EOF. It is an error to raise the highwater here. */ void -ColumnarSetReservedOffset(Relation rel, uint64 newOffset) +PgColumnarSetReservedOffset(Relation rel, uint64 newOffset) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); Assert(newOffset <= meta->reservedOffset); meta->reservedOffset = newOffset; @@ -349,7 +349,7 @@ ColumnarSetReservedOffset(Relation rel, uint64 newOffset) } /* - * ColumnarTruncateMainFork + * PgColumnarTruncateMainFork * Physically truncate the relation's MAIN fork to newnblocks, returning the * trailing blocks to the OS (physical end-truncation). Scoped to the main * fork only: the visibility-map fork is indexed by row-number-derived blocks, @@ -361,7 +361,7 @@ ColumnarSetReservedOffset(Relation rel, uint64 newOffset) * range. */ void -ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) +PgColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) { SMgrRelation srel = RelationGetSmgr(rel); ForkNumber fork = MAIN_FORKNUM; @@ -414,7 +414,7 @@ ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) } /* - * ColumnarWriteLogicalData + * PgColumnarWriteLogicalData * Write a contiguous logical byte range starting at a page-aligned * logical offset, splitting it across the physical pages it maps to * (spec 2.1). Blocks past the current end of the relation are extended; @@ -431,7 +431,7 @@ ColumnarTruncateMainFork(Relation rel, BlockNumber newnblocks) * relation extension lock, so the P_NEW extensions here are serialized. */ void -ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, +PgColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, char *data, uint64 length) { uint64 L = logicalOffset; @@ -521,12 +521,12 @@ ColumnarWriteLogicalData(Relation rel, uint64 logicalOffset, } /* - * ColumnarReadLogicalData + * PgColumnarReadLogicalData * Read a contiguous logical byte range into dest by walking the pages * it maps to (spec 2.1). */ void -ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, +PgColumnarReadLogicalData(Relation rel, uint64 logicalOffset, char *dest, uint64 length) { uint64 L = logicalOffset; @@ -537,7 +537,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, return; #if PG_VERSION_NUM >= 170000 - if (columnar_enable_read_stream) + if (pgcolumnar_enable_read_stream) { /* * The blocks map to a contiguous ascending range, so a read stream can @@ -546,7 +546,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, * so the same L/pageOffset walk drives the copy; the buffers are share- * locked here just as the synchronous path does. */ - ColumnarBlockRange range; + PgColumnarBlockRange range; ReadStream *stream; range.next = (BlockNumber) (L / COLUMNAR_BYTES_PER_PAGE); @@ -554,7 +554,7 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, stream = read_stream_begin_relation(READ_STREAM_SEQUENTIAL, NULL, rel, MAIN_FORKNUM, - columnar_read_stream_next, &range, 0); + pgcolumnar_read_stream_next, &range, 0); while (remaining > 0) { @@ -615,21 +615,21 @@ ColumnarReadLogicalData(Relation rel, uint64 logicalOffset, } /* - * ColumnarResetMetapage + * PgColumnarResetMetapage * Reset the reserved high-water marks to their initial values, keeping * the storage id. Used by non-transactional truncate. */ void -ColumnarResetMetapage(Relation rel) +PgColumnarResetMetapage(Relation rel) { Buffer buffer; Page page; - ColumnarMetapage *meta; + PgColumnarMetapage *meta; buffer = ReadBuffer(rel, COLUMNAR_METAPAGE_BLOCKNO); LockBuffer(buffer, BUFFER_LOCK_EXCLUSIVE); page = BufferGetPage(buffer); - meta = ColumnarMetapagePointer(page); + meta = PgColumnarMetapagePointer(page); meta->reservedStripeId = 1; meta->reservedRowNumber = COLUMNAR_FIRST_ROW_NUMBER; @@ -652,7 +652,7 @@ ColumnarResetMetapage(Relation rel) * Row-number <-> item-pointer mapping (spec 6). Row number 0 is invalid. */ void -ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) +PgColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) { BlockNumber blockno; OffsetNumber offset; @@ -665,7 +665,7 @@ ColumnarRowNumberToItemPointer(uint64 rowNumber, ItemPointer tid) } uint64 -ColumnarItemPointerToRowNumber(ItemPointer tid) +PgColumnarItemPointerToRowNumber(ItemPointer tid) { BlockNumber blockno = ItemPointerGetBlockNumber(tid); OffsetNumber offset = ItemPointerGetOffsetNumber(tid); diff --git a/src/columnar_tableam.c b/src/columnar_tableam.c index 08e1f19..b4c09e5 100644 --- a/src/columnar_tableam.c +++ b/src/columnar_tableam.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_tableam.c + * pgcolumnar_tableam.c * Table access method handler for pgColumnar and extension glue: * GUCs, the pre-commit flush hook, and drop-time metadata cleanup. * @@ -56,19 +56,19 @@ PG_MODULE_MAGIC; /* GUC-backed instance defaults (spec 8.3) */ -int columnar_stripe_row_limit = 150000; -int columnar_chunk_group_row_limit = 10000; -int columnar_encoding_sample_rows = 2048; +int pgcolumnar_stripe_row_limit = 150000; +int pgcolumnar_chunk_group_row_limit = 10000; +int pgcolumnar_encoding_sample_rows = 2048; -int columnar_compression = COLUMNAR_COMPRESSION_ZSTD; -int columnar_compression_level = 3; -int columnar_fsst_min_gain_percent = 5; -bool columnar_enable_qual_pushdown = true; -bool columnar_enable_column_projection = true; -bool columnar_enable_bloom_filter = true; +int pgcolumnar_compression = COLUMNAR_COMPRESSION_ZSTD; +int pgcolumnar_compression_level = 3; +int pgcolumnar_fsst_min_gain_percent = 5; +bool pgcolumnar_enable_qual_pushdown = true; +bool pgcolumnar_enable_column_projection = true; +bool pgcolumnar_enable_bloom_filter = true; /* value set for columnar.compression (spec 5, 8.3) */ -static const struct config_enum_entry columnar_compression_options[] = { +static const struct config_enum_entry pgcolumnar_compression_options[] = { {"none", COLUMNAR_COMPRESSION_NONE, false}, {"pglz", COLUMNAR_COMPRESSION_PGLZ, false}, {"lz4", COLUMNAR_COMPRESSION_LZ4, false}, @@ -77,7 +77,7 @@ static const struct config_enum_entry columnar_compression_options[] = { }; /* forward declaration of the AM routine so hooks can compare against it */ -static const TableAmRoutine columnar_am_methods; +static const TableAmRoutine pgcolumnar_am_methods; static object_access_hook_type prev_object_access_hook = NULL; static ProcessUtility_hook_type prev_process_utility_hook = NULL; @@ -89,7 +89,7 @@ static get_relation_info_hook_type prev_get_relation_info_hook = NULL; #endif /* cached OID of the "columnar" table access method (index-only-scan hook) */ -static Oid columnar_am_oid_cache = InvalidOid; +static Oid pgcolumnar_am_oid_cache = InvalidOid; /* our scan descriptor wraps the base scan and the reader state */ /* @@ -114,7 +114,7 @@ static Oid columnar_am_oid_cache = InvalidOid; * whole of every group core touches, which is what defeats the clustering trap. * A row is offered by exactly one block, so no row can be sampled twice. */ -typedef struct ColumnarAnalyzeState +typedef struct PgColumnarAnalyzeState { List *rowGroups; /* NativeRowGroupMetadata *, in row order */ Snapshot metaSnapshot; @@ -142,18 +142,18 @@ typedef struct ColumnarAnalyzeState * target was lowered, because the work is per row offered rather than per row * kept. */ - ColumnarReadState *rs; /* NULL until the first slice with rows */ + PgColumnarReadState *rs; /* NULL until the first slice with rows */ uint64 rsGroup; /* group number rs is restricted to */ bool rsHavePending; /* pendingRow/values hold an unconsumed row */ uint64 pendingRow; Datum *pendingValues; bool *pendingNulls; -} ColumnarAnalyzeState; +} PgColumnarAnalyzeState; -typedef struct ColumnarScanDescData +typedef struct PgColumnarScanDescData { TableScanDescData rs_base; - ColumnarReadState *readState; + PgColumnarReadState *readState; /* * The context the scan descriptor itself was allocated in. The read state @@ -162,11 +162,11 @@ typedef struct ColumnarScanDescData * instead and outlives the row that triggered it. */ MemoryContext scanContext; - ColumnarAnalyzeState *analyzeState; -} ColumnarScanDescData; -typedef struct ColumnarScanDescData *ColumnarScanDesc; + PgColumnarAnalyzeState *analyzeState; +} PgColumnarScanDescData; +typedef struct PgColumnarScanDescData *PgColumnarScanDesc; -PG_FUNCTION_INFO_V1(columnar_handler); +PG_FUNCTION_INFO_V1(pgcolumnar_handler); /* ------------------------------------------------------------------------- * slot / scan callbacks @@ -204,7 +204,7 @@ PG_FUNCTION_INFO_V1(columnar_handler); * slot_getsomeattrs never calls it. The full suite is run on an assert-enabled * build to keep that reasoning honest. */ -static TupleTableSlotOps ColumnarSlotOps; +static TupleTableSlotOps PgColumnarSlotOps; /* * A slot that can defer its decode (issue #157). @@ -225,7 +225,7 @@ static TupleTableSlotOps ColumnarSlotOps; * ExecStoreVirtualTuple, which sets tts_nvalid to the full count, so getsomeattrs * is never reached for those and their behaviour is unchanged. */ -typedef struct ColumnarSlot +typedef struct PgColumnarSlot { /* * VirtualTupleTableSlot, not TupleTableSlot, and it must come first. Every @@ -242,18 +242,18 @@ typedef struct ColumnarSlot Relation rel; Snapshot snapshot; uint64 rowNumber; -} ColumnarSlot; +} PgColumnarSlot; /* * The fields above must sit past everything the inherited callbacks touch. If * VirtualTupleTableSlot ever grows, this fails to compile rather than silently * aliasing. */ -StaticAssertDecl(offsetof(ColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlot), +StaticAssertDecl(offsetof(PgColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlot), "ColumnarSlot fields must not overlap VirtualTupleTableSlot"); /* - * columnar_slot_decode_upto + * pgcolumnar_slot_decode_upto * Materialise attributes 0 .. natts-1 of a deferred slot. * * Decodes a prefix because that is what slot_getsomeattrs asks for, and the @@ -261,9 +261,9 @@ StaticAssertDecl(offsetof(ColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlo * column 2 of 41 decodes two columns, not forty-one. */ static void -columnar_slot_decode_upto(TupleTableSlot *slot, int natts) +pgcolumnar_slot_decode_upto(TupleTableSlot *slot, int natts) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; Bitmapset *needed = NULL; int i; @@ -279,10 +279,10 @@ columnar_slot_decode_upto(TupleTableSlot *slot, int natts) * reconstructs the whole row, which is correct if not lazy, and it is bounded * by what one transaction has buffered. */ - if (!ColumnarReadRowByNumberCols(cslot->rel, cslot->snapshot, + if (!PgColumnarReadRowByNumberCols(cslot->rel, cslot->snapshot, cslot->rowNumber, slot->tts_values, slot->tts_isnull, needed)) - (void) ColumnarBufferedRowByNumber(cslot->rel, cslot->rowNumber, + (void) PgColumnarBufferedRowByNumber(cslot->rel, cslot->rowNumber, slot->tts_values, slot->tts_isnull); bms_free(needed); @@ -297,9 +297,9 @@ columnar_slot_decode_upto(TupleTableSlot *slot, int natts) } static void -columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) +pgcolumnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; if (!cslot->deferred) { @@ -311,7 +311,7 @@ columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) elog(ERROR, "getsomeattrs on a columnar slot that was filled eagerly"); } - columnar_slot_decode_upto(slot, natts); + pgcolumnar_slot_decode_upto(slot, natts); } /* @@ -319,24 +319,24 @@ columnar_slot_getsomeattrs(TupleTableSlot *slot, int natts) * -- has to finish the decode first. */ static void -columnar_slot_force_full(TupleTableSlot *slot) +pgcolumnar_slot_force_full(TupleTableSlot *slot) { - ColumnarSlot *cslot; + PgColumnarSlot *cslot; /* * Callers hand us slots that are not ours. copyslot in particular takes a * source of any type -- the executor copies an ordinary virtual slot into a - * columnar one on every INSERT -- and casting that to ColumnarSlot reads + * columnar one on every INSERT -- and casting that to PgColumnarSlot reads * past the end of it, so the deferred flag is whatever happened to be in * the next word and the relation pointer behind it is garbage. That is a * segfault on the plainest INSERT there is, which is how it was found. */ - if (slot->tts_ops != &ColumnarSlotOps) + if (slot->tts_ops != &PgColumnarSlotOps) return; - cslot = (ColumnarSlot *) slot; + cslot = (PgColumnarSlot *) slot; if (cslot->deferred && slot->tts_nvalid < slot->tts_tupleDescriptor->natts) - columnar_slot_decode_upto(slot, slot->tts_tupleDescriptor->natts); + pgcolumnar_slot_decode_upto(slot, slot->tts_tupleDescriptor->natts); } /* @@ -345,9 +345,9 @@ columnar_slot_force_full(TupleTableSlot *slot) * are there. */ static void -columnar_slot_init(TupleTableSlot *slot) +pgcolumnar_slot_init(TupleTableSlot *slot) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; TTSOpsVirtual.init(slot); cslot->deferred = false; @@ -357,11 +357,11 @@ columnar_slot_init(TupleTableSlot *slot) } static void -columnar_slot_clear(TupleTableSlot *slot) +pgcolumnar_slot_clear(TupleTableSlot *slot) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; - Assert(slot->tts_ops == &ColumnarSlotOps); + Assert(slot->tts_ops == &PgColumnarSlotOps); cslot->deferred = false; cslot->rel = NULL; cslot->snapshot = NULL; @@ -370,37 +370,37 @@ columnar_slot_clear(TupleTableSlot *slot) } static void -columnar_slot_materialize(TupleTableSlot *slot) +pgcolumnar_slot_materialize(TupleTableSlot *slot) { - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); TTSOpsVirtual.materialize(slot); } static void -columnar_slot_copyslot(TupleTableSlot *dstslot, TupleTableSlot *srcslot) +pgcolumnar_slot_copyslot(TupleTableSlot *dstslot, TupleTableSlot *srcslot) { - columnar_slot_force_full(srcslot); + pgcolumnar_slot_force_full(srcslot); TTSOpsVirtual.copyslot(dstslot, srcslot); } static MinimalTuple -columnar_slot_copy_minimal_tuple(COLUMNAR_COPY_MINIMAL_TUPLE_ARGS) +pgcolumnar_slot_copy_minimal_tuple(COLUMNAR_COPY_MINIMAL_TUPLE_ARGS) { - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); return TTSOpsVirtual.copy_minimal_tuple COLUMNAR_COPY_MINIMAL_TUPLE_FWD(slot); } /* - * ColumnarSlotStoreDeferred + * PgColumnarSlotStoreDeferred * Point the slot at a row without decoding it. The caller has already * established that the row is visible. */ static void -ColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, +PgColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, Snapshot snapshot, uint64 rowNumber) { - ColumnarSlot *cslot = (ColumnarSlot *) slot; + PgColumnarSlot *cslot = (PgColumnarSlot *) slot; ExecClearTuple(slot); cslot->deferred = true; @@ -413,13 +413,13 @@ ColumnarSlotStoreDeferred(TupleTableSlot *slot, Relation rel, } static HeapTuple -columnar_slot_copy_heap_tuple(TupleTableSlot *slot) +pgcolumnar_slot_copy_heap_tuple(TupleTableSlot *slot) { HeapTuple tuple; Assert(!TTS_EMPTY(slot)); - columnar_slot_force_full(slot); + pgcolumnar_slot_force_full(slot); tuple = heap_form_tuple(slot->tts_tupleDescriptor, slot->tts_values, slot->tts_isnull); @@ -430,30 +430,30 @@ columnar_slot_copy_heap_tuple(TupleTableSlot *slot) } static const TupleTableSlotOps * -columnar_slot_callbacks(Relation relation) +pgcolumnar_slot_callbacks(Relation relation) { - return &ColumnarSlotOps; + return &PgColumnarSlotOps; } static TableScanDesc -columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, +pgcolumnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, ScanKey key, ParallelTableScanDesc pscan, uint32 flags) { - ColumnarScanDesc scan; + PgColumnarScanDesc scan; RelationIncrementReferenceCount(rel); /* * Persist any data and delete marks written earlier in this transaction so * they reach the catalog before this scan reads it. The reader consults the - * catalog with a command-id-advanced snapshot (ColumnarCatalogSnapshot), so + * catalog with a command-id-advanced snapshot (PgColumnarCatalogSnapshot), so * these become visible to this same scan: same-transaction read-your-writes * (spec 9). */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); - scan = (ColumnarScanDesc) palloc0(sizeof(ColumnarScanDescData)); + scan = (PgColumnarScanDesc) palloc0(sizeof(PgColumnarScanDescData)); scan->rs_base.rs_rd = rel; scan->rs_base.rs_snapshot = snapshot; scan->rs_base.rs_nkeys = nkeys; @@ -488,7 +488,7 @@ columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, } /* - * columnar_scan_read_state + * pgcolumnar_scan_read_state * The scan's reader, built on first use against the descriptor the caller * is asking for. * @@ -500,19 +500,19 @@ columnar_scan_begin(Relation rel, Snapshot snapshot, int nkeys, * * The read state is allocated in the context the scan descriptor itself lives * in. The current context on first use is usually a per-tuple one that is reset - * before the scan ends, and ColumnarEndRead then frees an already-freed pointer. + * before the scan ends, and PgColumnarEndRead then frees an already-freed pointer. */ -static ColumnarReadState * -columnar_scan_read_state(ColumnarScanDesc scan, TupleDesc tupdesc) +static PgColumnarReadState * +pgcolumnar_scan_read_state(PgColumnarScanDesc scan, TupleDesc tupdesc) { if (scan->readState == NULL) { MemoryContext oldContext = MemoryContextSwitchTo(scan->scanContext); scan->readState = - ColumnarBeginReadWithStorage(scan->rs_base.rs_rd, + PgColumnarBeginReadWithStorage(scan->rs_base.rs_rd, scan->rs_base.rs_snapshot, - ColumnarStorageId(scan->rs_base.rs_rd), + PgColumnarStorageId(scan->rs_base.rs_rd), tupdesc, scan->rs_base.rs_parallel, NULL, scan->rs_base.rs_nkeys, @@ -524,17 +524,17 @@ columnar_scan_read_state(ColumnarScanDesc scan, TupleDesc tupdesc) } static void -columnar_scan_end(TableScanDesc sscan) +pgcolumnar_scan_end(TableScanDesc sscan) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; if (scan->readState != NULL) - ColumnarEndRead(scan->readState); + PgColumnarEndRead(scan->readState); if (scan->analyzeState != NULL) { if (scan->analyzeState->rs != NULL) - ColumnarEndRead(scan->analyzeState->rs); + PgColumnarEndRead(scan->analyzeState->rs); MemoryContextDelete(scan->analyzeState->cx); } @@ -547,48 +547,48 @@ columnar_scan_end(TableScanDesc sscan) } static void -columnar_scan_rescan(TableScanDesc sscan, ScanKey key, bool set_params, +pgcolumnar_scan_rescan(TableScanDesc sscan, ScanKey key, bool set_params, bool allow_strat, bool allow_sync, bool allow_pagemode) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; if (scan->readState != NULL) - ColumnarRescanRead(scan->readState); + PgColumnarRescanRead(scan->readState); } static bool -columnar_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, +pgcolumnar_scan_getnextslot(TableScanDesc sscan, ScanDirection direction, TupleTableSlot *slot) { - ColumnarScanDesc scan = (ColumnarScanDesc) sscan; + PgColumnarScanDesc scan = (PgColumnarScanDesc) sscan; uint64 rowNumber; ExecClearTuple(slot); - if (!ColumnarReadNextRow(columnar_scan_read_state(scan, + if (!PgColumnarReadNextRow(pgcolumnar_scan_read_state(scan, slot->tts_tupleDescriptor), slot->tts_values, slot->tts_isnull, &rowNumber)) return false; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(scan->rs_base.rs_rd); return true; } /* ------------------------------------------------------------------------- - * parallel scan: single-worker claim (see columnar_reader.c) + * parallel scan: single-worker claim (see pgcolumnar_reader.c) * ------------------------------------------------------------------------- */ static Size -columnar_parallelscan_estimate(Relation rel) +pgcolumnar_parallelscan_estimate(Relation rel) { return sizeof(ParallelBlockTableScanDescData); } static Size -columnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) +pgcolumnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) { ParallelBlockTableScanDesc bpscan = (ParallelBlockTableScanDesc) pscan; @@ -603,7 +603,7 @@ columnar_parallelscan_initialize(Relation rel, ParallelTableScanDesc pscan) } static void -columnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) +pgcolumnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) { ParallelBlockTableScanDesc bpscan = (ParallelBlockTableScanDesc) pscan; @@ -615,11 +615,11 @@ columnar_parallelscan_reinitialize(Relation rel, ParallelTableScanDesc pscan) * ------------------------------------------------------------------------- */ static void -columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, +pgcolumnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate) { - ColumnarWriteState *writeState = ColumnarGetWriteState(rel); + PgColumnarWriteState *writeState = PgColumnarGetWriteState(rel); uint64 rowNumber; slot_getallattrs(slot); @@ -629,13 +629,13 @@ columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, * the executor runs its btree uniqueness check on this row, so the check * runs only after any conflicting transaction has committed and flushed. */ - ColumnarLockUniqueKeys(rel, slot); + PgColumnarLockUniqueKeys(rel, slot); - rowNumber = ColumnarWriteRow(writeState, rel, slot->tts_values, + rowNumber = PgColumnarWriteRow(writeState, rel, slot->tts_values, slot->tts_isnull); /* fan the row out to every additional projection of this table (gap 26) */ - ColumnarProjectionFanoutRow(rel, writeState, rowNumber, slot->tts_values, + PgColumnarProjectionFanoutRow(rel, writeState, rowNumber, slot->tts_values, slot->tts_isnull); /* @@ -643,23 +643,23 @@ columnar_tuple_insert(Relation rel, TupleTableSlot *slot, CommandId cid, * index-only scan never skips the fetch for a block that just changed * (gap 28). A no-op unless a prior vacuum had marked the block visible. */ - ColumnarVMClearForRow(rel, rowNumber); + PgColumnarVMClearForRow(rel, rowNumber); /* * Publish the row's synthetic item pointer (spec 6) so the executor can * insert correct (index value, TID) entries into any indexes on this * relation and enforce unique constraints (spec 9). */ - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); } static void -columnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, +pgcolumnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate) { - ColumnarWriteState *writeState = ColumnarGetWriteState(rel); + PgColumnarWriteState *writeState = PgColumnarGetWriteState(rel); int i; for (i = 0; i < nslots; i++) @@ -667,27 +667,27 @@ columnar_multi_insert(Relation rel, TupleTableSlot **slots, int nslots, uint64 rowNumber; slot_getallattrs(slots[i]); - ColumnarLockUniqueKeys(rel, slots[i]); /* issue #5 */ - rowNumber = ColumnarWriteRow(writeState, rel, slots[i]->tts_values, + PgColumnarLockUniqueKeys(rel, slots[i]); /* issue #5 */ + rowNumber = PgColumnarWriteRow(writeState, rel, slots[i]->tts_values, slots[i]->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, rowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, rowNumber, slots[i]->tts_values, slots[i]->tts_isnull); - ColumnarVMClearForRow(rel, rowNumber); /* gap 28: block changed */ - ColumnarRowNumberToItemPointer(rowNumber, &slots[i]->tts_tid); + PgColumnarVMClearForRow(rel, rowNumber); /* gap 28: block changed */ + PgColumnarRowNumberToItemPointer(rowNumber, &slots[i]->tts_tid); slots[i]->tts_tableOid = RelationGetRelid(rel); } } static void -columnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) +pgcolumnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) { /* * End of a bulk-load path (COPY, CREATE TABLE AS, ALTER TABLE rewrite). * Flush now, under this operation's subtransaction, so the buffer never * spans a later statement or savepoint boundary (spec 9). */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(rel)); + PgColumnarFlushDeleteVectorForRelation(rel); } /* ------------------------------------------------------------------------- @@ -695,7 +695,7 @@ columnar_finish_bulk_insert(Relation rel, COLUMNAR_TABLE_OPTIONS options) * ------------------------------------------------------------------------- */ static void -columnar_relation_set_new_filelocator(Relation rel, +pgcolumnar_relation_set_new_filelocator(Relation rel, const RelFileLocator *newrlocator, char persistence, TransactionId *freezeXid, @@ -712,19 +712,19 @@ columnar_relation_set_new_filelocator(Relation rel, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("unlogged columnar tables are not supported"))); - srel = ColumnarRelationCreateStorage(*newrlocator, persistence); - storageId = ColumnarNextStorageId(); - ColumnarWriteNewMetapage(newrlocator, srel, persistence, storageId); + srel = PgColumnarRelationCreateStorage(*newrlocator, persistence); + storageId = PgColumnarNextStorageId(); + PgColumnarWriteNewMetapage(newrlocator, srel, persistence, storageId); } static void -columnar_relation_nontransactional_truncate(Relation rel) +pgcolumnar_relation_nontransactional_truncate(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); - ColumnarDeleteMetadata(storageId); + PgColumnarDeleteMetadata(storageId); RelationTruncate(rel, 2); - ColumnarResetMetapage(rel); + PgColumnarResetMetapage(rel); } /* ------------------------------------------------------------------------- @@ -732,7 +732,7 @@ columnar_relation_nontransactional_truncate(Relation rel) * ------------------------------------------------------------------------- */ static uint64 -columnar_relation_size(Relation rel, ForkNumber forkNumber) +pgcolumnar_relation_size(Relation rel, ForkNumber forkNumber) { SMgrRelation srel = RelationGetSmgr(rel); @@ -748,19 +748,19 @@ columnar_relation_size(Relation rel, ForkNumber forkNumber) } static bool -columnar_relation_needs_toast_table(Relation rel) +pgcolumnar_relation_needs_toast_table(Relation rel) { /* the writer detoasts and stores values inline in the value stream */ return false; } static void -columnar_relation_estimate_size(Relation rel, int32 *attr_widths, +pgcolumnar_relation_estimate_size(Relation rel, int32 *attr_widths, BlockNumber *pages, double *tuples, double *allvisfrac) { BlockNumber nblocks = RelationGetNumberOfBlocks(rel); - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Snapshot snapshot; List *rowGroupList; ListCell *lc; @@ -773,7 +773,7 @@ columnar_relation_estimate_size(Relation rel, int32 *attr_widths, * planner from mis-costing scans (spec 6, 9). */ snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); - rowGroupList = ColumnarReadRowGroupList(storageId, ColumnarCatalogSnapshot(snapshot)); + rowGroupList = PgColumnarReadRowGroupList(storageId, PgColumnarCatalogSnapshot(snapshot)); foreach(lc, rowGroupList) liveRows += (double) ((NativeRowGroupMetadata *) lfirst(lc))->rowCount; @@ -784,28 +784,28 @@ columnar_relation_estimate_size(Relation rel, int32 *attr_widths, } /* - * columnar_analyze_state + * pgcolumnar_analyze_state * The sampling state for this scan, built on first use. The row group list * is read once: ANALYZE holds ShareUpdateExclusiveLock, so no group is added * or retired under us, and re-reading it per block would put a catalog scan * in the middle of the sample loop. */ -static ColumnarAnalyzeState * -columnar_analyze_state(ColumnarScanDesc scan) +static PgColumnarAnalyzeState * +pgcolumnar_analyze_state(PgColumnarScanDesc scan) { - ColumnarAnalyzeState *st = scan->analyzeState; + PgColumnarAnalyzeState *st = scan->analyzeState; Relation rel = scan->rs_base.rs_rd; MemoryContext oldContext; if (st != NULL) return st; - st = palloc0(sizeof(ColumnarAnalyzeState)); + st = palloc0(sizeof(PgColumnarAnalyzeState)); st->cx = AllocSetContextCreate(CurrentMemoryContext, "columnar analyze", ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(st->cx); - st->metaSnapshot = ColumnarCatalogSnapshot(scan->rs_base.rs_snapshot); - st->rowGroups = ColumnarReadRowGroupList(ColumnarStorageId(rel), + st->metaSnapshot = PgColumnarCatalogSnapshot(scan->rs_base.rs_snapshot); + st->rowGroups = PgColumnarReadRowGroupList(PgColumnarStorageId(rel), st->metaSnapshot); st->values = palloc(sizeof(Datum) * RelationGetDescr(rel)->natts); st->nulls = palloc(sizeof(bool) * RelationGetDescr(rel)->natts); @@ -818,7 +818,7 @@ columnar_analyze_state(ColumnarScanDesc scan) } /* - * columnar_analyze_set_slice + * pgcolumnar_analyze_set_slice * Point the sampler at the rows a physical block stands for. * * The block's logical byte offset locates the row group it falls in; its @@ -830,7 +830,7 @@ columnar_analyze_state(ColumnarScanDesc scan) * as visited and offers nothing. */ static void -columnar_analyze_set_slice(ColumnarAnalyzeState *st, BlockNumber blockno) +pgcolumnar_analyze_set_slice(PgColumnarAnalyzeState *st, BlockNumber blockno) { uint64 logicalOffset; ListCell *lc; @@ -903,17 +903,17 @@ columnar_analyze_set_slice(ColumnarAnalyzeState *st, BlockNumber blockno) /* * The block comes from a read stream from PG17 and as a plain BlockNumber - * before that. columnar_compat.h supplies the parameter list and splits at the + * before that. pgcolumnar_compat.h supplies the parameter list and splits at the * same major; these two must agree, and when they did not, PG17 took the * pre-17 branch and failed to compile on a `blockno` its signature does not * have. */ #if PG_VERSION_NUM >= 170000 static bool -columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) +pgcolumnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; - ColumnarAnalyzeState *st = columnar_analyze_state(cscan); + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; + PgColumnarAnalyzeState *st = pgcolumnar_analyze_state(cscan); Buffer buf = read_stream_next_buffer(stream, NULL); if (!BufferIsValid(buf)) @@ -925,26 +925,26 @@ columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) * for are read through the fetch path instead. Release the pin at once rather * than holding it across the tuple loop as heap does. */ - columnar_analyze_set_slice(st, BufferGetBlockNumber(buf)); + pgcolumnar_analyze_set_slice(st, BufferGetBlockNumber(buf)); ReleaseBuffer(buf); return true; } #else static bool -columnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) +pgcolumnar_scan_analyze_next_block(COLUMNAR_ANALYZE_NEXT_BLOCK_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; - columnar_analyze_set_slice(columnar_analyze_state(cscan), blockno); + pgcolumnar_analyze_set_slice(pgcolumnar_analyze_state(cscan), blockno); return true; } #endif static bool -columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) +pgcolumnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) { - ColumnarScanDesc cscan = (ColumnarScanDesc) scan; - ColumnarAnalyzeState *st = columnar_analyze_state(cscan); + PgColumnarScanDesc cscan = (PgColumnarScanDesc) scan; + PgColumnarAnalyzeState *st = pgcolumnar_analyze_state(cscan); Relation rel = scan->rs_rd; TupleDesc tupdesc = RelationGetDescr(rel); uint64 sliceEnd = st->sliceFirstRow + st->sliceRows; @@ -959,9 +959,9 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) MemoryContext oldContext = MemoryContextSwitchTo(st->cx); if (st->rs != NULL) - ColumnarEndRead(st->rs); - st->rs = ColumnarBeginRead(rel, scan->rs_snapshot, NULL, NULL, 0, NULL); - ColumnarReadRestrictToGroups(st->rs, &st->sliceGroup, 1); + PgColumnarEndRead(st->rs); + st->rs = PgColumnarBeginRead(rel, scan->rs_snapshot, NULL, NULL, 0, NULL); + PgColumnarReadRestrictToGroups(st->rs, &st->sliceGroup, 1); st->rsGroup = st->sliceGroup; st->rsHavePending = false; MemoryContextSwitchTo(oldContext); @@ -981,7 +981,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) sizeof(Datum) * tupdesc->natts); memcpy(st->nulls, st->pendingNulls, sizeof(bool) * tupdesc->natts); } - else if (!ColumnarReadNextRow(st->rs, st->values, st->nulls, &rowNumber)) + else if (!PgColumnarReadNextRow(st->rs, st->values, st->nulls, &rowNumber)) { /* * The group is exhausted. Any rows of this slice not returned were @@ -1034,7 +1034,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) * is monotonic, so this is what makes the sorted order the physical order * -- which in turn is what makes the correlation statistic mean anything. */ - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); *liverows += 1; return true; @@ -1044,7 +1044,7 @@ columnar_scan_analyze_next_tuple(COLUMNAR_ANALYZE_NEXT_TUPLE_ARGS) /* VACUUM: mark all-visible groups in the VM fork and retire fully-deleted * groups online, both under ShareUpdateExclusiveLock */ static void -columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, +pgcolumnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, BufferAccessStrategy bstrategy) { /* @@ -1055,7 +1055,7 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, * concurrent with readers and writers. The space-reclaiming rewrite stays in * columnar.vacuum (AccessExclusiveLock, the VACUUM-FULL analog). */ - ColumnarVMSetVisibleForRelation(rel); + PgColumnarVMSetVisibleForRelation(rel); /* * Online compaction (Phase F3a): retire row groups that are fully deleted @@ -1065,7 +1065,7 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, * so a plain VACUUM / autovacuum reclaims fully-deleted groups online without * the AccessExclusiveLock rewrite. */ - ColumnarRetireFullyDeletedGroups(rel); + PgColumnarRetireFullyDeletedGroups(rel); } /* ------------------------------------------------------------------------- @@ -1079,33 +1079,33 @@ columnar_relation_vacuum(Relation rel, COLUMNAR_VACUUM_PARAMS params, feature))) /* our index-fetch descriptor is just the base plus nothing extra */ -typedef struct ColumnarIndexFetchData +typedef struct PgColumnarIndexFetchData { IndexFetchTableData xs_base; -} ColumnarIndexFetchData; +} PgColumnarIndexFetchData; static struct IndexFetchTableData * -columnar_index_fetch_begin(COLUMNAR_INDEX_FETCH_BEGIN_ARGS) +pgcolumnar_index_fetch_begin(COLUMNAR_INDEX_FETCH_BEGIN_ARGS) { - ColumnarIndexFetchData *scan = palloc0(sizeof(ColumnarIndexFetchData)); + PgColumnarIndexFetchData *scan = palloc0(sizeof(PgColumnarIndexFetchData)); scan->xs_base.rel = rel; return &scan->xs_base; } static void -columnar_index_fetch_reset(struct IndexFetchTableData *scan) +pgcolumnar_index_fetch_reset(struct IndexFetchTableData *scan) { } static void -columnar_index_fetch_end(struct IndexFetchTableData *scan) +pgcolumnar_index_fetch_end(struct IndexFetchTableData *scan) { pfree(scan); } /* - * columnar_index_fetch_tuple + * pgcolumnar_index_fetch_tuple * Fetch the columnar row addressed by an index item pointer (spec 6) into * the slot. Returns false when the row is marked deleted in the delete vector * or does not exist, so an index scan never returns a deleted row and a @@ -1120,12 +1120,12 @@ columnar_index_fetch_end(struct IndexFetchTableData *scan) * check path). */ static bool -columnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, +pgcolumnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot, bool *call_again, bool *all_dead) { Relation rel = scan->rel; - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); /* columnar rows are 1:1 with item pointers: no chain, never dead here */ *call_again = false; @@ -1161,31 +1161,31 @@ columnar_index_fetch_tuple(struct IndexFetchTableData *scan, ItemPointer tid, * transaction's write buffer, and that reader reconstructs whole rows, so it * is stored eagerly. */ - if (ColumnarRowIsLive(rel, snapshot, rowNumber)) - ColumnarSlotStoreDeferred(slot, rel, snapshot, rowNumber); - else if (ColumnarBufferedRowByNumber(rel, rowNumber, + if (PgColumnarRowIsLive(rel, snapshot, rowNumber)) + PgColumnarSlotStoreDeferred(slot, rel, snapshot, rowNumber); + else if (PgColumnarBufferedRowByNumber(rel, rowNumber, slot->tts_values, slot->tts_isnull)) ExecStoreVirtualTuple(slot); else return false; - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return true; } /* - * columnar_tuple_fetch_row_version + * pgcolumnar_tuple_fetch_row_version * Fetch the row addressed by tid into slot (spec 6). Used by UPDATE, which * re-fetches the old row by its item pointer. Returns false when the row * does not exist or is marked deleted. */ static bool -columnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, +pgcolumnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot) { - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); ExecClearTuple(slot); @@ -1210,33 +1210,33 @@ columnar_tuple_fetch_row_version(Relation rel, ItemPointer tid, * partial stripe to satisfy a read would fragment storage for the sake of * data already in hand. */ - if (!ColumnarReadRowByNumber(rel, snapshot, rowNumber, + if (!PgColumnarReadRowByNumber(rel, snapshot, rowNumber, slot->tts_values, slot->tts_isnull) && - !ColumnarBufferedRowByNumber(rel, rowNumber, + !PgColumnarBufferedRowByNumber(rel, rowNumber, slot->tts_values, slot->tts_isnull)) return false; ExecStoreVirtualTuple(slot); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); return true; } static bool -columnar_tuple_tid_valid(TableScanDesc scan, ItemPointer tid) +pgcolumnar_tuple_tid_valid(TableScanDesc scan, ItemPointer tid) { return true; } static void -columnar_tuple_get_latest_tid(TableScanDesc scan, ItemPointer tid) +pgcolumnar_tuple_get_latest_tid(TableScanDesc scan, ItemPointer tid) { COLUMNAR_UNSUPPORTED("get latest tid"); } static bool -columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, Snapshot snapshot) { /* stripes are visible per their metadata snapshot; slots are visible */ @@ -1244,9 +1244,9 @@ columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, } /* - * columnar_index_delete_tuples + * pgcolumnar_index_delete_tuples * Opportunistic index tuple deletion. An index entry is deletable exactly - * when its row is no longer visible, i.e. ColumnarReadRowByNumber cannot + * when its row is no longer visible, i.e. PgColumnarReadRowByNumber cannot * return it (deleted via the delete vector). Reporting deletability by actual * liveness is required for correctness: nbtree's deletion pass (including * bottom-up deletion of duplicate keys, which a same-key UPDATE produces) @@ -1266,14 +1266,14 @@ columnar_tuple_satisfies_snapshot(Relation rel, TupleTableSlot *slot, * correct and always-safe answer. */ static TransactionId -columnar_compute_xid_horizon_for_tuples(Relation rel, ItemPointerData *tids, +pgcolumnar_compute_xid_horizon_for_tuples(Relation rel, ItemPointerData *tids, int nitems) { return InvalidTransactionId; } #else static TransactionId -columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) +pgcolumnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) { Snapshot snapshot = ActiveSnapshotSet() ? GetActiveSnapshot() : GetTransactionSnapshot(); @@ -1282,16 +1282,16 @@ columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) for (i = 0; i < delstate->ndeltids; i++) { uint64 rowNumber = - ColumnarItemPointerToRowNumber(&delstate->deltids[i].tid); + PgColumnarItemPointerToRowNumber(&delstate->deltids[i].tid); /* - * Only liveness matters here, and ColumnarRowIsLive decodes nothing to + * Only liveness matters here, and PgColumnarRowIsLive decodes nothing to * answer it. This used to reconstruct every column of the row and then * free the result unread, once per candidate index tuple, on a path * nbtree drives during deletion (issue #157). */ delstate->status[delstate->deltids[i].id].knowndeletable = - !ColumnarRowIsLive(rel, snapshot, rowNumber); + !PgColumnarRowIsLive(rel, snapshot, rowNumber); } return InvalidTransactionId; @@ -1299,7 +1299,7 @@ columnar_index_delete_tuples(Relation rel, TM_IndexDeleteOp *delstate) #endif static void -columnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, CommandId cid, COLUMNAR_TABLE_OPTIONS options, struct BulkInsertStateData *bistate, uint32 specToken) @@ -1308,29 +1308,29 @@ columnar_tuple_insert_speculative(Relation rel, TupleTableSlot *slot, } static void -columnar_tuple_complete_speculative(Relation rel, TupleTableSlot *slot, +pgcolumnar_tuple_complete_speculative(Relation rel, TupleTableSlot *slot, uint32 specToken, bool succeeded) { COLUMNAR_UNSUPPORTED("speculative insert"); } /* - * columnar_tuple_delete + * pgcolumnar_tuple_delete * Mark the row addressed by tid as deleted in the delete vector (spec 9). The * stripe is not rewritten. The tid is the synthetic item pointer the scan * produced, which maps back to the row number. */ static TM_Result -columnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) +pgcolumnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) { - uint64 rowNumber = ColumnarItemPointerToRowNumber(tid); + uint64 rowNumber = PgColumnarItemPointerToRowNumber(tid); - ColumnarMarkRowDeleted(rel, rowNumber); + PgColumnarMarkRowDeleted(rel, rowNumber); return TM_Ok; } /* - * columnar_tuple_update + * pgcolumnar_tuple_update * Update is delete-plus-insert (spec 9): mark the old row deleted in the * delete vector and append the new tuple as a fresh row with a new row number. * The new row's item pointer is published on the slot and index @@ -1339,24 +1339,24 @@ columnar_tuple_delete(COLUMNAR_TUPLE_DELETE_ARGS) * row is now marked deleted (spec 6, 9). */ static TM_Result -columnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) +pgcolumnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) { - uint64 oldRowNumber = ColumnarItemPointerToRowNumber(otid); - ColumnarWriteState *writeState; + uint64 oldRowNumber = PgColumnarItemPointerToRowNumber(otid); + PgColumnarWriteState *writeState; uint64 rowNumber; - ColumnarMarkRowDeleted(rel, oldRowNumber); + PgColumnarMarkRowDeleted(rel, oldRowNumber); - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); slot_getallattrs(slot); /* the new row version is a fresh insert: serialize its unique keys too */ - ColumnarLockUniqueKeys(rel, slot); /* issue #5 */ + PgColumnarLockUniqueKeys(rel, slot); /* issue #5 */ - rowNumber = ColumnarWriteRow(writeState, rel, slot->tts_values, + rowNumber = PgColumnarWriteRow(writeState, rel, slot->tts_values, slot->tts_isnull); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); slot->tts_tableOid = RelationGetRelid(rel); *lockmode = LockTupleExclusive; @@ -1365,7 +1365,7 @@ columnar_tuple_update(COLUMNAR_TUPLE_UPDATE_ARGS) } static TM_Result -columnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, +pgcolumnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, TupleTableSlot *slot, CommandId cid, LockTupleMode mode, LockWaitPolicy wait_policy, uint8 flags, TM_FailureData *tmfd) @@ -1375,19 +1375,19 @@ columnar_tuple_lock(Relation rel, ItemPointer tid, Snapshot snapshot, } static void -columnar_relation_copy_data(Relation rel, const RelFileLocator *newrlocator) +pgcolumnar_relation_copy_data(Relation rel, const RelFileLocator *newrlocator) { COLUMNAR_UNSUPPORTED("relation copy (ALTER TABLE SET TABLESPACE)"); } static void -columnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) +pgcolumnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) { COLUMNAR_UNSUPPORTED("CLUSTER / VACUUM FULL"); } /* - * columnar_index_build_range_scan + * pgcolumnar_index_build_range_scan * Scan every live row of the columnar table and hand it to the index * build callback, so CREATE INDEX (btree or hash) works over a columnar * table (spec 9). Deleted rows (delete vector) are skipped by the reader, so @@ -1400,14 +1400,14 @@ columnar_relation_copy_for_cluster(COLUMNAR_COPY_FOR_CLUSTER_ARGS) * included in the build. */ static double -columnar_index_build_range_scan(Relation table_rel, Relation index_rel, +pgcolumnar_index_build_range_scan(Relation table_rel, Relation index_rel, struct IndexInfo *index_info, bool allow_sync, bool anyvisible, bool progress, BlockNumber start_blockno, BlockNumber numblocks, IndexBuildCallback callback, void *callback_state, TableScanDesc scan) { - ColumnarReadState *readState; + PgColumnarReadState *readState; bool ownReadState; EState *estate; ExprContext *econtext; @@ -1424,8 +1424,8 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, errmsg("columnar: partial-range index build is not supported"))); /* persist buffered rows and delete marks so the build sees them (spec 9) */ - ColumnarFlushWriteStateForRelation(RelationGetRelid(table_rel)); - ColumnarFlushDeleteVectorForRelation(table_rel); + PgColumnarFlushWriteStateForRelation(RelationGetRelid(table_rel)); + PgColumnarFlushDeleteVectorForRelation(table_rel); estate = CreateExecutorState(); econtext = GetPerTupleExprContext(estate); @@ -1439,7 +1439,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * Obtain the reader. A parallel index build passes the TableScanDesc it * opened with table_beginscan_parallel; that scan already holds a reader * bound to the shared parallel scan, whose single-participant claim (see - * columnar_read_start) makes exactly one participant read the whole table. + * pgcolumnar_read_start) makes exactly one participant read the whole table. * We must read through that reader, not a private one: a private full-table * reader in every participant would index every row once per participant, * producing duplicate (key, TID) entries. When no scan is supplied (a serial @@ -1455,7 +1455,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * indexing what the table is now, not what an in-flight rewrite is * converting away from. */ - readState = columnar_scan_read_state((ColumnarScanDesc) scan, + readState = pgcolumnar_scan_read_state((PgColumnarScanDesc) scan, RelationGetDescr(table_rel)); ownReadState = false; } @@ -1468,7 +1468,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, else snapshot = GetTransactionSnapshot(); - readState = ColumnarBeginRead(table_rel, snapshot, NULL, NULL, 0, NULL); + readState = PgColumnarBeginRead(table_rel, snapshot, NULL, NULL, 0, NULL); ownReadState = true; } @@ -1477,7 +1477,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, CHECK_FOR_INTERRUPTS(); ExecClearTuple(slot); - if (!ColumnarReadNextRow(readState, slot->tts_values, slot->tts_isnull, + if (!PgColumnarReadNextRow(readState, slot->tts_values, slot->tts_isnull, &rowNumber)) break; ExecStoreVirtualTuple(slot); @@ -1491,14 +1491,14 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, FormIndexDatum(index_info, slot, estate, indexValues, indexNulls); - ColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); + PgColumnarRowNumberToItemPointer(rowNumber, &slot->tts_tid); callback(index_rel, &slot->tts_tid, indexValues, indexNulls, true, callback_state); } if (ownReadState) - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(slot); FreeExecutorState(estate); @@ -1506,9 +1506,9 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, * The table AM contract makes index_build_range_scan the owner of a scan the * caller supplied: it must end it, exactly as heapam_index_build_range_scan * calls table_endscan on the passed scan whether or not it created the scan - * itself. columnar_scan_begin took a relation reference (and, for a worker, + * itself. pgcolumnar_scan_begin took a relation reference (and, for a worker, * a registered snapshot) and created the reader used above; table_endscan - * runs columnar_scan_end, which ends that reader and releases the reference. + * runs pgcolumnar_scan_end, which ends that reader and releases the reference. * Omitting this leaked one relation reference per build participant, which * surfaced at commit as "resource was not closed: relation". */ @@ -1519,7 +1519,7 @@ columnar_index_build_range_scan(Relation table_rel, Relation index_rel, } static void -columnar_index_validate_scan(Relation table_rel, Relation index_rel, +pgcolumnar_index_validate_scan(Relation table_rel, Relation index_rel, struct IndexInfo *index_info, Snapshot snapshot, struct ValidateIndexState *state) { @@ -1527,7 +1527,7 @@ columnar_index_validate_scan(Relation table_rel, Relation index_rel, } static bool -columnar_scan_sample_next_block(TableScanDesc scan, +pgcolumnar_scan_sample_next_block(TableScanDesc scan, struct SampleScanState *scanstate) { COLUMNAR_UNSUPPORTED("TABLESAMPLE"); @@ -1535,7 +1535,7 @@ columnar_scan_sample_next_block(TableScanDesc scan, } static bool -columnar_scan_sample_next_tuple(TableScanDesc scan, +pgcolumnar_scan_sample_next_tuple(TableScanDesc scan, struct SampleScanState *scanstate, TupleTableSlot *slot) { @@ -1547,67 +1547,67 @@ columnar_scan_sample_next_tuple(TableScanDesc scan, * the routine * ------------------------------------------------------------------------- */ -static const TableAmRoutine columnar_am_methods = { +static const TableAmRoutine pgcolumnar_am_methods = { .type = T_TableAmRoutine, - .slot_callbacks = columnar_slot_callbacks, + .slot_callbacks = pgcolumnar_slot_callbacks, - .scan_begin = columnar_scan_begin, - .scan_end = columnar_scan_end, - .scan_rescan = columnar_scan_rescan, - .scan_getnextslot = columnar_scan_getnextslot, + .scan_begin = pgcolumnar_scan_begin, + .scan_end = pgcolumnar_scan_end, + .scan_rescan = pgcolumnar_scan_rescan, + .scan_getnextslot = pgcolumnar_scan_getnextslot, - .parallelscan_estimate = columnar_parallelscan_estimate, - .parallelscan_initialize = columnar_parallelscan_initialize, - .parallelscan_reinitialize = columnar_parallelscan_reinitialize, + .parallelscan_estimate = pgcolumnar_parallelscan_estimate, + .parallelscan_initialize = pgcolumnar_parallelscan_initialize, + .parallelscan_reinitialize = pgcolumnar_parallelscan_reinitialize, - .index_fetch_begin = columnar_index_fetch_begin, - .index_fetch_reset = columnar_index_fetch_reset, - .index_fetch_end = columnar_index_fetch_end, - .index_fetch_tuple = columnar_index_fetch_tuple, + .index_fetch_begin = pgcolumnar_index_fetch_begin, + .index_fetch_reset = pgcolumnar_index_fetch_reset, + .index_fetch_end = pgcolumnar_index_fetch_end, + .index_fetch_tuple = pgcolumnar_index_fetch_tuple, - .tuple_fetch_row_version = columnar_tuple_fetch_row_version, - .tuple_tid_valid = columnar_tuple_tid_valid, - .tuple_get_latest_tid = columnar_tuple_get_latest_tid, - .tuple_satisfies_snapshot = columnar_tuple_satisfies_snapshot, + .tuple_fetch_row_version = pgcolumnar_tuple_fetch_row_version, + .tuple_tid_valid = pgcolumnar_tuple_tid_valid, + .tuple_get_latest_tid = pgcolumnar_tuple_get_latest_tid, + .tuple_satisfies_snapshot = pgcolumnar_tuple_satisfies_snapshot, #if PG_VERSION_NUM < 140000 - .COLUMNAR_AM_INDEX_DELETE_FIELD = columnar_compute_xid_horizon_for_tuples, + .COLUMNAR_AM_INDEX_DELETE_FIELD = pgcolumnar_compute_xid_horizon_for_tuples, #else - .COLUMNAR_AM_INDEX_DELETE_FIELD = columnar_index_delete_tuples, + .COLUMNAR_AM_INDEX_DELETE_FIELD = pgcolumnar_index_delete_tuples, #endif - .tuple_insert = columnar_tuple_insert, - .tuple_insert_speculative = columnar_tuple_insert_speculative, - .tuple_complete_speculative = columnar_tuple_complete_speculative, - .multi_insert = columnar_multi_insert, - .tuple_delete = columnar_tuple_delete, - .tuple_update = columnar_tuple_update, - .tuple_lock = columnar_tuple_lock, - .finish_bulk_insert = columnar_finish_bulk_insert, - - .COLUMNAR_AM_SET_NEW_FILE_FIELD = columnar_relation_set_new_filelocator, - .relation_nontransactional_truncate = columnar_relation_nontransactional_truncate, - .relation_copy_data = columnar_relation_copy_data, - .relation_copy_for_cluster = columnar_relation_copy_for_cluster, - .relation_vacuum = columnar_relation_vacuum, - .scan_analyze_next_block = columnar_scan_analyze_next_block, - .scan_analyze_next_tuple = columnar_scan_analyze_next_tuple, - .index_build_range_scan = columnar_index_build_range_scan, - .index_validate_scan = columnar_index_validate_scan, - - .relation_size = columnar_relation_size, - .relation_needs_toast_table = columnar_relation_needs_toast_table, - - .relation_estimate_size = columnar_relation_estimate_size, - - .scan_sample_next_block = columnar_scan_sample_next_block, - .scan_sample_next_tuple = columnar_scan_sample_next_tuple, + .tuple_insert = pgcolumnar_tuple_insert, + .tuple_insert_speculative = pgcolumnar_tuple_insert_speculative, + .tuple_complete_speculative = pgcolumnar_tuple_complete_speculative, + .multi_insert = pgcolumnar_multi_insert, + .tuple_delete = pgcolumnar_tuple_delete, + .tuple_update = pgcolumnar_tuple_update, + .tuple_lock = pgcolumnar_tuple_lock, + .finish_bulk_insert = pgcolumnar_finish_bulk_insert, + + .COLUMNAR_AM_SET_NEW_FILE_FIELD = pgcolumnar_relation_set_new_filelocator, + .relation_nontransactional_truncate = pgcolumnar_relation_nontransactional_truncate, + .relation_copy_data = pgcolumnar_relation_copy_data, + .relation_copy_for_cluster = pgcolumnar_relation_copy_for_cluster, + .relation_vacuum = pgcolumnar_relation_vacuum, + .scan_analyze_next_block = pgcolumnar_scan_analyze_next_block, + .scan_analyze_next_tuple = pgcolumnar_scan_analyze_next_tuple, + .index_build_range_scan = pgcolumnar_index_build_range_scan, + .index_validate_scan = pgcolumnar_index_validate_scan, + + .relation_size = pgcolumnar_relation_size, + .relation_needs_toast_table = pgcolumnar_relation_needs_toast_table, + + .relation_estimate_size = pgcolumnar_relation_estimate_size, + + .scan_sample_next_block = pgcolumnar_scan_sample_next_block, + .scan_sample_next_tuple = pgcolumnar_scan_sample_next_tuple, }; Datum -columnar_handler(PG_FUNCTION_ARGS) +pgcolumnar_handler(PG_FUNCTION_ARGS) { - PG_RETURN_POINTER(&columnar_am_methods); + PG_RETURN_POINTER(&pgcolumnar_am_methods); } /* ------------------------------------------------------------------------- @@ -1615,23 +1615,23 @@ columnar_handler(PG_FUNCTION_ARGS) * ------------------------------------------------------------------------- */ static void -columnar_xact_callback(XactEvent event, void *arg) +pgcolumnar_xact_callback(XactEvent event, void *arg) { switch (event) { case XACT_EVENT_PRE_COMMIT: case XACT_EVENT_PARALLEL_PRE_COMMIT: case XACT_EVENT_PREPARE: - ColumnarFlushAllPendingWrites(); - ColumnarFlushAllDeleteVectors(); + PgColumnarFlushAllPendingWrites(); + PgColumnarFlushAllDeleteVectors(); break; case XACT_EVENT_COMMIT: case XACT_EVENT_ABORT: case XACT_EVENT_PARALLEL_COMMIT: case XACT_EVENT_PARALLEL_ABORT: - ColumnarDiscardAllPendingWrites(); - ColumnarDiscardAllDeleteVectors(); - ColumnarDiscardFetchCache(); + PgColumnarDiscardAllPendingWrites(); + PgColumnarDiscardAllDeleteVectors(); + PgColumnarDiscardFetchCache(); break; default: break; @@ -1643,18 +1643,18 @@ columnar_xact_callback(XactEvent event, void *arg) * ------------------------------------------------------------------------- */ static void -columnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, +pgcolumnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, SubTransactionId parentSubid, void *arg) { switch (event) { case SUBXACT_EVENT_ABORT_SUB: - ColumnarWriteStateDiscardSubXact(mySubid); - ColumnarDeleteVectorDiscardSubXact(mySubid); + PgColumnarWriteStateDiscardSubXact(mySubid); + PgColumnarDeleteVectorDiscardSubXact(mySubid); break; case SUBXACT_EVENT_COMMIT_SUB: - ColumnarWriteStatePromoteSubXact(mySubid, parentSubid); - ColumnarDeleteVectorPromoteSubXact(mySubid, parentSubid); + PgColumnarWriteStatePromoteSubXact(mySubid, parentSubid); + PgColumnarDeleteVectorPromoteSubXact(mySubid, parentSubid); break; default: break; @@ -1674,15 +1674,15 @@ columnar_subxact_callback(SubXactEvent event, SubTransactionId mySubid, * ------------------------------------------------------------------------- */ static void -columnar_executor_end(QueryDesc *queryDesc) +pgcolumnar_executor_end(QueryDesc *queryDesc) { if (prev_executor_end_hook) prev_executor_end_hook(queryDesc); else standard_ExecutorEnd(queryDesc); - ColumnarFlushAllPendingWrites(); - ColumnarFlushAllDeleteVectors(); + PgColumnarFlushAllPendingWrites(); + PgColumnarFlushAllDeleteVectors(); /* * The fetch cache is scoped to a statement, so release it here rather than @@ -1690,7 +1690,7 @@ columnar_executor_end(QueryDesc *queryDesc) * slot pins them for the rest of the transaction, and a session sitting idle * in transaction after one UPDATE holds them indefinitely. */ - ColumnarDiscardFetchCache(); + PgColumnarDiscardFetchCache(); } /* ------------------------------------------------------------------------- @@ -1699,7 +1699,7 @@ columnar_executor_end(QueryDesc *queryDesc) * ------------------------------------------------------------------------- */ /* - * columnar_reject_fk_to_columnar + * pgcolumnar_reject_fk_to_columnar * Raise on a foreign key whose referenced side is a columnar table. * * The referential-integrity check reads the referenced row with FOR KEY SHARE, @@ -1726,7 +1726,7 @@ columnar_executor_end(QueryDesc *queryDesc) * through the ordinary insert path. */ static void -columnar_reject_fk_to_columnar(Oid constraintId) +pgcolumnar_reject_fk_to_columnar(Oid constraintId) { Relation conRel; SysScanDesc scan; @@ -1763,7 +1763,7 @@ columnar_reject_fk_to_columnar(Oid constraintId) if (get_rel_relkind(referenced) != RELKIND_RELATION) return; - if (ColumnarIsColumnarRelation(referenced)) + if (PgColumnarIsColumnarRelation(referenced)) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("cannot create a foreign key referencing columnar table \"%s\"", @@ -1775,14 +1775,14 @@ columnar_reject_fk_to_columnar(Oid constraintId) } /* - * columnar_am_is_columnar + * pgcolumnar_am_is_columnar * Does this table access method oid resolve to this extension's routine? * * By handler identity rather than by name, so a second name for the same access * method is still recognised. */ static bool -columnar_am_is_columnar(Oid amoid) +pgcolumnar_am_is_columnar(Oid amoid) { HeapTuple tup; Form_pg_am amform; @@ -1810,15 +1810,15 @@ columnar_am_is_columnar(Oid amoid) } ReleaseSysCache(tup); - return GetTableAmRoutine(handler) == &columnar_am_methods; + return GetTableAmRoutine(handler) == &pgcolumnar_am_methods; } /* - * columnar_fk_referencing + * pgcolumnar_fk_referencing * Name of some foreign key whose referenced side is relid, or NULL. */ static char * -columnar_fk_referencing(Oid relid) +pgcolumnar_fk_referencing(Oid relid) { Relation conRel; SysScanDesc scan; @@ -1852,11 +1852,11 @@ columnar_fk_referencing(Oid relid) } /* - * columnar_reject_set_am_to_columnar + * pgcolumnar_reject_set_am_to_columnar * Refuse ALTER TABLE ... SET ACCESS METHOD to columnar when the relation is * the referenced side of a foreign key. * - * columnar_reject_fk_to_columnar closes this door where the constraint is + * pgcolumnar_reject_fk_to_columnar closes this door where the constraint is * created. The same unusable configuration is reachable from the other end, by * making the referenced table columnar after the constraint already exists: * @@ -1883,7 +1883,7 @@ columnar_fk_referencing(Oid relid) * every supported major it executes this same statement. */ static void -columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) +pgcolumnar_reject_set_am_to_columnar(AlterTableStmt *stmt) { Oid relid; ListCell *lc; @@ -1934,10 +1934,10 @@ columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) /* SET ACCESS METHOD DEFAULT leaves the name unset (PG17+) */ amname = cmd->name ? cmd->name : default_table_access_method; - if (!columnar_am_is_columnar(get_table_am_oid(amname, true))) + if (!pgcolumnar_am_is_columnar(get_table_am_oid(amname, true))) continue; - conname = columnar_fk_referencing(relid); + conname = pgcolumnar_fk_referencing(relid); if (conname == NULL) continue; @@ -1954,7 +1954,7 @@ columnar_reject_set_am_to_columnar(AlterTableStmt *stmt) } static void -columnar_process_utility(PlannedStmt *pstmt, const char *queryString, +pgcolumnar_process_utility(PlannedStmt *pstmt, const char *queryString, bool readOnlyTree, ProcessUtilityContext context, ParamListInfo params, QueryEnvironment *queryEnv, DestReceiver *dest, QueryCompletion *qc) @@ -1963,7 +1963,7 @@ columnar_process_utility(PlannedStmt *pstmt, const char *queryString, /* read-only inspection, so readOnlyTree needs no copy of the tree */ if (parsetree != NULL && IsA(parsetree, AlterTableStmt)) - columnar_reject_set_am_to_columnar((AlterTableStmt *) parsetree); + pgcolumnar_reject_set_am_to_columnar((AlterTableStmt *) parsetree); if (prev_process_utility_hook) prev_process_utility_hook(pstmt, queryString, readOnlyTree, context, @@ -1974,14 +1974,14 @@ columnar_process_utility(PlannedStmt *pstmt, const char *queryString, } static void -columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, +pgcolumnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, int subId, void *arg) { if (prev_object_access_hook) prev_object_access_hook(access, classId, objectId, subId, arg); if (access == OAT_POST_CREATE && classId == ConstraintRelationId) - columnar_reject_fk_to_columnar(objectId); + pgcolumnar_reject_fk_to_columnar(objectId); if (access == OAT_DROP && classId == RelationRelationId && subId == 0) { @@ -1993,10 +1993,10 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, /* DROP already holds AccessExclusiveLock on the relation */ rel = relation_open(objectId, NoLock); - if (rel->rd_tableam == &columnar_am_methods) + if (rel->rd_tableam == &pgcolumnar_am_methods) { - uint64 storageId = ColumnarStorageId(rel); - List *projs = ColumnarListProjections(storageId); + uint64 storageId = PgColumnarStorageId(rel); + List *projs = PgColumnarListProjections(storageId); ListCell *lc; /* @@ -2005,20 +2005,20 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * projection's row groups, chunks, zone maps and bloom filters * behind with no relation to reach them from: metadata for a table * that no longer exists, accumulating one projection's worth per - * drop. This is the same loop columnar_vacuum.c runs when it + * drop. This is the same loop pgcolumnar_vacuum.c runs when it * rewrites into fresh storage. */ foreach(lc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != storageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(storageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(storageId, p->projectionId); } - ColumnarDeleteMetadata(storageId); - ColumnarDeleteOptions(objectId); + PgColumnarDeleteMetadata(storageId); + PgColumnarDeleteOptions(objectId); /* * And the projection declarations, for the same reason and in the * same place (#304). A declaration left behind holds a regclass that @@ -2026,7 +2026,7 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * rebuild_projections() aborts on, taking every other table in the * database with it. */ - ColumnarDeleteProjectionDeclarationsForRel(objectId); + PgColumnarDeleteProjectionDeclarationsForRel(objectId); } relation_close(rel, NoLock); @@ -2052,13 +2052,13 @@ columnar_object_access(ObjectAccessType access, Oid classId, Oid objectId, * ------------------------------------------------------------------------- */ static bool -columnar_relation_is_columnar(Oid relid) +pgcolumnar_relation_is_columnar(Oid relid) { - if (columnar_am_oid_cache == InvalidOid) - columnar_am_oid_cache = get_am_oid("pgcolumnar", true); + if (pgcolumnar_am_oid_cache == InvalidOid) + pgcolumnar_am_oid_cache = get_am_oid("pgcolumnar", true); - return OidIsValid(columnar_am_oid_cache) && - get_rel_relam(relid) == columnar_am_oid_cache; + return OidIsValid(pgcolumnar_am_oid_cache) && + get_rel_relam(relid) == pgcolumnar_am_oid_cache; } /* GUC: when on, allow the planner to build index-only-scan paths for columnar @@ -2067,22 +2067,22 @@ columnar_relation_is_columnar(Oid relid) * horizon accounts for open snapshots and every write clears the bit, both * WAL-logged), and a not-all-visible block always falls back to the * snapshot-checked fetch, so results are correct regardless. */ -bool columnar_enable_index_only_scan = true; +bool pgcolumnar_enable_index_only_scan = true; /* clear the "can return" flags of every index on a columnar relation */ static void -columnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) +pgcolumnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) { ListCell *lc; /* when index-only scans are enabled, leave the index canreturn flags intact * so the planner may choose an IOS; the VM fork (set by lazy vacuum) drives * whether the executor skips the fetch, and a not-all-visible block still - * falls back to columnar_index_fetch_tuple, so results are always correct. */ - if (columnar_enable_index_only_scan) + * falls back to pgcolumnar_index_fetch_tuple, so results are always correct. */ + if (pgcolumnar_enable_index_only_scan) return; - if (!OidIsValid(relid) || !columnar_relation_is_columnar(relid)) + if (!OidIsValid(relid) || !pgcolumnar_relation_is_columnar(relid)) return; foreach(lc, rel->indexlist) @@ -2100,24 +2100,24 @@ columnar_forbid_index_only_scan(Oid relid, RelOptInfo *rel) #if PG_VERSION_NUM >= 190000 static void -columnar_build_simple_rel(PlannerInfo *root, RelOptInfo *rel, +pgcolumnar_build_simple_rel(PlannerInfo *root, RelOptInfo *rel, RangeTblEntry *rte) { if (prev_build_simple_rel_hook) prev_build_simple_rel_hook(root, rel, rte); if (rte->rtekind == RTE_RELATION) - columnar_forbid_index_only_scan(rte->relid, rel); + pgcolumnar_forbid_index_only_scan(rte->relid, rel); } #else static void -columnar_get_relation_info(PlannerInfo *root, Oid relationObjectId, +pgcolumnar_get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent, RelOptInfo *rel) { if (prev_get_relation_info_hook) prev_get_relation_info_hook(root, relationObjectId, inhparent, rel); - columnar_forbid_index_only_scan(relationObjectId, rel); + pgcolumnar_forbid_index_only_scan(relationObjectId, rel); } #endif @@ -2130,22 +2130,22 @@ _PG_init(void) { /* * Virtual slot behaviour, except that a copied heap tuple keeps the slot's - * item pointer. See the comment on columnar_slot_copy_heap_tuple. + * item pointer. See the comment on pgcolumnar_slot_copy_heap_tuple. */ - ColumnarSlotOps = TTSOpsVirtual; - ColumnarSlotOps.base_slot_size = sizeof(ColumnarSlot); - ColumnarSlotOps.copy_heap_tuple = columnar_slot_copy_heap_tuple; - ColumnarSlotOps.init = columnar_slot_init; - ColumnarSlotOps.getsomeattrs = columnar_slot_getsomeattrs; - ColumnarSlotOps.clear = columnar_slot_clear; - ColumnarSlotOps.materialize = columnar_slot_materialize; - ColumnarSlotOps.copyslot = columnar_slot_copyslot; - ColumnarSlotOps.copy_minimal_tuple = columnar_slot_copy_minimal_tuple; + PgColumnarSlotOps = TTSOpsVirtual; + PgColumnarSlotOps.base_slot_size = sizeof(PgColumnarSlot); + PgColumnarSlotOps.copy_heap_tuple = pgcolumnar_slot_copy_heap_tuple; + PgColumnarSlotOps.init = pgcolumnar_slot_init; + PgColumnarSlotOps.getsomeattrs = pgcolumnar_slot_getsomeattrs; + PgColumnarSlotOps.clear = pgcolumnar_slot_clear; + PgColumnarSlotOps.materialize = pgcolumnar_slot_materialize; + PgColumnarSlotOps.copyslot = pgcolumnar_slot_copyslot; + PgColumnarSlotOps.copy_minimal_tuple = pgcolumnar_slot_copy_minimal_tuple; DefineCustomIntVariable("pgcolumnar.stripe_row_limit", "Maximum number of rows per stripe.", NULL, - &columnar_stripe_row_limit, + &pgcolumnar_stripe_row_limit, 150000, 1000, INT_MAX, PGC_USERSET, @@ -2155,7 +2155,7 @@ _PG_init(void) DefineCustomIntVariable("pgcolumnar.chunk_group_row_limit", "Maximum number of rows per chunk group.", NULL, - &columnar_chunk_group_row_limit, + &pgcolumnar_chunk_group_row_limit, 10000, 100, INT_MAX, PGC_USERSET, @@ -2171,7 +2171,7 @@ _PG_init(void) "value below 128 is treated as 0, because a sample that " "small cannot rank candidates: every candidate's fixed " "header would exceed the sample itself.", - &columnar_encoding_sample_rows, + &pgcolumnar_encoding_sample_rows, 2048, 0, INT_MAX, PGC_USERSET, @@ -2181,9 +2181,9 @@ _PG_init(void) DefineCustomEnumVariable("pgcolumnar.compression", "Default compression codec for new chunks.", NULL, - &columnar_compression, + &pgcolumnar_compression, COLUMNAR_COMPRESSION_ZSTD, - columnar_compression_options, + pgcolumnar_compression_options, PGC_USERSET, 0, NULL, NULL, NULL); @@ -2191,7 +2191,7 @@ _PG_init(void) DefineCustomIntVariable("pgcolumnar.compression_level", "Compression level for the zstd codec.", NULL, - &columnar_compression_level, + &pgcolumnar_compression_level, 3, 1, 22, PGC_USERSET, @@ -2212,7 +2212,7 @@ _PG_init(void) "and saves roughly a third of their load time; where FSST " "wins clearly it changes nothing. Set to 0 to keep FSST on " "any win at all.", - &columnar_fsst_min_gain_percent, + &pgcolumnar_fsst_min_gain_percent, 5, 0, 99, PGC_USERSET, @@ -2222,7 +2222,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_qual_pushdown", "Push scan qualifiers down for chunk-group skipping.", NULL, - &columnar_enable_qual_pushdown, + &pgcolumnar_enable_qual_pushdown, true, PGC_USERSET, 0, @@ -2234,7 +2234,7 @@ _PG_init(void) "read and decoded, as before the projection was honored. " "Provided as an escape hatch and as the A/B oracle the " "projection tests compare against.", - &columnar_enable_column_projection, + &pgcolumnar_enable_column_projection, true, PGC_USERSET, 0, @@ -2243,7 +2243,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_custom_scan", "Use the columnar custom scan path for columnar tables.", NULL, - &columnar_enable_custom_scan, + &pgcolumnar_enable_custom_scan, true, PGC_USERSET, 0, @@ -2252,7 +2252,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_vectorization", "Use the vectorized aggregate fast path.", NULL, - &columnar_enable_vectorization, + &pgcolumnar_enable_vectorization, true, PGC_USERSET, 0, @@ -2261,7 +2261,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_group_vectorization", "Use the vectorized aggregate fast path for GROUP BY queries.", NULL, - &columnar_enable_group_vectorization, + &pgcolumnar_enable_group_vectorization, false, PGC_USERSET, 0, @@ -2272,7 +2272,7 @@ _PG_init(void) "aggregates with a filter or sum/avg over " "int8/float/numeric.", NULL, - &columnar_enable_ungrouped_vector_agg, + &pgcolumnar_enable_ungrouped_vector_agg, false, PGC_USERSET, 0, @@ -2283,7 +2283,7 @@ _PG_init(void) "each worker folds distinct row groups and emits a partial " "aggregate a core Finalize combines.", NULL, - &columnar_enable_parallel_vector_agg, + &pgcolumnar_enable_parallel_vector_agg, false, PGC_USERSET, 0, @@ -2296,7 +2296,7 @@ _PG_init(void) "not the planner's estimate: over the cap the query errors " "rather than falling back, since the plan is fixed by then. " "Raise it, or turn off pgcolumnar.enable_group_vectorization.", - &columnar_groupagg_max_groups, + &pgcolumnar_groupagg_max_groups, 1000000, 1, INT_MAX, PGC_USERSET, 0, @@ -2305,7 +2305,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_bloom_filter", "Skip chunk groups on equality using per-chunk bloom filters.", NULL, - &columnar_enable_bloom_filter, + &pgcolumnar_enable_bloom_filter, true, PGC_USERSET, 0, @@ -2316,7 +2316,7 @@ _PG_init(void) "adjacent freed ranges, so compaction reclaims space " "under fragmentation. Off reverts to whole-range reuse.", NULL, - &columnar_reclaim_coalesce, + &pgcolumnar_reclaim_coalesce, true, PGC_USERSET, 0, @@ -2327,7 +2327,7 @@ _PG_init(void) "trailing reclaimed blocks to the OS. Off (the default) " "makes truncate() a no-op.", NULL, - &columnar_enable_end_truncation, + &pgcolumnar_enable_end_truncation, false, PGC_SUSET, 0, @@ -2336,7 +2336,7 @@ _PG_init(void) DefineCustomBoolVariable("pgcolumnar.enable_read_stream", "Prefetch block reads with the read stream API (PostgreSQL 17+).", NULL, - &columnar_enable_read_stream, + &pgcolumnar_enable_read_stream, true, PGC_USERSET, 0, @@ -2347,7 +2347,7 @@ _PG_init(void) "visibility-map fork (gap 28). On by default; set off to force " "a plain index scan.", NULL, - &columnar_enable_index_only_scan, + &pgcolumnar_enable_index_only_scan, true, PGC_USERSET, 0, @@ -2357,7 +2357,7 @@ _PG_init(void) "Let the planner scan a covering projection instead of the " "base table when one serves the query better (gap 26).", NULL, - &columnar_enable_projection_scan, + &pgcolumnar_enable_projection_scan, true, PGC_USERSET, 0, @@ -2370,7 +2370,7 @@ _PG_init(void) "lives in, so an unclustered ordered index scan can cost far " "more than core's per-page estimate. Off restores the " "unpenalized planner behaviour.", - &columnar_enable_index_fetch_penalty, + &pgcolumnar_enable_index_fetch_penalty, true, PGC_USERSET, 0, @@ -2385,7 +2385,7 @@ _PG_init(void) "when the creation lock guards nothing, so an ordinary write still " "behaves correctly. Off by default leaves the write path unchanged.", NULL, - &columnar_bulk_parallel_writer, + &pgcolumnar_bulk_parallel_writer, false, PGC_USERSET, GUC_NOT_IN_SAMPLE, @@ -2397,7 +2397,7 @@ _PG_init(void) "index key so overlapping same-key inserts conflict " "correctly (issue #5). Turning it off restores the " "prior racy behavior.", - &columnar_enable_unique_lock, + &pgcolumnar_enable_unique_lock, true, PGC_USERSET, 0, @@ -2414,7 +2414,7 @@ _PG_init(void) "backends inserting the same key must compute the " "same bucket, which they only do when they agree on " "this value.", - &columnar_unique_lock_buckets, + &pgcolumnar_unique_lock_buckets, 128, 1, 1048576, PGC_POSTMASTER, @@ -2423,17 +2423,17 @@ _PG_init(void) MarkGUCPrefixReserved("pgcolumnar"); - RegisterXactCallback(columnar_xact_callback, NULL); - RegisterSubXactCallback(columnar_subxact_callback, NULL); + RegisterXactCallback(pgcolumnar_xact_callback, NULL); + RegisterSubXactCallback(pgcolumnar_subxact_callback, NULL); prev_object_access_hook = object_access_hook; - object_access_hook = columnar_object_access; + object_access_hook = pgcolumnar_object_access; prev_process_utility_hook = ProcessUtility_hook; - ProcessUtility_hook = columnar_process_utility; + ProcessUtility_hook = pgcolumnar_process_utility; prev_executor_end_hook = ExecutorEnd_hook; - ExecutorEnd_hook = columnar_executor_end; + ExecutorEnd_hook = pgcolumnar_executor_end; /* * Forbid index-only scans on columnar tables. PG19 replaced @@ -2442,18 +2442,18 @@ _PG_init(void) */ #if PG_VERSION_NUM >= 190000 prev_build_simple_rel_hook = build_simple_rel_hook; - build_simple_rel_hook = columnar_build_simple_rel; + build_simple_rel_hook = pgcolumnar_build_simple_rel; #else prev_get_relation_info_hook = get_relation_info_hook; - get_relation_info_hook = columnar_get_relation_info; + get_relation_info_hook = pgcolumnar_get_relation_info; #endif /* register the custom scan provider and install the pathlist hook */ - ColumnarCustomScanInit(); + PgColumnarCustomScanInit(); /* install the vectorized-aggregate upper-path hook (spec 9) */ - ColumnarVectorInit(); + PgColumnarVectorInit(); /* register the unique-index cache invalidation callback (issue #5) */ - ColumnarUniqueInit(); + PgColumnarUniqueInit(); } diff --git a/src/columnar_thrift.c b/src/columnar_thrift.c index 718d49c..6fd48aa 100644 --- a/src/columnar_thrift.c +++ b/src/columnar_thrift.c @@ -1,9 +1,9 @@ /*------------------------------------------------------------------------- * - * columnar_thrift.c + * pgcolumnar_thrift.c * Thrift compact-protocol reader and writer. * - * See columnar_thrift.h. Nothing here knows about Parquet or about pgColumnar. + * See pgcolumnar_thrift.h. Nothing here knows about Parquet or about pgColumnar. * The two directions were previously in separate files, the reader inside the * Parquet import module and the writer inside the export module, which put the * encode and decode of the same wire format a thousand lines apart. @@ -25,7 +25,7 @@ * ------------------------------------------------------------------------- */ uint64 -ColumnarThriftVarint(TCReader *r) +PgColumnarThriftVarint(TCReader *r) { uint64 v = 0; int shift = 0; @@ -46,18 +46,18 @@ ColumnarThriftVarint(TCReader *r) } int64 -ColumnarThriftZigzag(TCReader *r) +PgColumnarThriftZigzag(TCReader *r) { - uint64 u = ColumnarThriftVarint(r); + uint64 u = PgColumnarThriftVarint(r); return (int64) (u >> 1) ^ -(int64) (u & 1); } /* read a binary/string field: returns pointer into the buffer and its length */ const uint8 * -ColumnarThriftBytes(TCReader *r, uint32 *outlen) +PgColumnarThriftBytes(TCReader *r, uint32 *outlen) { - uint64 n = ColumnarThriftVarint(r); + uint64 n = PgColumnarThriftVarint(r); const uint8 *p; /* @@ -88,7 +88,7 @@ ColumnarThriftBytes(TCReader *r, uint32 *outlen) * short-form delta encoding. */ void -ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) +PgColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) { uint8 b; @@ -108,13 +108,13 @@ ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId) if ((b >> 4) != 0) *fid = *lastId + (b >> 4); /* short-form delta */ else - *fid = (int) ColumnarThriftZigzag(r); /* long form */ + *fid = (int) PgColumnarThriftZigzag(r); /* long form */ *lastId = *fid; } /* skip a value of the given compact type (for fields we do not consume) */ void -ColumnarThriftSkip(TCReader *r, int ftype) +PgColumnarThriftSkip(TCReader *r, int ftype) { /* * A crafted footer can nest structs (or lists of structs) to any depth, and @@ -136,7 +136,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) case TC_I16: case TC_I32: case TC_I64: - (void) ColumnarThriftZigzag(r); + (void) PgColumnarThriftZigzag(r); break; case TC_DOUBLE: r->pos += 8; @@ -145,7 +145,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) { uint32 n; - (void) ColumnarThriftBytes(r, &n); + (void) PgColumnarThriftBytes(r, &n); break; } case TC_LIST: @@ -165,9 +165,9 @@ ColumnarThriftSkip(TCReader *r, int ftype) size = (sizeType >> 4) & 0x0f; et = sizeType & 0x0f; if (size == 0x0f) - size = (uint32) ColumnarThriftVarint(r); + size = (uint32) PgColumnarThriftVarint(r); for (i = 0; i < size && !r->error; i++) - ColumnarThriftSkip(r, et); + PgColumnarThriftSkip(r, et); break; } case TC_STRUCT: @@ -179,10 +179,10 @@ ColumnarThriftSkip(TCReader *r, int ftype) int ft, fid; - ColumnarThriftField(r, &ft, &fid, &lastId); + PgColumnarThriftField(r, &ft, &fid, &lastId); if (ft == TC_STOP || r->error) break; - ColumnarThriftSkip(r, ft); + PgColumnarThriftSkip(r, ft); } break; } @@ -193,7 +193,7 @@ ColumnarThriftSkip(TCReader *r, int ftype) /* list header: returns element count and element compact type */ uint32 -ColumnarThriftListHeader(TCReader *r, int *etype) +PgColumnarThriftListHeader(TCReader *r, int *etype) { uint8 b; uint32 size; @@ -208,14 +208,14 @@ ColumnarThriftListHeader(TCReader *r, int *etype) size = (b >> 4) & 0x0f; *etype = b & 0x0f; if (size == 0x0f) - size = (uint32) ColumnarThriftVarint(r); + size = (uint32) PgColumnarThriftVarint(r); return size; } /* ---- Thrift compact-protocol writer (into a StringInfo) ---- */ void -ColumnarThriftPutVarint(StringInfo b, uint64 v) +PgColumnarThriftPutVarint(StringInfo b, uint64 v) { while (v >= 0x80) { @@ -226,20 +226,20 @@ ColumnarThriftPutVarint(StringInfo b, uint64 v) } void -ColumnarThriftPutZigzag32(StringInfo b, int32 v) +PgColumnarThriftPutZigzag32(StringInfo b, int32 v) { - ColumnarThriftPutVarint(b, (uint32) ((v << 1) ^ (v >> 31))); + PgColumnarThriftPutVarint(b, (uint32) ((v << 1) ^ (v >> 31))); } void -ColumnarThriftPutZigzag64(StringInfo b, int64 v) +PgColumnarThriftPutZigzag64(StringInfo b, int64 v) { - ColumnarThriftPutVarint(b, (uint64) ((v << 1) ^ (v >> 63))); + PgColumnarThriftPutVarint(b, (uint64) ((v << 1) ^ (v >> 63))); } /* field header with delta-encoded id */ void -ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) +PgColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) { int delta = id - *lastId; @@ -248,49 +248,49 @@ ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type) else { appendStringInfoChar(b, (char) type); - ColumnarThriftPutZigzag32(b, id); + PgColumnarThriftPutZigzag32(b, id); } *lastId = id; } void -ColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v) +PgColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v) { - ColumnarThriftPutField(b, lastId, id, TC_I32); - ColumnarThriftPutZigzag32(b, v); + PgColumnarThriftPutField(b, lastId, id, TC_I32); + PgColumnarThriftPutZigzag32(b, v); } void -ColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v) +PgColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v) { - ColumnarThriftPutField(b, lastId, id, TC_I64); - ColumnarThriftPutZigzag64(b, v); + PgColumnarThriftPutField(b, lastId, id, TC_I64); + PgColumnarThriftPutZigzag64(b, v); } void -ColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len) +PgColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len) { - ColumnarThriftPutField(b, lastId, id, TC_BINARY); - ColumnarThriftPutVarint(b, (uint64) len); + PgColumnarThriftPutField(b, lastId, id, TC_BINARY); + PgColumnarThriftPutVarint(b, (uint64) len); if (len > 0) appendBinaryStringInfo(b, s, len); } /* list header; caller then appends the elements */ void -ColumnarThriftPutListHeader(StringInfo b, int size, int elemType) +PgColumnarThriftPutListHeader(StringInfo b, int size, int elemType) { if (size < 15) appendStringInfoChar(b, (char) ((size << 4) | elemType)); else { appendStringInfoChar(b, (char) (0xF0 | elemType)); - ColumnarThriftPutVarint(b, (uint64) size); + PgColumnarThriftPutVarint(b, (uint64) size); } } void -ColumnarThriftPutStop(StringInfo b) +PgColumnarThriftPutStop(StringInfo b) { appendStringInfoChar(b, (char) TC_STOP); } diff --git a/src/columnar_thrift.h b/src/columnar_thrift.h index 0d60041..d25309c 100644 --- a/src/columnar_thrift.h +++ b/src/columnar_thrift.h @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_thrift.h + * pgcolumnar_thrift.h * Thrift compact-protocol reader and writer. * * The protocol is the container Parquet uses for its file metadata, and nothing @@ -33,24 +33,24 @@ typedef struct TCReader bool error; } TCReader; -extern uint64 ColumnarThriftVarint(TCReader *r); -extern int64 ColumnarThriftZigzag(TCReader *r); -extern const uint8 *ColumnarThriftBytes(TCReader *r, uint32 *outlen); -extern void ColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId); -extern void ColumnarThriftSkip(TCReader *r, int ftype); -extern uint32 ColumnarThriftListHeader(TCReader *r, int *etype); +extern uint64 PgColumnarThriftVarint(TCReader *r); +extern int64 PgColumnarThriftZigzag(TCReader *r); +extern const uint8 *PgColumnarThriftBytes(TCReader *r, uint32 *outlen); +extern void PgColumnarThriftField(TCReader *r, int *ftype, int *fid, int *lastId); +extern void PgColumnarThriftSkip(TCReader *r, int ftype); +extern uint32 PgColumnarThriftListHeader(TCReader *r, int *etype); /* ---- writer: appends into a StringInfo ---- */ -extern void ColumnarThriftPutVarint(StringInfo b, uint64 v); -extern void ColumnarThriftPutZigzag32(StringInfo b, int32 v); -extern void ColumnarThriftPutZigzag64(StringInfo b, int64 v); -extern void ColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type); -extern void ColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v); -extern void ColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v); -extern void ColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, +extern void PgColumnarThriftPutVarint(StringInfo b, uint64 v); +extern void PgColumnarThriftPutZigzag32(StringInfo b, int32 v); +extern void PgColumnarThriftPutZigzag64(StringInfo b, int64 v); +extern void PgColumnarThriftPutField(StringInfo b, int16 *lastId, int16 id, int type); +extern void PgColumnarThriftPutI32Field(StringInfo b, int16 *lastId, int16 id, int32 v); +extern void PgColumnarThriftPutI64Field(StringInfo b, int16 *lastId, int16 id, int64 v); +extern void PgColumnarThriftPutStringField(StringInfo b, int16 *lastId, int16 id, const char *s, int len); -extern void ColumnarThriftPutListHeader(StringInfo b, int size, int elemType); -extern void ColumnarThriftPutStop(StringInfo b); +extern void PgColumnarThriftPutListHeader(StringInfo b, int size, int elemType); +extern void PgColumnarThriftPutStop(StringInfo b); #endif /* PGCOLUMNAR_THRIFT_H */ diff --git a/src/columnar_unique.c b/src/columnar_unique.c index 8ae9e2a..5367659 100644 --- a/src/columnar_unique.c +++ b/src/columnar_unique.c @@ -1,20 +1,20 @@ /*------------------------------------------------------------------------- * - * columnar_unique.c + * pgcolumnar_unique.c * Concurrent unique-key insert serialization for pgColumnar (issue #5). * * A columnar row's data is invisible to other backends until its stripe is * flushed at statement end, but its btree index entry (with the eagerly * reserved synthetic TID) is written immediately. PostgreSQL's dirty-snapshot * uniqueness check (_bt_check_unique -> table_index_fetch_tuple_check) resolves - * that TID through columnar_index_fetch_tuple, which returns false for a row + * that TID through pgcolumnar_index_fetch_tuple, which returns false for a row * still buffered in another backend's private write state. So two transactions * inserting the same unique key in overlapping windows can both miss the * conflict and both commit (see design/ISSUE_5_ANALYSIS.md, section A). * * The fix here serializes inserters of the SAME unique key. Before a row is * handed back to the executor's index maintenance, the table AM insert paths - * call ColumnarLockUniqueKeys, which takes a transaction-scoped advisory lock + * call PgColumnarLockUniqueKeys, which takes a transaction-scoped advisory lock * (the same SET_LOCKTAG_ADVISORY primitive used by the issue #4 delete_vector lock) * keyed by the row's unique key value(s). Because the lock is held to commit, * when a second inserter finally acquires it the first inserter has either @@ -55,8 +55,8 @@ #include "utils/typcache.h" /* GUCs (spec 8.3): default on, bounded bucket count to bound the lock budget */ -bool columnar_enable_unique_lock = true; -int columnar_unique_lock_buckets = 128; +bool pgcolumnar_enable_unique_lock = true; +int pgcolumnar_unique_lock_buckets = 128; /* * Advisory-lock discriminator in locktag_field4. The issue #4 delete_vector lock @@ -116,7 +116,7 @@ static HTAB *RelUniqueCache = NULL; * (an invalidation on an index carries the index's relid, not the table's). */ static void -columnar_unique_invalidate(Datum arg, Oid relid) +pgcolumnar_unique_invalidate(Datum arg, Oid relid) { HASH_SEQ_STATUS status; RelUniqueCacheEntry *entry; @@ -136,7 +136,7 @@ columnar_unique_invalidate(Datum arg, Oid relid) } static void -columnar_unique_cache_init(void) +pgcolumnar_unique_cache_init(void) { HASHCTL ctl; @@ -160,7 +160,7 @@ columnar_unique_cache_init(void) * no hash support. */ static bool -columnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, +pgcolumnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, UniqueKeyCol *out) { Oid keyType = indexRel->rd_opcintype[c]; @@ -191,7 +191,7 @@ columnar_key_col_hash(Relation indexRel, int c, MemoryContext cxt, /* Build (or rebuild) the cache entry for rel. */ static RelUniqueCacheEntry * -columnar_unique_build(Relation rel) +pgcolumnar_unique_build(Relation rel) { Oid relid = RelationGetRelid(rel); MemoryContext cxt; @@ -202,7 +202,7 @@ columnar_unique_build(Relation rel) int nIndexes = 0; bool found; - columnar_unique_cache_init(); + pgcolumnar_unique_cache_init(); cxt = AllocSetContextCreate(CacheMemoryContext, "columnar unique index info", @@ -254,7 +254,7 @@ columnar_unique_build(Relation rel) for (c = 0; c < uidx->nKeyCols; c++) { - if (!columnar_key_col_hash(indexRel, c, cxt, &uidx->cols[c])) + if (!pgcolumnar_key_col_hash(indexRel, c, cxt, &uidx->cols[c])) { uidx->coarse = true; break; @@ -290,7 +290,7 @@ columnar_unique_build(Relation rel) } static RelUniqueCacheEntry * -columnar_unique_lookup(Relation rel) +pgcolumnar_unique_lookup(Relation rel) { Oid relid = RelationGetRelid(rel); RelUniqueCacheEntry *entry = NULL; @@ -301,7 +301,7 @@ columnar_unique_lookup(Relation rel) if (entry != NULL) return entry; - return columnar_unique_build(rel); + return pgcolumnar_unique_build(rel); } /* ------------------------------------------------------------------------- @@ -310,7 +310,7 @@ columnar_unique_lookup(Relation rel) /* splitmix64/murmur3 finalizer, matching delete_vector_chunk_lock_key */ static inline uint64 -columnar_hash_finalize(uint64 h) +pgcolumnar_hash_finalize(uint64 h) { h ^= h >> 33; h *= UINT64CONST(0xff51afd7ed558ccd); @@ -321,7 +321,7 @@ columnar_hash_finalize(uint64 h) } static void -columnar_acquire_key_lock(Oid indexOid, uint32 bucket) +pgcolumnar_acquire_key_lock(Oid indexOid, uint32 bucket) { LOCKTAG tag; @@ -333,7 +333,7 @@ columnar_acquire_key_lock(Oid indexOid, uint32 bucket) } /* - * ColumnarLockUniqueKeys + * PgColumnarLockUniqueKeys * For a row about to be inserted into rel (through the table AM), take a * transaction-scoped advisory lock for each applicable unique index's key, * so a concurrent inserter of an equal key serializes behind this one until @@ -353,7 +353,7 @@ columnar_acquire_key_lock(Oid indexOid, uint32 bucket) * coarse per-index lock (over-serializes, always correct). */ void -ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) +PgColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) { RelUniqueCacheEntry *entry; ExprContext *econtext; @@ -361,14 +361,14 @@ ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) uint32 numBuckets; int i; - if (!columnar_enable_unique_lock) + if (!pgcolumnar_enable_unique_lock) return; - entry = columnar_unique_lookup(rel); + entry = pgcolumnar_unique_lookup(rel); if (entry->nIndexes == 0) return; - numBuckets = (uint32) Max(1, columnar_unique_lock_buckets); + numBuckets = (uint32) Max(1, pgcolumnar_unique_lock_buckets); econtext = GetPerTupleExprContext(entry->estate); saveScanTuple = econtext->ecxt_scantuple; @@ -434,25 +434,25 @@ ColumnarLockUniqueKeys(Relation rel, TupleTableSlot *slot) combined = (combined ^ (uint64) h) * COLUMNAR_FNV_PRIME; } - combined = columnar_hash_finalize(combined); + combined = pgcolumnar_hash_finalize(combined); bucket = (uint32) (combined % (uint64) numBuckets); } MemoryContextSwitchTo(oldcxt); - columnar_acquire_key_lock(uidx->indexOid, bucket); + pgcolumnar_acquire_key_lock(uidx->indexOid, bucket); } econtext->ecxt_scantuple = saveScanTuple; } /* - * ColumnarUniqueInit + * PgColumnarUniqueInit * Register the relcache invalidation callback that keeps the per-relation * unique-index cache coherent across DDL. Called once from _PG_init. */ void -ColumnarUniqueInit(void) +PgColumnarUniqueInit(void) { - CacheRegisterRelcacheCallback(columnar_unique_invalidate, (Datum) 0); + CacheRegisterRelcacheCallback(pgcolumnar_unique_invalidate, (Datum) 0); } diff --git a/src/columnar_vacuum.c b/src/columnar_vacuum.c index 50adb8e..70720b8 100644 --- a/src/columnar_vacuum.c +++ b/src/columnar_vacuum.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_vacuum.c + * pgcolumnar_vacuum.c * Compaction, statistics, and storage-id lookup functions for pgColumnar * (spec 8.2, 9). columnar.vacuum rewrites a columnar table's live rows * into fresh, full stripes: this combines many small stripes into few and @@ -49,30 +49,30 @@ #include "utils/tuplestore.h" #include "utils/typcache.h" -PG_FUNCTION_INFO_V1(columnar_relation_storageid); -PG_FUNCTION_INFO_V1(columnar_vacuum); -PG_FUNCTION_INFO_V1(columnar_vacuum_sorted); -PG_FUNCTION_INFO_V1(columnar_cluster); -PG_FUNCTION_INFO_V1(columnar_compact); -PG_FUNCTION_INFO_V1(columnar_compact_rewrite); -PG_FUNCTION_INFO_V1(columnar_recluster); -PG_FUNCTION_INFO_V1(columnar_truncate); -PG_FUNCTION_INFO_V1(columnar_debug_advance_reserved_offset); -PG_FUNCTION_INFO_V1(columnar_debug_set_metapage_version); +PG_FUNCTION_INFO_V1(pgcolumnar_relation_storageid); +PG_FUNCTION_INFO_V1(pgcolumnar_vacuum); +PG_FUNCTION_INFO_V1(pgcolumnar_vacuum_sorted); +PG_FUNCTION_INFO_V1(pgcolumnar_cluster); +PG_FUNCTION_INFO_V1(pgcolumnar_compact); +PG_FUNCTION_INFO_V1(pgcolumnar_compact_rewrite); +PG_FUNCTION_INFO_V1(pgcolumnar_recluster); +PG_FUNCTION_INFO_V1(pgcolumnar_truncate); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_advance_reserved_offset); +PG_FUNCTION_INFO_V1(pgcolumnar_debug_set_metapage_version); /* physical end-truncation opt-in (GUC), registered in _PG_init. Default off * until the abort/crash path is fully hardened and matrix-validated. */ -bool columnar_enable_end_truncation = false; +bool pgcolumnar_enable_end_truncation = false; /* - * ColumnarRequireTableOwner + * PgColumnarRequireTableOwner * Error unless the current user owns the relation (superusers pass). Every * maintenance and projection-DDL function gates on this: they rewrite data, * reclaim space, or take strong locks (truncate takes AccessExclusiveLock), * so they must be owner-only, like VACUUM and CLUSTER. */ void -ColumnarRequireTableOwner(Relation rel) +PgColumnarRequireTableOwner(Relation rel) { if (!COLUMNAR_TABLE_OWNERCHECK(RelationGetRelid(rel))) aclcheck_error(ACLCHECK_NOT_OWNER, OBJECT_TABLE, @@ -106,7 +106,7 @@ uint64_cmp(const void *a, const void *b) * rests on the protocol in design/PHASE_F3B_PLAN.md: the rewrite serializes with * deleters on the per-chunk-group advisory lock, and a delete that races a * rewrite of its group is aborted with a serialization failure by the - * conflict check in ColumnarUpsertDeleteVector. + * conflict check in PgColumnarUpsertDeleteVector. * ------------------------------------------------------------------------- */ /* the relation's ready indexes, opened once for a rewrite pass */ @@ -116,37 +116,37 @@ uint64_cmp(const void *a, const void *b) * on rel and has opened the indexes in ris. */ static int64 -rewrite_one_group(Relation rel, ColumnarIndexInsertState *ris, uint64 storageId, +rewrite_one_group(Relation rel, PgColumnarIndexInsertState *ris, uint64 storageId, uint64 groupNumber, uint64 firstRow, uint64 rowCount) { Oid relid = RelationGetRelid(rel); int natts = RelationGetDescr(rel)->natts; Datum *values = palloc(natts * sizeof(Datum)); bool *isnull = palloc(natts * sizeof(bool)); - ColumnarWriteState *ws; + PgColumnarWriteState *ws; Snapshot snap; - ColumnarReadState *rs; + PgColumnarReadState *rs; uint64 oldRn; int64 moved = 0; - /* serialize with concurrent deleters to this group (see ColumnarUpsertDeleteVector) */ - ColumnarLockChunkGroup(storageId, groupNumber); + /* serialize with concurrent deleters to this group (see PgColumnarUpsertDeleteVector) */ + PgColumnarLockChunkGroup(storageId, groupNumber); /* Read the group's live set under a snapshot taken after the lock, so every * delete committed before the lock is reflected. Register it (not just push * active): the reader derives a catalog snapshot copy from it, in - * ColumnarCatalogSnapshot, that must inherit a nonzero regd_count for PG18's + * PgColumnarCatalogSnapshot, that must inherit a nonzero regd_count for PG18's * heap-visibility assertion, which a merely-active GetLatestSnapshot does not * provide. */ snap = RegisterSnapshot(GetLatestSnapshot()); PushActiveSnapshot(snap); - ws = ColumnarGetWriteState(rel); + ws = PgColumnarGetWriteState(rel); /* * Stream the group rather than fetching its rows one at a time. * - * ColumnarReadRowByNumber decodes the whole group to return one value and + * PgColumnarReadRowByNumber decodes the whole group to return one value and * relies on the fetch cache to make the next call cheap. That cache holds * only what fits under COLUMNAR_FETCH_CACHE_MAX_BYTES, so a group whose * decoded form exceeds the cap re-decodes the columns that did not fit, once @@ -167,27 +167,27 @@ rewrite_one_group(Relation rel, ColumnarIndexInsertState *ris, uint64 storageId, * ANALYZE's sampler was moved off the same per-row fetch for the same * reason. */ - rs = ColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); - ColumnarReadRestrictToGroups(rs, &groupNumber, 1); + rs = PgColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); + PgColumnarReadRestrictToGroups(rs, &groupNumber, 1); - while (ColumnarReadNextRow(rs, values, isnull, &oldRn)) + while (PgColumnarReadNextRow(rs, values, isnull, &oldRn)) { uint64 newRn; CHECK_FOR_INTERRUPTS(); - newRn = ColumnarWriteRow(ws, rel, values, isnull); - ColumnarProjectionFanoutRow(rel, ws, newRn, values, isnull); - ColumnarIndexInsertRow(ris, rel, values, isnull, newRn); + newRn = PgColumnarWriteRow(ws, rel, values, isnull); + PgColumnarProjectionFanoutRow(rel, ws, newRn, values, isnull); + PgColumnarIndexInsertRow(ris, rel, values, isnull, newRn); moved++; } - ColumnarEndRead(rs); - ColumnarFlushWriteStateForRelation(relid); + PgColumnarEndRead(rs); + PgColumnarFlushWriteStateForRelation(relid); /* atomically (same transaction) the new group is now in the catalog; drop the * old one. Heap MVCC keeps the old group readable to older snapshots. */ - ColumnarRetireGroup(storageId, groupNumber); + PgColumnarRetireGroup(storageId, groupNumber); PopActiveSnapshot(); UnregisterSnapshot(snap); @@ -205,35 +205,35 @@ typedef struct RewriteCandidate } RewriteCandidate; /* - * columnar_rewrite_partial_groups + * pgcolumnar_rewrite_partial_groups * Rewrite up to maxGroups groups whose deleted fraction is at least * minDeletedFraction (and which are not fully dead -- F3a handles those). * maxGroups <= 0 means all. Returns the number of groups rewritten. */ static int64 -columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, +pgcolumnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, int maxGroups) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Oid relid = RelationGetRelid(rel); Snapshot snap; List *rgList; ListCell *lc; List *cands = NIL; - ColumnarIndexInsertState *ris; + PgColumnarIndexInsertState *ris; int64 rewritten = 0; /* persist own pending work so the group list and deletes are current */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* drop any free_space row overlapping a live group (the residual of a crash * in end-truncation's narrow window) before reusing anything */ - ColumnarReconcileFreeList(rel); + PgColumnarReconcileFreeList(rel); /* collect candidate groups first (do not mutate the catalog mid-scan) */ snap = RegisterSnapshot(GetLatestSnapshot()); - rgList = ColumnarReadRowGroupList(storageId, snap); + rgList = PgColumnarReadRowGroupList(storageId, snap); foreach(lc, rgList) { NativeRowGroupMetadata *rg = (NativeRowGroupMetadata *) lfirst(lc); @@ -243,7 +243,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, if (rg->rowCount == 0) continue; - rmList = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); + rmList = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); foreach(lc2, rmList) deleted += ((DeleteVectorMetadata *) lfirst(lc2))->deletedCount; @@ -264,7 +264,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, if (cands == NIL) return 0; - ris = ColumnarIndexInsertBegin(rel, false); + ris = PgColumnarIndexInsertBegin(rel, false); foreach(lc, cands) { RewriteCandidate *c = (RewriteCandidate *) lfirst(lc); @@ -275,14 +275,14 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, c->firstRow, c->rowCount); rewritten++; } - ColumnarIndexInsertEnd(ris); + PgColumnarIndexInsertEnd(ris); COLUMNAR_ASSERT_NO_OVERLAP(storageId); return rewritten; } /* - * columnar_compact_rewrite + * pgcolumnar_compact_rewrite * SQL: pgcolumnar.compact_rewrite(tablename regclass, * min_deleted_fraction float8 default 0.2, max_groups int default 0). * The lazy online space-reclaiming path (Phase F3b): rewrite partially @@ -314,7 +314,7 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, * An earlier version marked only an upper bound and tried to exclude the * below case by requiring the lowest live group to equal ours[0]. That * could not work (#342): the group list was read under the rewrite's own - * snapshot, taken before it read a row, and ColumnarCatalogSnapshot only + * snapshot, taken before it read a row, and PgColumnarCatalogSnapshot only * advances curcid rather than refreshing xmin/xmax, so a concurrent * inserter's group was invisible to the check whenever it committed. A * foreign group written just before the rewrite's first reservation was @@ -327,10 +327,10 @@ columnar_rewrite_partial_groups(Relation rel, double minDeletedFraction, */ static void record_online_sorted_extent(Relation rel, uint64 storageId, - ColumnarWriteState *writeState, int stripeMark) + PgColumnarWriteState *writeState, int stripeMark) { int nAll; - uint64 *all = ColumnarWriteStateStripeIds(writeState, &nAll); + uint64 *all = PgColumnarWriteStateStripeIds(writeState, &nAll); int nOurs = nAll - stripeMark; uint64 *ours; uint64 runEnd; @@ -354,7 +354,7 @@ record_online_sorted_extent(Relation rel, uint64 storageId, { int nProj = 0; - uint64 *projIds = ColumnarWriteStateProjStripeIds(writeState, &nProj); + uint64 *projIds = PgColumnarWriteStateProjStripeIds(writeState, &nProj); if (nProj > 0) { @@ -387,12 +387,12 @@ record_online_sorted_extent(Relation rel, uint64 storageId, runEnd = ours[i]; } - ColumnarSetSortedExtent(storageId, (int64) ours[0], (int64) runEnd); + PgColumnarSetSortedExtent(storageId, (int64) ours[0], (int64) runEnd); pfree(ours); } /* - * columnar_recluster_online + * pgcolumnar_recluster_online * Re-establish global Z-order clustering over the relation's live rows * online (Phase F3c): read all live rows under a snapshot taken after * advisory-locking every group, Morton-sort them, write them back as fresh @@ -402,9 +402,9 @@ record_online_sorted_extent(Relation rel, uint64 storageId, * conflict protocol. Returns the number of groups retired. */ static int64 -columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) +pgcolumnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); int natts = tupdesc->natts; @@ -425,23 +425,23 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) Oid byteaLt; Oid sortColl = InvalidOid; bool nullsFirst = false; - ColumnarReadState *readState; - ColumnarWriteState *writeState; - ColumnarIndexInsertState *ris; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; + PgColumnarIndexInsertState *ris; uint64 rowNumber; int stripeMark; /* persist own pending work so the group list and deletes are current */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* drop any free_space row overlapping a live group (the residual of a crash * in end-truncation's narrow window) before reusing anything */ - ColumnarReconcileFreeList(rel); + PgColumnarReconcileFreeList(rel); /* capture the current groups (retired at the end, after the new ones exist) */ listSnap = RegisterSnapshot(GetLatestSnapshot()); - rgList = ColumnarReadRowGroupList(storageId, listSnap); + rgList = PgColumnarReadRowGroupList(storageId, listSnap); oldGroups = palloc(sizeof(uint64) * (list_length(rgList) > 0 ? list_length(rgList) : 1)); foreach(lc, rgList) { @@ -463,7 +463,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) /* lock every group in ascending order (deadlock-safe), held to commit */ qsort(oldGroups, nGroups, sizeof(uint64), uint64_cmp); for (i = 0; i < nGroups; i++) - ColumnarLockChunkGroup(storageId, oldGroups[i]); + PgColumnarLockChunkGroup(storageId, oldGroups[i]); /* read all live rows into a Morton-keyed tuplesort (as in eager cluster). * Register the snapshot (not just push active) so the catalog snapshot copies @@ -492,8 +492,8 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) putSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsVirtual); augSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snap, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { bytea *zkey; @@ -509,41 +509,41 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) tuplesort_puttupleslot(tsort, putSlot); ExecClearTuple(putSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); ExecDropSingleTupleTableSlot(putSlot); tuplesort_performsort(tsort); /* write the sorted rows back as fresh groups, with online index maintenance */ - ris = ColumnarIndexInsertBegin(rel, false); - writeState = ColumnarGetWriteState(rel); + ris = PgColumnarIndexInsertBegin(rel, false); + writeState = PgColumnarGetWriteState(rel); /* * Note where this rewrite's own stripe reservations begin (#311). The write * state can already hold entries from earlier work in this transaction, so * only the tail from here on belongs to us. */ - stripeMark = ColumnarWriteStateStripeCount(writeState); + stripeMark = PgColumnarWriteStateStripeCount(writeState); while (tuplesort_gettupleslot(tsort, true, false, augSlot, NULL)) { uint64 newRn; CHECK_FOR_INTERRUPTS(); slot_getallattrs(augSlot); - newRn = ColumnarWriteRow(writeState, rel, augSlot->tts_values, + newRn = PgColumnarWriteRow(writeState, rel, augSlot->tts_values, augSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRn, augSlot->tts_values, + PgColumnarProjectionFanoutRow(rel, writeState, newRn, augSlot->tts_values, augSlot->tts_isnull); - ColumnarIndexInsertRow(ris, rel, augSlot->tts_values, + PgColumnarIndexInsertRow(ris, rel, augSlot->tts_values, augSlot->tts_isnull, newRn); } - ColumnarFlushWriteStateForRelation(relid); - ColumnarIndexInsertEnd(ris); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarIndexInsertEnd(ris); tuplesort_end(tsort); ExecDropSingleTupleTableSlot(augSlot); /* retire the old groups; heap MVCC keeps them readable to older snapshots */ for (i = 0; i < nGroups; i++) - ColumnarRetireGroup(storageId, oldGroups[i]); + PgColumnarRetireGroup(storageId, oldGroups[i]); /* record how far the reordered run reaches (#311) */ record_online_sorted_extent(rel, storageId, writeState, stripeMark); @@ -557,7 +557,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) } /* - * columnar_recluster + * pgcolumnar_recluster * SQL: pgcolumnar.recluster(tablename regclass, VARIADIC columns name[]). * The lazy online counterpart to cluster(): re-establish global Z-order * clustering under ShareUpdateExclusiveLock (concurrent reads and writes), @@ -565,7 +565,7 @@ columnar_recluster_online(Relation rel, int ncols, AttrNumber *atts) * number of groups reclustered. */ Datum -columnar_recluster(PG_FUNCTION_ARGS) +pgcolumnar_recluster(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); ArrayType *colArray; @@ -602,7 +602,7 @@ columnar_recluster(PG_FUNCTION_ARGS) /* the lazy lock: concurrent reads and writes during the recluster */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -611,7 +611,7 @@ columnar_recluster(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); tupdesc = RelationGetDescr(rel); atts = palloc(ncols * sizeof(AttrNumber)); @@ -654,14 +654,14 @@ columnar_recluster(PG_FUNCTION_ARGS) atts[i] = attno; } - reclustered = columnar_recluster_online(rel, ncols, atts); + reclustered = pgcolumnar_recluster_online(rel, ncols, atts); table_close(rel, NoLock); PG_RETURN_INT64(reclustered); } Datum -columnar_compact_rewrite(PG_FUNCTION_ARGS) +pgcolumnar_compact_rewrite(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); double minFrac = PG_ARGISNULL(1) ? 0.2 : PG_GETARG_FLOAT8(1); @@ -680,7 +680,7 @@ columnar_compact_rewrite(PG_FUNCTION_ARGS) rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -689,9 +689,9 @@ columnar_compact_rewrite(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - rewritten = columnar_rewrite_partial_groups(rel, minFrac, maxGroups); + rewritten = pgcolumnar_rewrite_partial_groups(rel, minFrac, maxGroups); table_close(rel, NoLock); PG_RETURN_INT64(rewritten); @@ -828,13 +828,13 @@ cluster_zorder_key(Datum *values, bool *isnull, AttrNumber *atts, int ncols, } /* - * columnar_relation_storageid + * pgcolumnar_relation_storageid * SQL: columnar.get_storage_id(regclass) -> bigint. Reads the relation's * metapage and returns its storage id (spec 3), so SQL-level functions * such as columnar.stats can join the metadata catalog by storage id. */ Datum -columnar_relation_storageid(PG_FUNCTION_ARGS) +pgcolumnar_relation_storageid(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; @@ -844,7 +844,7 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) if (rel == NULL) PG_RETURN_NULL(); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { relation_close(rel, AccessShareLock); ereport(ERROR, @@ -853,7 +853,7 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - storageId = ColumnarStorageId(rel); + storageId = PgColumnarStorageId(rel); relation_close(rel, AccessShareLock); PG_RETURN_INT64((int64) storageId); @@ -875,15 +875,15 @@ columnar_relation_storageid(PG_FUNCTION_ARGS) static void record_sorted_extent(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); + uint64 storageId = PgColumnarStorageId(rel); List *groups; uint64 lastGroup = 0; uint64 firstGroup = 0; bool haveGroup = false; ListCell *lc; - groups = ColumnarReadRowGroupList(storageId, - ColumnarCatalogSnapshot(GetActiveSnapshot())); + groups = PgColumnarReadRowGroupList(storageId, + PgColumnarCatalogSnapshot(GetActiveSnapshot())); foreach(lc, groups) { NativeRowGroupMetadata *rg = (NativeRowGroupMetadata *) lfirst(lc); @@ -897,11 +897,11 @@ record_sorted_extent(Relation rel) list_free_deep(groups); if (haveGroup) - ColumnarSetSortedExtent(storageId, (int64) firstGroup, (int64) lastGroup); + PgColumnarSetSortedExtent(storageId, (int64) firstGroup, (int64) lastGroup); } /* - * columnar_compact_relation + * pgcolumnar_compact_relation * Rewrite every live row of a columnar relation into fresh stripes. The * relation is already open with AccessExclusiveLock. * @@ -913,14 +913,14 @@ record_sorted_extent(Relation rel) * auto-maintained, so rows inserted afterward append in insert order. */ static void -columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) +pgcolumnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) { Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); uint64 oldStorageId; Snapshot snapshot; - ColumnarReadState *readState; - ColumnarWriteState *writeState; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; Tuplestorestate *tstore = NULL; Tuplesortstate *tsort = NULL; TupleTableSlot *readSlot; @@ -930,10 +930,10 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) List *oldProjs; /* persist any pending work so the read below sees it (spec 9) */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); - oldStorageId = ColumnarStorageId(rel); + oldStorageId = PgColumnarStorageId(rel); /* * Capture the table's projections (gap 26) before the storage swap. Compaction @@ -941,7 +941,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * compacted base; we re-record the definitions under the new storage id below * and the rewrite loop re-fans-out every live row into them. */ - oldProjs = ColumnarListProjections(oldStorageId); + oldProjs = PgColumnarListProjections(oldStorageId); /* * Take the read snapshot AFTER the AccessExclusiveLock the caller already @@ -950,7 +950,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * pre-lock snapshot's in-progress set, so it would be invisible to the row * enumeration below and silently discarded by the relfilenode swap -- data * loss. A fresh GetLatestSnapshot sees every commit as of now. Register it - * (not merely push active): ColumnarBeginRead derives a ColumnarCatalogSnapshot + * (not merely push active): PgColumnarBeginRead derives a PgColumnarCatalogSnapshot * copy that must inherit a nonzero regd_count for PG18's heap-visibility * assertion, exactly as the sibling rewrite/retire paths document. */ @@ -1002,8 +1002,8 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) readSlot = MakeSingleTupleTableSlot(tupdesc, &TTSOpsVirtual); writeSlot = MakeSingleTupleTableSlot(tupdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { CHECK_FOR_INTERRUPTS(); @@ -1014,7 +1014,7 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) tuplestore_puttupleslot(tstore, readSlot); ExecClearTuple(readSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); if (tsort != NULL) @@ -1027,8 +1027,8 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * still points at the old storage id) and remove the old metadata rows. */ RelationSetNewRelfilenumber(rel, rel->rd_rel->relpersistence); - ColumnarForgetWriteStateForRelation(relid); - ColumnarDeleteMetadata(oldStorageId); + PgColumnarForgetWriteStateForRelation(relid); + PgColumnarDeleteMetadata(oldStorageId); /* * Realign projections to the compacted base (gap 26): drop each old @@ -1040,31 +1040,31 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) */ if (oldProjs != NIL) { - uint64 newStorageId = ColumnarStorageId(rel); + uint64 newStorageId = PgColumnarStorageId(rel); ListCell *lc; foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != oldStorageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(oldStorageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(oldStorageId, p->projectionId); } foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); - ColumnarProjection np = *p; + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); + PgColumnarProjection np = *p; np.storageId = newStorageId; np.projStorageId = (p->projectionId == 0) ? newStorageId - : ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&np); + : PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&np); } } /* write the live rows back into the fresh storage, in sorted order if any */ - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); if (tsort != NULL) { while (tuplesort_gettupleslot(tsort, true, false, writeSlot, NULL)) @@ -1073,9 +1073,9 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) CHECK_FOR_INTERRUPTS(); slot_getallattrs(writeSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, writeSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, writeSlot->tts_values, writeSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, writeSlot->tts_values, writeSlot->tts_isnull); ExecClearTuple(writeSlot); } @@ -1088,14 +1088,14 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) CHECK_FOR_INTERRUPTS(); slot_getallattrs(writeSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, writeSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, writeSlot->tts_values, writeSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, writeSlot->tts_values, writeSlot->tts_isnull); ExecClearTuple(writeSlot); } } - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); /* * A sorted rewrite leaves the whole relation ordered, so record its extent. @@ -1117,22 +1117,22 @@ columnar_compact_relation(Relation rel, int nsortkeys, AttrNumber *sortAtts) * their synthetic item pointers (spec 6). A relation with no indexes is a * no-op here. */ - ColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); + PgColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); PopActiveSnapshot(); UnregisterSnapshot(snapshot); } /* - * columnar_compact_relation_zorder + * pgcolumnar_compact_relation_zorder * Rewrite every live row of a columnar relation ordered by the Z-order * (Morton) code over atts[0..ncols-1] (Phase F2). Mirrors - * columnar_compact_relation, but sorts by a computed key carried as a + * pgcolumnar_compact_relation, but sorts by a computed key carried as a * trailing bytea column of an augmented tuple, so the sort still spills to * disk through tuplesort. The relation is already open AccessExclusiveLock. */ static void -columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) +pgcolumnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) { Oid relid = RelationGetRelid(rel); TupleDesc tupdesc = RelationGetDescr(rel); @@ -1140,8 +1140,8 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) AttrNumber zAtt = (AttrNumber) (natts + 1); uint64 oldStorageId; Snapshot snapshot; - ColumnarReadState *readState; - ColumnarWriteState *writeState; + PgColumnarReadState *readState; + PgColumnarWriteState *writeState; Tuplesortstate *tsort; TupleDesc augdesc; TupleTableSlot *readSlot; @@ -1156,15 +1156,15 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) int i; /* persist pending work so the read below sees it (spec 9) */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); - oldStorageId = ColumnarStorageId(rel); - oldProjs = ColumnarListProjections(oldStorageId); + oldStorageId = PgColumnarStorageId(rel); + oldProjs = PgColumnarListProjections(oldStorageId); /* fresh snapshot AFTER the AccessExclusiveLock, not the caller's pre-lock one, * so a row group committed during the lock wait is copied rather than dropped * by the swap (#295); registered for PG18's catalog-snapshot regd_count. Same - * reasoning as columnar_compact_relation. */ + * reasoning as pgcolumnar_compact_relation. */ snapshot = RegisterSnapshot(GetLatestSnapshot()); PushActiveSnapshot(snapshot); @@ -1190,8 +1190,8 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) putSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsVirtual); augSlot = MakeSingleTupleTableSlot(augdesc, &TTSOpsMinimalTuple); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, readSlot->tts_values, + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, readSlot->tts_values, readSlot->tts_isnull, &rowNumber)) { bytea *zkey; @@ -1207,57 +1207,57 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) tuplesort_puttupleslot(tsort, putSlot); ExecClearTuple(putSlot); } - ColumnarEndRead(readState); + PgColumnarEndRead(readState); ExecDropSingleTupleTableSlot(readSlot); ExecDropSingleTupleTableSlot(putSlot); tuplesort_performsort(tsort); /* swap to fresh storage and drop old metadata (as in compact_relation) */ RelationSetNewRelfilenumber(rel, rel->rd_rel->relpersistence); - ColumnarForgetWriteStateForRelation(relid); - ColumnarDeleteMetadata(oldStorageId); + PgColumnarForgetWriteStateForRelation(relid); + PgColumnarDeleteMetadata(oldStorageId); /* realign projections to the compacted base (as in compact_relation) */ if (oldProjs != NIL) { - uint64 newStorageId = ColumnarStorageId(rel); + uint64 newStorageId = PgColumnarStorageId(rel); ListCell *lc; foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); if (p->projStorageId != oldStorageId) - ColumnarDeleteMetadata(p->projStorageId); - ColumnarDeleteProjectionRow(oldStorageId, p->projectionId); + PgColumnarDeleteMetadata(p->projStorageId); + PgColumnarDeleteProjectionRow(oldStorageId, p->projectionId); } foreach(lc, oldProjs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(lc); - ColumnarProjection np = *p; + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(lc); + PgColumnarProjection np = *p; np.storageId = newStorageId; np.projStorageId = (p->projectionId == 0) ? newStorageId - : ColumnarNextStorageId(); - ColumnarInsertProjectionRow(&np); + : PgColumnarNextStorageId(); + PgColumnarInsertProjectionRow(&np); } } /* write the live rows back in Z-order; the trailing key column is ignored */ - writeState = ColumnarGetWriteState(rel); + writeState = PgColumnarGetWriteState(rel); while (tuplesort_gettupleslot(tsort, true, false, augSlot, NULL)) { uint64 newRowNumber; CHECK_FOR_INTERRUPTS(); slot_getallattrs(augSlot); - newRowNumber = ColumnarWriteRow(writeState, rel, augSlot->tts_values, + newRowNumber = PgColumnarWriteRow(writeState, rel, augSlot->tts_values, augSlot->tts_isnull); - ColumnarProjectionFanoutRow(rel, writeState, newRowNumber, + PgColumnarProjectionFanoutRow(rel, writeState, newRowNumber, augSlot->tts_values, augSlot->tts_isnull); ExecClearTuple(augSlot); } - ColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushWriteStateForRelation(relid); /* Z-order is an order, so the same extent applies (see record_sorted_extent). */ record_sorted_extent(rel); @@ -1265,14 +1265,14 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) tuplesort_end(tsort); ExecDropSingleTupleTableSlot(augSlot); - ColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); + PgColumnarReindexRelation(relid, REINDEX_REL_PROCESS_TOAST); PopActiveSnapshot(); UnregisterSnapshot(snapshot); } /* - * columnar_vacuum + * pgcolumnar_vacuum * SQL: columnar.vacuum(tablename regclass, stripe_count int default 0). * Compacts a columnar table by combining its stripes and reclaiming the * space of deleted rows (spec 8.2, 9). stripe_count is accepted for @@ -1281,14 +1281,14 @@ columnar_compact_relation_zorder(Relation rel, int ncols, AttrNumber *atts) * recent stripes" contract. */ Datum -columnar_vacuum(PG_FUNCTION_ARGS) +pgcolumnar_vacuum(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1297,9 +1297,9 @@ columnar_vacuum(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - columnar_compact_relation(rel, 0, NULL); + pgcolumnar_compact_relation(rel, 0, NULL); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1308,7 +1308,7 @@ columnar_vacuum(PG_FUNCTION_ARGS) } /* - * columnar_vacuum_sorted + * pgcolumnar_vacuum_sorted * SQL: columnar.vacuum_sorted(tablename regclass, VARIADIC sort_columns name[]). * Like columnar.vacuum, but rewrites the live rows physically sorted * ascending / NULLS LAST on the named columns, in order (gap 26, piece 1). @@ -1324,7 +1324,7 @@ columnar_vacuum(PG_FUNCTION_ARGS) * cluster() path is numeric-only. */ Datum -columnar_vacuum_sorted(PG_FUNCTION_ARGS) +pgcolumnar_vacuum_sorted(PG_FUNCTION_ARGS) { Oid relid; Relation rel; @@ -1344,7 +1344,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1353,7 +1353,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); /* * Collect the sort-column names. Explicit columns win; when none are given @@ -1387,7 +1387,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) if (colNames == NIL) { - colNames = ColumnarReadSortBy(relid); + colNames = PgColumnarReadSortBy(relid); fromPersisted = true; } @@ -1452,7 +1452,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) sortAtts[i++] = attno; } - columnar_compact_relation(rel, ncols, sortAtts); + pgcolumnar_compact_relation(rel, ncols, sortAtts); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1461,7 +1461,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) } /* - * columnar_cluster + * pgcolumnar_cluster * SQL: pgcolumnar.cluster(tablename regclass, VARIADIC columns name[]). * Physically reorders a columnar table by the Z-order (Morton) space-filling * curve over the named columns (Phase F2, spec 9). Unlike vacuum_sorted's @@ -1479,7 +1479,7 @@ columnar_vacuum_sorted(PG_FUNCTION_ARGS) * path. */ Datum -columnar_cluster(PG_FUNCTION_ARGS) +pgcolumnar_cluster(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); ArrayType *colArray; @@ -1514,7 +1514,7 @@ columnar_cluster(PG_FUNCTION_ARGS) rel = table_open(relid, AccessExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, AccessExclusiveLock); ereport(ERROR, @@ -1523,7 +1523,7 @@ columnar_cluster(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); tupdesc = RelationGetDescr(rel); atts = palloc(ncols * sizeof(AttrNumber)); @@ -1569,7 +1569,7 @@ columnar_cluster(PG_FUNCTION_ARGS) atts[i] = attno; } - columnar_compact_relation_zorder(rel, ncols, atts); + pgcolumnar_compact_relation_zorder(rel, ncols, atts); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1578,7 +1578,7 @@ columnar_cluster(PG_FUNCTION_ARGS) } /* - * columnar_compact + * pgcolumnar_compact * SQL: pgcolumnar.compact(tablename regclass) -> bigint. The LAZY / online * maintenance path (Phase F3a): retire every row group that is fully deleted * as-of the oldest-xmin horizon, dropping its catalog rows so scans no longer @@ -1590,7 +1590,7 @@ columnar_cluster(PG_FUNCTION_ARGS) * Phase F3b. */ Datum -columnar_compact(PG_FUNCTION_ARGS) +pgcolumnar_compact(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); Relation rel; @@ -1604,7 +1604,7 @@ columnar_compact(PG_FUNCTION_ARGS) /* the lazy lock: concurrent reads and writes are allowed during compaction */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -1613,15 +1613,15 @@ columnar_compact(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); /* self-heal a truncate crash-residual so the no-overlap assert holds eagerly, - * even though compact does not reuse (see ColumnarReconcileFreeList) */ - ColumnarReconcileFreeList(rel); + * even though compact does not reuse (see PgColumnarReconcileFreeList) */ + PgColumnarReconcileFreeList(rel); - retired = ColumnarRetireFullyDeletedGroups(rel); + retired = PgColumnarRetireFullyDeletedGroups(rel); - COLUMNAR_ASSERT_NO_OVERLAP(ColumnarStorageId(rel)); + COLUMNAR_ASSERT_NO_OVERLAP(PgColumnarStorageId(rel)); /* keep the lock until end of transaction */ table_close(rel, NoLock); @@ -1630,16 +1630,16 @@ columnar_compact(PG_FUNCTION_ARGS) } /* - * columnar_end_truncation_storages + * pgcolumnar_end_truncation_storages * Collect the distinct storage ids that share this relation's file: the base * storage plus every projection's own storage. Returns a list of palloc'd * uint64. All must be considered when computing the safe truncation point, * because they all place data in the one shared file. */ static List * -columnar_end_truncation_storages(uint64 base) +pgcolumnar_end_truncation_storages(uint64 base) { - List *projs = ColumnarListProjections(base); + List *projs = PgColumnarListProjections(base); List *result = NIL; ListCell *lc; uint64 *b = palloc(sizeof(uint64)); @@ -1649,7 +1649,7 @@ columnar_end_truncation_storages(uint64 base) foreach(lc, projs) { - ColumnarProjection *pr = (ColumnarProjection *) lfirst(lc); + PgColumnarProjection *pr = (PgColumnarProjection *) lfirst(lc); ListCell *rc; bool dup = false; @@ -1671,7 +1671,7 @@ columnar_end_truncation_storages(uint64 base) } /* - * columnar_do_end_truncation + * pgcolumnar_do_end_truncation * Compute the safe truncation point and, if the trailing region is entirely * reclaimable, physically shrink the main fork. The caller holds * AccessExclusiveLock, so no reader or writer is concurrent. Returns the @@ -1691,7 +1691,7 @@ columnar_end_truncation_storages(uint64 base) * window (after the truncate, before the highwater is lowered) leaves * "highwater still high + free_space restored + file short", which the * gap-tolerant write path self-heals. The function also runs outside a - * transaction block (see columnar_truncate), so a user ROLLBACK cannot land + * transaction block (see pgcolumnar_truncate), so a user ROLLBACK cannot land * in the residual window between lowering the highwater and commit; and it * first purges any free_space row at or above the current highwater, which * under the exclusive lock is stale by definition and would otherwise be the @@ -1699,20 +1699,20 @@ columnar_end_truncation_storages(uint64 base) * live groups, none of which are in the truncated region. */ static int64 -columnar_do_end_truncation(Relation rel) +pgcolumnar_do_end_truncation(Relation rel) { - uint64 base = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *storages = columnar_end_truncation_storages(base); + uint64 base = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *storages = pgcolumnar_end_truncation_storages(base); ListCell *lc; uint64 liveEnd = COLUMNAR_FIRST_LOGICAL_OFFSET; - ColumnarMetapage meta; + PgColumnarMetapage meta; uint64 highwater; Snapshot snap; BlockNumber oldnblocks; BlockNumber truncBlock; - ColumnarReadMetapage(rel, &meta); + PgColumnarReadMetapage(rel, &meta); highwater = meta.reservedOffset; /* @@ -1725,15 +1725,15 @@ columnar_do_end_truncation(Relation rel) * the end-of-run no-overlap assert valid. */ foreach(lc, storages) - ColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), highwater); - ColumnarReconcileFreeList(rel); + PgColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), highwater); + PgColumnarReconcileFreeList(rel); /* highest live-data end across all storages, in the latest committed state */ snap = RegisterSnapshot(GetLatestSnapshot()); foreach(lc, storages) { uint64 sid = *(uint64 *) lfirst(lc); - List *rgs = ColumnarReadRowGroupList(sid, snap); + List *rgs = PgColumnarReadRowGroupList(sid, snap); ListCell *g; foreach(g, rgs) @@ -1756,16 +1756,16 @@ columnar_do_end_truncation(Relation rel) /* the trailing region must be entirely behind the oldest-xmin horizon */ foreach(lc, storages) - if (!ColumnarTrailingFreeSpaceSafe(*(uint64 *) lfirst(lc), liveEnd, + if (!PgColumnarTrailingFreeSpaceSafe(*(uint64 *) lfirst(lc), liveEnd, oldestXmin)) return 0; /* a recent retirement is in the tail; retry later */ /* drop the trailing free ranges, shrink the file, THEN lower the highwater */ foreach(lc, storages) - ColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), liveEnd); + PgColumnarDeleteFreeSpaceAtOrAbove(*(uint64 *) lfirst(lc), liveEnd); CommandCounterIncrement(); - ColumnarTruncateMainFork(rel, truncBlock); - ColumnarSetReservedOffset(rel, liveEnd); + PgColumnarTruncateMainFork(rel, truncBlock); + PgColumnarSetReservedOffset(rel, liveEnd); /* stale offset-keyed cache entries for this relation must go */ CacheInvalidateRelcacheByRelid(RelationGetRelid(rel)); @@ -1775,7 +1775,7 @@ columnar_do_end_truncation(Relation rel) } /* - * columnar_truncate + * pgcolumnar_truncate * SQL: pgcolumnar.truncate(regclass) -> bigint (blocks returned to the OS). * Physically shrinks a columnar table's file by dropping trailing blocks that * reclaim has freed. Opt-in (gated by pgcolumnar.enable_end_truncation) and @@ -1784,7 +1784,7 @@ columnar_do_end_truncation(Relation rel) * 0 without waiting if the table is busy, so it never blocks concurrent load. */ Datum -columnar_truncate(PG_FUNCTION_ARGS) +pgcolumnar_truncate(PG_FUNCTION_ARGS) { Oid relid; Relation rel; @@ -1813,7 +1813,7 @@ columnar_truncate(PG_FUNCTION_ARGS) /* serialize with other lazy maintenance (compact/recluster also take SUEL) */ rel = table_open(relid, ShareUpdateExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, ShareUpdateExclusiveLock); ereport(ERROR, @@ -1822,11 +1822,11 @@ columnar_truncate(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarRequireTableOwner(rel); + PgColumnarRequireTableOwner(rel); - if (columnar_enable_end_truncation && + if (pgcolumnar_enable_end_truncation && ConditionalLockRelation(rel, AccessExclusiveLock)) - result = columnar_do_end_truncation(rel); + result = pgcolumnar_do_end_truncation(rel); /* keep the locks until end of transaction */ table_close(rel, NoLock); @@ -1834,14 +1834,14 @@ columnar_truncate(PG_FUNCTION_ARGS) } /* - * columnar_debug_advance_reserved_offset + * pgcolumnar_debug_advance_reserved_offset * SQL test hook: advance a columnar table's write highwater by N pages * without writing data, leaving a gap between the physical EOF and the * highwater so the next write exercises the gap-tolerant path. Not bound in * the shipped catalog; the gap test creates the SQL binding itself. */ Datum -columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) +pgcolumnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); int32 npages = PG_GETARG_INT32(1); @@ -1853,7 +1853,7 @@ columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) errmsg("npages must be non-negative"))); rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1862,21 +1862,21 @@ columnar_debug_advance_reserved_offset(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarAdvanceReservedOffset(rel, (uint64) npages * COLUMNAR_BYTES_PER_PAGE); + PgColumnarAdvanceReservedOffset(rel, (uint64) npages * COLUMNAR_BYTES_PER_PAGE); table_close(rel, NoLock); PG_RETURN_VOID(); } /* - * columnar_debug_set_metapage_version + * pgcolumnar_debug_set_metapage_version * Test hook: overwrite a columnar table's stored metapage format version so * a subsequent read exercises the unsupported-version rejection in - * ColumnarReadMetapage. Not bound in the shipped catalog; the format suite + * PgColumnarReadMetapage. Not bound in the shipped catalog; the format suite * creates the binding when it needs it (like the advance helper above). */ Datum -columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) +pgcolumnar_debug_set_metapage_version(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); int32 major = PG_GETARG_INT32(1); @@ -1884,7 +1884,7 @@ columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) Relation rel; rel = table_open(relid, RowExclusiveLock); - if (!ColumnarIsColumnarRelation(relid)) + if (!PgColumnarIsColumnarRelation(relid)) { table_close(rel, RowExclusiveLock); ereport(ERROR, @@ -1893,7 +1893,7 @@ columnar_debug_set_metapage_version(PG_FUNCTION_ARGS) RelationGetRelationName(rel)))); } - ColumnarDebugSetMetapageVersion(rel, (uint32) major, (uint32) minor); + PgColumnarDebugSetMetapageVersion(rel, (uint32) major, (uint32) minor); table_close(rel, NoLock); PG_RETURN_VOID(); diff --git a/src/columnar_vector.c b/src/columnar_vector.c index 13d5f7f..eb81d38 100644 --- a/src/columnar_vector.c +++ b/src/columnar_vector.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_vector.c + * pgcolumnar_vector.c * Vectorized execution for pgColumnar (spec 9): a column-at-a-time filter * and vectorized aggregates over decoded chunk-group arrays. * @@ -23,8 +23,8 @@ * tests can assert that equality. * * The aggregate custom scan reuses the same registered CustomScanMethods as the - * base custom scan (so both show as "Custom Scan (ColumnarScan)"); the shared - * create-state callback in columnar_customscan.c dispatches to the aggregate + * base custom scan (so both show as "Custom Scan (PgColumnarScan)"); the shared + * create-state callback in pgcolumnar_customscan.c dispatches to the aggregate * variant when the plan is a scanrelid==0 upper node. * * Independent MIT implementation built from @@ -82,17 +82,17 @@ #include "utils/typcache.h" /* GUC: use the vectorized aggregate path (spec 8.3 scan control) */ -bool columnar_enable_vectorization = true; +bool pgcolumnar_enable_vectorization = true; /* * GUC: extend the vectorized aggregate to GROUP BY (#289). Default off while the * grouped path is built out incrementally; the ungrouped path is unaffected. * groupagg_max_groups caps the ACTUAL group count at execution time; exceeding - * it raises an error (see columnar_groupagg_lookup) rather than routing to core + * it raises an error (see pgcolumnar_groupagg_lookup) rather than routing to core * HashAgg. It is not a plan-time gate. */ -bool columnar_enable_group_vectorization = false; -int columnar_groupagg_max_groups = 1000000; +bool pgcolumnar_enable_group_vectorization = false; +int pgcolumnar_groupagg_max_groups = 1000000; /* * GUC: extend the ungrouped vectorized aggregate to the shapes a zone map cannot @@ -101,7 +101,7 @@ int columnar_groupagg_max_groups = 1000000; * over the columnar scan instead. Default off while the path is proven and * benchmarked; the metadata-answerable no-filter path is unaffected. */ -bool columnar_enable_ungrouped_vector_agg = false; +bool pgcolumnar_enable_ungrouped_vector_agg = false; /* * GUC: make the ungrouped vectorized batch fold parallel-aware (#289 phase 5/6). @@ -112,7 +112,7 @@ bool columnar_enable_ungrouped_vector_agg = false; * the batch-eligible count/sum/avg-float shapes take the parallel arm; everything * else keeps the serial node or the ordinary parallel core Agg. */ -bool columnar_enable_parallel_vector_agg = false; +bool pgcolumnar_enable_parallel_vector_agg = false; /* ------------------------------------------------------------------------- * shared column-at-a-time filter @@ -124,18 +124,18 @@ bool columnar_enable_parallel_vector_agg = false; * an array of these. The machinery that evaluated them over a decoded chunk * group had no call site and is deleted (issue #200). */ -typedef struct ColumnarVecPredicate +typedef struct PgColumnarVecPredicate { int attidx; /* 0-based column index */ bool varOnLeft; /* column op const, else const op column */ FmgrInfo opFn; /* the operator function (returns bool) */ Datum constValue; Oid collation; -} ColumnarVecPredicate; +} PgColumnarVecPredicate; /* - * columnar_clause_to_predicate + * pgcolumnar_clause_to_predicate * Turn one "column op const" (or "const op column") clause into a predicate * we can evaluate row by row. Requires a strict boolean operator and a * non-null constant, so that a null column value or a failed comparison @@ -143,8 +143,8 @@ typedef struct ColumnarVecPredicate * other clause. */ static bool -columnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, - ColumnarVecPredicate *pred) +pgcolumnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, + PgColumnarVecPredicate *pred) { OpExpr *op; Node *leftop; @@ -204,7 +204,7 @@ columnar_clause_to_predicate(Node *clause, Index scanrelid, TupleDesc tupdesc, } static void -ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, +PgColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, int *nconvertible, bool *allConvertible) { ListCell *lc; @@ -217,9 +217,9 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, foreach(lc, qual) { - ColumnarVecPredicate scratch; + PgColumnarVecPredicate scratch; - if (columnar_clause_to_predicate((Node *) lfirst(lc), scanrelid, tupdesc, + if (pgcolumnar_clause_to_predicate((Node *) lfirst(lc), scanrelid, tupdesc, &scratch)) n++; else @@ -236,7 +236,7 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, * in the tree; both are deleted (issue #200). Recoverable from history if a * filtered aggregate path is ever built. What should not be recovered with them is their gating, which * was none: the scalar path's predicates are gated on - * pgcolumnar.enable_qual_pushdown inside ColumnarBeginRead and these never were, + * pgcolumnar.enable_qual_pushdown inside PgColumnarBeginRead and these never were, * so wiring them up as they stood would have filtered rows while EXPLAIN * reported no pushdown at all. */ @@ -246,7 +246,7 @@ ColumnarCountConvertibleQuals(List *qual, Index scanrelid, TupleDesc tupdesc, * vectorized aggregate: classification * ------------------------------------------------------------------------- */ -typedef enum ColumnarAggKind +typedef enum PgColumnarAggKind { COLUMNAR_AGG_COUNT_STAR, COLUMNAR_AGG_COUNT_COL, @@ -256,7 +256,7 @@ typedef enum ColumnarAggKind COLUMNAR_AGG_MAX, /* * Extended kinds used by the grouped path and the ungrouped scan-fold path - * (#289). The metadata-fold path (columnar_fill_native_metadata_agg) still + * (#289). The metadata-fold path (pgcolumnar_fill_native_metadata_agg) still * never produces these: a query using one is routed to scan-fold instead. */ COLUMNAR_AGG_SUM_INT8, /* sum(int8) -> numeric */ @@ -265,11 +265,11 @@ typedef enum ColumnarAggKind COLUMNAR_AGG_AVG_INT8, /* avg(int8) -> numeric */ COLUMNAR_AGG_AVG_FLOAT, /* avg(float4/float8) -> float8 */ COLUMNAR_AGG_AVG_NUMERIC /* avg(numeric) -> numeric */ -} ColumnarAggKind; +} PgColumnarAggKind; -typedef struct ColumnarAggSpec +typedef struct PgColumnarAggSpec { - ColumnarAggKind kind; + PgColumnarAggKind kind; int attidx; /* 0-based column, or -1 for count(*) */ Oid inputType; /* column type (min/max/sum/avg) */ @@ -288,10 +288,10 @@ typedef struct ColumnarAggSpec float8 fsxx; /* avg(float): Youngs-Cramer Sxx, for overflow parity */ Datum nsum; /* numeric running total (in resultContext) */ bool nsumSet; /* nsum initialized */ -} ColumnarAggSpec; +} PgColumnarAggSpec; /* - * columnar_classify_aggref + * pgcolumnar_classify_aggref * Decide whether an Aggref is one we can compute vectorized, and if so fill * its spec. expectedVarno is the scan relation's range-table index at plan * time (to check the argument Var), or a negative value at execution time @@ -299,8 +299,8 @@ typedef struct ColumnarAggSpec * scalar fallback. */ static bool -columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, - bool allowPartial, ColumnarAggSpec *spec) +pgcolumnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, + bool allowPartial, PgColumnarAggSpec *spec) { char *name; Oid nsp; @@ -427,7 +427,7 @@ columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, } /* - * columnar_group_key_unsupported_walker + * pgcolumnar_group_key_unsupported_walker * Reject any node in a candidate GROUP BY key the grouped path cannot * evaluate against a bare base-relation slot: aggregates, grouping-set * constructs, window functions, sublinks/subplans, external parameters, @@ -437,7 +437,7 @@ columnar_classify_aggref(Aggref *agg, int expectedVarno, bool allowExtended, * to the volatility and varno checks the caller also applies. */ static bool -columnar_group_key_unsupported_walker(Node *node, void *context) +pgcolumnar_group_key_unsupported_walker(Node *node, void *context) { if (node == NULL) return false; @@ -465,12 +465,12 @@ columnar_group_key_unsupported_walker(Node *node, void *context) break; } - return expression_tree_walker(node, columnar_group_key_unsupported_walker, + return expression_tree_walker(node, pgcolumnar_group_key_unsupported_walker, context); } /* - * columnar_classify_group_keys + * pgcolumnar_classify_group_keys * Decide whether every GROUP BY key can be computed and grouped by the * grouped vectorized path, and if so return copies of the key expressions * (original varnos, one per grouping column). A key must be computable from @@ -481,7 +481,7 @@ columnar_group_key_unsupported_walker(Node *node, void *context) * false (add no path, run the ordinary Agg) on anything unsupported. */ static bool -columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, +pgcolumnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, List **keysOut) { Query *parse = root->parse; @@ -512,7 +512,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, return false; if (expression_returns_set(expr)) return false; - if (columnar_group_key_unsupported_walker(expr, NULL)) + if (pgcolumnar_group_key_unsupported_walker(expr, NULL)) return false; type = exprType(expr); @@ -524,7 +524,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, return false; if (!OidIsValid(tce->eq_opr_finfo.fn_oid)) return false; - if (OidIsValid(coll) && !ColumnarCollationIsDeterministic(coll)) + if (OidIsValid(coll) && !PgColumnarCollationIsDeterministic(coll)) return false; keys = lappend(keys, copyObject(expr)); @@ -540,7 +540,7 @@ columnar_classify_group_keys(PlannerInfo *root, RelOptInfo *input_rel, * vectorized aggregate: executor state * ------------------------------------------------------------------------- */ -typedef struct ColumnarAggScanState +typedef struct PgColumnarAggScanState { CustomScanState css; @@ -548,7 +548,7 @@ typedef struct ColumnarAggScanState List *quals; /* restriction clauses (original varnos) */ Index scanrelid; /* their range-table index */ - ColumnarAggSpec *specs; + PgColumnarAggSpec *specs; int naggs; int npreds; /* pushed-down predicate count, for EXPLAIN */ @@ -593,16 +593,16 @@ typedef struct ColumnarAggScanState uint64 groupsRead; uint64 groupsSkipped; uint64 groupsTotal; -} ColumnarAggScanState; +} PgColumnarAggScanState; -static const CustomExecMethods columnar_agg_exec_methods; -static const CustomExecMethods columnar_agg_parallel_exec_methods; +static const CustomExecMethods pgcolumnar_agg_exec_methods; +static const CustomExecMethods pgcolumnar_agg_parallel_exec_methods; /* ------------------------------------------------------------------------- * grouped vectorized aggregate (#289): executor state * * Fires for SELECT , agg(col) ... [WHERE ...] GROUP BY over a - * single columnar relation. The reader (ColumnarReadNextRow) applies WHERE + * single columnar relation. The reader (PgColumnarReadNextRow) applies WHERE * pushdown for group/vector skipping; each surviving row is rechecked against * the full WHERE, its group keys are evaluated, and it is scattered into an * open-addressing hash table whose per-group accumulators fold in scan order -- @@ -611,7 +611,7 @@ static const CustomExecMethods columnar_agg_parallel_exec_methods; * scale, and deterministic-collation text all group exactly as core does. * ------------------------------------------------------------------------- */ -typedef struct ColumnarGroupKey +typedef struct PgColumnarGroupKey { Expr *expr; /* key expression (original varnos) */ ExprState *exprState; /* evaluates it against the base slot */ @@ -621,18 +621,18 @@ typedef struct ColumnarGroupKey bool byval; FmgrInfo hashFn; /* type hash function */ FmgrInfo eqFn; /* type equality operator function */ -} ColumnarGroupKey; +} PgColumnarGroupKey; -typedef struct ColumnarGroupEntry +typedef struct PgColumnarGroupEntry { uint32 hash; bool used; Datum *keys; /* nkeys key values, in keyContext */ bool *keyNulls; /* nkeys null flags */ - ColumnarAggSpec *specs; /* naggs accumulators, in specContext */ -} ColumnarGroupEntry; + PgColumnarAggSpec *specs; /* naggs accumulators, in specContext */ +} PgColumnarGroupEntry; -typedef struct ColumnarGroupAggScanState +typedef struct PgColumnarGroupAggScanState { CustomScanState css; @@ -641,10 +641,10 @@ typedef struct ColumnarGroupAggScanState Index scanrelid; /* their range-table index */ int nkeys; - ColumnarGroupKey *keys; + PgColumnarGroupKey *keys; int naggs; - ColumnarAggSpec *aggTemplate; /* classified once; copied per new group */ + PgColumnarAggSpec *aggTemplate; /* classified once; copied per new group */ int nout; /* output tuple width */ int *outMap; /* per output pos: >=0 key index, else agg -(v)-1 */ @@ -653,10 +653,10 @@ typedef struct ColumnarGroupAggScanState TupleTableSlot *baseSlot; /* holds each read row for key/qual eval */ ExprState *whereState; /* residual WHERE recheck, or NULL */ - ColumnarGroupEntry *entries; /* open-addressing table (power-of-two) */ + PgColumnarGroupEntry *entries; /* open-addressing table (power-of-two) */ int capacity; int nGroups; - int maxGroups; /* GUC cap enforced at execution (columnar_groupagg_lookup) */ + int maxGroups; /* GUC cap enforced at execution (pgcolumnar_groupagg_lookup) */ MemoryContext keyContext; /* copied key Datums */ MemoryContext specContext; /* per-group specs + running min/max/numeric */ @@ -686,13 +686,13 @@ typedef struct ColumnarGroupAggScanState uint64 groupsRead; uint64 groupsSkipped; uint64 groupsTotal; -} ColumnarGroupAggScanState; +} PgColumnarGroupAggScanState; -static const CustomExecMethods columnar_groupagg_exec_methods; -static const CustomExecMethods columnar_groupagg_parallel_exec_methods; -static void ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, +static const CustomExecMethods pgcolumnar_groupagg_exec_methods; +static const CustomExecMethods pgcolumnar_groupagg_parallel_exec_methods; +static void PgColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra); -static bool columnar_batch_shape_eligible(ColumnarAggScanState *state, +static bool pgcolumnar_batch_shape_eligible(PgColumnarAggScanState *state, TupleDesc tupdesc, ScanKey *keysOut, int *nkeysOut); @@ -701,7 +701,7 @@ static bool columnar_batch_shape_eligible(ColumnarAggScanState *state, * ------------------------------------------------------------------------- */ static Plan * -ColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, +PgColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, List *tlist, List *clauses, List *custom_plans) { CustomScan *cscan = makeNode(CustomScan); @@ -714,29 +714,29 @@ ColumnarPlanAggPath(PlannerInfo *root, RelOptInfo *rel, CustomPath *best_path, cscan->custom_exprs = NIL; cscan->custom_private = best_path->custom_private; cscan->custom_scan_tlist = tlist; /* defines the output tuple shape */ - cscan->methods = &columnar_scan_methods; /* shared registered methods */ + cscan->methods = &pgcolumnar_scan_methods; /* shared registered methods */ return &cscan->scan.plan; } -static const CustomPathMethods columnar_agg_path_methods = { +static const CustomPathMethods pgcolumnar_agg_path_methods = { .CustomName = "ColumnarAgg", - .PlanCustomPath = ColumnarPlanAggPath, + .PlanCustomPath = PgColumnarPlanAggPath, .ReparameterizeCustomPathByChild = NULL, }; static create_upper_paths_hook_type prev_create_upper_paths_hook = NULL; /* - * columnar_agg_metadata_answerable + * pgcolumnar_agg_metadata_answerable * True when this aggregate kind is answerable from whole-chunk zone maps * with no data scan: count, count(col), sum/avg over int2/int4 (the zone * stores an int sum), min and max. The extended kinds (sum/avg over * int8/float/numeric) are not, so a query using one must scan and fold. See - * columnar_fill_native_metadata_agg. + * pgcolumnar_fill_native_metadata_agg. */ static bool -columnar_agg_metadata_answerable(ColumnarAggKind kind) +pgcolumnar_agg_metadata_answerable(PgColumnarAggKind kind) { switch (kind) { @@ -759,7 +759,7 @@ columnar_agg_metadata_answerable(ColumnarAggKind kind) } /* - * columnar_parallel_agg_ok + * pgcolumnar_parallel_agg_ok * Kinds whose transition state is a plain, non-internal value the batch fold * already holds and a core Finalize can combine (#289 phase 5/6): count(*), * count(col), and sum/avg over int2/int4/float4/float8. The transition types: @@ -769,7 +769,7 @@ columnar_agg_metadata_answerable(ColumnarAggKind kind) * and stay on the serial node or the ordinary core Agg. */ static bool -columnar_parallel_agg_ok(ColumnarAggKind kind) +pgcolumnar_parallel_agg_ok(PgColumnarAggKind kind) { switch (kind) { @@ -786,15 +786,15 @@ columnar_parallel_agg_ok(ColumnarAggKind kind) } /* - * ColumnarCreateUpperPaths - * create_upper_paths_hook: for a plain SELECT agg(col) FROM columnar_table + * PgColumnarCreateUpperPaths + * create_upper_paths_hook: for a plain SELECT agg(col) FROM pgcolumnar_table * [WHERE simple quals] with no grouping or HAVING, add a custom path that * computes the aggregates vectorized. Every aggregate, column type and * filter clause must be fully supported, or we add nothing and the ordinary * Agg plan runs, so results are never at risk. */ static void -ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, +PgColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra) { @@ -805,7 +805,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, ListCell *lc; int naggs; int i; - ColumnarAggSpec *specs; + PgColumnarAggSpec *specs; List *quals; int npreds; bool allConvertible; @@ -819,7 +819,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (stage != UPPERREL_GROUP_AGG) return; - if (!columnar_enable_vectorization || !columnar_enable_custom_scan) + if (!pgcolumnar_enable_vectorization || !pgcolumnar_enable_custom_scan) return; if (!parse->hasAggs) @@ -835,14 +835,14 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, parse->havingQual != NULL || parse->distinctClause != NIL || parse->hasWindowFuncs || parse->hasTargetSRFs) { - if (columnar_enable_group_vectorization && + if (pgcolumnar_enable_group_vectorization && parse->groupClause != NIL && parse->groupingSets == NIL && parse->havingQual == NULL && parse->distinctClause == NIL && !parse->hasWindowFuncs && !parse->hasTargetSRFs) - ColumnarTryGroupAggPath(root, input_rel, output_rel, extra); + PgColumnarTryGroupAggPath(root, input_rel, output_rel, extra); return; } @@ -860,7 +860,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (rte == NULL || rte->rtekind != RTE_RELATION || rte->relkind != RELKIND_RELATION) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; relid = rte->relid; @@ -868,7 +868,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, naggs = list_length(tlist); if (naggs == 0) return; - specs = (ColumnarAggSpec *) palloc0(sizeof(ColumnarAggSpec) * naggs); + specs = (PgColumnarAggSpec *) palloc0(sizeof(PgColumnarAggSpec) * naggs); i = 0; foreach(lc, tlist) { @@ -876,7 +876,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, if (!IsA(expr, Aggref)) return; - if (!columnar_classify_aggref((Aggref *) expr, (int) input_rel->relid, + if (!pgcolumnar_classify_aggref((Aggref *) expr, (int) input_rel->relid, true, false, &specs[i])) return; i++; @@ -893,7 +893,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, needsScan = (quals != NIL); for (i = 0; i < naggs; i++) - if (!columnar_agg_metadata_answerable(specs[i].kind)) + if (!pgcolumnar_agg_metadata_answerable(specs[i].kind)) needsScan = true; if (needsScan) @@ -907,7 +907,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, Bitmapset *whereAtts = NULL; int m = -1; - if (!columnar_enable_ungrouped_vector_agg) + if (!pgcolumnar_enable_ungrouped_vector_agg) return; /* @@ -937,7 +937,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, Relation rel = table_open(relid, AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); - ColumnarCountConvertibleQuals(quals, input_rel->relid, tupdesc, + PgColumnarCountConvertibleQuals(quals, input_rel->relid, tupdesc, &npreds, &allConvertible); table_close(rel, AccessShareLock); if (!allConvertible) @@ -1007,8 +1007,8 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, } else { - ColumnarOptions opts; - int limit = columnar_stripe_row_limit; + PgColumnarOptions opts; + int limit = pgcolumnar_stripe_row_limit; double rows = input_rel->tuples; double ngroups; double dirtyFraction = 0.0; @@ -1024,7 +1024,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * model getting a right answer, and it stops being harmless as soon as the * clamp is not what decides. */ - if (ColumnarReadOptions(relid, &opts) && + if (PgColumnarReadOptions(relid, &opts) && opts.stripeRowLimitSet && opts.stripeRowLimit > 0) limit = opts.stripeRowLimit; if (limit <= 0) @@ -1057,8 +1057,8 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, { Relation frel = table_open(relid, AccessShareLock); - if (ColumnarStorageHasDeleteVector(ColumnarStorageId(frel), - ColumnarCatalogSnapshot(snap))) + if (PgColumnarStorageHasDeleteVector(PgColumnarStorageId(frel), + PgColumnarCatalogSnapshot(snap))) dirtyFraction = 0.25; table_close(frel, AccessShareLock); } @@ -1092,12 +1092,12 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, copyObject(quals), makeConst(OIDOID, -1, InvalidOid, sizeof(Oid), ObjectIdGetDatum(relid), false, true)); - cpath->methods = &columnar_agg_path_methods; + cpath->methods = &pgcolumnar_agg_path_methods; /* * Parallel arm (#289 phase 5/6). When the shape is one whose transition state * the fold already holds and can be combined by a core Finalize - * (columnar_parallel_agg_ok), and the base relation has a partial (parallel) + * (pgcolumnar_parallel_agg_ok), and the base relation has a partial (parallel) * path, add a parallel-aware partial version of this node under Gather -> * Finalize Aggregate. Each worker claims distinct row groups through the shared * gap-23 counter and emits one per-worker transition-state tuple; the core @@ -1110,7 +1110,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * cheap Gather cost by #133, would wrongly out-cost the genuinely parallel * plan. Opt-in while it is proven and benchmarked. */ - if (needsScan && columnar_enable_parallel_vector_agg) + if (needsScan && pgcolumnar_enable_parallel_vector_agg) { GroupPathExtraData *gpe = (GroupPathExtraData *) extra; bool parallelOk = (gpe != NULL && @@ -1120,7 +1120,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, input_rel->partial_pathlist != NIL); for (i = 0; parallelOk && i < naggs; i++) - if (!columnar_parallel_agg_ok(specs[i].kind)) + if (!pgcolumnar_parallel_agg_ok(specs[i].kind)) parallelOk = false; if (parallelOk) @@ -1175,7 +1175,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, copyObject(quals), makeConst(OIDOID, -1, InvalidOid, sizeof(Oid), ObjectIdGetDatum(relid), false, true)); - ppath->methods = &columnar_agg_path_methods; + ppath->methods = &pgcolumnar_agg_path_methods; /* * Gather this partial node directly rather than add_partial_path'ing @@ -1202,7 +1202,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, } /* - * columnar_groupagg_outmap + * pgcolumnar_groupagg_outmap * Map one target list onto this node's output: per position, the group-key * index (>= 0) or the aggregate index encoded as -(index + 1). False when an * entry is neither a supported aggregate nor a bare reference to a group key, @@ -1216,7 +1216,7 @@ ColumnarCreateUpperPaths(PlannerInfo *root, UpperRelationKind stage, * allowPartial admits the INITIAL_SERIAL aggrefs of the second. */ static bool -columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, +pgcolumnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, bool allowPartial, List **outMapOut, int *naggsOut) { List *outMap = NIL; @@ -1229,9 +1229,9 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, if (IsA(oexpr, Aggref)) { - ColumnarAggSpec spec; + PgColumnarAggSpec spec; - if (!columnar_classify_aggref((Aggref *) oexpr, (int) scanrelid, + if (!pgcolumnar_classify_aggref((Aggref *) oexpr, (int) scanrelid, true, allowPartial, &spec)) return false; outMap = lappend(outMap, makeInteger(-(aggIdx + 1))); @@ -1274,13 +1274,13 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, } /* - * ColumnarTryGroupAggPath + * PgColumnarTryGroupAggPath * Add a grouped vectorized aggregate path (#289) when the query is one we * can answer exactly: a single columnar base relation, an optional WHERE, * every output entry either a supported aggregate or a bare reference to a * supported GROUP BY key. On anything unsupported it adds nothing and the * ordinary Agg plan runs. The group-count cap is enforced only at - * execution (columnar_groupagg_lookup). + * execution (pgcolumnar_groupagg_lookup). * * When the shape also qualifies for the parallel arm (#349) this adds * Finalize HashAggregate -> Gather -> parallel-aware partial node instead of @@ -1288,7 +1288,7 @@ columnar_groupagg_outmap(List *exprs, List *groupKeys, Index scanrelid, * displacing a parallel plan with a single-threaded one. */ static void -ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, +PgColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, RelOptInfo *output_rel, void *extra) { RangeTblEntry *rte; @@ -1324,12 +1324,12 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ if (rte->inh) return; - if (!OidIsValid(rte->relid) || !ColumnarIsColumnarRelation(rte->relid)) + if (!OidIsValid(rte->relid) || !PgColumnarIsColumnarRelation(rte->relid)) return; relid = rte->relid; /* every GROUP BY key must be one we can evaluate and group exactly */ - if (!columnar_classify_group_keys(root, input_rel, &groupKeys)) + if (!pgcolumnar_classify_group_keys(root, input_rel, &groupKeys)) return; /* @@ -1337,7 +1337,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * one of the group keys. An output expression built on top of a key (a * function of a grouping column) is not handled here and forces the fallback. */ - if (!columnar_groupagg_outmap(output_rel->reltarget->exprs, groupKeys, + if (!pgcolumnar_groupagg_outmap(output_rel->reltarget->exprs, groupKeys, input_rel->relid, false, &outMap, &naggs)) return; @@ -1349,7 +1349,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * path on exactly the large tables it helps. The unbounded-hash-table guard * is the execution-time cap on the actual group count * (pgcolumnar.groupagg_max_groups), which errors with guidance -- see - * columnar_groupagg_lookup -- rather than silently declining the feature. + * pgcolumnar_groupagg_lookup -- rather than silently declining the feature. */ groupExprs = groupKeys; dNumGroups = estimate_num_groups(root, groupExprs, @@ -1439,7 +1439,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * Nothing serial survived, which is the common case rather than the rare * one: add_path drops the serial columnar scan once a Gather over the * partial path dominates it. Cost the scan this node performs directly, - * the same way ColumnarSetRelPathlist costs its own fallback, instead of + * the same way PgColumnarSetRelPathlist costs its own fallback, instead of * borrowing whatever path happened to survive. */ QualCost qcost = input_rel->baserestrictcost; @@ -1529,7 +1529,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, ObjectIdGetDatum(relid), false, true), groupKeys, outMap); - cpath->methods = &columnar_agg_path_methods; + cpath->methods = &pgcolumnar_agg_path_methods; /* * Parallel arm (#349). The serial node above folds vectors instead of @@ -1544,7 +1544,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * tuple per group. A core Finalize re-aggregates across workers by key, * exactly as it does for an ordinary parallel grouped aggregate. */ - if (columnar_enable_parallel_vector_agg) + if (pgcolumnar_enable_parallel_vector_agg) { GroupPathExtraData *gpe = (GroupPathExtraData *) extra; bool parallelOk = (gpe != NULL && @@ -1561,16 +1561,16 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ foreach(lc, output_rel->reltarget->exprs) { - ColumnarAggSpec spec; + PgColumnarAggSpec spec; if (!parallelOk) break; if (!IsA(lfirst(lc), Aggref)) continue; - if (!columnar_classify_aggref((Aggref *) lfirst(lc), + if (!pgcolumnar_classify_aggref((Aggref *) lfirst(lc), (int) input_rel->relid, true, false, &spec) || - !columnar_parallel_agg_ok(spec.kind)) + !pgcolumnar_parallel_agg_ok(spec.kind)) parallelOk = false; } @@ -1604,7 +1604,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, */ if (partialScan != NULL && partialScan->parallel_workers >= 1 && pgr->reltarget != NULL && - columnar_groupagg_outmap(pgr->reltarget->exprs, groupKeys, + pgcolumnar_groupagg_outmap(pgr->reltarget->exprs, groupKeys, input_rel->relid, true, &partialMap, &partialAggs)) { @@ -1649,7 +1649,7 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, ObjectIdGetDatum(relid), false, true), groupKeys, partialMap); - ppath->methods = &columnar_agg_path_methods; + ppath->methods = &pgcolumnar_agg_path_methods; /* * Gather this partial node directly rather than add_partial_path'ing @@ -1698,10 +1698,10 @@ ColumnarTryGroupAggPath(PlannerInfo *root, RelOptInfo *input_rel, * ------------------------------------------------------------------------- */ Node * -ColumnarCreateAggScanState(CustomScan *cscan) +PgColumnarCreateAggScanState(CustomScan *cscan) { - ColumnarAggScanState *state = - (ColumnarAggScanState *) palloc0(sizeof(ColumnarAggScanState)); + PgColumnarAggScanState *state = + (PgColumnarAggScanState *) palloc0(sizeof(PgColumnarAggScanState)); int naggs = list_length(cscan->custom_scan_tlist); ListCell *lc; int i = 0; @@ -1730,19 +1730,19 @@ ColumnarCreateAggScanState(CustomScan *cscan) state->isPartial = true; } state->css.methods = state->isPartial - ? &columnar_agg_parallel_exec_methods - : &columnar_agg_exec_methods; + ? &pgcolumnar_agg_parallel_exec_methods + : &pgcolumnar_agg_exec_methods; /* rebuild the aggregate specs from the output tuple's aggregates */ state->naggs = naggs; - state->specs = (ColumnarAggSpec *) palloc0(sizeof(ColumnarAggSpec) * naggs); + state->specs = (PgColumnarAggSpec *) palloc0(sizeof(PgColumnarAggSpec) * naggs); foreach(lc, cscan->custom_scan_tlist) { TargetEntry *tle = (TargetEntry *) lfirst(lc); /* classified successfully at plan time; -1 skips the varno check. * allowPartial accepts the partial arm's INITIAL_SERIAL aggrefs. */ - (void) columnar_classify_aggref((Aggref *) tle->expr, -1, true, true, + (void) pgcolumnar_classify_aggref((Aggref *) tle->expr, -1, true, true, &state->specs[i]); i++; } @@ -1750,20 +1750,20 @@ ColumnarCreateAggScanState(CustomScan *cscan) /* * Scan-fold mode (#289) matches the planner's routing: a residual filter, or * any aggregate a zone map cannot answer. The metadata-answerable no-filter - * case keeps the zone-map path in ColumnarExecAggScan. + * case keeps the zone-map path in PgColumnarExecAggScan. */ state->scanFold = (state->quals != NIL); for (i = 0; i < naggs; i++) - if (!columnar_agg_metadata_answerable(state->specs[i].kind)) + if (!pgcolumnar_agg_metadata_answerable(state->specs[i].kind)) state->scanFold = true; return (Node *) state; } static void -ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; Relation rel; TupleDesc tupdesc; bool allConvertible; @@ -1784,7 +1784,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) */ if (state->scanFold) state->batchEligible = - columnar_batch_shape_eligible(state, tupdesc, NULL, NULL); + pgcolumnar_batch_shape_eligible(state, tupdesc, NULL, NULL); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) { @@ -1794,20 +1794,20 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) /* * Guard the native format version before folding any aggregate (#240). The - * plain read path checks it in ColumnarBeginReadWithStorage, but the + * plain read path checks it in PgColumnarBeginReadWithStorage, but the * zone-map-only aggregate path answers count/min/max from metadata without * ever opening a read state, so the check must also sit here -- otherwise an * unsupported-format table answers from bytes this build may not decode - * correctly. ColumnarStorageId reads the metapage, so its version is checked + * correctly. PgColumnarStorageId reads the metapage, so its version is checked * here too. */ - ColumnarCheckNativeFormatVersion(ColumnarStorageId(rel), + PgColumnarCheckNativeFormatVersion(PgColumnarStorageId(rel), RelationGetRelationName(rel)); /* finish setting up min/max comparison info now that we have the tupdesc */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->kind == COLUMNAR_AGG_MIN || spec->kind == COLUMNAR_AGG_MAX) { @@ -1834,7 +1834,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) * stops being true without anyone noticing. Computing it costs one walk of * an empty list. */ - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, &state->npreds, &allConvertible); /* @@ -1883,7 +1883,7 @@ ColumnarBeginAggScan(CustomScanState *node, EState *estate, int eflags) * arithmetic is bit-for-bit what float4pl/float8pl compute. */ static inline float8 -columnar_float8_pl(float8 a, float8 b) +pgcolumnar_float8_pl(float8 a, float8 b) { float8 r = a + b; @@ -1895,7 +1895,7 @@ columnar_float8_pl(float8 a, float8 b) } static inline float4 -columnar_float4_pl(float4 a, float4 b) +pgcolumnar_float4_pl(float4 a, float4 b) { float4 r = a + b; @@ -1907,13 +1907,13 @@ columnar_float4_pl(float4 a, float4 b) } /* - * columnar_apply_one + * pgcolumnar_apply_one * Fold one value (or a null) into an aggregate accumulator. This is the * reference per-row semantics, shared by the ungrouped scan path - * (columnar_native_scan_agg) and the grouped path (columnar_groupagg_build). + * (pgcolumnar_native_scan_agg) and the grouped path (pgcolumnar_groupagg_build). */ static void -columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, +pgcolumnar_apply_one(MemoryContext resultContext, PgColumnarAggSpec *spec, Datum val, bool isnull) { switch (spec->kind) @@ -1969,10 +1969,10 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, ? (float8) DatumGetFloat4(val) : DatumGetFloat8(val); else if (spec->inputType == FLOAT4OID) - spec->fsum = (float8) columnar_float4_pl((float4) spec->fsum, + spec->fsum = (float8) pgcolumnar_float4_pl((float4) spec->fsum, DatumGetFloat4(val)); else - spec->fsum = columnar_float8_pl(spec->fsum, + spec->fsum = pgcolumnar_float8_pl(spec->fsum, DatumGetFloat8(val)); spec->sawValue = true; } @@ -2098,7 +2098,7 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, /* - * columnar_agg_emit_partial + * pgcolumnar_agg_emit_partial * A parallel partial node (#289 phase 5/6) emits each aggregate's transition * state, not its finalized value, for a core Finalize Aggregate to combine. * The transition types match core's own partial aggregate exactly, so @@ -2107,7 +2107,7 @@ columnar_apply_one(MemoryContext resultContext, ColumnarAggSpec *spec, * re-derives and re-checks the Youngs-Cramer Sxx we pass through. */ static Datum -columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) +pgcolumnar_agg_emit_partial(PgColumnarAggSpec *spec, bool *isnull) { *isnull = false; @@ -2198,7 +2198,7 @@ columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) /* * The parallel arm is gated to the kinds above - * (columnar_parallel_agg_ok), so this is unreachable; fail loudly rather + * (pgcolumnar_parallel_agg_ok), so this is unreachable; fail loudly rather * than emit a value of the wrong transition type. */ elog(ERROR, "columnar parallel partial: unsupported aggregate kind %d", @@ -2208,12 +2208,12 @@ columnar_agg_emit_partial(ColumnarAggSpec *spec, bool *isnull) } /* - * columnar_agg_finalize + * pgcolumnar_agg_finalize * Turn one accumulator into its output Datum, reproducing PostgreSQL's * aggregate result types and empty-input behaviour exactly. */ static Datum -columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) +pgcolumnar_agg_finalize(PgColumnarAggSpec *spec, bool *isnull) { *isnull = false; @@ -2302,7 +2302,7 @@ columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) } /* - * columnar_group_deleted_count + * pgcolumnar_group_deleted_count * How many of this row group's rows are deleted, under the given catalog * snapshot. A group can have several delete_vector rows, whose bitmaps * overlap, so they are OR'd before counting rather than summed -- summing @@ -2311,7 +2311,7 @@ columnar_agg_finalize(ColumnarAggSpec *spec, bool *isnull) * Bits past the group's row count are ignored. */ static uint64 -columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, +pgcolumnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, Snapshot snap) { uint32 want = (uint32) ((rg->rowCount + 7) / 8); @@ -2321,7 +2321,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, uint64 deleted = 0; uint32 b; - rml = ColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); + rml = PgColumnarReadDeleteVectorList(storageId, rg->groupNumber, snap); if (rml == NIL) return 0; @@ -2356,7 +2356,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, } /* - * columnar_fill_native_metadata_agg + * pgcolumnar_fill_native_metadata_agg * Answer an ungrouped, unfiltered aggregate over a native (PGCN v1) table * from its whole-chunk zone maps (native spec 7.1, D5b): count(*) from * row-group row counts, count(col) and the avg count from value_count, sum @@ -2379,7 +2379,7 @@ columnar_group_deleted_count(uint64 storageId, NativeRowGroupMetadata *rg, * data pages even when the group has deletes. */ static uint64 * -columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) +pgcolumnar_fill_native_metadata_agg(PgColumnarAggScanState *state, int *ndirty) { EState *estate = state->css.ss.ps.state; Relation rel; @@ -2410,11 +2410,11 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) } } - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); rel = table_open(state->relid, AccessShareLock); tupdesc = RelationGetDescr(rel); - snap = ColumnarCatalogSnapshot(estate->es_snapshot); - storageId = ColumnarStorageId(rel); + snap = PgColumnarCatalogSnapshot(estate->es_snapshot); + storageId = PgColumnarStorageId(rel); /* * One storage-wide probe first. When nothing is deleted no group can have @@ -2422,9 +2422,9 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * keeps a clean table at exactly the catalog traffic it had before this * change, which for a count(*) is the row group list and nothing else. */ - anyDeletes = ColumnarStorageHasDeleteVector(storageId, snap); + anyDeletes = PgColumnarStorageHasDeleteVector(storageId, snap); - groups = ColumnarReadRowGroupList(storageId, snap); + groups = PgColumnarReadRowGroupList(storageId, snap); dirty = palloc(sizeof(uint64) * (list_length(groups) > 0 ? list_length(groups) : 1)); *ndirty = 0; @@ -2437,7 +2437,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) int a; if (anyDeletes) - deleted = columnar_group_deleted_count(storageId, rg, snap); + deleted = pgcolumnar_group_deleted_count(storageId, rg, snap); if (deleted > 0) { @@ -2464,7 +2464,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) if (needZones) { - List *zones = ColumnarReadZoneMapList(storageId, + List *zones = PgColumnarReadZoneMapList(storageId, rg->groupNumber, snap); ListCell *zc; @@ -2520,7 +2520,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; NativeZoneMapMetadata *z = (byCol != NULL && spec->attidx >= 0 && spec->attidx < tupdesc->natts) @@ -2562,7 +2562,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) MemoryContextSwitchTo(state->resultContext); char *cur = (spec->kind == COLUMNAR_AGG_MIN) ? (char *) z->minimum : (char *) z->maximum; - Datum v = ColumnarDecodeValue(att, &cur, + Datum v = PgColumnarDecodeValue(att, &cur, state->resultContext); if (!spec->sawValue) @@ -2608,7 +2608,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * once per row. For a full-scan ungrouped aggregate that per-row tax is the whole * cost. The batch fold instead walks each loaded group's packed value streams, * evaluates a pushable WHERE inline, and folds each surviving value through the - * same columnar_apply_one -- so accumulators are byte-identical to the row path, + * same pgcolumnar_apply_one -- so accumulators are byte-identical to the row path, * floats included -- with none of the per-row Datum, context, or executor cost. * It runs only when every aggregate and the whole WHERE are batch-eligible; a * false return means it folded nothing and the caller runs the always-correct @@ -2619,7 +2619,7 @@ columnar_fill_native_metadata_agg(ColumnarAggScanState *state, int *ndirty) * -0.0 == 0.0, matching float8_cmp_internal so an inline compare equals the * operator ExecQual would call. */ static inline int -columnar_batch_float_cmp(double a, double b) +pgcolumnar_batch_float_cmp(double a, double b) { if (isnan(a)) return isnan(b) ? 0 : 1; @@ -2631,7 +2631,7 @@ columnar_batch_float_cmp(double a, double b) /* Whether one fixed-width numeric column value (known non-null) satisfies one * btree scan key. */ static bool -columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) +pgcolumnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) { int c; @@ -2644,9 +2644,9 @@ columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) case INT8OID: { int64 a = DatumGetInt64(val), b = DatumGetInt64(arg); c = (a < b) ? -1 : (a > b) ? 1 : 0; break; } case FLOAT4OID: - c = columnar_batch_float_cmp((double) DatumGetFloat4(val), (double) DatumGetFloat4(arg)); break; + c = pgcolumnar_batch_float_cmp((double) DatumGetFloat4(val), (double) DatumGetFloat4(arg)); break; case FLOAT8OID: - c = columnar_batch_float_cmp(DatumGetFloat8(val), DatumGetFloat8(arg)); break; + c = pgcolumnar_batch_float_cmp(DatumGetFloat8(val), DatumGetFloat8(arg)); break; default: return false; } @@ -2663,17 +2663,17 @@ columnar_batch_key_pass(Oid coltype, Datum val, StrategyNumber strat, Datum arg) /* A fixed-width by-value numeric column type the batch fold can read directly. */ static bool -columnar_batch_type_ok(Oid typ) +pgcolumnar_batch_type_ok(Oid typ) { return typ == INT2OID || typ == INT4OID || typ == INT8OID || typ == FLOAT4OID || typ == FLOAT8OID; } -/* An aggregate kind the batch fold accumulates (folded via columnar_apply_one +/* An aggregate kind the batch fold accumulates (folded via pgcolumnar_apply_one * over a fixed-width by-value numeric column, or count). int8/numeric sum/avg and * min/max stay on the row path. */ static bool -columnar_batch_agg_ok(ColumnarAggKind kind) +pgcolumnar_batch_agg_ok(PgColumnarAggKind kind) { switch (kind) { @@ -2690,7 +2690,7 @@ columnar_batch_agg_ok(ColumnarAggKind kind) } /* - * columnar_batch_shape_eligible + * pgcolumnar_batch_shape_eligible * Whether this aggregate's shape can use the batch fold: every aggregate is * batch-accumulable, the whole WHERE converts to scan keys (no residual), and * every key is a supported btree comparison on a batch-readable column. When @@ -2699,7 +2699,7 @@ columnar_batch_agg_ok(ColumnarAggKind kind) * Begin can report it in EXPLAIN before execution. */ static bool -columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, +pgcolumnar_batch_shape_eligible(PgColumnarAggScanState *state, TupleDesc tupdesc, ScanKey *keysOut, int *nkeysOut) { ScanKey keys; @@ -2711,15 +2711,15 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, bool ok = true; for (a = 0; a < state->naggs; a++) - if (!columnar_batch_agg_ok(state->specs[a].kind)) + if (!pgcolumnar_batch_agg_ok(state->specs[a].kind)) return false; - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, tupdesc, &npred, &allConvertible); if (!allConvertible) return false; - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nkeys); + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nkeys); for (k = 0; k < nkeys; k++) { ScanKey key = &keys[k]; @@ -2729,7 +2729,7 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, if (key->sk_flags != 0 || attidx < 0 || attidx >= tupdesc->natts) { ok = false; break; } coltype = TupleDescAttr(tupdesc, attidx)->atttypid; - if (!columnar_batch_type_ok(coltype)) + if (!pgcolumnar_batch_type_ok(coltype)) { ok = false; break; } if (key->sk_subtype != InvalidOid && key->sk_subtype != coltype) { ok = false; break; } @@ -2750,14 +2750,14 @@ columnar_batch_shape_eligible(ColumnarAggScanState *state, TupleDesc tupdesc, /* Reset the accumulators to their initial state (for a clean fall-back). */ static void -columnar_agg_specs_reset(ColumnarAggScanState *state) +pgcolumnar_agg_specs_reset(PgColumnarAggScanState *state) { int a; MemoryContextReset(state->resultContext); for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; spec->count = 0; spec->sum = 0; @@ -2771,13 +2771,13 @@ columnar_agg_specs_reset(ColumnarAggScanState *state) } /* - * columnar_native_batch_fold + * pgcolumnar_native_batch_fold * Fold the whole scan column-at-a-time. Returns false (having folded and * reset nothing that the caller cannot redo) when the shape is not eligible * or a group is missing a needed column, so the caller runs the row path. */ static bool -columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, +pgcolumnar_native_batch_fold(PgColumnarAggScanState *state, Relation rel, TupleDesc tupdesc) { EState *estate = state->css.ss.ps.state; @@ -2787,7 +2787,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int k; int col; int natts = tupdesc->natts; - ColumnarReadState *rs; + PgColumnarReadState *rs; const char **cvalidity = (const char **) palloc0(sizeof(char *) * natts); const char **cpacked = (const char **) palloc0(sizeof(char *) * natts); int16 *cattlen = (int16 *) palloc0(sizeof(int16) * natts); @@ -2796,7 +2796,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, Datum *cval = (Datum *) palloc0(sizeof(Datum) * natts); bool *cisnull = (bool *) palloc0(sizeof(bool) * natts); - if (!columnar_batch_shape_eligible(state, tupdesc, &keys, &nkeys)) + if (!pgcolumnar_batch_shape_eligible(state, tupdesc, &keys, &nkeys)) return false; col = -1; @@ -2814,11 +2814,11 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * * Per-vector skipping WITHIN a surviving group is a separate refinement the * fold does not take: it would need to step the present index past a skipped - * vector. That is safe to leave out because columnar_native_load_group builds + * vector. That is safe to leave out because pgcolumnar_native_load_group builds * the packed present-value stream whole regardless of the skip vector, so * walking all rows and advancing the present index over each is correct. */ - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, nkeys, keys); /* @@ -2830,14 +2830,14 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * InitializeDSM callback runs even leader-only), so a NULL here is a bug. */ if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); else if (state->isPartial) { - ColumnarEndRead(rs); + PgColumnarEndRead(rs); elog(ERROR, "parallel columnar aggregate ran without a shared group counter"); } - while (ColumnarReadFoldNextGroup(rs)) + while (PgColumnarReadFoldNextGroup(rs)) { uint64 nrows; const char *dmask; @@ -2847,7 +2847,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int vcount; uint64 r; - ColumnarReadFoldGroupInfo(rs, &nrows, &dmask, &dlen, + PgColumnarReadFoldGroupInfo(rs, &nrows, &dmask, &dlen, &skipVec, &vecStart, &vcount); for (col = 0; col < natts; col++) @@ -2860,7 +2860,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, cpresent[col] = 0; if (!cneeded[col]) continue; - if (!ColumnarReadFoldColumn(rs, col, &vbits, &pk, &al, &vrl)) + if (!PgColumnarReadFoldColumn(rs, col, &vbits, &pk, &al, &vrl)) { /* * The column is absent from this group (a later ADD COLUMN). The @@ -2874,8 +2874,8 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * cleanly rather than return a wrong answer. Rare (an old row group * predating an ADD COLUMN); turn the parallel GUC off for the table. */ - ColumnarEndRead(rs); - columnar_agg_specs_reset(state); + PgColumnarEndRead(rs); + pgcolumnar_agg_specs_reset(state); if (state->isPartial) elog(ERROR, "parallel columnar aggregate cannot fold a relation " "with a column added after some row groups; " @@ -2921,7 +2921,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, int attidx = key->sk_attno - 1; if (cisnull[attidx] || - !columnar_batch_key_pass(TupleDescAttr(tupdesc, attidx)->atttypid, + !pgcolumnar_batch_key_pass(TupleDescAttr(tupdesc, attidx)->atttypid, cval[attidx], key->sk_strategy, key->sk_argument)) { @@ -2934,29 +2934,29 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->resultContext, spec, + pgcolumnar_apply_one(state->resultContext, spec, cval[spec->attidx], cisnull[spec->attidx]); else - columnar_apply_one(state->resultContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->resultContext, spec, (Datum) 0, true); } } } - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; state->batchFolded = true; - ColumnarEndRead(rs); + PgColumnarEndRead(rs); return true; } /* - * columnar_native_scan_agg + * pgcolumnar_native_scan_agg * Fold an ungrouped, unfiltered aggregate over a native table by scanning it - * one row at a time (ColumnarReadNextRow applies the delete mask), for the + * one row at a time (PgColumnarReadNextRow applies the delete mask), for the * case where the zone-map-only path cannot be used because the storage has * deletes (D6b). No quals: the upper-path hook only adds the native agg path * when there is no filter. @@ -2966,7 +2966,7 @@ columnar_native_batch_fold(ColumnarAggScanState *state, Relation rel, * already folded from their zone maps by the caller. */ static void -columnar_native_scan_agg(ColumnarAggScanState *state, +pgcolumnar_native_scan_agg(PgColumnarAggScanState *state, const uint64 *restrictGroups, int nRestrictGroups) { EState *estate = state->css.ss.ps.state; @@ -2974,7 +2974,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, Relation rel = table_open(state->relid, AccessShareLock); TupleDesc tupdesc = RelationGetDescr(rel); Bitmapset *projected; - ColumnarReadState *rs; + PgColumnarReadState *rs; ScanKey keys = NULL; int nScanKeys = 0; Datum *values = (Datum *) palloc(sizeof(Datum) * tupdesc->natts); @@ -2999,8 +2999,8 @@ columnar_native_scan_agg(ColumnarAggScanState *state, projected = bms_make_singleton(0); /* count(*) only: one column */ } - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(rel); /* * Batch fold when the shape allows it (#289): the whole scan folds @@ -3009,7 +3009,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, * row path. */ if (state->scanFold && restrictGroups == NULL && - columnar_native_batch_fold(state, rel, tupdesc)) + pgcolumnar_native_batch_fold(state, rel, tupdesc)) { table_close(rel, AccessShareLock); return; @@ -3017,7 +3017,7 @@ columnar_native_scan_agg(ColumnarAggScanState *state, /* * The row path reached here because the shape is not batch-foldable (e.g. a - * NULL test or a non-btree filter): columnar_native_batch_fold returned false + * NULL test or a non-btree filter): pgcolumnar_native_batch_fold returned false * before claiming any group, so the shared counter is untouched and a parallel * partial node can fold correctly here too -- each worker just claims distinct * groups through the same atomic and applies the WHERE recheck per row. (The @@ -3027,21 +3027,21 @@ columnar_native_scan_agg(ColumnarAggScanState *state, /* push the WHERE down for group and vector pruning; the recheck is exact */ if (state->scanFold && state->quals != NIL) - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, tupdesc, &nScanKeys); - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, projected, nScanKeys, keys); if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); if (restrictGroups != NULL) - ColumnarReadRestrictToGroups(rs, restrictGroups, nRestrictGroups); + PgColumnarReadRestrictToGroups(rs, restrictGroups, nRestrictGroups); /* columns outside the projection stay null in the recheck slot */ if (state->whereState != NULL) memset(state->baseSlot->tts_isnull, true, sizeof(bool) * tupdesc->natts); - while (ColumnarReadNextRow(rs, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(rs, values, nulls, &rowNumber)) { /* * Recheck the whole WHERE per row: the scan keys only prune groups and @@ -3066,31 +3066,31 @@ columnar_native_scan_agg(ColumnarAggScanState *state, for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->resultContext, spec, + pgcolumnar_apply_one(state->resultContext, spec, values[spec->attidx], nulls[spec->attidx]); else - columnar_apply_one(state->resultContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->resultContext, spec, (Datum) 0, true); } } if (state->scanFold) { - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; } - ColumnarEndRead(rs); + PgColumnarEndRead(rs); table_close(rel, AccessShareLock); } static TupleTableSlot * -ColumnarExecAggScan(CustomScanState *node) +PgColumnarExecAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; TupleTableSlot *scanSlot = node->ss.ss_ScanTupleSlot; ExprContext *econtext = node->ss.ps.ps_ExprContext; TupleTableSlot *result; @@ -3116,24 +3116,24 @@ ColumnarExecAggScan(CustomScanState *node) * a zone map that counts the rows this transaction has already removed. */ frel = table_open(state->relid, AccessShareLock); - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); if (state->scanFold) { /* * A filter, or a sum/avg no zone map answers (#289): scan every row once - * and fold it. columnar_native_scan_agg builds the scan keys, rechecks the + * and fold it. pgcolumnar_native_scan_agg builds the scan keys, rechecks the * WHERE, and captures the EXPLAIN stats. */ - columnar_native_scan_agg(state, NULL, 0); + pgcolumnar_native_scan_agg(state, NULL, 0); } else { - dirtyGroups = columnar_fill_native_metadata_agg(state, &nDirtyGroups); + dirtyGroups = pgcolumnar_fill_native_metadata_agg(state, &nDirtyGroups); if (nDirtyGroups > 0) - columnar_native_scan_agg(state, dirtyGroups, nDirtyGroups); + pgcolumnar_native_scan_agg(state, dirtyGroups, nDirtyGroups); state->haveStats = false; } @@ -3145,8 +3145,8 @@ ColumnarExecAggScan(CustomScanState *node) ExecClearTuple(scanSlot); for (a = 0; a < state->naggs; a++) scanSlot->tts_values[a] = state->isPartial - ? columnar_agg_emit_partial(&state->specs[a], &scanSlot->tts_isnull[a]) - : columnar_agg_finalize(&state->specs[a], &scanSlot->tts_isnull[a]); + ? pgcolumnar_agg_emit_partial(&state->specs[a], &scanSlot->tts_isnull[a]) + : pgcolumnar_agg_finalize(&state->specs[a], &scanSlot->tts_isnull[a]); ExecStoreVirtualTuple(scanSlot); /* @@ -3166,21 +3166,21 @@ ColumnarExecAggScan(CustomScanState *node) } static void -ColumnarEndAggScan(CustomScanState *node) +PgColumnarEndAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; if (state->baseSlot != NULL) ExecDropSingleTupleTableSlot(state->baseSlot); state->baseSlot = NULL; - /* the reader is ended inside ColumnarExecAggScan; the memory contexts are + /* the reader is ended inside PgColumnarExecAggScan; the memory contexts are * children of es_query_cxt and freed with it */ } static void -ColumnarReScanAggScan(CustomScanState *node) +PgColumnarReScanAggScan(CustomScanState *node) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; int a; state->done = false; @@ -3189,7 +3189,7 @@ ColumnarReScanAggScan(CustomScanState *node) MemoryContextReset(state->resultContext); for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->specs[a]; + PgColumnarAggSpec *spec = &state->specs[a]; spec->count = 0; spec->sum = 0; @@ -3210,9 +3210,9 @@ ColumnarReScanAggScan(CustomScanState *node) } static void -ColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) +PgColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; ExplainPropertyInteger("Columnar Vectorized Aggregates", NULL, state->naggs, es); @@ -3233,13 +3233,13 @@ ColumnarExplainAggScan(CustomScanState *node, List *ancestors, ExplainState *es) } } -static const CustomExecMethods columnar_agg_exec_methods = { +static const CustomExecMethods pgcolumnar_agg_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginAggScan, - .ExecCustomScan = ColumnarExecAggScan, - .EndCustomScan = ColumnarEndAggScan, - .ReScanCustomScan = ColumnarReScanAggScan, - .ExplainCustomScan = ColumnarExplainAggScan, + .BeginCustomScan = PgColumnarBeginAggScan, + .ExecCustomScan = PgColumnarExecAggScan, + .EndCustomScan = PgColumnarEndAggScan, + .ReScanCustomScan = PgColumnarReScanAggScan, + .ExplainCustomScan = PgColumnarExplainAggScan, }; /* ------------------------------------------------------------------------- @@ -3251,20 +3251,20 @@ static const CustomExecMethods columnar_agg_exec_methods = { * and passes its address as `coordinate` to the DSM/worker init callbacks. Our * agg node opens its reader lazily during Exec, strictly after both DSM-init and * Worker-init, so the callbacks only need to record the counter on the state; - * ColumnarBeginRead wiring happens at fold time. + * PgColumnarBeginRead wiring happens at fold time. * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMAggScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMAggScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); @@ -3277,17 +3277,17 @@ ColumnarInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, * rows the leader deleted earlier in this transaction. The Exec-time flush in * every backend then only ever flushes its own (empty) buffers. */ - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); { Relation frel = table_open(state->relid, AccessShareLock); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); } } static void -ColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -3297,26 +3297,26 @@ ColumnarReInitializeDSMAggScan(CustomScanState *node, ParallelContext *pcxt, } static void -ColumnarInitializeWorkerAggScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerAggScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarAggScanState *state = (ColumnarAggScanState *) node; + PgColumnarAggScanState *state = (PgColumnarAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; state->parallelCounter = counter; } -static const CustomExecMethods columnar_agg_parallel_exec_methods = { +static const CustomExecMethods pgcolumnar_agg_parallel_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginAggScan, - .ExecCustomScan = ColumnarExecAggScan, - .EndCustomScan = ColumnarEndAggScan, - .ReScanCustomScan = ColumnarReScanAggScan, - .ExplainCustomScan = ColumnarExplainAggScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMAggScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMAggScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMAggScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerAggScan, + .BeginCustomScan = PgColumnarBeginAggScan, + .ExecCustomScan = PgColumnarExecAggScan, + .EndCustomScan = PgColumnarEndAggScan, + .ReScanCustomScan = PgColumnarReScanAggScan, + .ExplainCustomScan = PgColumnarExplainAggScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMAggScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMAggScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMAggScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerAggScan, }; /* ------------------------------------------------------------------------- @@ -3324,10 +3324,10 @@ static const CustomExecMethods columnar_agg_parallel_exec_methods = { * ------------------------------------------------------------------------- */ Node * -ColumnarCreateGroupAggScanState(CustomScan *cscan) +PgColumnarCreateGroupAggScanState(CustomScan *cscan) { - ColumnarGroupAggScanState *state = - (ColumnarGroupAggScanState *) palloc0(sizeof(ColumnarGroupAggScanState)); + PgColumnarGroupAggScanState *state = + (PgColumnarGroupAggScanState *) palloc0(sizeof(PgColumnarGroupAggScanState)); List *groupKeys; List *outMapList; ListCell *lc; @@ -3354,8 +3354,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) state->isPartial = true; } state->css.methods = state->isPartial - ? &columnar_groupagg_parallel_exec_methods - : &columnar_groupagg_exec_methods; + ? &pgcolumnar_groupagg_parallel_exec_methods + : &pgcolumnar_groupagg_exec_methods; /* custom_private: rti, quals, relid, group-key exprs, output map (length 5) */ state->scanrelid = (Index) intVal(linitial(cscan->custom_private)); @@ -3366,8 +3366,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) outMapList = (List *) list_nth(cscan->custom_private, 4); state->nkeys = list_length(groupKeys); - state->keys = (ColumnarGroupKey *) - palloc0(sizeof(ColumnarGroupKey) * Max(state->nkeys, 1)); + state->keys = (PgColumnarGroupKey *) + palloc0(sizeof(PgColumnarGroupKey) * Max(state->nkeys, 1)); i = 0; foreach(lc, groupKeys) state->keys[i++].expr = (Expr *) lfirst(lc); @@ -3377,8 +3377,8 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) if (IsA(((TargetEntry *) lfirst(lc))->expr, Aggref)) naggs++; state->naggs = naggs; - state->aggTemplate = (ColumnarAggSpec *) - palloc0(sizeof(ColumnarAggSpec) * Max(naggs, 1)); + state->aggTemplate = (PgColumnarAggSpec *) + palloc0(sizeof(PgColumnarAggSpec) * Max(naggs, 1)); i = 0; foreach(lc, cscan->custom_scan_tlist) { @@ -3387,7 +3387,7 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) if (IsA(tle->expr, Aggref)) { /* allowPartial accepts the parallel arm's INITIAL_SERIAL aggrefs */ - (void) columnar_classify_aggref((Aggref *) tle->expr, -1, true, true, + (void) pgcolumnar_classify_aggref((Aggref *) tle->expr, -1, true, true, &state->aggTemplate[i]); i++; } @@ -3399,7 +3399,7 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) foreach(lc, outMapList) state->outMap[i++] = intVal(lfirst(lc)); - state->maxGroups = columnar_groupagg_max_groups; + state->maxGroups = pgcolumnar_groupagg_max_groups; state->capacity = 0; state->nGroups = 0; state->entries = NULL; @@ -3408,9 +3408,9 @@ ColumnarCreateGroupAggScanState(CustomScan *cscan) } static void -ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) +PgColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; Relation rel; TupleDesc basedesc; Bitmapset *proj = NULL; @@ -3444,7 +3444,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) * Count pushable filters for EXPLAIN before the EXPLAIN-only early return, so * a plain EXPLAIN reports the real pushed-down filter count instead of 0. */ - ColumnarCountConvertibleQuals(state->quals, state->scanrelid, basedesc, + PgColumnarCountConvertibleQuals(state->quals, state->scanrelid, basedesc, &state->npreds, &allConvertible); if (eflags & EXEC_FLAG_EXPLAIN_ONLY) @@ -3454,7 +3454,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) } /* guard the native format before decoding any value (#240) */ - ColumnarCheckNativeFormatVersion(ColumnarStorageId(rel), + PgColumnarCheckNativeFormatVersion(PgColumnarStorageId(rel), RelationGetRelationName(rel)); /* a virtual slot holding each read row for key and qual evaluation */ @@ -3464,7 +3464,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) /* group-key ExprStates and their hash/equality machinery */ for (k = 0; k < state->nkeys; k++) { - ColumnarGroupKey *key = &state->keys[k]; + PgColumnarGroupKey *key = &state->keys[k]; Oid type = exprType((Node *) key->expr); TypeCacheEntry *tce = lookup_type_cache(type, TYPECACHE_HASH_PROC_FINFO | @@ -3487,7 +3487,7 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) /* finish min/max comparison setup on the per-agg template */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &state->aggTemplate[a]; + PgColumnarAggSpec *spec = &state->aggTemplate[a]; if (spec->kind == COLUMNAR_AGG_MIN || spec->kind == COLUMNAR_AGG_MAX) { @@ -3525,14 +3525,14 @@ ColumnarBeginGroupAggScan(CustomScanState *node, EState *estate, int eflags) } /* - * columnar_groupagg_keys_equal + * pgcolumnar_groupagg_keys_equal * Whether a probing row's keys match a stored group's, by SQL grouping * semantics: two nulls are equal, and non-nulls compare with the key type's * equality operator (with collation) -- exactly how core groups. */ static bool -columnar_groupagg_keys_equal(ColumnarGroupAggScanState *state, - ColumnarGroupEntry *e, +pgcolumnar_groupagg_keys_equal(PgColumnarGroupAggScanState *state, + PgColumnarGroupEntry *e, Datum *keyvals, bool *keynulls) { int k; @@ -3552,17 +3552,17 @@ columnar_groupagg_keys_equal(ColumnarGroupAggScanState *state, } /* - * columnar_groupagg_grow + * pgcolumnar_groupagg_grow * Double the open-addressing table and reinsert live entries. Entry structs * (and the key/spec pointers they carry) move by value; the pointed-at key * Datums and accumulators stay put in their own contexts. */ static void -columnar_groupagg_grow(ColumnarGroupAggScanState *state) +pgcolumnar_groupagg_grow(PgColumnarGroupAggScanState *state) { int oldCap = state->capacity; int newCap = (oldCap <= 0) ? 1024 : oldCap * 2; - ColumnarGroupEntry *newEntries; + PgColumnarGroupEntry *newEntries; MemoryContext old; int i; @@ -3578,15 +3578,15 @@ columnar_groupagg_grow(ColumnarGroupAggScanState *state) * via pgcolumnar.groupagg_max_groups. */ old = MemoryContextSwitchTo(state->hashContext); - newEntries = (ColumnarGroupEntry *) + newEntries = (PgColumnarGroupEntry *) MemoryContextAllocExtended(state->hashContext, - sizeof(ColumnarGroupEntry) * (Size) newCap, + sizeof(PgColumnarGroupEntry) * (Size) newCap, MCXT_ALLOC_HUGE | MCXT_ALLOC_ZERO); MemoryContextSwitchTo(old); for (i = 0; i < oldCap; i++) { - ColumnarGroupEntry *e = &state->entries[i]; + PgColumnarGroupEntry *e = &state->entries[i]; uint32 idx; if (!e->used) @@ -3604,23 +3604,23 @@ columnar_groupagg_grow(ColumnarGroupAggScanState *state) } /* - * columnar_groupagg_lookup + * pgcolumnar_groupagg_lookup * Find the group for this row's keys, inserting a fresh one (with the key * Datums copied into keyContext and accumulators seeded from the template) * when it is new. */ -static ColumnarGroupEntry * -columnar_groupagg_lookup(ColumnarGroupAggScanState *state, +static PgColumnarGroupEntry * +pgcolumnar_groupagg_lookup(PgColumnarGroupAggScanState *state, Datum *keyvals, bool *keynulls) { uint32 hash = 0; uint32 idx; int k; - ColumnarGroupEntry *e; + PgColumnarGroupEntry *e; /* grow before probing so the index is computed against the final table */ if ((int64) (state->nGroups + 1) * 10 >= (int64) state->capacity * 7) - columnar_groupagg_grow(state); + pgcolumnar_groupagg_grow(state); for (k = 0; k < state->nkeys; k++) { @@ -3642,7 +3642,7 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, if (!e->used) break; if (e->hash == hash && - columnar_groupagg_keys_equal(state, e, keyvals, keynulls)) + pgcolumnar_groupagg_keys_equal(state, e, keyvals, keynulls)) return e; idx = (idx + 1) & (uint32) (state->capacity - 1); } @@ -3682,10 +3682,10 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, { MemoryContext oldc = MemoryContextSwitchTo(state->specContext); - e->specs = (ColumnarAggSpec *) - palloc(sizeof(ColumnarAggSpec) * Max(state->naggs, 1)); + e->specs = (PgColumnarAggSpec *) + palloc(sizeof(PgColumnarAggSpec) * Max(state->naggs, 1)); memcpy(e->specs, state->aggTemplate, - sizeof(ColumnarAggSpec) * state->naggs); + sizeof(PgColumnarAggSpec) * state->naggs); MemoryContextSwitchTo(oldc); } state->nGroups++; @@ -3693,14 +3693,14 @@ columnar_groupagg_lookup(ColumnarGroupAggScanState *state, } /* - * columnar_groupagg_build + * pgcolumnar_groupagg_build * Scan the relation once and fold every surviving row into its group. The * reader prunes groups and vectors with the pushed-down WHERE; each row is * rechecked against the whole WHERE, its keys evaluated, and its values * folded in scan order so accumulators match the scalar Agg byte for byte. */ static void -columnar_groupagg_build(ColumnarGroupAggScanState *state) +pgcolumnar_groupagg_build(PgColumnarGroupAggScanState *state) { EState *estate = state->css.ss.ps.state; ExprContext *econtext = state->css.ss.ps.ps_ExprContext; @@ -3711,17 +3711,17 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) bool *nulls = (bool *) palloc(sizeof(bool) * natts); Datum *keyvals = (Datum *) palloc(sizeof(Datum) * Max(state->nkeys, 1)); bool *keynulls = (bool *) palloc(sizeof(bool) * Max(state->nkeys, 1)); - ColumnarReadState *rs; + PgColumnarReadState *rs; ScanKey keys; int nScanKeys = 0; uint64 rowNumber; - ColumnarFlushWriteStateForRelation(state->relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushDeleteVectorForRelation(rel); - keys = ColumnarBuildScanKeys(state->quals, state->scanrelid, basedesc, + keys = PgColumnarBuildScanKeys(state->quals, state->scanrelid, basedesc, &nScanKeys); - rs = ColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, + rs = PgColumnarBeginRead(rel, estate->es_snapshot, NULL, state->projected, nScanKeys, keys); /* @@ -3731,17 +3731,17 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) * case the reader walks every group as before. */ if (state->parallelCounter != NULL) - ColumnarReadSetParallelCounter(rs, state->parallelCounter); + PgColumnarReadSetParallelCounter(rs, state->parallelCounter); /* columns outside the projection stay null in the base slot */ memset(state->baseSlot->tts_isnull, true, sizeof(bool) * natts); - while (ColumnarReadNextRow(rs, values, nulls, &rowNumber)) + while (PgColumnarReadNextRow(rs, values, nulls, &rowNumber)) { int x; int k; int a; - ColumnarGroupEntry *e; + PgColumnarGroupEntry *e; ResetExprContext(econtext); @@ -3770,46 +3770,46 @@ columnar_groupagg_build(ColumnarGroupAggScanState *state) keyvals[k] = ExecEvalExprSwitchContext(state->keys[k].exprState, econtext, &keynulls[k]); - e = columnar_groupagg_lookup(state, keyvals, keynulls); + e = pgcolumnar_groupagg_lookup(state, keyvals, keynulls); /* fold this row's values into the group's accumulators */ for (a = 0; a < state->naggs; a++) { - ColumnarAggSpec *spec = &e->specs[a]; + PgColumnarAggSpec *spec = &e->specs[a]; if (spec->attidx >= 0) - columnar_apply_one(state->specContext, spec, + pgcolumnar_apply_one(state->specContext, spec, values[spec->attidx], nulls[spec->attidx]); else - columnar_apply_one(state->specContext, spec, (Datum) 0, true); + pgcolumnar_apply_one(state->specContext, spec, (Datum) 0, true); } } - ColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, + PgColumnarReadStats(rs, &state->groupsRead, &state->groupsSkipped, &state->groupsTotal); state->haveStats = true; - ColumnarEndRead(rs); + PgColumnarEndRead(rs); table_close(rel, AccessShareLock); } static TupleTableSlot * -ColumnarExecGroupAggScan(CustomScanState *node) +PgColumnarExecGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; TupleTableSlot *scanSlot = node->ss.ss_ScanTupleSlot; ExprContext *econtext = node->ss.ps.ps_ExprContext; if (!state->started) { - columnar_groupagg_build(state); + pgcolumnar_groupagg_build(state); state->started = true; state->emitPos = 0; } while (state->emitPos < state->capacity) { - ColumnarGroupEntry *e = &state->entries[state->emitPos++]; + PgColumnarGroupEntry *e = &state->entries[state->emitPos++]; int p; if (!e->used) @@ -3839,9 +3839,9 @@ ColumnarExecGroupAggScan(CustomScanState *node) * expects -- not one empty group. */ scanSlot->tts_values[p] = state->isPartial - ? columnar_agg_emit_partial(&e->specs[a], + ? pgcolumnar_agg_emit_partial(&e->specs[a], &scanSlot->tts_isnull[p]) - : columnar_agg_finalize(&e->specs[a], + : pgcolumnar_agg_finalize(&e->specs[a], &scanSlot->tts_isnull[p]); } } @@ -3859,9 +3859,9 @@ ColumnarExecGroupAggScan(CustomScanState *node) } static void -ColumnarEndGroupAggScan(CustomScanState *node) +PgColumnarEndGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; if (state->baseSlot != NULL) ExecDropSingleTupleTableSlot(state->baseSlot); @@ -3870,9 +3870,9 @@ ColumnarEndGroupAggScan(CustomScanState *node) } static void -ColumnarReScanGroupAggScan(CustomScanState *node) +PgColumnarReScanGroupAggScan(CustomScanState *node) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; state->started = false; state->emitPos = 0; @@ -3886,10 +3886,10 @@ ColumnarReScanGroupAggScan(CustomScanState *node) } static void -ColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, +PgColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, ExplainState *es) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; ExplainPropertyInteger("Columnar Vectorized Group Keys", NULL, state->nkeys, es); @@ -3909,13 +3909,13 @@ ColumnarExplainGroupAggScan(CustomScanState *node, List *ancestors, } } -static const CustomExecMethods columnar_groupagg_exec_methods = { +static const CustomExecMethods pgcolumnar_groupagg_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginGroupAggScan, - .ExecCustomScan = ColumnarExecGroupAggScan, - .EndCustomScan = ColumnarEndGroupAggScan, - .ReScanCustomScan = ColumnarReScanGroupAggScan, - .ExplainCustomScan = ColumnarExplainGroupAggScan, + .BeginCustomScan = PgColumnarBeginGroupAggScan, + .ExecCustomScan = PgColumnarExecGroupAggScan, + .EndCustomScan = PgColumnarEndGroupAggScan, + .ReScanCustomScan = PgColumnarReScanGroupAggScan, + .ExplainCustomScan = PgColumnarExplainGroupAggScan, }; /* ------------------------------------------------------------------------- @@ -3924,23 +3924,23 @@ static const CustomExecMethods columnar_groupagg_exec_methods = { * The same shared pg_atomic_uint32 the ungrouped partial node uses (gap 23), * handing out row-group indices so each worker folds distinct groups. These * mirror the ungrouped four and cannot share their bodies: those cast the node - * to ColumnarAggScanState, and the grouped node is a different struct. + * to PgColumnarAggScanState, and the grouped node is a different struct. * * The grouped node opens its reader lazily in Exec, strictly after both DSM-init * and Worker-init, so the callbacks need only record the counter on the state. * ------------------------------------------------------------------------- */ static Size -ColumnarEstimateDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt) +PgColumnarEstimateDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt) { return sizeof(pg_atomic_uint32); } static void -ColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; pg_atomic_init_u32(counter, 0); @@ -3961,17 +3961,17 @@ ColumnarInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, * a future path that reaches here with unflushed state would silently give * workers a stale view; no test covers its removal, and none claims to. */ - ColumnarFlushWriteStateForRelation(state->relid); + PgColumnarFlushWriteStateForRelation(state->relid); { Relation frel = table_open(state->relid, AccessShareLock); - ColumnarFlushDeleteVectorForRelation(frel); + PgColumnarFlushDeleteVectorForRelation(frel); table_close(frel, AccessShareLock); } } static void -ColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, +PgColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt, void *coordinate) { pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; @@ -3981,26 +3981,26 @@ ColumnarReInitializeDSMGroupAggScan(CustomScanState *node, ParallelContext *pcxt } static void -ColumnarInitializeWorkerGroupAggScan(CustomScanState *node, shm_toc *toc, +PgColumnarInitializeWorkerGroupAggScan(CustomScanState *node, shm_toc *toc, void *coordinate) { - ColumnarGroupAggScanState *state = (ColumnarGroupAggScanState *) node; + PgColumnarGroupAggScanState *state = (PgColumnarGroupAggScanState *) node; pg_atomic_uint32 *counter = (pg_atomic_uint32 *) coordinate; state->parallelCounter = counter; } -static const CustomExecMethods columnar_groupagg_parallel_exec_methods = { +static const CustomExecMethods pgcolumnar_groupagg_parallel_exec_methods = { .CustomName = "ColumnarScan", - .BeginCustomScan = ColumnarBeginGroupAggScan, - .ExecCustomScan = ColumnarExecGroupAggScan, - .EndCustomScan = ColumnarEndGroupAggScan, - .ReScanCustomScan = ColumnarReScanGroupAggScan, - .ExplainCustomScan = ColumnarExplainGroupAggScan, - .EstimateDSMCustomScan = ColumnarEstimateDSMGroupAggScan, - .InitializeDSMCustomScan = ColumnarInitializeDSMGroupAggScan, - .ReInitializeDSMCustomScan = ColumnarReInitializeDSMGroupAggScan, - .InitializeWorkerCustomScan = ColumnarInitializeWorkerGroupAggScan, + .BeginCustomScan = PgColumnarBeginGroupAggScan, + .ExecCustomScan = PgColumnarExecGroupAggScan, + .EndCustomScan = PgColumnarEndGroupAggScan, + .ReScanCustomScan = PgColumnarReScanGroupAggScan, + .ExplainCustomScan = PgColumnarExplainGroupAggScan, + .EstimateDSMCustomScan = PgColumnarEstimateDSMGroupAggScan, + .InitializeDSMCustomScan = PgColumnarInitializeDSMGroupAggScan, + .ReInitializeDSMCustomScan = PgColumnarReInitializeDSMGroupAggScan, + .InitializeWorkerCustomScan = PgColumnarInitializeWorkerGroupAggScan, }; /* ------------------------------------------------------------------------- @@ -4008,8 +4008,8 @@ static const CustomExecMethods columnar_groupagg_parallel_exec_methods = { * ------------------------------------------------------------------------- */ void -ColumnarVectorInit(void) +PgColumnarVectorInit(void) { prev_create_upper_paths_hook = create_upper_paths_hook; - create_upper_paths_hook = ColumnarCreateUpperPaths; + create_upper_paths_hook = PgColumnarCreateUpperPaths; } diff --git a/src/columnar_visibilitymap.c b/src/columnar_visibilitymap.c index 5070850..2433e8a 100644 --- a/src/columnar_visibilitymap.c +++ b/src/columnar_visibilitymap.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_visibilitymap.c + * pgcolumnar_visibilitymap.c * Visibility-map fork maintenance for pgColumnar index-only scans * (gap 28 direction 1, phase 1: the load-bearing prototype). * @@ -46,8 +46,8 @@ #include "storage/procarray.h" #include "utils/rel.h" -PG_FUNCTION_INFO_V1(columnar_vm_selftest); -PG_FUNCTION_INFO_V1(columnar_vm_is_visible); +PG_FUNCTION_INFO_V1(pgcolumnar_vm_selftest); +PG_FUNCTION_INFO_V1(pgcolumnar_vm_is_visible); /* * Visibility-map on-disk layout. These mirror the private macros in @@ -67,14 +67,14 @@ PG_FUNCTION_INFO_V1(columnar_vm_is_visible); (((x) % COLUMNAR_VM_BLOCKS_PER_BYTE) * COLUMNAR_VM_BITS_PER_BLOCK) /* - * ColumnarVMSetVisible + * PgColumnarVMSetVisible * Mark the synthetic block `blk` all-visible in the relation's VM fork, * WAL-logged. Idempotent: a no-op if the bit is already set. This writes * only the VM fork -- there is no heap page to flag -- which is why it does * not go through visibilitymap_set(). */ void -ColumnarVMSetVisible(Relation rel, BlockNumber blk) +PgColumnarVMSetVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; Page page; @@ -108,12 +108,12 @@ ColumnarVMSetVisible(Relation rel, BlockNumber blk) } /* - * ColumnarVMClearVisible + * PgColumnarVMClearVisible * Clear the all-visible (and all-frozen) bits for `blk`, WAL-logged. Used * by write paths so a modified range is never reported all-visible. */ void -ColumnarVMClearVisible(Relation rel, BlockNumber blk) +PgColumnarVMClearVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; Page page; @@ -153,28 +153,28 @@ ColumnarVMClearVisible(Relation rel, BlockNumber blk) } /* - * ColumnarVMClearForRow + * PgColumnarVMClearForRow * Clear the all-visible bit for the synthetic block that holds `rowNumber`. - * The block is computed the same way ColumnarRowNumberToItemPointer derives + * The block is computed the same way PgColumnarRowNumberToItemPointer derives * a TID block, so it matches the block the index-only-scan executor probes. * Called by every write path (insert/delete/update) so a modified block is * never left all-visible. Cheap when no bit is set (a short-circuit read). */ void -ColumnarVMClearForRow(Relation rel, uint64 rowNumber) +PgColumnarVMClearForRow(Relation rel, uint64 rowNumber) { BlockNumber blk = (BlockNumber) (rowNumber / COLUMNAR_VALID_ITEMPOINTER_OFFSETS); - ColumnarVMClearVisible(rel, blk); + PgColumnarVMClearVisible(rel, blk); } /* - * ColumnarVMIsVisible + * PgColumnarVMIsVisible * True if `blk` is marked all-visible in the VM fork. Thin wrapper over the * stock reader (the same call the index-only-scan executor makes). */ bool -ColumnarVMIsVisible(Relation rel, BlockNumber blk) +PgColumnarVMIsVisible(Relation rel, BlockNumber blk) { Buffer vmbuf = InvalidBuffer; uint8 status = visibilitymap_get_status(rel, blk, &vmbuf); @@ -188,8 +188,8 @@ ColumnarVMIsVisible(Relation rel, BlockNumber blk) static int rowrange_cmp(const void *a, const void *b) { - uint64 fa = ((const ColumnarRowRange *) a)->firstRowNumber; - uint64 fb = ((const ColumnarRowRange *) b)->firstRowNumber; + uint64 fa = ((const PgColumnarRowRange *) a)->firstRowNumber; + uint64 fb = ((const PgColumnarRowRange *) b)->firstRowNumber; if (fa < fb) return -1; @@ -199,7 +199,7 @@ rowrange_cmp(const void *a, const void *b) } /* - * ColumnarVMSetVisibleForRelation + * PgColumnarVMSetVisibleForRelation * Lazy vacuum step (gap 28 phase 3): mark all-visible chunk groups in the * VM fork. Computes the all-visible groups (stripe committed past the * oldest-xmin horizon, no committed-or-in-progress deletes), merges @@ -211,25 +211,25 @@ rowrange_cmp(const void *a, const void *b) * and clear-on-write removes any bit for a row changed after this runs. */ void -ColumnarVMSetVisibleForRelation(Relation rel) +PgColumnarVMSetVisibleForRelation(Relation rel) { - uint64 storageId = ColumnarStorageId(rel); - TransactionId oldestXmin = ColumnarOldestXmin(rel); - List *groups = ColumnarComputeAllVisibleGroups(storageId, oldestXmin); + uint64 storageId = PgColumnarStorageId(rel); + TransactionId oldestXmin = PgColumnarOldestXmin(rel); + List *groups = PgColumnarComputeAllVisibleGroups(storageId, oldestXmin); uint64 K = COLUMNAR_VALID_ITEMPOINTER_OFFSETS; int n = list_length(groups); - ColumnarRowRange *arr; + PgColumnarRowRange *arr; ListCell *lc; int i; if (n == 0) return; - arr = palloc(sizeof(ColumnarRowRange) * n); + arr = palloc(sizeof(PgColumnarRowRange) * n); i = 0; foreach(lc, groups) - arr[i++] = *(ColumnarRowRange *) lfirst(lc); - qsort(arr, n, sizeof(ColumnarRowRange), rowrange_cmp); + arr[i++] = *(PgColumnarRowRange *) lfirst(lc); + qsort(arr, n, sizeof(PgColumnarRowRange), rowrange_cmp); i = 0; while (i < n) @@ -254,7 +254,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) b = (BlockNumber) ((lo + K - 1) / K); bend = (BlockNumber) (hi / K); for (; b < bend; b++) - ColumnarVMSetVisible(rel, b); + PgColumnarVMSetVisible(rel, b); i = m; } @@ -263,7 +263,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) } /* - * columnar_vm_selftest(rel regclass, blk int) -> bool + * pgcolumnar_vm_selftest(rel regclass, blk int) -> bool * Phase-1 proof: set the all-visible bit for a synthetic block on a * columnar relation, then read it back through the backend's own * visibilitymap_get_status. Returns true iff the round trip succeeds, @@ -271,7 +271,7 @@ ColumnarVMSetVisibleForRelation(Relation rel) * reader the index-only-scan executor uses. */ Datum -columnar_vm_selftest(PG_FUNCTION_ARGS) +pgcolumnar_vm_selftest(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); BlockNumber blk = (BlockNumber) PG_GETARG_INT32(1); @@ -281,9 +281,9 @@ columnar_vm_selftest(PG_FUNCTION_ARGS) rel = table_open(relid, RowExclusiveLock); - before = ColumnarVMIsVisible(rel, blk); - ColumnarVMSetVisible(rel, blk); - after = ColumnarVMIsVisible(rel, blk); + before = PgColumnarVMIsVisible(rel, blk); + PgColumnarVMSetVisible(rel, blk); + after = PgColumnarVMIsVisible(rel, blk); table_close(rel, RowExclusiveLock); @@ -292,18 +292,18 @@ columnar_vm_selftest(PG_FUNCTION_ARGS) } /* - * columnar_vm_is_visible(rel regclass, blk int) -> bool + * pgcolumnar_vm_is_visible(rel regclass, blk int) -> bool * Read-only probe: is the synthetic block marked all-visible in the VM * fork? Used by the phase-3 tests to check that lazy vacuum sets bits for * all-visible groups and clear-on-write removes them for modified ones. */ Datum -columnar_vm_is_visible(PG_FUNCTION_ARGS) +pgcolumnar_vm_is_visible(PG_FUNCTION_ARGS) { Oid relid = PG_GETARG_OID(0); BlockNumber blk = (BlockNumber) PG_GETARG_INT32(1); Relation rel = table_open(relid, AccessShareLock); - bool vis = ColumnarVMIsVisible(rel, blk); + bool vis = PgColumnarVMIsVisible(rel, blk); table_close(rel, AccessShareLock); PG_RETURN_BOOL(vis); diff --git a/src/columnar_write_state.c b/src/columnar_write_state.c index c94e4eb..e7700f0 100644 --- a/src/columnar_write_state.c +++ b/src/columnar_write_state.c @@ -1,6 +1,6 @@ /*------------------------------------------------------------------------- * - * columnar_write_state.c + * pgcolumnar_write_state.c * The columnar writer: batch rows into chunk groups and stripes, and * flush a stripe (data pages + catalog rows) when it fills or at * transaction pre-commit (spec 4, 9). @@ -73,21 +73,21 @@ * predicate could take the shortcut, because it does not build stored bounds * that a later scan trusts; this one does, so it cannot. */ -typedef enum ColumnarFastCmp +typedef enum PgColumnarFastCmp { COLUMNAR_FASTCMP_NONE = 0, COLUMNAR_FASTCMP_I16, COLUMNAR_FASTCMP_I32, COLUMNAR_FASTCMP_I64 -} ColumnarFastCmp; +} PgColumnarFastCmp; /* per-column, per-write-state facts needed for the min/max skip list */ -typedef struct ColumnarColumnDef +typedef struct PgColumnarColumnDef { bool orderable; /* type has a default btree comparison proc */ FmgrInfo cmpFn; /* the comparison proc, when orderable */ Oid collation; /* collation to compare under */ - ColumnarFastCmp fastCmp; /* direct comparison, when the type allows */ + PgColumnarFastCmp fastCmp; /* direct comparison, when the type allows */ /* * int2/int4 column: its exact sum fits an int64 accumulator, so the zone map @@ -104,7 +104,7 @@ typedef struct ColumnarColumnDef bool bloomable; FmgrInfo hashFn; Oid hashCollation; /* collation to hash under (InvalidOid if none) */ -} ColumnarColumnDef; +} PgColumnarColumnDef; /* one column's two streams within one chunk group */ typedef struct ColumnChunkBuffer @@ -146,7 +146,7 @@ typedef struct ChunkGroupBuffer ColumnChunkBuffer *columns; /* array [natts] */ } ChunkGroupBuffer; -struct ColumnarWriteState +struct PgColumnarWriteState { Oid relid; SubTransactionId subid; /* subtransaction that owns the buffer */ @@ -159,7 +159,7 @@ struct ColumnarWriteState bool bloomEnabled; /* columnar.enable_bloom_filter at open time */ int encodeEffort; /* per-table encode_effort at open time */ uint64 storageId; - ColumnarColumnDef *colDefs; /* array [natts], in writeContext */ + PgColumnarColumnDef *colDefs; /* array [natts], in writeContext */ MemoryContext writeContext; /* lives for the transaction */ MemoryContext stripeContext; /* reset after each stripe flush */ @@ -204,28 +204,28 @@ struct ColumnarWriteState * lookup that builds the list. */ bool projInited; - List *projWriters; /* list of ColumnarProjWriter * */ + List *projWriters; /* list of PgColumnarProjWriter * */ }; -/* per-backend registry of pending write states, in ColumnarWriteContext */ -static MemoryContext ColumnarWriteContext = NULL; -static List *ColumnarWriteStates = NIL; +/* per-backend registry of pending write states, in PgColumnarWriteContext */ +static MemoryContext PgColumnarWriteContext = NULL; +static List *PgColumnarWriteStates = NIL; -static void columnar_flush_row_group(ColumnarWriteState *writeState); -static void flush_ws_projections(ColumnarWriteState *writeState); -static ChunkGroupBuffer *columnar_start_chunk_group(ColumnarWriteState *writeState); +static void pgcolumnar_flush_row_group(PgColumnarWriteState *writeState); +static void flush_ws_projections(PgColumnarWriteState *writeState); +static ChunkGroupBuffer *pgcolumnar_start_chunk_group(PgColumnarWriteState *writeState); static uint64 *grow_uint64_array(uint64 *arr, int oldSize, int newSize); -static void columnar_init_col_defs(ColumnarWriteState *writeState); +static void pgcolumnar_init_col_defs(PgColumnarWriteState *writeState); /* - * columnar_cmp_value + * pgcolumnar_cmp_value * Compare two values of a column under its ordering, taking the direct * route when the type allows one and fmgr otherwise. The integer kinds * reproduce their btree comparison exactly, so which route is taken can * never change the answer. */ static inline int32 -columnar_cmp_value(ColumnarColumnDef *def, Datum a, Datum b) +pgcolumnar_cmp_value(PgColumnarColumnDef *def, Datum a, Datum b) { switch (def->fastCmp) { @@ -258,18 +258,18 @@ columnar_cmp_value(ColumnarColumnDef *def, Datum a, Datum b) } /* - * columnar_init_col_defs + * pgcolumnar_init_col_defs * Allocate and fill writeState->colDefs: for each column, resolve the btree * comparison proc (for the per-chunk min/max skip list, spec 7.2) and the * hash proc (for the per-chunk bloom filter, I7). Shared by the base writer * and the projection writer so both carry skip metadata. */ static void -columnar_init_col_defs(ColumnarWriteState *writeState) +pgcolumnar_init_col_defs(PgColumnarWriteState *writeState) { int c; - writeState->colDefs = palloc0(sizeof(ColumnarColumnDef) * writeState->natts); + writeState->colDefs = palloc0(sizeof(PgColumnarColumnDef) * writeState->natts); for (c = 0; c < writeState->natts; c++) { @@ -286,7 +286,7 @@ columnar_init_col_defs(ColumnarWriteState *writeState) { writeState->colDefs[c].orderable = true; fmgr_info_copy(&writeState->colDefs[c].cmpFn, - &tce->cmp_proc_finfo, ColumnarWriteContext); + &tce->cmp_proc_finfo, PgColumnarWriteContext); writeState->colDefs[c].collation = att->attcollation; /* @@ -327,38 +327,38 @@ columnar_init_col_defs(ColumnarWriteState *writeState) */ if (writeState->bloomEnabled && OidIsValid(tce->hash_proc_finfo.fn_oid) && - ColumnarCollationIsDeterministic(att->attcollation)) + PgColumnarCollationIsDeterministic(att->attcollation)) { writeState->colDefs[c].bloomable = true; fmgr_info_copy(&writeState->colDefs[c].hashFn, - &tce->hash_proc_finfo, ColumnarWriteContext); + &tce->hash_proc_finfo, PgColumnarWriteContext); writeState->colDefs[c].hashCollation = att->attcollation; } } } /* - * ColumnarWriteStateStripeCount + * PgColumnarWriteStateStripeCount * How many stripe reservations this write state has taken so far (#311). * * A caller that wants to know which groups IT wrote records this before - * its writes and takes the tail afterwards, because ColumnarGetWriteState + * its writes and takes the tail afterwards, because PgColumnarGetWriteState * can hand back a state that already holds another statement's entries. */ int -ColumnarWriteStateStripeCount(ColumnarWriteState *ws) +PgColumnarWriteStateStripeCount(PgColumnarWriteState *ws) { return ws->nReservedStripeIds; } /* - * ColumnarWriteStateStripeIds + * PgColumnarWriteStateStripeIds * The stripe ids this write state has reserved, in reservation order. * Points at the write state's own array, which stays valid until the * transaction ends. */ uint64 * -ColumnarWriteStateStripeIds(ColumnarWriteState *ws, int *n) +PgColumnarWriteStateStripeIds(PgColumnarWriteState *ws, int *n) { *n = ws->nReservedStripeIds; return ws->reservedStripeIds; @@ -378,17 +378,17 @@ grow_uint64_array(uint64 *arr, int oldSize, int newSize) } /* - * ColumnarGetWriteState + * PgColumnarGetWriteState * Find or create the pending write state for a relation. */ -ColumnarWriteState * -ColumnarGetWriteState(Relation rel) +PgColumnarWriteState * +PgColumnarGetWriteState(Relation rel) { Oid relid = RelationGetRelid(rel); SubTransactionId subid = GetCurrentSubTransactionId(); ListCell *lc; MemoryContext oldContext; - ColumnarWriteState *writeState; + PgColumnarWriteState *writeState; /* * A write state is keyed by (relation, subtransaction) so that a buffer @@ -396,9 +396,9 @@ ColumnarGetWriteState(Relation rel) * rollback of a subtransaction a simple matter of dropping its buffers * (spec 9). */ - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - writeState = (ColumnarWriteState *) lfirst(lc); + writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid || writeState->subid != subid) continue; @@ -418,23 +418,23 @@ ColumnarGetWriteState(Relation rel) return writeState; if (writeState->stripeRowCount > 0) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - ColumnarWriteStates = list_delete_ptr(ColumnarWriteStates, writeState); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + PgColumnarWriteStates = list_delete_ptr(PgColumnarWriteStates, writeState); MemoryContextSwitchTo(oldContext); break; } - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); - writeState = palloc0(sizeof(ColumnarWriteState)); + writeState = palloc0(sizeof(PgColumnarWriteState)); writeState->relid = relid; writeState->subid = subid; /* CopyConstr (not CopyEntry) so attgenerated is preserved: the flush skips @@ -442,19 +442,19 @@ ColumnarGetWriteState(Relation rel) * CreateTupleDescCopy would clear. */ writeState->tupdesc = CreateTupleDescCopyConstr(RelationGetDescr(rel)); writeState->natts = writeState->tupdesc->natts; - writeState->stripeRowLimit = columnar_stripe_row_limit; - writeState->chunkGroupRowLimit = columnar_chunk_group_row_limit; - writeState->compressionType = columnar_compression; - writeState->compressionLevel = columnar_compression_level; + writeState->stripeRowLimit = pgcolumnar_stripe_row_limit; + writeState->chunkGroupRowLimit = pgcolumnar_chunk_group_row_limit; + writeState->compressionType = pgcolumnar_compression; + writeState->compressionLevel = pgcolumnar_compression_level; /* * Whether to build bloom filters at all, captured here with the other * write-time settings rather than consulted per row, so one stripe is * written under one decision. */ - writeState->bloomEnabled = columnar_enable_bloom_filter; + writeState->bloomEnabled = pgcolumnar_enable_bloom_filter; writeState->encodeEffort = COLUMNAR_ENCODE_EFFORT_FULL; - writeState->storageId = ColumnarStorageId(rel); + writeState->storageId = PgColumnarStorageId(rel); /* * Per-table options (spec 7.4) override the instance-wide GUC defaults for @@ -463,9 +463,9 @@ ColumnarGetWriteState(Relation rel) * inserts (spec 9). */ { - ColumnarOptions opts; + PgColumnarOptions opts; - if (ColumnarReadOptions(relid, &opts)) + if (PgColumnarReadOptions(relid, &opts)) { if (opts.stripeRowLimitSet) writeState->stripeRowLimit = opts.stripeRowLimit; @@ -480,12 +480,12 @@ ColumnarGetWriteState(Relation rel) } } - columnar_init_col_defs(writeState); + pgcolumnar_init_col_defs(writeState); - writeState->stripeContext = AllocSetContextCreate(ColumnarWriteContext, + writeState->stripeContext = AllocSetContextCreate(PgColumnarWriteContext, "columnar stripe", ALLOCSET_DEFAULT_SIZES); - writeState->writeContext = ColumnarWriteContext; + writeState->writeContext = PgColumnarWriteContext; writeState->chunkGroups = NIL; writeState->currentGroup = NULL; writeState->stripeRowCount = 0; @@ -496,7 +496,7 @@ ColumnarGetWriteState(Relation rel) writeState->nReservedStripeIds = 0; writeState->reservedStripeIdsSize = 0; - ColumnarWriteStates = lappend(ColumnarWriteStates, writeState); + PgColumnarWriteStates = lappend(PgColumnarWriteStates, writeState); MemoryContextSwitchTo(oldContext); @@ -504,31 +504,31 @@ ColumnarGetWriteState(Relation rel) } /* - * ColumnarEnsureStorageRow + * PgColumnarEnsureStorageRow * Create the native storage catalog row for `rel` if it does not exist, * with exactly the metadata a normal flush would record. Used by * pgcolumnar.parallel_copy's coordinator to pre-create and commit the storage * row before launching concurrent loaders, so each loader (with - * columnar_bulk_parallel_writer set) sees it committed and skips the - * storage-row creation lock. Idempotent -- ColumnarInsertNativeStorageRow + * pgcolumnar_bulk_parallel_writer set) sees it committed and skips the + * storage-row creation lock. Idempotent -- PgColumnarInsertNativeStorageRow * returns if the row already exists. */ void -ColumnarEnsureStorageRow(Relation rel) +PgColumnarEnsureStorageRow(Relation rel) { NativeStorageMetadata s; - ColumnarOptions opts; - int stripeRowLimit = columnar_stripe_row_limit; + PgColumnarOptions opts; + int stripeRowLimit = pgcolumnar_stripe_row_limit; - if (ColumnarReadOptions(RelationGetRelid(rel), &opts) && opts.stripeRowLimitSet) + if (PgColumnarReadOptions(RelationGetRelid(rel), &opts) && opts.stripeRowLimitSet) stripeRowLimit = opts.stripeRowLimit; - s.storageId = ColumnarStorageId(rel); + s.storageId = PgColumnarStorageId(rel); s.relationOid = RelationGetRelid(rel); s.formatVersion = COLUMNAR_NATIVE_VERSION_MAJOR; s.vectorLength = COLUMNAR_NATIVE_VECTOR_LENGTH; s.rowGroupLimit = stripeRowLimit; - ColumnarInsertNativeStorageRow(&s); + PgColumnarInsertNativeStorageRow(&s); } /* @@ -592,7 +592,7 @@ buffered_build_offsets(ColumnChunkBuffer *col, Form_pg_attribute att, * Bounded by chunk_group_row_limit, which is user-settable, so this pass * is a cancellation point for the same reason the decoders are. It is * also on the unique-check fetch path, which had no interrupt check at - * all before #220 and still has none below columnar_fetch_row. + * all before #220 and still has none below pgcolumnar_fetch_row. */ CHECK_FOR_INTERRUPTS(); @@ -600,19 +600,19 @@ buffered_build_offsets(ColumnChunkBuffer *col, Form_pg_attribute att, (uint32) (cursor - col->valueStream.data); if (col->existsStream.data[i]) - (void) ColumnarDecodeValue(att, &cursor, scratch); + (void) PgColumnarDecodeValue(att, &cursor, scratch); } MemoryContextDelete(scratch); } /* - * columnar_start_chunk_group + * pgcolumnar_start_chunk_group * Begin a new chunk group inside the current stripe, allocated in the * stripe memory context. */ static ChunkGroupBuffer * -columnar_start_chunk_group(ColumnarWriteState *writeState) +pgcolumnar_start_chunk_group(PgColumnarWriteState *writeState) { MemoryContext oldContext = MemoryContextSwitchTo(writeState->stripeContext); ChunkGroupBuffer *group = palloc0(sizeof(ChunkGroupBuffer)); @@ -636,14 +636,14 @@ columnar_start_chunk_group(ColumnarWriteState *writeState) } /* - * ColumnarWriteRow + * PgColumnarWriteRow * Append one row to the current stripe, opening a new chunk group when * the current one is full and flushing the stripe when it reaches the * stripe row limit. Returns the stable 1-based row number assigned to the * row (spec 6), so the caller can set the row's item pointer for indexing. */ uint64 -ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, +PgColumnarWriteRow(PgColumnarWriteState *writeState, Relation rel, Datum *values, bool *nulls) { ChunkGroupBuffer *group = writeState->currentGroup; @@ -660,7 +660,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, */ if (!writeState->haveReservation) { - ColumnarReserveRowNumbers(rel, (uint64) writeState->stripeRowLimit, + PgColumnarReserveRowNumbers(rel, (uint64) writeState->stripeRowLimit, &writeState->stripeId, &writeState->stripeFirstRowNumber); writeState->haveReservation = true; @@ -693,7 +693,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, if (group == NULL || group->rowCount >= (uint64) writeState->chunkGroupRowLimit) - group = columnar_start_chunk_group(writeState); + group = pgcolumnar_start_chunk_group(writeState); for (c = 0; c < writeState->natts; c++) { @@ -715,7 +715,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, else { appendStringInfoChar(&col->existsStream, 1); - ColumnarEncodeValue(&col->valueStream, att, values[c]); + PgColumnarEncodeValue(&col->valueStream, att, values[c]); col->valueCount++; /* accumulate the value's hash for the per-chunk bloom filter (I7) */ @@ -738,7 +738,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, /* maintain the per-chunk min/max for orderable types */ if (writeState->colDefs[c].orderable) { - ColumnarColumnDef *def = &writeState->colDefs[c]; + PgColumnarColumnDef *def = &writeState->colDefs[c]; MemoryContext oldContext = MemoryContextSwitchTo(writeState->stripeContext); @@ -760,7 +760,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, * of a serial or timestamp column produces -- cost one * comparison per value instead of two. */ - int32 cmpMax = columnar_cmp_value(def, values[c], + int32 cmpMax = pgcolumnar_cmp_value(def, values[c], col->maxValue); if (cmpMax > 0) @@ -770,7 +770,7 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, col->maxValue = datumCopy(values[c], att->attbyval, att->attlen); } - else if (columnar_cmp_value(def, values[c], + else if (pgcolumnar_cmp_value(def, values[c], col->minValue) < 0) { if (!att->attbyval) @@ -789,13 +789,13 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, writeState->stripeRowCount++; if (writeState->stripeRowCount >= (uint64) writeState->stripeRowLimit) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); return rowNumber; } /* - * ColumnarBufferedRowByNumber + * PgColumnarBufferedRowByNumber * Reconstruct a single row that is still held in an unflushed write buffer, * addressed by its row number (spec 6). Returns true and fills values/nulls * (by-reference values copied into the current memory context) when the row @@ -809,16 +809,16 @@ ColumnarWriteRow(ColumnarWriteState *writeState, Relation rel, * to call while the caller holds an index buffer lock. */ bool -ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, +PgColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, Datum *values, bool *nulls) { Oid relid = RelationGetRelid(rel); MemoryContext target = CurrentMemoryContext; ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *ws = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *ws = (PgColumnarWriteState *) lfirst(lc); uint64 offset; uint64 accumulated; ListCell *glc; @@ -876,7 +876,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, if (existsBytes[posInGroup]) { - values[c] = ColumnarDecodeValue(att, &cursor, target); + values[c] = PgColumnarDecodeValue(att, &cursor, target); nulls[c] = false; } else @@ -894,7 +894,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, } /* - * columnar_flush_row_group + * pgcolumnar_flush_row_group * Native-format (PGCN v1) flush. Lay out the accumulated rows as one row * group: each column is a column chunk of [validity bitmap][values], where * the validity bitmap is one bit per row (LSB-first) and the values are the @@ -905,7 +905,7 @@ ColumnarBufferedRowByNumber(Relation rel, uint64 rowNumber, * column_chunk, zone_map, bloom). */ static void -columnar_flush_row_group(ColumnarWriteState *writeState) +pgcolumnar_flush_row_group(PgColumnarWriteState *writeState) { MemoryContext flushContext; MemoryContext oldContext; @@ -988,7 +988,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) char *finalData; uint32 finalLen; int blockCodec = COLUMNAR_COMPRESSION_NONE; - ColumnarColumnDef *def = &writeState->colDefs[c]; + PgColumnarColumnDef *def = &writeState->colDefs[c]; int vec = 0; bool chunkHasMinMax = false; Datum chunkMin = (Datum) 0; @@ -1090,8 +1090,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) */ if (corpus.len > 0 && writeState->encodeEffort != COLUMNAR_ENCODE_EFFORT_FAST && - !ColumnarFsstDictWins(corpus.data, (uint32) corpus.len)) - ColumnarFsstBuildChunkTable(corpus.data, sampleLen, att, + !PgColumnarFsstDictWins(corpus.data, (uint32) corpus.len)) + PgColumnarFsstBuildChunkTable(corpus.data, sampleLen, att, &fsstTable, &fsstTableLen); /* @@ -1112,7 +1112,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) * imprecise but inverted, so no margin on the sample would be safe. */ if (fsstTable != NULL && - !ColumnarFsstHelpsCompressed(corpus.data, (uint32) corpus.len, + !PgColumnarFsstHelpsCompressed(corpus.data, (uint32) corpus.len, fsstTable, fsstTableLen, writeState->compressionType, writeState->compressionLevel)) @@ -1137,7 +1137,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) uint32 entryValueCount; uint32 entryRawLen; - encType = ColumnarEncodeChunk(col->valueStream.data, + encType = PgColumnarEncodeChunk(col->valueStream.data, col->valueStream.len, att, col->valueCount, fsstTable, fsstTableLen, &encData, &encLen); @@ -1178,8 +1178,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) initStringInfo(&mn); initStringInfo(&mx); - ColumnarEncodeValue(&mn, att, col->minValue); - ColumnarEncodeValue(&mx, att, col->maxValue); + PgColumnarEncodeValue(&mn, att, col->minValue); + PgColumnarEncodeValue(&mx, att, col->maxValue); z->hasMinMax = true; z->minimum = mn.data; z->minimumLen = (uint32) mn.len; @@ -1249,8 +1249,8 @@ columnar_flush_row_group(ColumnarWriteState *writeState) initStringInfo(&mn); initStringInfo(&mx); - ColumnarEncodeValue(&mn, att, chunkMin); - ColumnarEncodeValue(&mx, att, chunkMax); + PgColumnarEncodeValue(&mn, att, chunkMin); + PgColumnarEncodeValue(&mx, att, chunkMax); z->hasMinMax = true; z->minimum = mn.data; z->minimumLen = (uint32) mn.len; @@ -1279,7 +1279,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) col->hashBuf.len); } if (hashes.len > 0 && - ColumnarBloomBuild((const uint32 *) hashes.data, + PgColumnarBloomBuild((const uint32 *) hashes.data, hashes.len / sizeof(uint32), &bloom, &bloomLen)) { @@ -1305,7 +1305,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) int usedType; int usedLevel; - ColumnarCompressValueStream(encoded->data, encoded->len, + PgColumnarCompressValueStream(encoded->data, encoded->len, writeState->compressionType, writeState->compressionLevel, &compData, &compLen, @@ -1343,14 +1343,14 @@ columnar_flush_row_group(ColumnarWriteState *writeState) reusedOffset = false; if (dataLength > 0 && CheckRelationLockedByMe(rel, ShareUpdateExclusiveLock, false)) - reusedOffset = ColumnarAllocateFreeSpace(ColumnarStorageId(rel), dataLength, - ColumnarOldestXmin(rel), &fileOffset); + reusedOffset = PgColumnarAllocateFreeSpace(PgColumnarStorageId(rel), dataLength, + PgColumnarOldestXmin(rel), &fileOffset); LockRelationForExtension(rel, ExclusiveLock); if (!reusedOffset) - ColumnarReserveOffset(rel, dataLength, &fileOffset); + PgColumnarReserveOffset(rel, dataLength, &fileOffset); if (dataLength > 0) - ColumnarWriteLogicalData(rel, fileOffset, data->data, dataLength); + PgColumnarWriteLogicalData(rel, fileOffset, data->data, dataLength); UnlockRelationForExtension(rel, ExclusiveLock); { @@ -1361,7 +1361,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) s.formatVersion = COLUMNAR_NATIVE_VERSION_MAJOR; s.vectorLength = COLUMNAR_NATIVE_VECTOR_LENGTH; s.rowGroupLimit = writeState->stripeRowLimit; - ColumnarInsertNativeStorageRow(&s); + PgColumnarInsertNativeStorageRow(&s); } { NativeRowGroupMetadata rg; @@ -1372,7 +1372,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) rg.rowCount = rowCount; rg.byteLength = dataLength; rg.firstRowNumber = writeState->stripeFirstRowNumber; - ColumnarInsertRowGroupRow(&rg); + PgColumnarInsertRowGroupRow(&rg); } for (c = 0; c < natts; c++) { @@ -1391,12 +1391,12 @@ columnar_flush_row_group(ColumnarWriteState *writeState) cc.blockCodec = chunkBlockCodec[c]; cc.pageOffset = fileOffset + chunkOffset[c]; cc.pageLength = chunkLength[c]; - ColumnarInsertColumnChunkRow(&cc); + PgColumnarInsertColumnChunkRow(&cc); } foreach(lc, zoneRows) - ColumnarInsertZoneMapRow((NativeZoneMapMetadata *) lfirst(lc)); + PgColumnarInsertZoneMapRow((NativeZoneMapMetadata *) lfirst(lc)); foreach(lc, bloomRows) - ColumnarInsertBloomRow((NativeBloomMetadata *) lfirst(lc)); + PgColumnarInsertBloomRow((NativeBloomMetadata *) lfirst(lc)); table_close(rel, RowExclusiveLock); @@ -1421,7 +1421,7 @@ columnar_flush_row_group(ColumnarWriteState *writeState) * table's relation file and row-number space. On insert, the projected columns * plus the base row number are buffered; at flush the batch is sorted on the * projection's sort key and written as a stripe to the projection's storage, - * reusing the base stripe encoder (ColumnarWriteRow + columnar_flush_row_group). + * reusing the base stripe encoder (PgColumnarWriteRow + pgcolumnar_flush_row_group). * The base row number is stored as a leading int8 column so the projection can * be joined back to the base; deletes/visibility come from the base delete_vector, so * only INSERT fans out (see design/gaps/26-IMPL-projections-phase2-plan.md). @@ -1434,7 +1434,7 @@ typedef struct ProjRow bool *nulls; } ProjRow; -typedef struct ColumnarProjWriter +typedef struct PgColumnarProjWriter { uint64 projStorageId; int ncols; /* number of projection columns (K) */ @@ -1455,14 +1455,14 @@ typedef struct ColumnarProjWriter int nrows; MemoryContext ctx; /* persists: struct arrays, projTupdesc */ MemoryContext rowCtx; /* reset after each stripe flush: row datums */ - ColumnarWriteState *innerWs; /* reused stripe encoder for this projection */ -} ColumnarProjWriter; + PgColumnarWriteState *innerWs; /* reused stripe encoder for this projection */ +} PgColumnarProjWriter; /* - * ColumnarWriteStateProjStripeIds + * PgColumnarWriteStateProjStripeIds * The stripe ids this write state's projection fan-out drew (#345). * * A projection writes through its own inner write state but reserves from - * the BASE relation's stripe counter, because ColumnarWriteRow is called + * the BASE relation's stripe counter, because PgColumnarWriteRow is called * with the base relation (see flush_proj_writer). Its groups are recorded * under the projection's own storage id, so they never appear in the base * relation's row group list. @@ -1478,7 +1478,7 @@ typedef struct ColumnarProjWriter * state has no projection writers. */ uint64 * -ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) +PgColumnarWriteStateProjStripeIds(PgColumnarWriteState *ws, int *n) { ListCell *lc; uint64 *ids = NULL; @@ -1491,7 +1491,7 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) foreach(lc, ws->projWriters) { - ColumnarProjWriter *w = (ColumnarProjWriter *) lfirst(lc); + PgColumnarProjWriter *w = (PgColumnarProjWriter *) lfirst(lc); if (w->innerWs != NULL) total += w->innerWs->nReservedStripeIds; @@ -1502,7 +1502,7 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) ids = (uint64 *) palloc(sizeof(uint64) * total); foreach(lc, ws->projWriters) { - ColumnarProjWriter *w = (ColumnarProjWriter *) lfirst(lc); + PgColumnarProjWriter *w = (PgColumnarProjWriter *) lfirst(lc); int i; if (w->innerWs == NULL) @@ -1516,28 +1516,28 @@ ColumnarWriteStateProjStripeIds(ColumnarWriteState *ws, int *n) /* - * columnar_build_write_state + * pgcolumnar_build_write_state * Allocate a standalone stripe encoder for the given tuple descriptor and - * storage id, not registered in ColumnarWriteStates. Used for a + * storage id, not registered in PgColumnarWriteStates. Used for a * projection's inner writer; carries the same per-chunk min/max and bloom * skip metadata as the base writer so a sorted projection gives tight * min/max ranges for the planner (gap 26). */ -static ColumnarWriteState * -columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, +static PgColumnarWriteState * +pgcolumnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, int stripeRowLimit, int chunkGroupRowLimit, int compType, int compLevel) { MemoryContext oldContext; - ColumnarWriteState *ws; + PgColumnarWriteState *ws; - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); - ws = palloc0(sizeof(ColumnarWriteState)); + ws = palloc0(sizeof(PgColumnarWriteState)); ws->relid = relid; ws->subid = GetCurrentSubTransactionId(); ws->tupdesc = CreateTupleDescCopy(srcTupdesc); @@ -1552,7 +1552,7 @@ columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, * relation. Leaving this unset would zero it, silently dropping bloom * filters from projections while the setting was on. */ - ws->bloomEnabled = columnar_enable_bloom_filter; + ws->bloomEnabled = pgcolumnar_enable_bloom_filter; /* * And under the same encode_effort as its base, for the same reason: a @@ -1561,17 +1561,17 @@ columnar_build_write_state(Oid relid, TupleDesc srcTupdesc, uint64 storageId, */ ws->encodeEffort = COLUMNAR_ENCODE_EFFORT_FULL; { - ColumnarOptions opts; + PgColumnarOptions opts; - if (ColumnarReadOptions(relid, &opts) && opts.encodeEffortSet) + if (PgColumnarReadOptions(relid, &opts) && opts.encodeEffortSet) ws->encodeEffort = opts.encodeEffort; } ws->storageId = storageId; - columnar_init_col_defs(ws); /* min/max + bloom skip metadata for projections */ - ws->stripeContext = AllocSetContextCreate(ColumnarWriteContext, + pgcolumnar_init_col_defs(ws); /* min/max + bloom skip metadata for projections */ + ws->stripeContext = AllocSetContextCreate(PgColumnarWriteContext, "columnar proj stripe", ALLOCSET_DEFAULT_SIZES); - ws->writeContext = ColumnarWriteContext; + ws->writeContext = PgColumnarWriteContext; ws->chunkGroups = NIL; ws->currentGroup = NULL; ws->stripeRowCount = 0; @@ -1587,7 +1587,7 @@ proj_row_cmp(const void *a, const void *b, void *arg) { const ProjRow *ra = (const ProjRow *) a; const ProjRow *rb = (const ProjRow *) b; - ColumnarProjWriter *w = (ColumnarProjWriter *) arg; + PgColumnarProjWriter *w = (PgColumnarProjWriter *) arg; int i; for (i = 0; i < w->nsort; i++) @@ -1617,7 +1617,7 @@ proj_row_cmp(const void *a, const void *b, void *arg) * stripe to the projection's storage, then reset the buffer. */ static void -flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) +flush_proj_writer(PgColumnarProjWriter *w, Relation tableRel) { int i; @@ -1628,17 +1628,17 @@ flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) qsort_arg(w->rows, w->nrows, sizeof(ProjRow), proj_row_cmp, w); if (w->innerWs == NULL) - w->innerWs = columnar_build_write_state(RelationGetRelid(tableRel), + w->innerWs = pgcolumnar_build_write_state(RelationGetRelid(tableRel), w->projTupdesc, w->projStorageId, w->stripeRowLimit, w->chunkGroupRowLimit, w->compType, w->compLevel); for (i = 0; i < w->nrows; i++) - ColumnarWriteRow(w->innerWs, tableRel, w->rows[i].values, w->rows[i].nulls); + PgColumnarWriteRow(w->innerWs, tableRel, w->rows[i].values, w->rows[i].nulls); if (w->innerWs->stripeRowCount > 0) - columnar_flush_row_group(w->innerWs); + pgcolumnar_flush_row_group(w->innerWs); MemoryContextReset(w->rowCtx); w->nrows = 0; @@ -1646,24 +1646,24 @@ flush_proj_writer(ColumnarProjWriter *w, Relation tableRel) /* * build_proj_writer - * Construct a ColumnarProjWriter for one projection catalog row. + * Construct a PgColumnarProjWriter for one projection catalog row. */ -static ColumnarProjWriter * -build_proj_writer(Relation rel, const ColumnarProjection *proj, +static PgColumnarProjWriter * +build_proj_writer(Relation rel, const PgColumnarProjection *proj, int stripeRowLimit, int chunkGroupRowLimit, int compType, int compLevel) { TupleDesc tableDesc = RelationGetDescr(rel); MemoryContext ctx; MemoryContext oldContext; - ColumnarProjWriter *w; + PgColumnarProjWriter *w; int i; - ctx = AllocSetContextCreate(ColumnarWriteContext, "columnar proj writer", + ctx = AllocSetContextCreate(PgColumnarWriteContext, "columnar proj writer", ALLOCSET_DEFAULT_SIZES); oldContext = MemoryContextSwitchTo(ctx); - w = palloc0(sizeof(ColumnarProjWriter)); + w = palloc0(sizeof(PgColumnarProjWriter)); w->projStorageId = proj->projStorageId; w->ncols = proj->columnsLen; w->stripeRowLimit = stripeRowLimit; @@ -1734,7 +1734,7 @@ build_proj_writer(Relation rel, const ColumnarProjection *proj, * insert fan-out and the add-projection back-fill. */ static void -append_proj_row(ColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, +append_proj_row(PgColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, uint64 rowNumber, Datum *values, bool *nulls) { MemoryContext oldContext = MemoryContextSwitchTo(w->rowCtx); @@ -1770,14 +1770,14 @@ append_proj_row(ColumnarProjWriter *w, Relation rel, TupleDesc tableDesc, } /* - * ColumnarProjectionFanoutRow + * PgColumnarProjectionFanoutRow * Buffer a freshly inserted row into each additional projection of the - * relation. rowNumber is the base row number returned by ColumnarWriteRow. + * relation. rowNumber is the base row number returned by PgColumnarWriteRow. * The projection writers hang off the base write state, so they share its * (relid, subid) lifecycle. */ void -ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, +PgColumnarProjectionFanoutRow(Relation rel, PgColumnarWriteState *baseWs, uint64 rowNumber, Datum *values, bool *nulls) { TupleDesc tableDesc = RelationGetDescr(rel); @@ -1785,13 +1785,13 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, if (!baseWs->projInited) { - List *projs = ColumnarListProjections(baseWs->storageId); - MemoryContext oldContext = MemoryContextSwitchTo(ColumnarWriteContext); + List *projs = PgColumnarListProjections(baseWs->storageId); + MemoryContext oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); ListCell *pc; foreach(pc, projs) { - ColumnarProjection *p = (ColumnarProjection *) lfirst(pc); + PgColumnarProjection *p = (PgColumnarProjection *) lfirst(pc); if (p->projectionId == 0) continue; /* base projection is the table itself */ @@ -1810,12 +1810,12 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, return; foreach(lc, baseWs->projWriters) - append_proj_row((ColumnarProjWriter *) lfirst(lc), rel, tableDesc, + append_proj_row((PgColumnarProjWriter *) lfirst(lc), rel, tableDesc, rowNumber, values, nulls); } /* - * ColumnarBackfillProjection + * PgColumnarBackfillProjection * Populate a newly declared projection from the table's existing live rows * (gap 26): scan the base and buffer-sort-flush each row into the * projection's storage. Called by add_projection so a projection added to a @@ -1823,28 +1823,28 @@ ColumnarProjectionFanoutRow(Relation rel, ColumnarWriteState *baseWs, * concurrent writers (ShareLock) so no row is missed. */ void -ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) +PgColumnarBackfillProjection(Relation rel, const PgColumnarProjection *proj) { TupleDesc tableDesc = RelationGetDescr(rel); Oid relid = RelationGetRelid(rel); - int stripeRowLimit = columnar_stripe_row_limit; - int chunkGroupRowLimit = columnar_chunk_group_row_limit; - int compType = columnar_compression; - int compLevel = columnar_compression_level; - ColumnarOptions opts; - ColumnarProjWriter *w; - ColumnarReadState *readState; + int stripeRowLimit = pgcolumnar_stripe_row_limit; + int chunkGroupRowLimit = pgcolumnar_chunk_group_row_limit; + int compType = pgcolumnar_compression; + int compLevel = pgcolumnar_compression_level; + PgColumnarOptions opts; + PgColumnarProjWriter *w; + PgColumnarReadState *readState; Snapshot snapshot; Datum *values; bool *nulls; uint64 rowNumber; - if (ColumnarWriteContext == NULL) - ColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, + if (PgColumnarWriteContext == NULL) + PgColumnarWriteContext = AllocSetContextCreate(TopTransactionContext, "columnar write", ALLOCSET_DEFAULT_SIZES); - if (ColumnarReadOptions(relid, &opts)) + if (PgColumnarReadOptions(relid, &opts)) { if (opts.stripeRowLimitSet) stripeRowLimit = opts.stripeRowLimit; @@ -1857,8 +1857,8 @@ ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) } /* flush any pending base writes so the scan sees this transaction's rows */ - ColumnarFlushWriteStateForRelation(relid); - ColumnarFlushDeleteVectorForRelation(rel); + PgColumnarFlushWriteStateForRelation(relid); + PgColumnarFlushDeleteVectorForRelation(rel); w = build_proj_writer(rel, proj, stripeRowLimit, chunkGroupRowLimit, compType, compLevel); @@ -1867,58 +1867,58 @@ ColumnarBackfillProjection(Relation rel, const ColumnarProjection *proj) values = palloc(sizeof(Datum) * tableDesc->natts); nulls = palloc(sizeof(bool) * tableDesc->natts); - readState = ColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); - while (ColumnarReadNextRow(readState, values, nulls, &rowNumber)) + readState = PgColumnarBeginRead(rel, snapshot, NULL, NULL, 0, NULL); + while (PgColumnarReadNextRow(readState, values, nulls, &rowNumber)) append_proj_row(w, rel, tableDesc, rowNumber, values, nulls); - ColumnarEndRead(readState); + PgColumnarEndRead(readState); flush_proj_writer(w, rel); } /* Flush all projection writers hanging off a base write state. */ static void -flush_ws_projections(ColumnarWriteState *ws) +flush_ws_projections(PgColumnarWriteState *ws) { ListCell *lc; Relation rel; bool any = false; foreach(lc, ws->projWriters) - if (((ColumnarProjWriter *) lfirst(lc))->nrows > 0) + if (((PgColumnarProjWriter *) lfirst(lc))->nrows > 0) any = true; if (!any) return; rel = table_open(ws->relid, RowExclusiveLock); foreach(lc, ws->projWriters) - flush_proj_writer((ColumnarProjWriter *) lfirst(lc), rel); + flush_proj_writer((PgColumnarProjWriter *) lfirst(lc), rel); table_close(rel, RowExclusiveLock); } /* - * ColumnarFlushWriteStateForRelation + * PgColumnarFlushWriteStateForRelation * Flush any pending partial stripe for a single relation. Used at scan * start so data written earlier in this transaction is persisted. */ void -ColumnarFlushWriteStateForRelation(Oid relid) +PgColumnarFlushWriteStateForRelation(Oid relid) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid) continue; if (writeState->stripeRowCount > 0) - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); } } /* - * ColumnarForgetWriteStateForRelation + * PgColumnarForgetWriteStateForRelation * Drop the cached write state for a relation without flushing it. Used * after the relation's storage is swapped (columnar.vacuum): the cached * state holds the old storage id, so it must be discarded and a fresh one @@ -1926,102 +1926,102 @@ ColumnarFlushWriteStateForRelation(Oid relid) * buffered rows still needed persisting. */ void -ColumnarForgetWriteStateForRelation(Oid relid) +PgColumnarForgetWriteStateForRelation(Oid relid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarWriteStates == NIL) + if (PgColumnarWriteStates == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - foreach(lc, ColumnarWriteStates) + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->relid != relid) kept = lappend(kept, writeState); } MemoryContextSwitchTo(oldContext); - ColumnarWriteStates = kept; + PgColumnarWriteStates = kept; } /* - * ColumnarFlushAllPendingWrites + * PgColumnarFlushAllPendingWrites * Flush every pending write state. Called at transaction pre-commit. */ void -ColumnarFlushAllPendingWrites(void) +PgColumnarFlushAllPendingWrites(void) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); - columnar_flush_row_group(writeState); + pgcolumnar_flush_row_group(writeState); flush_ws_projections(writeState); } } /* - * ColumnarDiscardAllPendingWrites + * PgColumnarDiscardAllPendingWrites * Forget all pending write states. The backing memory is freed with the * transaction context, so we only clear our static references. */ void -ColumnarDiscardAllPendingWrites(void) +PgColumnarDiscardAllPendingWrites(void) { - ColumnarWriteStates = NIL; - ColumnarWriteContext = NULL; + PgColumnarWriteStates = NIL; + PgColumnarWriteContext = NULL; } /* - * ColumnarWriteStateDiscardSubXact + * PgColumnarWriteStateDiscardSubXact * Drop buffered (unflushed) writes made in an aborting subtransaction. * Stripes already flushed by that subtransaction are made invisible by * the subtransaction abort itself (their catalog rows), so only the * in-memory buffers need discarding here (spec 9). */ void -ColumnarWriteStateDiscardSubXact(SubTransactionId subid) +PgColumnarWriteStateDiscardSubXact(SubTransactionId subid) { List *kept = NIL; ListCell *lc; MemoryContext oldContext; - if (ColumnarWriteStates == NIL) + if (PgColumnarWriteStates == NIL) return; - oldContext = MemoryContextSwitchTo(ColumnarWriteContext); - foreach(lc, ColumnarWriteStates) + oldContext = MemoryContextSwitchTo(PgColumnarWriteContext); + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->subid != subid) kept = lappend(kept, writeState); } MemoryContextSwitchTo(oldContext); - ColumnarWriteStates = kept; + PgColumnarWriteStates = kept; } /* - * ColumnarWriteStatePromoteSubXact + * PgColumnarWriteStatePromoteSubXact * On subtransaction commit, reassign its buffered writes to the parent so * they are flushed when the parent (eventually the top transaction) * commits. */ void -ColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent) +PgColumnarWriteStatePromoteSubXact(SubTransactionId subid, SubTransactionId parent) { ListCell *lc; - foreach(lc, ColumnarWriteStates) + foreach(lc, PgColumnarWriteStates) { - ColumnarWriteState *writeState = (ColumnarWriteState *) lfirst(lc); + PgColumnarWriteState *writeState = (PgColumnarWriteState *) lfirst(lc); if (writeState->subid == subid) writeState->subid = parent; From 636827ac65fcdd5f89f5148cb85712c1c5574cad Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 19:48:50 +0000 Subject: [PATCH 02/10] Follow the rename in the suites that grep the C source Several suites assert a property by grepping src/ for a function name, so the rename moved the thing their patterns were looking for. Eight went red on 18 and 19: encode_invariants, native_fetch_position, native_gap, native_format, wal_envelope, decode_interrupts, native_fetch_interrupt and native_fetch_projection. The patterns are updated from a map derived from what actually changed in src/, rather than by rerunning the regex over test/. A regex cannot tell a renamed function from the two kinds of token that did not move: source file names such as columnar_reader.c, and the CustomScan node names ColumnarScan and ColumnarAgg. Both are asserted intact. test/pbt builds against our headers, so its stub and its test follow too. Co-Authored-By: Claude Opus 5 (1M context) --- test/analyze_stats.sh | 4 ++-- test/audit.sh | 2 +- test/build_san.sh | 2 +- test/decode_interrupts.sh | 4 ++-- test/encode_invariants.sh | 2 +- test/fsst_margin.sh | 2 +- test/fuzz_parquet.sh | 2 +- test/index_only.sh | 2 +- test/native_cancel.sh | 4 ++-- test/native_fastdecode.sh | 6 +++--- test/native_fetch_cache.sh | 4 ++-- test/native_fetch_interrupt.sh | 12 ++++++------ test/native_fetch_position.sh | 2 +- test/native_fetch_projection.sh | 10 +++++----- test/native_format.sh | 14 +++++++------- test/native_gap.sh | 4 ++-- test/native_groupagg.sh | 2 +- test/native_index.sh | 4 ++-- test/native_parquet_stack.sh | 6 +++--- test/native_reclaim_cycles.sh | 2 +- test/native_reclaim_frag.sh | 2 +- test/native_reclaim_reconcile.sh | 2 +- test/native_rewrite.sh | 4 ++-- test/native_vacuum_race.sh | 4 ++-- test/parallel_vector_agg.sh | 2 +- test/pbt/columnar.h | 16 ++++++++-------- test/pbt/test_encoding.c | 16 ++++++++-------- test/pushdown_report.sh | 4 ++-- test/replication.sh | 2 +- test/rewrite_group_scan.sh | 2 +- test/row_triggers.sh | 2 +- test/run_san.sh | 2 +- test/ungrouped_vector_agg.sh | 2 +- test/unique_conc.sh | 2 +- test/wal_envelope.sh | 6 +++--- 35 files changed, 79 insertions(+), 79 deletions(-) diff --git a/test/analyze_stats.sh b/test/analyze_stats.sh index 6c3f5bb..9f1d79a 100755 --- a/test/analyze_stats.sh +++ b/test/analyze_stats.sh @@ -334,7 +334,7 @@ check "ANALYZE on a wide table is not many times a full scan of it" \ # by number, and each fetch decodes the whole row group the row lives in. When the # ordering column is unclustered those rows are scattered across every group, so the # scan decodes the table many times over -- but core prices the fetch as a page or -# two and picks the index to avoid a sort. columnar_index_fetch_penalty adds the +# two and picks the index to avoid a sort. pgcolumnar_index_fetch_penalty adds the # decode cost, so a sort over the scan wins instead. Measured on the bench, an # unclustered ORDER BY that took minutes on the index dropped to seconds once it # sorted. @@ -443,7 +443,7 @@ check "the penalty is applied before the columnar path is offered, so it can sti # The checks above vary how many rows a plan fetches. This varies *which column* it # references, holding everything else fixed -- same row count, same emitted width, # same plan shape. The deferred index-fetch slot decodes the attribute prefix -# 0..max-referenced (columnar_tableam.c, columnar_slot_decode_upto), so referencing +# 0..max-referenced (columnar_tableam.c, pgcolumnar_slot_decode_upto), so referencing # a late column decodes every column before it. # # Sizing that decode from rel->reltarget->width cannot see the difference: it is diff --git a/test/audit.sh b/test/audit.sh index 55c72bf..04f8943 100755 --- a/test/audit.sh +++ b/test/audit.sh @@ -29,7 +29,7 @@ # group yet; if that fetch cannot answer, _bt_doinsert retries forever. # Cheap standalone cover for what unique_conc scenario 7 exercises. # 4. CREATE INDEX must not leak a relation reference. A parallel index build -# opens a TableScanDesc per participant through columnar_scan_begin (which +# opens a TableScanDesc per participant through pgcolumnar_scan_begin (which # takes a relation reference); the index_build_range_scan callback owns that # scan and must end it, or each participant leaks a reference that surfaces # at transaction commit as "resource was not closed: relation". The callback diff --git a/test/build_san.sh b/test/build_san.sh index b25a7e0..406cb2d 100644 --- a/test/build_san.sh +++ b/test/build_san.sh @@ -16,7 +16,7 @@ # 2. clang, not gcc. This looks like a free choice and is not. The defect this # gate exists to catch (#225) is a 4-byte varlena header read at an unaligned # address, and the read sits behind a 1-byte-header branch that tests the same -# byte (ColumnarVarSizeAnyUnaligned). gcc's -fsanitize=alignment silently +# byte (PgColumnarVarSizeAnyUnaligned). gcc's -fsanitize=alignment silently # drops the check on that guarded load at -O1 -- verified: the load happens # 45,000 times on a low-cardinality-text INSERT, gcc reports none, clang # reports it. A gcc build here would run clean and prove nothing, which is the diff --git a/test/decode_interrupts.sh b/test/decode_interrupts.sh index a043259..b67429f 100755 --- a/test/decode_interrupts.sh +++ b/test/decode_interrupts.sh @@ -109,8 +109,8 @@ check "the stride is not so large that a check never lands" \ # leave a whole group load or a whole run of skipped rows uninterruptible. RD="$SRC/columnar_reader.c" check "the group load checks for interrupts per column chunk" \ - "$(awk '/^columnar_native_load_group\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" + "$(awk '/^pgcolumnar_native_load_group\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" check "the row loop checks for interrupts" \ - "$(awk '/^columnar_native_next_row\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" + "$(awk '/^pgcolumnar_native_next_row\(/,/^}/' "$RD" | grep -c 'CHECK_FOR_INTERRUPTS')" "1" pgc_summary diff --git a/test/encode_invariants.sh b/test/encode_invariants.sh index a262883..d54d1ff 100755 --- a/test/encode_invariants.sh +++ b/test/encode_invariants.sh @@ -47,7 +47,7 @@ ROWS="${PGC_ENCINV_ROWS:-4096}" # directly and compares each against a reference implementation of the algorithm # it replaced, which is the promise the rewrites actually made: the same bytes. psql_run "CREATE FUNCTION pgcolumnar.debug_encoding_selftest() - RETURNS SETOF text AS 'pgcolumnar', 'columnar_debug_encoding_selftest' + RETURNS SETOF text AS 'pgcolumnar', 'pgcolumnar_debug_encoding_selftest' LANGUAGE C;" >/dev/null 2>&1 selftest="$(q "SELECT * FROM pgcolumnar.debug_encoding_selftest();")" diff --git a/test/fsst_margin.sh b/test/fsst_margin.sh index e7457ec..88793fb 100755 --- a/test/fsst_margin.sh +++ b/test/fsst_margin.sh @@ -2,7 +2,7 @@ # # The FSST keep/drop cost margin (#155, #271). # -# ColumnarFsstHelpsCompressed used to keep FSST on any compressed win at all, +# PgColumnarFsstHelpsCompressed used to keep FSST on any compressed win at all, # however small, and the per-vector FSST encode is a dominant cost of a text or # varlena load. pgcolumnar.fsst_min_gain_percent requires the win to clear a # margin before FSST is kept. diff --git a/test/fuzz_parquet.sh b/test/fuzz_parquet.sh index 4fdf412..277b46c 100755 --- a/test/fuzz_parquet.sh +++ b/test/fuzz_parquet.sh @@ -10,7 +10,7 @@ # a malformed file makes the backend raise an ERROR, never die. # # That is the property #210 broke. A crafted footer reached unbounded recursion -# in ColumnarThriftSkip, and a crafted schema chain reached a second one in +# in PgColumnarThriftSkip, and a crafted schema chain reached a second one in # walk_schema, either of which takes down the whole cluster rather than the # session. Both were found by reading code. This suite is the machine that was # missing. diff --git a/test/index_only.sh b/test/index_only.sh index a0fe6b2..a34d6fe 100644 --- a/test/index_only.sh +++ b/test/index_only.sh @@ -59,7 +59,7 @@ check "iv row count" "$(q "SELECT count(*) FROM iv;")" "50000" # Freshly written groups are never all-visible until a vacuum runs. check "before vacuum: block 50 not all-visible" "$(q "SELECT pgcolumnar.vm_is_visible('iv', 50);")" "f" -# Plain VACUUM (ShareUpdateExclusiveLock) drives columnar_relation_vacuum, which +# Plain VACUUM (ShareUpdateExclusiveLock) drives pgcolumnar_relation_vacuum, which # marks the all-visible groups. psql_run "VACUUM iv;" check "after vacuum: block 10 all-visible" "$(q "SELECT pgcolumnar.vm_is_visible('iv', 10);")" "t" diff --git a/test/native_cancel.sh b/test/native_cancel.sh index f8c867d..57d259b 100755 --- a/test/native_cancel.sh +++ b/test/native_cancel.sh @@ -4,7 +4,7 @@ # # The executor checks for interrupts once per tuple it receives, which is no help # while a scan is doing work without producing a tuple. The expensive case is -# loading a row group: columnar_native_load_group() reads and decodes every +# loading a row group: pgcolumnar_native_load_group() reads and decodes every # column chunk of the group before the row loop can iterate once, and a vector # holds up to pgcolumnar.chunk_group_row_limit values, which is user-settable and # unbounded. Without interrupt checks inside that load, statement_timeout, @@ -82,7 +82,7 @@ check "the short timeout is what fired" \ # Which guard this actually proves, established by removing each one rather than # from the description above: it is COLUMNAR_DECODE_INTERRUPT in # columnar_encoding.c, the per-value decode-loop check on a 65536 stride, not the -# per-column-chunk CHECK_FOR_INTERRUPTS in columnar_native_load_group(). Deleting +# per-column-chunk CHECK_FOR_INTERRUPTS in pgcolumnar_native_load_group(). Deleting # the per-chunk check leaves this suite green, because a two-column group reaches # it only twice; disabling the decode-loop macro makes cancel converge on full # (151 ms against 150) and fails this check, which is the behaviour the paragraph diff --git a/test/native_fastdecode.sh b/test/native_fastdecode.sh index 9eb9e2a..961689a 100755 --- a/test/native_fastdecode.sh +++ b/test/native_fastdecode.sh @@ -2,11 +2,11 @@ # # pgColumnar #289: fast decode of attbyval fixed-width columns. # -# The read path inlines the by-value decode in columnar_native_next_row: for an -# attbyval column it does the same fetch_att + advance ColumnarDecodeValue does, +# The read path inlines the by-value decode in pgcolumnar_native_next_row: for an +# attbyval column it does the same fetch_att + advance PgColumnarDecodeValue does, # but without the out-of-line call and its own attbyval branch (the per-row # decode dispatch #289 profiled as hot). By-reference (uuid) and varlena (text, -# numeric) columns keep the ColumnarDecodeValue path and serve as controls. This +# numeric) columns keep the PgColumnarDecodeValue path and serve as controls. This # test proves the inlined values are identical to the call path across every # byval fixed type, every NULL pattern, every encoding, per-vector skipping, # deletes and ADD COLUMN, plus adversarial bit patterns. diff --git a/test/native_fetch_cache.sh b/test/native_fetch_cache.sh index e6b3c8d..820f226 100755 --- a/test/native_fetch_cache.sh +++ b/test/native_fetch_cache.sh @@ -2,7 +2,7 @@ # # pgColumnar fetch-by-row-number cache (issue #143). # -# ColumnarReadRowByNumber() used to read and decode a whole row group per row +# PgColumnarReadRowByNumber() used to read and decode a whole row group per row # returned, so fetching N rows out of one group cost N times the group. A # statement-scoped cache of the decoded group removes the repeat. # @@ -94,7 +94,7 @@ check "a hit re-checks the group geometry it was filled with" \ "$(grep -cE 'entry->fileOffset != rg->fileOffset' "$SRC/columnar_reader.c")" "1" check "the cache is released at executor end, not only at transaction end" \ - "$(grep -c 'ColumnarDiscardFetchCache' "$SRC/columnar_tableam.c")" "2" + "$(grep -c 'PgColumnarDiscardFetchCache' "$SRC/columnar_tableam.c")" "2" # --- #353: a wide group's decode scratch must not blow the fetch cap ---------- # The by-row-number decode allocated its intermediates -- the decompressed region diff --git a/test/native_fetch_interrupt.sh b/test/native_fetch_interrupt.sh index a9b962f..67cdcad 100755 --- a/test/native_fetch_interrupt.sh +++ b/test/native_fetch_interrupt.sh @@ -2,7 +2,7 @@ # # pgColumnar fetch-path interrupt guard (#212). # -# columnar_fetch_row is reached once per candidate item pointer by +# pgcolumnar_fetch_row is reached once per candidate item pointer by # _bt_check_unique() during a unique INSERT, and each call reads the row-group # list out of the catalog. Before #212 that path had no CHECK_FOR_INTERRUPTS -- # the three checks already in columnar_reader.c are all on the scan/decode path, @@ -30,21 +30,21 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" -# The body of columnar_fetch_row, from its definition to its closing brace. -body="$(awk '/^columnar_fetch_row\(/{p=1} p{print} p&&/^}/{exit}' "$SRC/columnar_reader.c")" +# The body of pgcolumnar_fetch_row, from its definition to its closing brace. +body="$(awk '/^pgcolumnar_fetch_row\(/{p=1} p{print} p&&/^}/{exit}' "$SRC/columnar_reader.c")" # The guard has to be in this function, not merely in the file. The interrupt # checks elsewhere in columnar_reader.c are on the scan/decode path, which the # unique liveness fetch never enters, so counting the file would pass vacuously. -check "columnar_fetch_row carries an interrupt check" \ +check "pgcolumnar_fetch_row carries an interrupt check" \ "$(printf '%s\n' "$body" | grep -c 'CHECK_FOR_INTERRUPTS' | awk '{print ($1>=1)?"yes":"no"}')" \ "yes" # And it has to run before the per-fetch catalog read it guards, or the fetch # does its work before ever reaching a cancellation point -- which is the state -# #212 was in. Assert the check precedes the ColumnarReadRowGroupList call. +# #212 was in. Assert the check precedes the PgColumnarReadRowGroupList call. cfi_line="$(printf '%s\n' "$body" | grep -n 'CHECK_FOR_INTERRUPTS' | head -1 | cut -d: -f1)" -rgl_line="$(printf '%s\n' "$body" | grep -n 'ColumnarReadRowGroupList' | head -1 | cut -d: -f1)" +rgl_line="$(printf '%s\n' "$body" | grep -n 'PgColumnarReadRowGroupList' | head -1 | cut -d: -f1)" check "the interrupt check precedes the per-fetch catalog read" \ "$( [ -n "$cfi_line" ] && [ -n "$rgl_line" ] && [ "$cfi_line" -lt "$rgl_line" ] && echo yes || echo no )" \ "yes" diff --git a/test/native_fetch_position.sh b/test/native_fetch_position.sh index af988ae..1680ef5 100755 --- a/test/native_fetch_position.sh +++ b/test/native_fetch_position.sh @@ -228,7 +228,7 @@ check "doubling the row group does not double the cost of the same fetches" \ SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" check "the rank comes from a prefix rather than a loop over earlier rows" \ - "$(grep -c 'present = columnar_rank_before' "$SRC/columnar_reader.c")" "1" + "$(grep -c 'present = pgcolumnar_rank_before' "$SRC/columnar_reader.c")" "1" check "a varying-length column reaches its value through an offset table" \ "$(grep -c 'entry->valOffset\[c\]\[present\]' "$SRC/columnar_reader.c")" "1" diff --git a/test/native_fetch_projection.sh b/test/native_fetch_projection.sh index 8283133..0e3db3a 100755 --- a/test/native_fetch_projection.sh +++ b/test/native_fetch_projection.sh @@ -140,22 +140,22 @@ check "and its values are right with nothing decoded from the base" \ SRC="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/src" check "the visibility-only caller decodes nothing" \ - "$(grep -c 'ColumnarRowIsLive(rel, snap, baseRow)' "$SRC/columnar_projection.c")" "1" + "$(grep -c 'PgColumnarRowIsLive(rel, snap, baseRow)' "$SRC/columnar_projection.c")" "1" check "the reconstruct caller asks only for uncovered columns" \ - "$(grep -c 'ColumnarReadRowByNumberCols(rel, snap, baseRow' "$SRC/columnar_projection.c")" "1" + "$(grep -c 'PgColumnarReadRowByNumberCols(rel, snap, baseRow' "$SRC/columnar_projection.c")" "1" # Scoped to the function rather than counting a string across the file: the # string appears legitimately elsewhere now that the index fetch also asks only # for liveness, and a whole-file count turned that correct second use into a # failure. -deltuples="$(awk '/^columnar_index_delete_tuples\(/,/^}/' "$SRC/columnar_tableam.c")" +deltuples="$(awk '/^pgcolumnar_index_delete_tuples\(/,/^}/' "$SRC/columnar_tableam.c")" check "index deletion asks whether the row is live" \ - "$(case "$deltuples" in *ColumnarRowIsLive*) echo yes ;; *) echo no ;; esac)" "yes" + "$(case "$deltuples" in *PgColumnarRowIsLive*) echo yes ;; *) echo no ;; esac)" "yes" check "and does not decode the row to find out" \ - "$(case "$deltuples" in *ColumnarReadRowByNumber*) echo "no (still decodes)" ;; + "$(case "$deltuples" in *PgColumnarReadRowByNumber*) echo "no (still decodes)" ;; *) echo yes ;; esac)" "yes" # and the convention that made an empty set mean its opposite stays gone: the diff --git a/test/native_format.sh b/test/native_format.sh index 6146e60..8c2636d 100755 --- a/test/native_format.sh +++ b/test/native_format.sh @@ -5,7 +5,7 @@ # Two layers carry a version. The native data format stamps a major version into # pgcolumnar.storage.format_version (PGCN v1); the physical metapage stamps # versionMajor/versionMinor into block 0. Only the metapage version is checked on -# read -- ColumnarReadMetapage rejects a version it does not understand -- so that +# read -- PgColumnarReadMetapage rejects a version it does not understand -- so that # guard is the thing standing between a future, incompatible layout and a silent # misread of old bytes. This suite pins both stamps and proves the guard fires. # @@ -14,7 +14,7 @@ # accidental bump goes red here rather than shipping unnoticed; # 2. a metapage version this build does not understand is REJECTED with a clean # error and a surviving backend -- not misread as valid data. A test-only -# hook (columnar_debug_set_metapage_version, bound here rather than shipped, +# hook (pgcolumnar_debug_set_metapage_version, bound here rather than shipped, # like the gap suite's advance hook) plants the bad version; # 3. within a version, a diverse-typed table round-trips byte-for-byte against a # heap mirror, so "same version" genuinely means "same data back". @@ -29,7 +29,7 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" # Bind the internal test hook (deliberately not in the shipped catalog): it # overwrites the metapage version so we can confirm the read-side guard. psql_run "CREATE FUNCTION pgcolumnar.debug_set_metapage_version(regclass, int, int) - RETURNS void AS 'pgcolumnar', 'columnar_debug_set_metapage_version' + RETURNS void AS 'pgcolumnar', 'pgcolumnar_debug_set_metapage_version' LANGUAGE C;" # Error text (stderr) of a failing statement; empty when it succeeds. Used to @@ -71,7 +71,7 @@ check "table reads at the version this build wrote" "$(q 'SELECT count(*) FROM b # Pin the metapage major -- the version that is actually enforced on read, and # the one this suite exists to guard (format_version above is stamped but not -# checked). ColumnarReadMetapage rejects on versionMajor != COLUMNAR_VERSION_MAJOR, +# checked). PgColumnarReadMetapage rejects on versionMajor != COLUMNAR_VERSION_MAJOR, # so re-stamping the current major must be a no-op the read accepts. Bump # COLUMNAR_VERSION_MAJOR by accident and this goes red; on a deliberate bump, # change the 2 below in the same commit. @@ -95,7 +95,7 @@ check "backend survives the rejection" "$(q 'SELECT 1;')" "1" # --- 4: native format_version is now enforced on read too (#240 decision) ----- # The metapage version above guards the physical layout; format_version is the # independent data-format stamp. It used to be written and never read; it is now a -# read-side guard (ColumnarCheckNativeFormatVersion at scan open), so a future +# read-side guard (PgColumnarCheckNativeFormatVersion at scan open), so a future # PGCN version that keeps the metapage layout but changes the encoding is rejected # rather than misread. A catalog UPDATE stands in for that future version -- the # value is read from pgcolumnar.storage, so no on-disk bytes need forging. @@ -106,8 +106,8 @@ check "table reads at native format_version 1" "$(q 'SELECT count(*) FROM fv;')" fvsid="$(storage_id_of fv)" psql_run "UPDATE pgcolumnar.storage SET format_version = 99 WHERE storage_id = $fvsid;" # Three decode shapes must all reject. A seq scan opens a read state -# (ColumnarBeginReadWithStorage); the zone-map-only aggregate answers from metadata -# without one (ColumnarBeginAggScan); and an index-scan fetch of a non-key column +# (PgColumnarBeginReadWithStorage); the zone-map-only aggregate answers from metadata +# without one (PgColumnarBeginAggScan); and an index-scan fetch of a non-key column # decodes through the by-row-number fetch path, which reaches neither scan-open # guard -- it is the case the first cut missed. All are keyed to the same catalog # UPDATE standing in for a future format version. diff --git a/test/native_gap.sh b/test/native_gap.sh index 4c898d7..155a47e 100644 --- a/test/native_gap.sh +++ b/test/native_gap.sh @@ -7,7 +7,7 @@ # leaves (the file was shortened but the highwater was not). A write whose target # block is beyond EOF must fill the gap with empty pages and succeed, so the state # self-heals on the next write. This suite forces that state with a test-only hook -# (columnar_debug_advance_reserved_offset, bound here rather than shipped) and +# (pgcolumnar_debug_advance_reserved_offset, bound here rather than shipped) and # asserts writes across the gap succeed, data stays correct against a heap mirror, # the file physically materializes the gap, and the table is fully usable after. # @@ -20,7 +20,7 @@ pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" # bind the internal test hook (deliberately not in the shipped catalog). psql_run "CREATE FUNCTION pgcolumnar.debug_advance_reserved_offset(regclass, int) - RETURNS void AS 'pgcolumnar', 'columnar_debug_advance_reserved_offset' + RETURNS void AS 'pgcolumnar', 'pgcolumnar_debug_advance_reserved_offset' LANGUAGE C;" psql_run "CREATE TABLE h (id int, v text);" diff --git a/test/native_groupagg.sh b/test/native_groupagg.sh index fab035e..5327d4f 100644 --- a/test/native_groupagg.sh +++ b/test/native_groupagg.sh @@ -4,7 +4,7 @@ # # The grouped path fires for SELECT , agg(col) ... [WHERE ...] GROUP BY # over one columnar relation. It reads each surviving row with -# ColumnarReadNextRow (WHERE pushed down for group/vector skipping), rechecks the +# PgColumnarReadNextRow (WHERE pushed down for group/vector skipping), rechecks the # full WHERE, evaluates the group keys, and scatters the row into an # open-addressing hash table whose per-group accumulators fold in scan order. # diff --git a/test/native_index.sh b/test/native_index.sh index 58d761a..2c59088 100644 --- a/test/native_index.sh +++ b/test/native_index.sh @@ -2,7 +2,7 @@ # # pgColumnar native fetch-by-row-number (Phase D6a): index scan, bitmap scan and # unique/primary-key enforcement on a native-format (PGCN v1) table. All route -# through ColumnarReadRowByNumber, which before D6a had only a 2.2 path (index +# through PgColumnarReadRowByNumber, which before D6a had only a 2.2 path (index # scans returned 0 rows, unique was silently unenforced). This suite proves parity # with a heap mirror under a forced index path. Index-only scan (visibility map) # is D6c; native projection storage is D6d; delete/update is D6b. @@ -26,7 +26,7 @@ check "row count" "$(q 'SELECT count(*) FROM n;')" "8000" # Force the index path (seqscan off) in a single session (two -c, quiet so the SET # tag is not printed), and confirm scalar results match the heap oracle. Filtered # aggregates fall back from the zone-map path to an index scan that fetches each -# row via ColumnarReadRowByNumber. +# row via PgColumnarReadRowByNumber. iscan() { env PATH="$PGC_BINDIR:$PATH" psql -h 127.0.0.1 -p "$PGC_PORT" -U postgres \ -d "$PGC_DB" -q -At -c "SET enable_seqscan=off" -c "$1" 2>/dev/null diff --git a/test/native_parquet_stack.sh b/test/native_parquet_stack.sh index 3d62a43..1269b8d 100755 --- a/test/native_parquet_stack.sh +++ b/test/native_parquet_stack.sh @@ -11,7 +11,7 @@ # The two recursions are independent and each needs its own guard -- one does not # cover the other, because they are reached at different points: # -# 1. ColumnarThriftSkip recurses through nested structs (and lists of structs). +# 1. PgColumnarThriftSkip recurses through nested structs (and lists of structs). # Every unrecognised metadata field is skipped through it, so it is reached # straight from the footer bytes, before any schema is interpreted. A footer # that is just N struct openers nests N deep. @@ -24,7 +24,7 @@ # Both are guarded with check_stack_depth(), which turns the crash into a caught # ERROR (SQLSTATE 54001, statement_too_complex). That each guard is load-bearing # for its own vector was proven by removal at the depths that actually overflow an -# 8 MB stack: with only the ColumnarThriftSkip guard present a schema chain still +# 8 MB stack: with only the PgColumnarThriftSkip guard present a schema chain still # SIGSEGVs, and with only the walk_schema guard present a nested-struct footer # still SIGSEGVs. # @@ -71,7 +71,7 @@ def wrap(meta): # Vector 1: N nested struct openers. 0x1c is a compact-protocol field header # meaning (delta 1, type TC_STRUCT) -- "open a struct". N of them nest N deep -# through ColumnarThriftSkip. This never parses a schema; it crashes on the +# through PgColumnarThriftSkip. This never parses a schema; it crashes on the # top-level field skip. def nested(n): return wrap(b'\x1c' * n) diff --git a/test/native_reclaim_cycles.sh b/test/native_reclaim_cycles.sh index 9068970..153e58c 100644 --- a/test/native_reclaim_cycles.sh +++ b/test/native_reclaim_cycles.sh @@ -3,7 +3,7 @@ # pgColumnar physical reclaim, repeated compact_rewrite cycles (Phase F). # # Regression guard for the free-space allocator self-conflict fixed in PR #84: -# ColumnarAllocateFreeSpace consumed a free_space row without a +# PgColumnarAllocateFreeSpace consumed a free_space row without a # CommandCounterIncrement, so a compact_rewrite that wrote MORE THAN ONE group in # one command (many allocations) re-selected the just-consumed row and died with # "tuple already updated by self". It only fires once reusable free space exists diff --git a/test/native_reclaim_frag.sh b/test/native_reclaim_frag.sh index 0e94d51..76f1bc3 100644 --- a/test/native_reclaim_frag.sh +++ b/test/native_reclaim_frag.sh @@ -12,7 +12,7 @@ # GUC on they collapse to FEWER free_space rows than with it off (direct, # deterministic evidence the coalesce path runs); and # * coalesce is never worse than whole-range reuse for the final file size. -# The assert-build no-overlap validator (ColumnarCheckFreeSpaceNoOverlap) runs at +# The assert-build no-overlap validator (PgColumnarCheckFreeSpaceNoOverlap) runs at # the end of every compaction here, so this also exercises the tiling invariant # on variable-size ranges. # diff --git a/test/native_reclaim_reconcile.sh b/test/native_reclaim_reconcile.sh index 6393510..b94a5f4 100644 --- a/test/native_reclaim_reconcile.sh +++ b/test/native_reclaim_reconcile.sh @@ -4,7 +4,7 @@ # transactional, so normal retirement is atomic. The one seam is physical # end-truncation: a crash in its narrow window can leave a free_space row that # overlaps a live group (the highwater was lowered but the row's delete rolled -# back, and a later insert placed a live group there). ColumnarReconcileFreeList +# back, and a later insert placed a live group there). PgColumnarReconcileFreeList # runs at the start of every reuse op (compact_rewrite, recluster) and drops any # free_space row overlapping a live row-group footprint, so it cannot be handed # out on top of a live group. diff --git a/test/native_rewrite.sh b/test/native_rewrite.sh index e4d6260..9fec35b 100755 --- a/test/native_rewrite.sh +++ b/test/native_rewrite.sh @@ -95,13 +95,13 @@ check "compact_rewrite holds no AccessExclusiveLock" "${locks##*|}" "0" # --------------------------------------------------------------------------- # What makes the statement-scoped fetch cache safe (#143, #148). # -# ColumnarReadRowByNumber caches a decoded row group keyed by +# PgColumnarReadRowByNumber caches a decoded row group keyed by # (storage_id, group_number). That is only sound because such a pair is never # re-served with different bytes, which rests on two allocator properties rather # than on any locking: # # group numbers come from the monotonic metapage counter reserved in -# ColumnarReserveRowNumbers (meta->reservedStripeId += 1), never from the +# PgColumnarReserveRowNumbers (meta->reservedStripeId += 1), never from the # catalog maximum, so retiring a group does not free its number for reuse; # # a rewrite writes into freshly allocated storage rather than over the old. diff --git a/test/native_vacuum_race.sh b/test/native_vacuum_race.sh index 632722d..d6b752e 100644 --- a/test/native_vacuum_race.sh +++ b/test/native_vacuum_race.sh @@ -2,12 +2,12 @@ # # Regression for #295: compaction must not drop rows committed by another # transaction while it waited for (or before it took) its read snapshot. -# columnar_compact_relation used the caller's pre-lock statement snapshot, so a +# pgcolumnar_compact_relation used the caller's pre-lock statement snapshot, so a # row group committed after that snapshot was invisible to the rewrite and was # destroyed by the relfilenode swap. The fix takes a fresh snapshot after the # lock. Here session B pins a REPEATABLE READ snapshot, session A commits 100 # rows, then B runs the maintenance op: all 150 rows must survive. -# Covers vacuum() and vacuum_sorted() (columnar_compact_relation) and cluster() +# Covers vacuum() and vacuum_sorted() (pgcolumnar_compact_relation) and cluster() # (its Z-order twin). Written fresh for pgColumnar. # # Usage: test/native_vacuum_race.sh [PG_CONFIG] diff --git a/test/parallel_vector_agg.sh b/test/parallel_vector_agg.sh index 14cecd5..b70c977 100644 --- a/test/parallel_vector_agg.sh +++ b/test/parallel_vector_agg.sh @@ -218,7 +218,7 @@ check "grouped few-groups-many-workers: parallel == serial (#349)" "$GW_VEC" "$G # require the parallel answer to match the serial one. # # What this does NOT establish, measured rather than assumed: it does not prove -# the leader-side flush in ColumnarInitializeDSMGroupAggScan. Removing that flush +# the leader-side flush in PgColumnarInitializeDSMGroupAggScan. Removing that flush # -- and the ungrouped one this is modelled on -- leaves both H2 checks green, # with in-transaction INSERT and DELETE and a confirmed parallel plan, because # the write and delete buffers are already flushed at the command boundary before diff --git a/test/pbt/columnar.h b/test/pbt/columnar.h index acc3e60..2e4599d 100644 --- a/test/pbt/columnar.h +++ b/test/pbt/columnar.h @@ -24,30 +24,30 @@ #define PG_UINT32_MAX 0xFFFFFFFFU #endif -typedef struct ColumnarBlockReader +typedef struct PgColumnarBlockReader { const char *raw; uint64 valueCount; int width; uint64 pos; -} ColumnarBlockReader; +} PgColumnarBlockReader; -extern int ColumnarEncodeChunk(const char *raw, uint32 rawLen, +extern int PgColumnarEncodeChunk(const char *raw, uint32 rawLen, Form_pg_attribute att, uint64 valueCount, const char *fsstTable, uint32 fsstTableLen, char **out, uint32 *outLen); -extern char *ColumnarDecodeChunk(const char *enc, uint32 encLen, +extern char *PgColumnarDecodeChunk(const char *enc, uint32 encLen, int encodingType, Form_pg_attribute att, uint64 valueCount, uint32 rawLen, const char *fsstTable, uint32 fsstTableLen, MemoryContext cx); -extern bool ColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, +extern bool PgColumnarFsstBuildChunkTable(const char *corpus, uint32 corpusLen, Form_pg_attribute att, char **tableOut, uint32 *tableLenOut); -extern const char *ColumnarEncodingName(int encodingType); -extern void ColumnarBlockReaderInit(ColumnarBlockReader *br, const char *raw, +extern const char *PgColumnarEncodingName(int encodingType); +extern void PgColumnarBlockReaderInit(PgColumnarBlockReader *br, const char *raw, uint64 valueCount, int width); -extern bool ColumnarBlockNextRun(ColumnarBlockReader *br, +extern bool PgColumnarBlockNextRun(PgColumnarBlockReader *br, const char **valBytes, uint64 *runLen); #endif /* PGCOLUMNAR_PBT_COLUMNAR_H */ diff --git a/test/pbt/test_encoding.c b/test/pbt/test_encoding.c index 25a3765..42f6e14 100644 --- a/test/pbt/test_encoding.c +++ b/test/pbt/test_encoding.c @@ -3,7 +3,7 @@ * standalone against the test/pbt PostgreSQL shim. * * The governing property is round-trip: for any raw value stream, - * ColumnarDecodeChunk(ColumnarEncodeChunk(raw)) reproduces the exact bytes. It + * PgColumnarDecodeChunk(PgColumnarEncodeChunk(raw)) reproduces the exact bytes. It * is exercised over randomized data shaped to hit each encoding (constant, * alternating, monotonic, clustered, low-cardinality, runs, random) across all * fixed widths, floats (gorilla), and varlena (dict), plus explicit boundary @@ -64,8 +64,8 @@ check_fixed(int w, Oid typid, uint32 n, const char *raw) att.attbyval = true; att.atttypid = typid; - code = ColumnarEncodeChunk(raw, rawLen, &att, n, NULL, 0, &enc, &encLen); - dec = ColumnarDecodeChunk(enc, encLen, code, &att, n, rawLen, NULL, 0, NULL); + code = PgColumnarEncodeChunk(raw, rawLen, &att, n, NULL, 0, &enc, &encLen); + dec = PgColumnarDecodeChunk(enc, encLen, code, &att, n, rawLen, NULL, 0, NULL); checks++; if (rawLen > 0 && memcmp(dec, raw, rawLen) != 0) @@ -73,7 +73,7 @@ check_fixed(int w, Oid typid, uint32 n, const char *raw) failures++; fprintf(stderr, "FAIL fixed w=%d typid=%u n=%u code=%s(%d) encLen=%u\n", - w, typid, n, ColumnarEncodingName(code), code, encLen); + w, typid, n, PgColumnarEncodingName(code), code, encLen); } } @@ -272,20 +272,20 @@ gen_varlena(uint32 n, int shape) { char *tbl = NULL; uint32 tblLen = 0; - bool haveTbl = ColumnarFsstBuildChunkTable(s.data, (uint32) s.len, + bool haveTbl = PgColumnarFsstBuildChunkTable(s.data, (uint32) s.len, &att, &tbl, &tblLen); - code = ColumnarEncodeChunk(s.data, (uint32) s.len, &att, n, + code = PgColumnarEncodeChunk(s.data, (uint32) s.len, &att, n, haveTbl ? tbl : NULL, haveTbl ? tblLen : 0, &enc, &encLen); - dec = ColumnarDecodeChunk(enc, encLen, code, &att, n, (uint32) s.len, + dec = PgColumnarDecodeChunk(enc, encLen, code, &att, n, (uint32) s.len, haveTbl ? tbl : NULL, haveTbl ? tblLen : 0, NULL); checks++; if (s.len > 0 && memcmp(dec, s.data, s.len) != 0) { failures++; fprintf(stderr, "FAIL varlena n=%u code=%s rawLen=%d\n", - n, ColumnarEncodingName(code), s.len); + n, PgColumnarEncodingName(code), s.len); } if (haveTbl) free(tbl); diff --git a/test/pushdown_report.sh b/test/pushdown_report.sh index 2c2e854..026f11a 100755 --- a/test/pushdown_report.sh +++ b/test/pushdown_report.sh @@ -3,8 +3,8 @@ # pgColumnar: EXPLAIN must report the pushdown the scan performs, not the # pushdown the planner offered it (#191). # -# columnar_enable_qual_pushdown gates columnar_build_predicates in -# ColumnarBeginRead, so with the setting off the reader builds no predicates and +# pgcolumnar_enable_qual_pushdown gates pgcolumnar_build_predicates in +# PgColumnarBeginRead, so with the setting off the reader builds no predicates and # skips no chunk groups. cstate->nScanKeys is the planner's count and does not # move, so EXPLAIN reported the same "Columnar Pushed-Down Filters: 1" either # way -- telling someone who had just turned the setting off to test a theory diff --git a/test/replication.sh b/test/replication.sh index 0e9951a..df8aa45 100755 --- a/test/replication.sh +++ b/test/replication.sh @@ -446,7 +446,7 @@ psql_run "SELECT pgcolumnar.compact('r');" >/dev/null 2>&1 sync_or_fail "sync 6" check "compact replays identically" "$(hash_standby r)" "$(hash_primary r)" -# ColumnarTruncateMainFork is the one direct XLogInsert in the tree +# PgColumnarTruncateMainFork is the one direct XLogInsert in the tree # (RM_SMGR_ID / XLOG_SMGR_TRUNCATE). wal_envelope.sh asserts the envelope around # it by reading the source; nothing has ever executed the record, let alone # replayed it. diff --git a/test/rewrite_group_scan.sh b/test/rewrite_group_scan.sh index 96d849b..00c665f 100755 --- a/test/rewrite_group_scan.sh +++ b/test/rewrite_group_scan.sh @@ -2,7 +2,7 @@ # # pgColumnar: a rewrite must read each group once, not once per row (#196). # -# rewrite_one_group used ColumnarReadRowByNumber for every row. That call +# rewrite_one_group used PgColumnarReadRowByNumber for every row. That call # decodes the whole group to return one value and depends on the fetch cache to # make the next call cheap -- and the cache drops any group whose decoded form # exceeds COLUMNAR_FETCH_CACHE_MAX_BYTES, after every fetch. So a group over the diff --git a/test/row_triggers.sh b/test/row_triggers.sh index 3388c75..15f16e5 100755 --- a/test/row_triggers.sh +++ b/test/row_triggers.sh @@ -100,7 +100,7 @@ both "a multi-row insert fires once per row with the right values" 2 # disk before any trigger runs # 2. the first trigger's fetch therefore succeeds, and its body runs # 3. the body is itself a statement, and pgColumnar's ExecutorEnd hook calls -# ColumnarFlushAllPendingWrites when it ends -- flushing the outer INSERT's +# PgColumnarFlushAllPendingWrites when it ends -- flushing the outer INSERT's # remaining buffered rows # 4. every later fetch then finds its row on disk # diff --git a/test/run_san.sh b/test/run_san.sh index 50ea33c..0cfcaa6 100644 --- a/test/run_san.sh +++ b/test/run_san.sh @@ -16,7 +16,7 @@ # # Prove it works the way #224 asks: undo the fix it guards and confirm this gate # goes red while the ordinary matrix stays green. Undo it in its TRUE pre-#225 -# form -- delete the ColumnarVarSizeAnyUnaligned helper and restore VARSIZE_ANY at +# form -- delete the PgColumnarVarSizeAnyUnaligned helper and restore VARSIZE_ANY at # the three call sites (columnar_encoding.c, columnar_reader.c x2). That reports # the misalignment (about 20 of the 23 suites, at columnar_encoding.c:1135). # Do NOT instead rewrite the helper's body to the cast while keeping the inline diff --git a/test/ungrouped_vector_agg.sh b/test/ungrouped_vector_agg.sh index 0d26e79..96ce5c0 100644 --- a/test/ungrouped_vector_agg.sh +++ b/test/ungrouped_vector_agg.sh @@ -6,7 +6,7 @@ # zone map cannot answer -- one with a WHERE filter, or sum/avg over # int8/float/numeric -- to a single-pass scan-fold node instead of the row-wise # core Agg. This suite proves the new path returns byte-for-byte what core Agg -# returns (the fold reuses columnar_apply_one in scan order, so floats match +# returns (the fold reuses pgcolumnar_apply_one in scan order, so floats match # exactly), across types, nulls, filters, empty and all-null inputs; and it # ASSERTS THE PREMISE that the new path actually runs with the GUC on and does # not with it off, so the A/B is never vacuous. diff --git a/test/unique_conc.sh b/test/unique_conc.sh index 077510e..40e0489 100755 --- a/test/unique_conc.sh +++ b/test/unique_conc.sh @@ -7,7 +7,7 @@ # synthetic TID) is written immediately. So while transaction T1's inserting # statement is still in flight (row buffered, not flushed), a second transaction # T2 inserting the SAME key finds T1's index entry but cannot resolve the row: -# columnar_index_fetch_tuple returns false for a row still in T1's private write +# pgcolumnar_index_fetch_tuple returns false for a row still in T1's private write # buffer. The btree dirty-snapshot uniqueness check then treats the entry as dead # and T2 inserts a duplicate -- two live rows with the same unique key. # diff --git a/test/wal_envelope.sh b/test/wal_envelope.sh index 8bd82ca..ebc8bdd 100755 --- a/test/wal_envelope.sh +++ b/test/wal_envelope.sh @@ -59,12 +59,12 @@ check "every other WAL emitter is a core full-page-image helper" \ # ---- 2. the envelope around the one direct record --------------------------- FN="$SRC/columnar_storage.c" -# line range of ColumnarTruncateMainFork: its header to the next top-level function -start="$(grep -n '^ColumnarTruncateMainFork(' "$FN" | cut -d: -f1)" +# line range of PgColumnarTruncateMainFork: its header to the next top-level function +start="$(grep -n '^PgColumnarTruncateMainFork(' "$FN" | cut -d: -f1)" end="$(awk -v s="$start" 'NR > s && /^[A-Za-z_][A-Za-z0-9_]*\(/ {print NR; exit}' "$FN")" [ -z "$end" ] && end="$(wc -l < "$FN")" -check "ColumnarTruncateMainFork was found" "$([ -n "$start" ] && echo yes || echo no)" "yes" +check "PgColumnarTruncateMainFork was found" "$([ -n "$start" ] && echo yes || echo no)" "yes" # Line number of the first line inside the function that CONTAINS the literal # string, or empty. From 23d769a51f08b5ce45021d36a796d0ea10b6627f Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 13:52:54 -0600 Subject: [PATCH 03/10] Match the static assertion message to the renamed type The type became PgColumnarSlot but its StaticAssertDecl message still said ColumnarSlot, so a failure would have named a type that no longer exists. Co-Authored-By: Claude Opus 5 (1M context) --- src/columnar_tableam.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/columnar_tableam.c b/src/columnar_tableam.c index b4c09e5..259d30a 100644 --- a/src/columnar_tableam.c +++ b/src/columnar_tableam.c @@ -250,7 +250,7 @@ typedef struct PgColumnarSlot * aliasing. */ StaticAssertDecl(offsetof(PgColumnarSlot, deferred) >= sizeof(VirtualTupleTableSlot), - "ColumnarSlot fields must not overlap VirtualTupleTableSlot"); + "PgColumnarSlot fields must not overlap VirtualTupleTableSlot"); /* * pgcolumnar_slot_decode_upto From e3b1b6ebc2c39e346de1b425badfb20ca3a90e04 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:13:42 -0600 Subject: [PATCH 04/10] Add the 1.0-dev to 1.0-alpha upgrade script so existing installs survive Renaming the C link names orphans every pg_proc row that recorded the old ones. Replacing only the shared library, which is what a package upgrade does, leaves the extension inert rather than degraded: reading an existing columnar table fails with "could not find function columnar_handler", and so does creating a new one. jdatcmd measured this on #389. There was also no way out. default_version was still 1.0-dev, so ALTER EXTENSION pgcolumnar UPDATE had nothing to run, and the only remaining route was DROP EXTENSION CASCADE, which takes the user's tables with it. v1.0-alpha shipped carrying the old link names, so this is a live install base, not a hypothetical. My error in the first version of this PR was reading "no upgrade scripts exist" as "every install is fresh". It means the opposite. With no upgrade script an existing install has no way to repair itself. So: default_version 1.0-dev -> 1.0-alpha pgcolumnar--1.0-dev.sql renamed to pgcolumnar--1.0-alpha.sql new pgcolumnar--1.0-dev--1.0-alpha.sql The upgrade script is 27 CREATE OR REPLACE FUNCTION declarations covering 26 distinct link symbols, vacuum_sorted having two overloads. CREATE OR REPLACE keeps each function's OID, so the CREATE ACCESS METHOD binding and every dependency survive and only prosrc moves. No signature, permission, catalog or on-disk format change. The declarations are lifted verbatim from the install script rather than retyped, so the two cannot drift, and the generator refuses to emit anything if a link name still lacks the new prefix. This also makes SELECT extversion report 1.0-alpha, matching VERSION. That contradicts a sentence in my open #383, which I will update rather than leave wrong. Co-Authored-By: Claude Opus 5 (1M context) --- Makefile | 2 +- ...--1.0-dev.sql => pgcolumnar--1.0-alpha.sql | 0 pgcolumnar--1.0-dev--1.0-alpha.sql | 192 ++++++++++++++++++ pgcolumnar.control | 2 +- 4 files changed, 194 insertions(+), 2 deletions(-) rename pgcolumnar--1.0-dev.sql => pgcolumnar--1.0-alpha.sql (100%) create mode 100644 pgcolumnar--1.0-dev--1.0-alpha.sql diff --git a/Makefile b/Makefile index a1c9e0d..f206f2b 100644 --- a/Makefile +++ b/Makefile @@ -30,7 +30,7 @@ OBJS = \ src/columnar_parallel_export.o EXTENSION = pgcolumnar -DATA = pgcolumnar--1.0-dev.sql +DATA = pgcolumnar--1.0-alpha.sql pgcolumnar--1.0-dev--1.0-alpha.sql PGFILEDESC = "pgColumnar - column-oriented table access method" # make installcheck. Not the project's gate -- that is test/run_all_versions.sh, diff --git a/pgcolumnar--1.0-dev.sql b/pgcolumnar--1.0-alpha.sql similarity index 100% rename from pgcolumnar--1.0-dev.sql rename to pgcolumnar--1.0-alpha.sql diff --git a/pgcolumnar--1.0-dev--1.0-alpha.sql b/pgcolumnar--1.0-dev--1.0-alpha.sql new file mode 100644 index 0000000..4cedaa2 --- /dev/null +++ b/pgcolumnar--1.0-dev--1.0-alpha.sql @@ -0,0 +1,192 @@ +/* + * pgcolumnar 1.0-dev -> 1.0-alpha + * + * The C link names moved into the pgcolumnar namespace (#382), so a build from this + * version no longer exports the symbols an existing pg_proc recorded. Without this + * script, replacing the shared library leaves the extension inert: reading an existing + * columnar table fails with "could not find function columnar_handler", and so does + * creating a new one. + * + * CREATE OR REPLACE keeps each function's OID, so the CREATE ACCESS METHOD binding and + * every dependency survive. Only prosrc changes. No signature, name, or permission + * changes here, and no catalog or on-disk format change. + */ + +\echo Use "ALTER EXTENSION pgcolumnar UPDATE" to load this file. \quit + +CREATE OR REPLACE FUNCTION pgcolumnar.columnar_handler(internal) + RETURNS table_am_handler + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_handler'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.get_storage_id(rel regclass) + RETURNS bigint + LANGUAGE C STABLE STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_relation_storageid'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.add_projection( + rel regclass, + name text, + columns text[], + sort_key text[] DEFAULT '{}') + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_add_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.drop_projection(rel regclass, name text) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_drop_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.read_projection(rel regclass, name text) + RETURNS SETOF text + LANGUAGE C STABLE + AS 'MODULE_PATHNAME', 'pgcolumnar_read_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.reconstruct_via_projection(rel regclass, name text) + RETURNS SETOF text + LANGUAGE C STABLE + AS 'MODULE_PATHNAME', 'pgcolumnar_reconstruct_via_projection'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum(tablename regclass, stripe_count int DEFAULT 0) + RETURNS void + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum_sorted( + tablename regclass, + VARIADIC sort_columns name[]) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vacuum_sorted(tablename regclass) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vacuum_sorted'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.cluster( + tablename regclass, + VARIADIC columns name[]) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_cluster'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.compact(tablename regclass) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_compact'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.truncate(tablename regclass) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_truncate'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.compact_rewrite( + tablename regclass, + min_deleted_fraction float8 DEFAULT 0.2, + max_groups int DEFAULT 0) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_compact_rewrite'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.recluster( + tablename regclass, + VARIADIC columns name[]) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_recluster'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.export_arrow(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_export_arrow'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.export_parquet(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_export_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parallel_export_parquet(target regclass, path text, + workers int DEFAULT NULL) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_export_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.import_arrow(rel regclass, path text) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_import_arrow'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.import_parquet(rel regclass, path text) + RETURNS bigint + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_import_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_schema(path text) + RETURNS TABLE(column_name text, data_type text, nullable boolean) + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_schema'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.read_parquet(path text) + RETURNS SETOF record + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_read_parquet'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_fdw_handler() + RETURNS fdw_handler + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_fdw_handler'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parquet_fdw_validator(text[], oid) + RETURNS void + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parquet_fdw_validator'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vm_selftest(rel regclass, blk int) + RETURNS boolean + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_selftest'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.vm_is_visible(rel regclass, blk int) + RETURNS boolean + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_vm_is_visible'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.file_split_offsets(path text, workers int) + RETURNS bigint[] + LANGUAGE C STRICT + AS 'MODULE_PATHNAME', 'pgcolumnar_file_split_offsets'; + + +CREATE OR REPLACE FUNCTION pgcolumnar.parallel_copy(target regclass, filename text, + workers int DEFAULT NULL) + RETURNS bigint + LANGUAGE C + AS 'MODULE_PATHNAME', 'pgcolumnar_parallel_copy'; + diff --git a/pgcolumnar.control b/pgcolumnar.control index 4420323..66e34a6 100644 --- a/pgcolumnar.control +++ b/pgcolumnar.control @@ -4,7 +4,7 @@ # independent MIT implementation; the re-origination line builds from # design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md. comment = 'Columnar storage for PostgreSQL (pgColumnar)' -default_version = '1.0-dev' +default_version = '1.0-alpha' module_pathname = '$libdir/pgcolumnar' relocatable = false schema = pgcolumnar From 618790b26d61d0a0ea5a5482778bfde325b051f6 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:17:31 -0600 Subject: [PATCH 05/10] Gate the upgrade path, which no build or suite could see The bug jdatcmd caught on #389 compiles, links, and passes every suite on a fresh install, while leaving every existing install inert. CI cannot see it because CI only ever creates the extension from scratch. test/extension_upgrade.sh installs a released build, creates a columnar table with rows in it, installs the tree under test over the top, runs ALTER EXTENSION UPDATE, and then requires that reads, writes, table creation and a maintenance function all still work. Two checks are deliberately general rather than about this rename: - if default_version moved and no matching upgrade script was installed, that is the failure, because an existing install then has no route forward that keeps its tables - every C function owned by the extension must have a prosrc that resolves in the namespace, which catches the next rename as well as this one The old build is made in a throwaway clone, so the caller's working tree is never checked out from under it. Not in run_all_versions.sh, for the same reason pg_upgrade.sh is not: the matrix builds one tree per invocation and this needs two at once. It is a second gate beside the matrix. Co-Authored-By: Claude Opus 5 (1M context) --- test/extension_upgrade.sh | 145 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100755 test/extension_upgrade.sh diff --git a/test/extension_upgrade.sh b/test/extension_upgrade.sh new file mode 100755 index 0000000..94a5722 --- /dev/null +++ b/test/extension_upgrade.sh @@ -0,0 +1,145 @@ +#!/usr/bin/env bash +# +# pgColumnar extension-upgrade gate (#382). +# +# The failure this exists to catch is invisible to a build and to CI. Every SQL-callable +# function records its C link name in pg_proc.prosrc at CREATE EXTENSION time. Rename a +# link name in C and the old rows point at a symbol the new library no longer exports. +# Replacing only the shared library, which is exactly what a package upgrade does, then +# leaves the extension inert rather than degraded: reading an existing columnar table +# fails with "could not find function columnar_handler", and so does creating one. +# +# It compiles, it links, every suite passes on a fresh install, and every existing +# install is broken. That combination is why this needs its own gate. +# +# Not in run_all_versions.sh, for the same reason pg_upgrade.sh is not: the matrix builds +# one tree per invocation and this needs two builds of the extension at once. It is a +# second gate beside the matrix, run explicitly. +# +# Usage: +# test/extension_upgrade.sh [PG_CONFIG] [OLD_REF] +# +# OLD_REF defaults to the previous release, and may be any ref that still has the old +# link names. The old build is made in a throwaway clone, so the working tree is never +# checked out from under the caller. +set -uo pipefail + +PG_CONFIG=${1:-pg_config} +OLD_REF=${2:-v1.0-alpha} +SRCDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) + +command -v "$PG_CONFIG" >/dev/null 2>&1 || { echo "FATAL: $PG_CONFIG not found"; exit 1; } +BINDIR=$($PG_CONFIG --bindir) +SHAREDIR=$($PG_CONFIG --sharedir) +PGMAJ=$($PG_CONFIG --version | grep -oE '[0-9]+' | head -1) + +TMP=$(mktemp -d /tmp/pgc-extupg.XXXXXX) +DATA=$TMP/data +PORT=$(( 55000 + RANDOM % 8000 )) +LOG=$TMP/server.log +fail=0 + +cleanup () { + runuser -u postgres -- "$BINDIR/pg_ctl" -D "$DATA" -w stop >/dev/null 2>&1 + rm -rf "$TMP" +} +trap cleanup EXIT + +runpg () { runuser -u postgres -- "$@"; } +q () { runpg "$BINDIR/psql" -h /tmp -p "$PORT" -d extupg -X -Atc "$1" 2>&1; } + +echo "== extension_upgrade: PG$PGMAJ, old ref $OLD_REF" + +# ---- 1. build and install the old extension, from a throwaway clone ------------------ +if ! git -C "$SRCDIR" rev-parse --verify -q "$OLD_REF^{commit}" >/dev/null; then + echo " SKIP $OLD_REF is not present in this repository" + exit 0 +fi +git clone -q --shared "$SRCDIR" "$TMP/old" || { echo "FATAL: clone failed"; exit 1; } +git -C "$TMP/old" checkout -q --detach "$OLD_REF" || { echo "FATAL: checkout $OLD_REF failed"; exit 1; } +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_old.log" 2>&1 \ + || { echo "FAIL old build"; tail -20 "$TMP/build_old.log"; exit 1; } +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ + || { echo "FAIL old install"; exit 1; } + +chown -R postgres "$TMP" 2>/dev/null +runpg "$BINDIR/initdb" -D "$DATA" --locale=C -U postgres >/dev/null 2>&1 \ + || { echo "FAIL initdb"; exit 1; } +{ + echo "shared_preload_libraries='pgcolumnar'" + echo "port=$PORT" +} >> "$DATA/postgresql.conf" +runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w start >/dev/null 2>&1 \ + || { echo "FAIL start"; tail -10 "$LOG"; exit 1; } +runpg "$BINDIR/createdb" -h /tmp -p "$PORT" extupg >/dev/null 2>&1 + +q "CREATE EXTENSION pgcolumnar" >/dev/null +q "CREATE TABLE t (id int, v text) USING pgcolumnar" >/dev/null +q "INSERT INTO t SELECT g, 'v'||g FROM generate_series(1,1000) g" >/dev/null +before_rows=$(q "SELECT count(*) FROM t") +old_ver=$(q "SELECT extversion FROM pg_extension WHERE extname='pgcolumnar'") +echo " old install: $before_rows rows, extversion $old_ver" +[ "$before_rows" = "1000" ] || { echo "FAIL old install did not store rows"; exit 1; } + +# ---- 2. install the tree under test over it, library and scripts --------------------- +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_new.log" 2>&1 \ + || { echo "FAIL new build"; tail -20 "$TMP/build_new.log"; exit 1; } +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ + || { echo "FAIL new install"; exit 1; } +runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w restart >/dev/null 2>&1 + +# ---- 3. upgrade, and require that it be available at all ---------------------------- +# If the link names did not move, nothing is broken here and the upgrade is a no-op. If +# they did move, ALTER EXTENSION UPDATE is the only route back that keeps user tables, +# so its absence is itself the failure. +new_default=$(grep -oE "default_version = '[^']+'" "$SRCDIR/pgcolumnar.control" | sed "s/.*'\(.*\)'/\1/") +if [ "$new_default" != "$old_ver" ] && \ + [ ! -f "$SHAREDIR/extension/pgcolumnar--$old_ver--$new_default.sql" ]; then + echo " FAIL default_version moved $old_ver -> $new_default with no" + echo " pgcolumnar--$old_ver--$new_default.sql, so an existing install cannot upgrade" + fail=1 +fi + +upd=$(q "ALTER EXTENSION pgcolumnar UPDATE") +case "$upd" in + *ERROR*) echo " FAIL ALTER EXTENSION UPDATE: $upd"; fail=1 ;; + *) echo " ok ALTER EXTENSION UPDATE -> $(q "SELECT extversion FROM pg_extension WHERE extname='pgcolumnar'")" ;; +esac + +# ---- 4. the extension must work on the upgraded install ----------------------------- +chk () { + local label=$1 want=$2 got + got=$(q "$3") + if [ "$got" = "$want" ]; then + echo " PASS $label: $got" + else + echo " FAIL $label: got [$got] want [$want]" + fail=1 + fi +} +chk "existing rows still readable" "1000" "SELECT count(*) FROM t" +q "INSERT INTO t VALUES (0,'x')" >/dev/null +chk "insert into an existing table" "1001" "SELECT count(*) FROM t" +q "CREATE TABLE t2 (a int) USING pgcolumnar" >/dev/null +q "INSERT INTO t2 SELECT generate_series(1,3)" >/dev/null +chk "new columnar table creatable" "3" "SELECT count(*) FROM t2" +chk "access method still bound" "pgcolumnar" "SELECT amname FROM pg_am WHERE amname='pgcolumnar'" +chk "maintenance function callable" "" "SELECT pgcolumnar.vacuum('t')" + +# Every C function's recorded link name must resolve in the library we just installed. +# This is the general form of the bug, so it catches the next rename as well as this one. +missing=$(q "SELECT string_agg(p.proname||' -> '||p.prosrc, ', ') + FROM pg_proc p + JOIN pg_depend d ON d.objid = p.oid AND d.deptype = 'e' + JOIN pg_extension e ON e.oid = d.refobjid AND e.extname = 'pgcolumnar' + WHERE p.prolang = (SELECT oid FROM pg_language WHERE lanname='c') + AND p.prosrc !~ '^pgcolumnar'") +if [ -n "$missing" ]; then + echo " FAIL link names left outside the pgcolumnar namespace: $missing" + fail=1 +else + echo " PASS every C function's link name is namespaced" +fi + +echo "== extension_upgrade: $([ $fail -eq 0 ] && echo PASS || echo FAIL)" +exit $fail From e2be0ab8e578b7a28ca61783151583c59152efea Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:18:58 -0600 Subject: [PATCH 06/10] Make a missing old ref fail the upgrade gate rather than skip it Found by running the gate against the broken commit to check it actually fails there. It did not. It skipped and exited 0, because the ref lookup ran from the wrong directory. A skip that exits 0 is the same shape as the bug this gate exists to catch: everything green, nothing checked. The gate is invoked deliberately, so a missing ref is a setup error worth stopping on, not a reason to pass. Co-Authored-By: Claude Opus 5 (1M context) --- test/extension_upgrade.sh | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/test/extension_upgrade.sh b/test/extension_upgrade.sh index 94a5722..7554eff 100755 --- a/test/extension_upgrade.sh +++ b/test/extension_upgrade.sh @@ -51,9 +51,13 @@ q () { runpg "$BINDIR/psql" -h /tmp -p "$PORT" -d extupg -X -Atc "$1" 2>&1; } echo "== extension_upgrade: PG$PGMAJ, old ref $OLD_REF" # ---- 1. build and install the old extension, from a throwaway clone ------------------ -if ! git -C "$SRCDIR" rev-parse --verify -q "$OLD_REF^{commit}" >/dev/null; then - echo " SKIP $OLD_REF is not present in this repository" - exit 0 +# A missing ref must fail, not skip. This gate is invoked deliberately, and a skip that +# exits 0 would let it go inert the moment someone clones without tags. That is the same +# shape as the bug it exists to catch: everything green, nothing checked. +if ! git -C "$SRCDIR" rev-parse --verify -q "$OLD_REF^{commit}" >/dev/null 2>&1; then + echo " FAIL $OLD_REF is not present. Fetch tags, or pass an explicit ref:" + echo " git fetch --tags && test/extension_upgrade.sh $PG_CONFIG " + exit 1 fi git clone -q --shared "$SRCDIR" "$TMP/old" || { echo "FATAL: clone failed"; exit 1; } git -C "$TMP/old" checkout -q --detach "$OLD_REF" || { echo "FATAL: checkout $OLD_REF failed"; exit 1; } From 7e853e666312eaddf6d5d1c5c0cf32221d42fcfa Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:20:17 -0600 Subject: [PATCH 07/10] Derive the install-script name in server_file_privilege instead of hardcoding it The suite read pgcolumnar--1.0-dev.sql by name, so renaming the install script for the version bump broke it. It now reads default_version from the control file and asserts the file exists, which also survives the next bump. Co-Authored-By: Claude Opus 5 (1M context) --- test/server_file_privilege.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/test/server_file_privilege.sh b/test/server_file_privilege.sh index bcb0e8c..7f72d6a 100644 --- a/test/server_file_privilege.sh +++ b/test/server_file_privilege.sh @@ -37,7 +37,12 @@ set -uo pipefail export PGC_EXTRA_CONF=$'max_prepared_transactions=4\nmax_worker_processes=8' pgc_setup "${1:-/usr/local/pg17/bin/pg_config}" -SQLFILE="$(dirname "${BASH_SOURCE[0]}")/../pgcolumnar--1.0-dev.sql" +# Derived from the control file, not hardcoded. This broke once when the install script +# was renamed for a version bump, and a hardcoded name would break again at the next one. +_root="$(dirname "${BASH_SOURCE[0]}")/.." +_ver=$(grep -oE "default_version = '[^']+'" "$_root/pgcolumnar.control" | sed "s/.*'\(.*\)'/\1/") +SQLFILE="$_root/pgcolumnar--$_ver.sql" +[ -f "$SQLFILE" ] || { echo "FAIL install script $SQLFILE not found"; exit 1; } nope="/tmp/pgc_sfp_does_not_exist.parquet" out="$PGC_WORKDIR/sfp_out" outdir="$PGC_WORKDIR/sfp_dir" From c85d1e004c5369df1fa2cbac8b116a442fe6e9b1 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:27:41 -0600 Subject: [PATCH 08/10] Document the upgrade step, and follow the harness conventions Docs. The Upgrade section told users to run make install and restart, which is exactly the sequence that leaves them broken. It now has a third step, ALTER EXTENSION pgcolumnar UPDATE, with a query to list the databases that need it, the error they see if they miss it, and the fact that their data is untouched. It also says plainly not to reach for DROP EXTENSION, which would take their tables with it. CHANGELOG. The header claimed default_version is pinned at 1.0-dev because no upgrade script exists. That was true until this branch. Replaced, and an Unreleased section records the rename and the upgrade step. Harness. harness_selftest caught two real problems with the new gate: - it was unregistered. Registered as an exemption beside pg_upgrade and run_san, which are also second gates rather than matrix suites. - it drew a port from inside the kernel's ephemeral range, so the kernel could hand the same port to something else between the choice and the bind. It now draws from the band portlib.sh carves below that floor, and probes it. The second one is a real defect in my test, not a convention quibble, and it is the kind that fails once a month with nothing to show for it. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 38 ++++++++++++++++++++++++++++++---- docs/installation.md | 43 +++++++++++++++++++++++++++++++++++++++ test/extension_upgrade.sh | 8 +++++++- test/harness_selftest.sh | 2 +- 4 files changed, 85 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b3be1ec..11158bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,10 +6,40 @@ pre-release; the version marker is `1.0-alpha`, recorded in `VERSION`. New table are written in the native on-disk format, PGCN v1. For the forward-looking plan see [design/ROADMAP.md](design/ROADMAP.md); for full history see the git log. -The extension's own `default_version` is still `1.0-dev`. That is deliberate: it -governs `ALTER EXTENSION UPDATE`, and moving it needs an upgrade script that does -not exist yet. `SELECT extversion FROM pg_extension` therefore reports `1.0-dev` -on a 1.0-alpha build. +The extension's `default_version` is `1.0-alpha`, and an upgrade script ships with +it. Older notes in this file describe `default_version` as pinned at `1.0-dev`, +which was true until that script existed. + +## [Unreleased] + +### Changed + +- The extension's exported C symbols are namespaced under `pgcolumnar` (#382). + Two extensions that both call themselves `columnar` could define the same + symbol. `columnar_handler` and `columnar_relation_storageid` collided with + Citus columnar. Four settings variables such as `columnar_stripe_row_limit` + also shared names with the same settings there. That case binds one library's + setting to the other's storage. +- `default_version` moves from `1.0-dev` to `1.0-alpha`, so + `SELECT extversion FROM pg_extension` now agrees with `VERSION`. + +### Upgrading + +**Run `ALTER EXTENSION pgcolumnar UPDATE;` in every database that has the +extension, after installing this build.** + +The rename moves the C symbol names that each installed function recorded when it +was created. Replace the shared library without this step and those records +point at symbols the new library does not export. The extension then stops +working until the catalog is updated. Reading an existing columnar table fails +with `could not find function "columnar_handler"`. + +Nothing happens to your data, and no conversion runs. The upgrade replaces +catalog entries only, and keeps each function's identity, so the access method +binding and every dependency survive. The SQL you write does not change. + +See [Upgrade](docs/installation.md#upgrade) for the commands, including how to +list the databases that need it. ## [1.0-alpha] - 2026-08-04 diff --git a/docs/installation.md b/docs/installation.md index dbd0277..7986117 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -110,6 +110,49 @@ To install a new build of the extension: 1. Run `make install` with the same `PG_CONFIG`. 2. Start the server again, so that it loads the new library. +3. Run `ALTER EXTENSION pgcolumnar UPDATE;` **in every database that has the + extension**. + +Step 3 is not optional, and it is easy to miss because nothing prompts for it. +The first two steps replace the shared library. The third updates the catalog to +match it. + +```sql +-- in each database that ran CREATE EXTENSION pgcolumnar +ALTER EXTENSION pgcolumnar UPDATE; +SELECT extversion FROM pg_extension WHERE extname = 'pgcolumnar'; +``` + +To find the databases that need it: + +```sql +SELECT datname FROM pg_database WHERE datallowconn + AND EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'pgcolumnar'); +``` + +### If you skipped step 3 + +Every function the extension installs records the name of a C symbol. When those +names change between builds, the recorded names no longer resolve, and the +extension stops working until the catalog is updated. Reading an existing +columnar table then fails: + +``` +ERROR: could not find function "columnar_handler" in file ".../pgcolumnar.so" +``` + +The fix is step 3. Run `ALTER EXTENSION pgcolumnar UPDATE;` in that database and +the error goes away. **Your data is not affected.** The tables are intact and no +conversion happens. Only the catalog entry is stale. + +Do not run `DROP EXTENSION`. It removes your columnar tables with it. + +### Upgrading from 1.0-dev to 1.0-alpha + +This release renames the extension's C symbols into the `pgcolumnar` namespace, +so that two extensions named `columnar` can be loaded without colliding. That is +the change step 3 applies. The SQL you write does not change. Function names, +settings and table syntax are all the same. The source records the on-disk format version. The specification also records it, in [../design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md](https://github.com/commandprompt/pgcolumnar/blob/main/design/NATIVE_FORMAT_AND_INTERFACE_SPEC.md). diff --git a/test/extension_upgrade.sh b/test/extension_upgrade.sh index 7554eff..8e8fbc6 100755 --- a/test/extension_upgrade.sh +++ b/test/extension_upgrade.sh @@ -28,6 +28,12 @@ PG_CONFIG=${1:-pg_config} OLD_REF=${2:-v1.0-alpha} SRCDIR=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd) +# The cluster binds a port, so it must come from the band portlib.sh carves BELOW the +# kernel's ephemeral range, probed rather than assumed free. Picking out of the +# ephemeral range means the kernel can hand the same port to something else between +# the choice and the bind. harness_selftest enforces this, and caught it here. +. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh" + command -v "$PG_CONFIG" >/dev/null 2>&1 || { echo "FATAL: $PG_CONFIG not found"; exit 1; } BINDIR=$($PG_CONFIG --bindir) SHAREDIR=$($PG_CONFIG --sharedir) @@ -35,7 +41,7 @@ PGMAJ=$($PG_CONFIG --version | grep -oE '[0-9]+' | head -1) TMP=$(mktemp -d /tmp/pgc-extupg.XXXXXX) DATA=$TMP/data -PORT=$(( 55000 + RANDOM % 8000 )) +PORT=$(pgc_pick_free_port "$PGC_AUX_PORT_LO" "$PGC_AUX_PORT_HI" "$$") LOG=$TMP/server.log fail=0 diff --git a/test/harness_selftest.sh b/test/harness_selftest.sh index 6853a0e..ca73439 100755 --- a/test/harness_selftest.sh +++ b/test/harness_selftest.sh @@ -180,7 +180,7 @@ RUNNER="$TESTDIR/run_all_versions.sh" # not a suite the ordinary five-major matrix can carry. not_a_suite() { case "$1" in - lib|portlib|run_all_versions|build_all_versions|devloop|rebuild|native_scale|build_san|run_san|run_coverage|pg_upgrade) return 0 ;; + lib|portlib|run_all_versions|build_all_versions|devloop|rebuild|native_scale|build_san|run_san|run_coverage|pg_upgrade|extension_upgrade) return 0 ;; *) return 1 ;; esac } From 637724970009a77ab66147954a117ecc1caab527 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 14:35:02 -0600 Subject: [PATCH 09/10] Build clean in the upgrade gate, and check the restart Two defects in the gate, both found by running it after a preflight that had built the tree against another major. make was handed objects compiled for PG19 and relinked them into a .so for a PG18 server, which then refused to start. Both builds now clean first. The restart was unchecked, so a postmaster that never came back produced six connection errors instead of one statement of what went wrong. It now fails immediately and prints the server log. Co-Authored-By: Claude Opus 5 (1M context) --- test/extension_upgrade.sh | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/test/extension_upgrade.sh b/test/extension_upgrade.sh index 8e8fbc6..efb2c58 100755 --- a/test/extension_upgrade.sh +++ b/test/extension_upgrade.sh @@ -67,6 +67,7 @@ if ! git -C "$SRCDIR" rev-parse --verify -q "$OLD_REF^{commit}" >/dev/null 2>&1; fi git clone -q --shared "$SRCDIR" "$TMP/old" || { echo "FATAL: clone failed"; exit 1; } git -C "$TMP/old" checkout -q --detach "$OLD_REF" || { echo "FATAL: checkout $OLD_REF failed"; exit 1; } +make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" clean >/dev/null 2>&1 make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_old.log" 2>&1 \ || { echo "FAIL old build"; tail -20 "$TMP/build_old.log"; exit 1; } make -C "$TMP/old" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ @@ -92,11 +93,20 @@ echo " old install: $before_rows rows, extversion $old_ver" [ "$before_rows" = "1000" ] || { echo "FAIL old install did not store rows"; exit 1; } # ---- 2. install the tree under test over it, library and scripts --------------------- +# Clean first. This tree may have last been built against another major, and make +# would happily relink those objects into a .so this server cannot load. That is how +# this gate first failed: a pg19 build silently relinked for pg18, the postmaster +# refused to start, and every check below reported a connection error instead. +make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" clean >/dev/null 2>&1 make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" -j"$(nproc)" >"$TMP/build_new.log" 2>&1 \ || { echo "FAIL new build"; tail -20 "$TMP/build_new.log"; exit 1; } make -C "$SRCDIR" PG_CONFIG="$PG_CONFIG" install >/dev/null 2>&1 \ || { echo "FAIL new install"; exit 1; } -runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w restart >/dev/null 2>&1 +if ! runpg "$BINDIR/pg_ctl" -D "$DATA" -l "$LOG" -w restart >/dev/null 2>&1; then + echo " FAIL server did not come back after installing the new build" + tail -15 "$LOG" + exit 1 +fi # ---- 3. upgrade, and require that it be available at all ---------------------------- # If the link names did not move, nothing is broken here and the upgrade is a no-op. If From 6d817e6bba285cbd4a9bb11197407828dababe67 Mon Sep 17 00:00:00 2001 From: "Joshua (D) Drake" Date: Tue, 4 Aug 2026 15:39:58 -0600 Subject: [PATCH 10/10] Carry the upgrade step into limitations.md too (#392 review) Reviewing jdatcmd's #392 turned up a second copy of the instruction I rewrote in docs/installation.md, 85 lines below the paragraph that PR fixes. limitations.md said the pre-release ships no ALTER EXTENSION UPDATE scripts, which this branch makes false, and then told the reader they can replace the shared library and restart without one. That is precisely the sequence that leaves the extension inert. It then pointed at DROP EXTENSION, which takes the user's columnar tables with it. A user who hits the broken state and follows the status badge lands on that page, so it was the worst remaining copy. The paragraph now states that replacing the library is not sufficient on its own, gives the command, explains why an un-updated catalog fails and with which error, and says plainly that the data is untouched. The DROP EXTENSION paragraph is scoped to a build no upgrade script covers, and says it is not the remedy for this case. I missed this in #383 and again when I rewrote installation.md. Sweeping for a second copy is the step I skipped both times. Co-Authored-By: Claude Opus 5 (1M context) --- docs/limitations.md | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-) diff --git a/docs/limitations.md b/docs/limitations.md index 2983ab4..9f31fb7 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -89,16 +89,28 @@ A physical copy does not replace the source across a version change. The same posture covers the extension's own catalog, not only the on-disk data format. The install script of a build defines the `pgcolumnar` catalog tables for a fresh -`CREATE EXTENSION`. The pre-release ships no `ALTER EXTENSION UPDATE` scripts, so -there is no in-place catalog migration either. +`CREATE EXTENSION`. An `ALTER EXTENSION UPDATE` script ships when a build needs one. +The 1.0-dev to 1.0-alpha script is the first. -You can replace the shared library and the SQL script, and then restart, without -a new `CREATE EXTENSION`. This can leave a catalog table without a column that a -newer build needs. For example, `sort_by` was added to `pgcolumnar.options`. A function that +**Replacing the shared library is not sufficient on its own.** After installing a new +build, run `ALTER EXTENSION pgcolumnar UPDATE;` in every database that has the extension. +The library and the catalog have to agree, and only that command updates the catalog. + +Skipping it can leave the catalog describing the previous build. Each installed function +records the name of a C symbol. 1.0-alpha moved those names, so an un-updated catalog +names symbols the new library does not export. Reading an existing columnar table then +fails with `could not find function "columnar_handler"`. The data is untouched, and the +command above fixes it. See [Upgrade](installation.md#upgrade). + +A catalog can also lack a column that a newer build needs, where no upgrade script covers +the gap. For example, `sort_by` was added to `pgcolumnar.options`. A function that uses that column fails against an `options` table that an older build created. -Across an incompatible build, recreate the extension with `DROP EXTENSION` and -`CREATE EXTENSION`, and load the data again. Do not replace the files in place. A +Across an incompatible build, meaning one where no upgrade script covers the change, +recreate the extension with `DROP EXTENSION` and `CREATE EXTENSION`, and load the data +again. This is not the remedy for the un-updated catalog described above, where +`ALTER EXTENSION pgcolumnar UPDATE` is enough. `DROP EXTENSION` removes your columnar +tables with it. Do not replace the files in place. A dump that exists still restores into a newer build. `pg_dump` writes an explicit column list for the configuration tables of the extension, and a new column takes its default of NULL.