CAS: 04070_no_crash_extreme_compress_block_size crashes the server

[alsugiliazova]

CaContentWriteBuffer aborts on extreme adaptive_write_buffer_initial_size (CAS-default stateless lane)

Branch: cas-gc-rebuild
Commit: 08ea8d1200e49f8928a0d71356782eb3bcfb3d4e
Binary: locally built on 20.07.26
Harness: Praktika functional lane
Stateless tests (arm_binary, content_addressed storage, parallel)
Severity: HIGH — release-build logical error aborts the whole ClickHouse server mid-suite (not a soft query failure). Violates the regression contract of 04070_no_crash_extreme_compress_block_size ("extreme write-buffer size settings must not crash the server").
Component (suspected): Cas::CaContentWriteBuffer in ContentAddressedTransaction.cpp — missing the same adaptive-initial-size clamp that WriteBufferFromFileDescriptor already applies.

---

Summary

While running the full CAS-as-default MergeTree stateless suite, the server aborted with:

Logical error: 'Too large size (9223372036854775870) passed to allocator. It indicates an error.'

on the query:

INSERT INTO t_extreme_compress SELECT number, toString(number) FROM numbers(1000);

from test 04070_no_crash_extreme_compress_block_size.

The stack is entirely on the CAS write path:

MergeTreeDataPartWriterWide::addStreams
→ MergeTreeWriterStream
→ DataPartStorageOnDiskFull::writeFile
→ ContentAddressedTransaction::writeFile
→ Cas::CaContentWriteBuffer::CaContentWriteBuffer
→ WriteBufferFromFileBase / Allocator::alloc

The allocated size 9223372036854775870 is 2^63 - 1 + 63 (INT64_MAX plus typical buffer alignment padding). That matches Test 7 of 04070, which sets:

SETTINGS adaptive_write_buffer_initial_size = 9223372036854775807,
         min_columns_to_activate_adaptive_write_buffer = 1,
         min_bytes_for_wide_part = 0;

Stdout from the aborted run contains six 1000 lines (Tests 1–6 completed); the reference expects nine. The abort happens on the first INSERT that enables an extreme adaptive initial buffer size under CAS.

This is a CAS-specific regression relative to plain local disks: the non-CAS write buffer already clamps adaptive_buffer_initial_size to buf_size before allocating. CaContentWriteBuffer does not.

The abort killed the parallel suite (~8590 / 10985 completed).

---

Evidence from logs

Fatal message (server err log / test stderr capture)

Source artifacts:

• ci/tmp/var/log/clickhouse-server/clickhouse-server.err.log (tail around 2026.07.21 13:36:51)
• tests/queries/0_stateless/04070_no_crash_extreme_compress_block_size.4395.stderr
• tests/queries/0_stateless/04070_no_crash_extreme_compress_block_size.4395.stdout (six 1000 lines)
• ci/tmp/test_result.txt (suite stopped after hung check)

2026.07.21 13:36:51.506202 [ 3185 ] {47b6e114-f6f1-4582-bc3d-d75d19028039} <Fatal> :
  Logical error: 'Too large size (9223372036854775870) passed to allocator. It indicates an error.'.

2026.07.21 13:36:51.507231 [ 2086 ] <Fatal> BaseDaemon: ########################################
2026.07.21 13:36:51.507309 [ 2086 ] <Fatal> BaseDaemon:
  (version 26.6.1.20000.altinityantalya,
   build id: A06BB0B1F15BF39DCE41EE5EEA95CB8E60EF28CD,
   git hash: 08ea8d1200e49f8928a0d71356782eb3bcfb3d4e)
  (from thread 3185)
  (query_id: 47b6e114-f6f1-4582-bc3d-d75d19028039)
  (query: INSERT INTO t_extreme_compress SELECT number, toString(number) FROM numbers(1000);)
  Received signal Aborted (6)

Stack (shortened to CAS-relevant frames)

5.  Allocator<false, false>::alloc
6.  DB::Memory<Allocator<false, false>>::alloc
7.  DB::WriteBufferFromFileBase::WriteBufferFromFileBase
8.  DB::Cas::CaContentWriteBuffer::CaContentWriteBuffer   ← CAS outer buffer ctor
9.  DB::ContentAddressedTransaction::writeFile
10. DB::ContentAddressedTransaction::tryCreateWriteBuffer
11. DB::DiskObjectStorageTransaction::writeFileImpl
12. DB::DataPartStorageOnDiskFull::writeFile
13. DB::MergeTreeWriterStream::MergeTreeWriterStream
…
17. DB::MergeTreeDataPartWriterWide::addStreams
…
23. DB::MergeTreeSink::consume

BaseDaemon also recorded log_comment = '04070_no_crash_extreme_compress_block_size.sql-test_fla8u902'.

Suite-level fallout

[ 8585 / 10985] 04327_reader_executor_metrics: [ FAIL ]   # unrelated metrics mismatch earlier
…
[ 8591 / 10985] 02677_analyzer_compound_expressions: [ OK ]
Aborted (core dumped)
Hung check failed: server is not responding
Unable to locate any ClickHouse server process. It must have crashed or exited prematurely!
Fail [Stateless tests …]  Failed: 5, Passed: 8522, Skipped: 64
  Fail [Server died]

Other listed FAILs in the same run (02479_mysql_connect_to_self, 02784_connection_string, 02420_stracktrace_debug_symbols, 01945_show_debug_warning, 04327_reader_executor_metrics) are orthogonal to this abort and should not be conflated with the CAS write-buffer bug.

---

Root-cause analysis

What the regression test is for

tests/queries/0_stateless/04070_no_crash_extreme_compress_block_size.sql (tag no-random-merge-tree-settings) is an explicit no-crash regression for STID 0883:

extreme write-buffer size settings must not crash the server.
Compress block sizes are clamped to 256 MiB; the adaptive write buffer initial size is
clamped to its own maximum (the compress block size) so an out-of-range value never
reaches the allocator.

It exercises nine cases. Cases 1–6 / 3–5 set extreme max_compress_block_size / marks / PK / min compress sizes. Cases 7–9 set extreme adaptive_write_buffer_initial_size (including 2^63).

Why cases 1–6 survive under CAS

MergeTreeWriterSettings clamps compress-block settings to MAX_COMPRESS_BLOCK_SIZE (256 MiB) in MergeTreeIOSettings.cpp:

max_compress_block_size(std::min<size_t>(..., MAX_COMPRESS_BLOCK_SIZE))
min_compress_block_size(std::min<size_t>(..., MAX_COMPRESS_BLOCK_SIZE))
marks_compress_block_size(std::min<size_t>(..., MAX_COMPRESS_BLOCK_SIZE))
primary_key_compress_block_size(std::min<size_t>(..., MAX_COMPRESS_BLOCK_SIZE))

Those clamped values become the buf_size passed into writeFile / MergeTreeWriterStream. Under CAS, CaContentWriteBuffer can therefore still allocate a sane buf_size for Tests 1–6 → six successful count()=1000 lines.

Why case 7 dies under CAS

adaptive_write_buffer_initial_size is not clamped in MergeTreeIOSettings.cpp — it is copied through verbatim:

, adaptive_write_buffer_initial_size((*storage_settings)[MergeTreeSetting::adaptive_write_buffer_initial_size])

On the non-CAS path, safety lives one layer down in WriteBufferFromFileDescriptor:

/// The adaptive buffer grows from the initial size up to buf_size (the max), so the
/// initial allocation must not exceed it. An out-of-range initial size would otherwise
/// be passed straight to the allocator (e.g. a fuzzed adaptive_write_buffer_initial_size).
: WriteBufferFromFileBase(
      use_adaptive_buffer_size_ ? std::min(adaptive_buffer_initial_size, buf_size) : buf_size,
      ...)

On the CAS path, CaContentWriteBuffer (both Local-scratch and S3-staging constructors) does:

: WriteBufferFromFileBase(
      use_adaptive_buffer_size ? adaptive_buffer_initial_size : buf_size,
      nullptr, 0)

i.e. it uses the raw adaptive initial size when adaptive mode is on — no std::min(..., buf_size).

Test 7 enables adaptive writes (min_columns_to_activate_adaptive_write_buffer = 1) and sets adaptive_write_buffer_initial_size = 9223372036854775807 (2^63 - 1). CaContentWriteBuffer then asks the allocator for that size (plus alignment padding → 9223372036854775870), which trips:

Too large size (...) passed to allocator. It indicates an error.

→ abortOnFailedAssertion → SIGABRT → server death → suite abort.

Size arithmetic check

Quantity	Value
Table setting adaptive_write_buffer_initial_size	9223372036854775807 (2^63 - 1)
Allocator rejection size in log	9223372036854775870
Difference	+63 (consistent with BufferWithOwnMemory / SIMD padding)

Why this is specifically a CAS lane bug

• Same SQL on a local default disk is supposed to be safe (that is what STID 0883 / 04070 already proved for non-CAS).
• CAS replaces the disk writeFile buffer with CaContentWriteBuffer, which re-implements the adaptive-size choice without copying the WriteBufferFromFileDescriptor clamp.
• The CA-default stateless lane (--content-addressed-storage) makes every MergeTree table hit this path, so 04070 becomes a server-killing test instead of a soft regression.

---

Reproduction steps

Prerequisites

1. Build clickhouse on cas-gc-rebuild (or any CAS-enabled tree containing CaContentWriteBuffer).
2. Symlink the binary for Praktika:

cd /root/workspace-alsu/ClickHouse
mkdir -p ci/tmp
ln -sfn "$(pwd)/build/programs/clickhouse" ci/tmp/clickhouse

Minimal targeted repro (preferred)

cd /root/workspace-alsu/ClickHouse
python -m ci.praktika run \
  "Stateless tests (arm_binary, content_addressed storage, parallel)" \
  --test 04070_no_crash_extreme_compress_block_size

Expected (bug present): server logical-error abort during Test 7 INSERT; test fails / suite reports server death.
Expected (bug fixed): all nine 1000 lines; test [ OK ]; server stays up.

Optional single-statement sketch (after CA disk is mounted)

Equivalent to Test 7 alone:

DROP TABLE IF EXISTS t_extreme_compress;
CREATE TABLE t_extreme_compress (x UInt64, s String)
    ENGINE = MergeTree() ORDER BY x
    SETTINGS adaptive_write_buffer_initial_size = 9223372036854775807,
             min_columns_to_activate_adaptive_write_buffer = 1,
             min_bytes_for_wide_part = 0;
INSERT INTO t_extreme_compress SELECT number, toString(number) FROM numbers(1000);
-- Bug: Logical error / server abort on CAS disk
-- Fixed: INSERT succeeds; SELECT count() → 1000

Full-lane context (how it was originally found)

python -m ci.praktika run \
  "Stateless tests (arm_binary, content_addressed storage, parallel)"

Progress reached ~8590/10985 before abort. Do not re-run the full lane until disk is cleaned and the targeted repro is understood.

---

Suspected fix direction (not yet implemented)

Mirror the non-CAS clamp inside both CaContentWriteBuffer constructors:

WriteBufferFromFileBase(
    use_adaptive_buffer_size ? std::min(adaptive_buffer_initial_size, buf_size) : buf_size,
    nullptr, 0)

Possibly also clamp adaptive_write_buffer_initial_size once in MergeTreeIOSettings (defense in depth), but the direct hole vs STID 0883 is the missing std::min in CaContentWriteBuffer.

Also verify:

• Local-scratch constructor and S3-staging constructor (both currently pass the unclamped adaptive size into WriteBufferFromFileBase).
• Tests 8–9 of 04070 (adaptive + PK compression; adaptive size exactly 2^63) after the clamp.
• Whether WriteBufferFromFile created as the Local spill sink needs the same treatment when constructed with raw buf_size / adaptive flags (secondary; the fatal stack shows the outer CaContentWriteBuffer → WriteBufferFromFileBase allocation first).

Regression gate: keep 04070_no_crash_extreme_compress_block_size green on both:

• content_addressed storage (local object storage)
• content_addressed s3 storage (RustFS)

---