Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,9 @@ endif ()
option(BUILD_TOOLS "Build the tsfile command-line tools" ON)
message("cmake using: BUILD_TOOLS=${BUILD_TOOLS}")

option(BUILD_BENCHMARK "Build the read-backend benchmark" OFF)
message("cmake using: BUILD_BENCHMARK=${BUILD_BENCHMARK}")

option(ENABLE_ANTLR4 "Enable ANTLR4 runtime" ON)
message("cmake using: ENABLE_ANTLR4=${ENABLE_ANTLR4}")

Expand Down Expand Up @@ -378,6 +381,15 @@ if (NOT "${_TSFILE_PROJECT_DEPENDENCIES}" STREQUAL "")
endif ()

add_subdirectory(src)
if (BUILD_BENCHMARK)
find_package(Threads REQUIRED)
add_executable(read_backend_benchmark
bench_mark/bench_mark_src/read_backend_benchmark.cc)
target_include_directories(read_backend_benchmark PRIVATE
${PROJECT_SRC_DIR})
target_link_libraries(read_backend_benchmark PRIVATE
tsfile Threads::Threads)
endif ()
if (BUILD_TOOLS)
add_subdirectory(tools)
endif ()
Expand Down
18 changes: 18 additions & 0 deletions cpp/README-zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,24 @@ storage::set_write_thread_count(4);

默认情况下,当机器 CPU 核数大于 1 时自动启用并行写入,线程数设为硬件核数(上限 64)。

### 本地文件读取后端

Reader 可以为本地文件选择内存映射 I/O 或传统的定位读取路径。配置会在
reader 打开文件时确定,因此修改配置不会影响已经打开的 reader。

```cpp
#include "common/global.h"

common::set_file_read_backend(common::FileReadBackend::AUTO); // 默认值
common::set_file_read_backend(common::FileReadBackend::MMAP); // 必须使用 mmap
common::set_file_read_backend(common::FileReadBackend::PREAD); // 兼容旧读取路径
```

C API 可通过 `tsfile_set_file_read_backend(TSFILE_READ_BACKEND_*)` 设置相同
选项。`AUTO` 会优先映射受支持的普通文件,映射不可用时自动回退到 `pread`;
`MMAP` 不回退:输入不受支持时返回 `RET_NOT_SUPPORT`,映射失败时返回
`RET_FILE_MAP_ERR`。映射 reader 打开期间不得修改或截断文件。

---

## 使用 TsFile
Expand Down
21 changes: 21 additions & 0 deletions cpp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,27 @@ storage::set_write_thread_count(4);

By default, parallel write is enabled when the machine has more than one CPU core, and the thread count is set to the number of hardware cores (capped at 64).

### Local File Read Backend

Readers can use memory-mapped I/O or the traditional positioned-read path for
local files. The setting is captured when a reader opens a file, so changing it
does not affect readers that are already open.

```cpp
#include "common/global.h"

common::set_file_read_backend(common::FileReadBackend::AUTO); // default
common::set_file_read_backend(common::FileReadBackend::MMAP); // require mmap
common::set_file_read_backend(common::FileReadBackend::PREAD); // legacy path
```

The C API exposes the same setting through
`tsfile_set_file_read_backend(TSFILE_READ_BACKEND_*)`. `AUTO` prefers memory
mapping for supported regular files and falls back to `pread` if mapping is not
available. `MMAP` does not fall back: unsupported inputs return
`RET_NOT_SUPPORT`, while mapping failures return `RET_FILE_MAP_ERR`. Files must
not be modified or truncated while a mapped reader is open.

## Use TsFile

You can find examples on how to read and write data in `demo_read.cpp` and `demo_write.cpp` located under `./examples/cpp_examples`. There are also examples under `./examples/c_examples` on how to use a C-style API to read and write data in a C environment. The examples will be built automatically when you run the main build command.
Expand Down
56 changes: 56 additions & 0 deletions cpp/bench_mark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
<!--

Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.

-->

# C++ Benchmarks

`read_backend_benchmark` compares `PREAD` and `MMAP` without enforcing a
performance threshold. It reports sequential 64 KiB reads, deterministic
random 4 KiB reads, repeated parsing of real TsFile metadata, random bounded
`queryByRow` queries, and one concurrent random-read worker per input file. The
query-planning scan is performed before the timed random-query interval.
Files with no queryable rows still participate in the byte-read and metadata
workloads, but are skipped for the random-query workload.

Build it through the main CMake project so it links the exact SDK under test:

```bash
cmake -S cpp -B cpp/target/read-backend-benchmark \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_BENCHMARK=ON -DBUILD_TEST=OFF -DBUILD_TOOLS=OFF \
-DTSFILE_BUILD_SHARED=OFF
cmake --build cpp/target/read-backend-benchmark \
--target read_backend_benchmark --config Release
```

With a single-config generator such as Ninja, the executable is
`cpp/target/read-backend-benchmark/read_backend_benchmark` (plus `.exe` on
Windows). Run it with one or, preferably, several representative TsFiles:

```bash
./read_backend_benchmark data-1.tsfile data-2.tsfile data-3.tsfile
./read_backend_benchmark --mmap-first data-1.tsfile data-2.tsfile data-3.tsfile
```

The second form reverses backend order to expose warm page-cache bias. For
meaningful results, repeat both forms and record filesystem, storage device,
file sizes, compiler flags, and whether the OS page cache was warm. The checksum
makes accidental short reads, metadata-load failures, or optimizer removal
visible; it is not a TsFile-content checksum.
Loading
Loading