From 0d48b07657542e882d92a069029d5fed8167005b Mon Sep 17 00:00:00 2001 From: Dongpo Liu Date: Tue, 14 Jul 2026 12:21:14 +0200 Subject: [PATCH 1/7] This is an automated cherry-pick of #21938 Signed-off-by: ti-chi-bot --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 30 +++- system-variables.md | 146 +++++++++++++++++- ...v8.1-performance-benchmarking-with-tpcc.md | 123 +++++++++++++++ ...v8.5-performance-benchmarking-with-tpcc.md | 123 +++++++++++++++ troubleshoot-tidb-oom.md | 3 +- 6 files changed, 417 insertions(+), 10 deletions(-) create mode 100644 tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md create mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index 7c60c79023022..a98271087b2da 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. + You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). * Reverse index scan diff --git a/statistics.md b/statistics.md index 7c52ed64ded9b..e74eb98966808 100644 --- a/statistics.md +++ b/statistics.md @@ -412,14 +412,37 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` +<<<<<<< HEAD Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. +======= +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. -#### `tidb_distsql_scan_concurrency` +#### `tidb_build_sampling_stats_concurrency` +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency for merging samples collected from different Regions. +- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +The default value is `2`. + +#### `tidb_analyze_partition_concurrency` + +This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) + +#### `tidb_analyze_distsql_scan_concurrency` + +<<<<<<< HEAD When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. +======= +This variable controls the following aspects of `ANALYZE` concurrency: +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) -#### `tidb_index_serial_scan_concurrency` +- The concurrency of scanning TiKV Regions. +- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). +<<<<<<< HEAD When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. ### Persist ANALYZE configurations @@ -864,6 +887,9 @@ mysql> show warnings; +-------+------+-----------------------------------------------------------------+ 1 row in set (0.00 sec) ``` +======= +The default value is `4`. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index 417777b17a3ee..4dba0beba6eda 100644 --- a/system-variables.md +++ b/system-variables.md @@ -735,6 +735,42 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. +<<<<<<< HEAD +======= +### tidb_analyze_column_options New in v8.3.0 + +> **Note:** +> +> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. +> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. +> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. +> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Enumeration +- Default value: `ALL` +- Value options:`ALL`, `PREDICATE` +- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. + +### tidb_analyze_distsql_scan_concurrency New in v7.6.0 + +> **Note:** +> +> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. + +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Default value: `4` +- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency of scanning TiKV Regions. + - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency > **Warning:** @@ -743,8 +779,15 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes +<<<<<<< HEAD - Default value: `1` - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. +======= +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. +- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. +- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -759,6 +802,76 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - If your cluster is upgraded from an earlier version, the default value of `tidb_analyze_version` does not change after the upgrade. - For detailed introduction about this variable, see [Introduction to Statistics](/statistics.md). +<<<<<<< HEAD +======= +### tidb_analyze_skip_column_types New in v7.2.0 + +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Default value: "json,blob,mediumblob,longblob,mediumtext,longtext". Before v8.2.0, the default value is "json,blob,mediumblob,longblob". +- Possible values: "json,blob,mediumblob,longblob,text,mediumtext,longtext" +- This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`. + +``` +mysql> SHOW CREATE TABLE t; ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Table | Create Table | ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| t | CREATE TABLE `t` ( + `a` int DEFAULT NULL, + `b` varchar(10) DEFAULT NULL, + `c` json DEFAULT NULL, + `d` blob DEFAULT NULL, + `e` longblob DEFAULT NULL +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | ++-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +1 row in set (0.00 sec) + +mysql> SELECT @@tidb_analyze_skip_column_types; ++----------------------------------+ +| @@tidb_analyze_skip_column_types | ++----------------------------------+ +| json,blob,mediumblob,longblob,mediumtext,longtext | ++----------------------------------+ +1 row in set (0.00 sec) + +mysql> ANALYZE TABLE t; +Query OK, 0 rows affected, 1 warning (0.05 sec) + +mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; ++---------------------------------------------------------------------+ +| job_info | ++---------------------------------------------------------------------+ +| analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate | ++---------------------------------------------------------------------+ +1 row in set (0.00 sec) + +mysql> ANALYZE TABLE t COLUMNS a, c; +Query OK, 0 rows affected, 1 warning (0.04 sec) + +mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; ++------------------------------------------------------------------+ +| job_info | ++------------------------------------------------------------------+ +| analyze table columns a with 256 buckets, 500 topn, 1 samplerate | ++------------------------------------------------------------------+ +1 row in set (0.00 sec) +``` + +### tidb_auto_analyze_concurrency New in v8.4.0 + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Default value: `3` +- Range: `[1, 2147483647]` +- This variable controls the number of concurrent auto-analyze operations that can run in a TiDB cluster. To accelerate statistics collection tasks, you can increase this concurrency based on the available resources in your cluster. +- Before v8.4.0, this concurrency is fixed at `1`. +- Starting from v8.5.7 and v9.0.0, the default value changes from `1` to `3`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_auto_analyze_end_time - Scope: GLOBAL @@ -805,7 +918,12 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Type: Integer - Default value: `1` - Range: `[1, 256]` +<<<<<<< HEAD - This variable is used to set the concurrency of executing the automatic update of statistics. +======= +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously +- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_backoff_lock_fast @@ -899,9 +1017,24 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. + +<<<<<<< HEAD +======= +### tidb_build_sampling_stats_concurrency New in v7.5.0 +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Unit: Threads +- Default value: `2` +- Range: `[1, 256]` +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency for merging samples collected from different Regions. + - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_capture_plan_baselines New in v4.0 - Scope: GLOBAL @@ -2435,14 +2568,17 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. ### tidb_init_chunk_size @@ -2749,7 +2885,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- This variable controls the concurrency for merging TopN results of partitioned tables. ### tidb_metric_query_range_duration New in v4.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md new file mode 100644 index 0000000000000..79e74aec086dd --- /dev/null +++ b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md @@ -0,0 +1,123 @@ +--- +title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 +summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. +--- + +# TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 + +This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.1.0 clusters. + +## Test overview + +This test aims at showing the TPC-C performance of TiDB v8.1.0 in the Online Transactional Processing (OLTP) scenario. + +## Test environment + +### TiDB cluster + +The test is conducted on a TiDB cluster with the following settings: + +- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) +- Cluster version: v8.1.0 +- Cloud provider: AWS (us-west-2) +- Cluster configuration: + + | Node type | Node size | Node quantity | Node storage | + |:----------|:----------|:----------|:----------| + | TiDB | 16 vCPU, 32 GiB | 2 | N/A | + | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | + +### Parameter configuration + +> **Note:** +> +> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. + +The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: + +```yaml +raft-engine.prefill-for-recycle = true +``` + +### Benchmark executor + +The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: + +- Machine type: Amazon EC2 (us-west-2) +- Instance type: c6a.2xlarge + +## Test steps + +This section introduces how to perform the TPC-C performance test step by step. + +1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. + + For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). + +2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. + + To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). + + To create the `tpcc` database, execute the following SQL statement: + + ```sql + CREATE DATABASE tpcc; + ``` + +3. Load TPC-C data to the `tpcc` database. + + 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: + + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh + ``` + + 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. + + ```shell + go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: + + ```sql + ANALYZE TABLE customer; + ANALYZE TABLE district; + ANALYZE TABLE history; + ANALYZE TABLE item; + ANALYZE TABLE new_order; + ANALYZE TABLE order_line; + ANALYZE TABLE orders; + ANALYZE TABLE stock; + ANALYZE TABLE warehouse; + ``` + + To accelerate the collection of statistics, execute the following SQL statements before collecting: + + ```sql + SET tidb_build_stats_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; + ``` + +5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. + + ```shell + go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +6. Extract the tpmC data of `NEW_ORDER` from the result. + + TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. + +## Test results + +The TPC-C performance of v8.1.0 in the [test environment](#test-environment) is as follows: + +| Threads | v8.1.0 tpmC | +|:--------|:----------| +| 50 | 43,660 | +| 100 | 75,495 | +| 200 | 102,013 | + +![TPC-C](/media/tidb-cloud/v8.1.0_tpcc.png) diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md new file mode 100644 index 0000000000000..3bd5bbd2a576a --- /dev/null +++ b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md @@ -0,0 +1,123 @@ +--- +title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 +summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. +--- + +# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 + +This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. + +## Test overview + +This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. + +## Test environment + +### TiDB cluster + +The test is conducted on a TiDB cluster with the following settings: + +- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) +- Cluster version: v8.5.0 +- Cloud provider: AWS (us-west-2) +- Cluster configuration: + + | Node type | Node size | Node quantity | Node storage | + |:----------|:----------|:----------|:----------| + | TiDB | 16 vCPU, 32 GiB | 2 | N/A | + | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | + +### Parameter configuration + +> **Note:** +> +> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. + +The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: + +```yaml +raft-engine.prefill-for-recycle = true +``` + +### Benchmark executor + +The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: + +- Machine type: Amazon EC2 (us-west-2) +- Instance type: c6a.2xlarge + +## Test steps + +This section introduces how to perform the TPC-C performance test step by step. + +1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. + + For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). + +2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. + + To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). + + To create the `tpcc` database, execute the following SQL statement: + + ```sql + CREATE DATABASE tpcc; + ``` + +3. Load TPC-C data to the `tpcc` database. + + 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: + + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh + ``` + + 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. + + ```shell + go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: + + ```sql + ANALYZE TABLE customer; + ANALYZE TABLE district; + ANALYZE TABLE history; + ANALYZE TABLE item; + ANALYZE TABLE new_order; + ANALYZE TABLE order_line; + ANALYZE TABLE orders; + ANALYZE TABLE stock; + ANALYZE TABLE warehouse; + ``` + + To accelerate the collection of statistics, execute the following SQL statements before collecting: + + ```sql + SET tidb_build_stats_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; + ``` + +5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. + + ```shell + go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +6. Extract the tpmC data of `NEW_ORDER` from the result. + + TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. + +## Test results + +The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: + +| Threads | v8.5.0 tpmC | +|:--------|:----------| +| 50 | 43,146 | +| 100 | 73,875 | +| 200 | 103,395 | + +![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From f66083250197a47a0a524972e23a5a24050520ca Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:24:34 +0800 Subject: [PATCH 2/7] docs: skip inapplicable release-6.5 cherry-pick --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 30 +--- system-variables.md | 146 +----------------- ...v8.1-performance-benchmarking-with-tpcc.md | 123 --------------- ...v8.5-performance-benchmarking-with-tpcc.md | 123 --------------- troubleshoot-tidb-oom.md | 3 +- 6 files changed, 10 insertions(+), 417 deletions(-) delete mode 100644 tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md delete mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index a98271087b2da..7c60c79023022 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). + You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. * Reverse index scan diff --git a/statistics.md b/statistics.md index e74eb98966808..7c52ed64ded9b 100644 --- a/statistics.md +++ b/statistics.md @@ -412,37 +412,14 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` -<<<<<<< HEAD Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. -======= -This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. -#### `tidb_build_sampling_stats_concurrency` +#### `tidb_distsql_scan_concurrency` -This variable controls the following aspects of `ANALYZE` concurrency: - -- The concurrency for merging samples collected from different Regions. -- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. - -The default value is `2`. - -#### `tidb_analyze_partition_concurrency` - -This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) - -#### `tidb_analyze_distsql_scan_concurrency` - -<<<<<<< HEAD When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. -======= -This variable controls the following aspects of `ANALYZE` concurrency: ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) -- The concurrency of scanning TiKV Regions. -- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). +#### `tidb_index_serial_scan_concurrency` -<<<<<<< HEAD When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. ### Persist ANALYZE configurations @@ -887,9 +864,6 @@ mysql> show warnings; +-------+------+-----------------------------------------------------------------+ 1 row in set (0.00 sec) ``` -======= -The default value is `4`. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index 4dba0beba6eda..417777b17a3ee 100644 --- a/system-variables.md +++ b/system-variables.md @@ -735,42 +735,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. -<<<<<<< HEAD -======= -### tidb_analyze_column_options New in v8.3.0 - -> **Note:** -> -> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. -> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. -> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. -> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. - -- Scope: GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Enumeration -- Default value: `ALL` -- Value options:`ALL`, `PREDICATE` -- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. - -### tidb_analyze_distsql_scan_concurrency New in v7.6.0 - -> **Note:** -> -> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Default value: `4` -- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. -- This variable controls the following aspects of `ANALYZE` concurrency: - - The concurrency of scanning TiKV Regions. - - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency > **Warning:** @@ -779,15 +743,8 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes -<<<<<<< HEAD - Default value: `1` - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. -======= -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. -- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. -- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -802,76 +759,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - If your cluster is upgraded from an earlier version, the default value of `tidb_analyze_version` does not change after the upgrade. - For detailed introduction about this variable, see [Introduction to Statistics](/statistics.md). -<<<<<<< HEAD -======= -### tidb_analyze_skip_column_types New in v7.2.0 - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Default value: "json,blob,mediumblob,longblob,mediumtext,longtext". Before v8.2.0, the default value is "json,blob,mediumblob,longblob". -- Possible values: "json,blob,mediumblob,longblob,text,mediumtext,longtext" -- This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`. - -``` -mysql> SHOW CREATE TABLE t; -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Table | Create Table | -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| t | CREATE TABLE `t` ( - `a` int DEFAULT NULL, - `b` varchar(10) DEFAULT NULL, - `c` json DEFAULT NULL, - `d` blob DEFAULT NULL, - `e` longblob DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | -+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -1 row in set (0.00 sec) - -mysql> SELECT @@tidb_analyze_skip_column_types; -+----------------------------------+ -| @@tidb_analyze_skip_column_types | -+----------------------------------+ -| json,blob,mediumblob,longblob,mediumtext,longtext | -+----------------------------------+ -1 row in set (0.00 sec) - -mysql> ANALYZE TABLE t; -Query OK, 0 rows affected, 1 warning (0.05 sec) - -mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; -+---------------------------------------------------------------------+ -| job_info | -+---------------------------------------------------------------------+ -| analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate | -+---------------------------------------------------------------------+ -1 row in set (0.00 sec) - -mysql> ANALYZE TABLE t COLUMNS a, c; -Query OK, 0 rows affected, 1 warning (0.04 sec) - -mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; -+------------------------------------------------------------------+ -| job_info | -+------------------------------------------------------------------+ -| analyze table columns a with 256 buckets, 500 topn, 1 samplerate | -+------------------------------------------------------------------+ -1 row in set (0.00 sec) -``` - -### tidb_auto_analyze_concurrency New in v8.4.0 - -- Scope: GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Default value: `3` -- Range: `[1, 2147483647]` -- This variable controls the number of concurrent auto-analyze operations that can run in a TiDB cluster. To accelerate statistics collection tasks, you can increase this concurrency based on the available resources in your cluster. -- Before v8.4.0, this concurrency is fixed at `1`. -- Starting from v8.5.7 and v9.0.0, the default value changes from `1` to `3`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_auto_analyze_end_time - Scope: GLOBAL @@ -918,12 +805,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Type: Integer - Default value: `1` - Range: `[1, 256]` -<<<<<<< HEAD - This variable is used to set the concurrency of executing the automatic update of statistics. -======= -- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously -- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_backoff_lock_fast @@ -1017,24 +899,9 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. - -<<<<<<< HEAD -======= -### tidb_build_sampling_stats_concurrency New in v7.5.0 - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Unit: Threads -- Default value: `2` -- Range: `[1, 256]` -- This variable controls the following aspects of `ANALYZE` concurrency: - - The concurrency for merging samples collected from different Regions. - - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. +- This variable is used to set the concurrency of executing the `ANALYZE` statement. +- When the variable is set to a larger value, the execution performance of other queries is affected. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_capture_plan_baselines New in v4.0 - Scope: GLOBAL @@ -2568,17 +2435,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency -> **Warning:** -> -> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. +- This variable is used to set the concurrency of the `serial scan` operation. +- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. ### tidb_init_chunk_size @@ -2885,7 +2749,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable controls the concurrency for merging TopN results of partitioned tables. +- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. ### tidb_metric_query_range_duration New in v4.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md deleted file mode 100644 index 79e74aec086dd..0000000000000 --- a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 -summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. ---- - -# TiDB Cloud TPC-C Performance Test Report for TiDB v8.1.0 - -This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.1.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.1.0 clusters. - -## Test overview - -This test aims at showing the TPC-C performance of TiDB v8.1.0 in the Online Transactional Processing (OLTP) scenario. - -## Test environment - -### TiDB cluster - -The test is conducted on a TiDB cluster with the following settings: - -- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) -- Cluster version: v8.1.0 -- Cloud provider: AWS (us-west-2) -- Cluster configuration: - - | Node type | Node size | Node quantity | Node storage | - |:----------|:----------|:----------|:----------| - | TiDB | 16 vCPU, 32 GiB | 2 | N/A | - | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | - -### Parameter configuration - -> **Note:** -> -> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. - -The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: - -```yaml -raft-engine.prefill-for-recycle = true -``` - -### Benchmark executor - -The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: - -- Machine type: Amazon EC2 (us-west-2) -- Instance type: c6a.2xlarge - -## Test steps - -This section introduces how to perform the TPC-C performance test step by step. - -1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. - - For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). - -2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. - - To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). - - To create the `tpcc` database, execute the following SQL statement: - - ```sql - CREATE DATABASE tpcc; - ``` - -3. Load TPC-C data to the `tpcc` database. - - 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: - - ```shell - curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh - ``` - - 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. - - ```shell - go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: - - ```sql - ANALYZE TABLE customer; - ANALYZE TABLE district; - ANALYZE TABLE history; - ANALYZE TABLE item; - ANALYZE TABLE new_order; - ANALYZE TABLE order_line; - ANALYZE TABLE orders; - ANALYZE TABLE stock; - ANALYZE TABLE warehouse; - ``` - - To accelerate the collection of statistics, execute the following SQL statements before collecting: - - ```sql - SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; - ``` - -5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. - - ```shell - go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -6. Extract the tpmC data of `NEW_ORDER` from the result. - - TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. - -## Test results - -The TPC-C performance of v8.1.0 in the [test environment](#test-environment) is as follows: - -| Threads | v8.1.0 tpmC | -|:--------|:----------| -| 50 | 43,660 | -| 100 | 75,495 | -| 200 | 102,013 | - -![TPC-C](/media/tidb-cloud/v8.1.0_tpcc.png) diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md deleted file mode 100644 index 3bd5bbd2a576a..0000000000000 --- a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 -summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. ---- - -# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 - -This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. - -## Test overview - -This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. - -## Test environment - -### TiDB cluster - -The test is conducted on a TiDB cluster with the following settings: - -- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) -- Cluster version: v8.5.0 -- Cloud provider: AWS (us-west-2) -- Cluster configuration: - - | Node type | Node size | Node quantity | Node storage | - |:----------|:----------|:----------|:----------| - | TiDB | 16 vCPU, 32 GiB | 2 | N/A | - | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | - -### Parameter configuration - -> **Note:** -> -> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. - -The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: - -```yaml -raft-engine.prefill-for-recycle = true -``` - -### Benchmark executor - -The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: - -- Machine type: Amazon EC2 (us-west-2) -- Instance type: c6a.2xlarge - -## Test steps - -This section introduces how to perform the TPC-C performance test step by step. - -1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. - - For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). - -2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. - - To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). - - To create the `tpcc` database, execute the following SQL statement: - - ```sql - CREATE DATABASE tpcc; - ``` - -3. Load TPC-C data to the `tpcc` database. - - 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: - - ```shell - curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh - ``` - - 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. - - ```shell - go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: - - ```sql - ANALYZE TABLE customer; - ANALYZE TABLE district; - ANALYZE TABLE history; - ANALYZE TABLE item; - ANALYZE TABLE new_order; - ANALYZE TABLE order_line; - ANALYZE TABLE orders; - ANALYZE TABLE stock; - ANALYZE TABLE warehouse; - ``` - - To accelerate the collection of statistics, execute the following SQL statements before collecting: - - ```sql - SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; - ``` - -5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. - - ```shell - go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -6. Extract the tpmC data of `NEW_ORDER` from the result. - - TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. - -## Test results - -The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: - -| Threads | v8.5.0 tpmC | -|:--------|:----------| -| 50 | 43,146 | -| 100 | 73,875 | -| 200 | 103,395 | - -![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -194,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file From f480447e13edbc76a8b924643c2abe5e37ea7d07 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:23:30 +0800 Subject: [PATCH 3/7] docs: adapt cherry-pick for release-6.5 --- statistics.md | 2 +- system-variables.md | 17 ++++++++++------- troubleshoot-tidb-oom.md | 3 +-- 3 files changed, 12 insertions(+), 10 deletions(-) diff --git a/statistics.md b/statistics.md index 7c52ed64ded9b..7d21e2e2cd172 100644 --- a/statistics.md +++ b/statistics.md @@ -412,7 +412,7 @@ When you run the `ANALYZE` statement, you can adjust the concurrency using the f #### `tidb_build_stats_concurrency` -Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `4`. #### `tidb_distsql_scan_concurrency` diff --git a/system-variables.md b/system-variables.md index 417777b17a3ee..ada155efa476e 100644 --- a/system-variables.md +++ b/system-variables.md @@ -744,7 +744,8 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. +- Range: `[1, 18446744073709551615]` +- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ### tidb_analyze_version New in v5.1.0 @@ -805,7 +806,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Type: Integer - Default value: `1` - Range: `[1, 256]` -- This variable is used to set the concurrency of executing the automatic update of statistics. +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_backoff_lock_fast @@ -899,8 +900,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `4` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_capture_plan_baselines New in v4.0 @@ -2435,14 +2435,17 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50). + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans. ### tidb_init_chunk_size @@ -2749,7 +2752,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- This variable controls the concurrency for merging TopN results of partitioned tables. ### tidb_metric_query_range_duration New in v4.0 diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From a39ec5c1ffed6a0682028162cd3062cd5bc0d84d Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:31:03 +0800 Subject: [PATCH 4/7] docs: retain release-6.5 index scan concurrency --- system-variables.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/system-variables.md b/system-variables.md index ada155efa476e..eede835f20da0 100644 --- a/system-variables.md +++ b/system-variables.md @@ -2435,17 +2435,14 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency -> **Warning:** -> -> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50). - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans. +- This variable is used to set the concurrency of the `serial scan` operation. +- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. ### tidb_init_chunk_size From 96337d8bcd11e51af41e421a182d88f4ed922cd2 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:42:26 +0800 Subject: [PATCH 5/7] docs: retain release-6.5 OOM concurrency guidance --- troubleshoot-tidb-oom.md | 1 + 1 file changed, 1 insertion(+) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..545ec8c1d0c16 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. From f14be7363055f0ba73dfd023e556765d086c86d3 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:44:26 +0800 Subject: [PATCH 6/7] docs: exclude no-op OOM guidance from release-6.5 --- troubleshoot-tidb-oom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 545ec8c1d0c16..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -195,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file From ad79aafb6a6fae52d9c566d291f9c411f9b51931 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 16 Jul 2026 16:02:18 +0800 Subject: [PATCH 7/7] Update system-variables.md --- system-variables.md | 1 - 1 file changed, 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index eede835f20da0..ceca067245411 100644 --- a/system-variables.md +++ b/system-variables.md @@ -744,7 +744,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Default value: `1` -- Range: `[1, 18446744073709551615]` - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ### tidb_analyze_version New in v5.1.0