Skip to content

Document per-node metric collection in single_endpoint_mode for ClickHouse Cloud#38103

Open
sangeetashivaji wants to merge 7 commits into
masterfrom
sangeeta.shivajirao/clickhouse-single-endpoint-per-node-docs
Open

Document per-node metric collection in single_endpoint_mode for ClickHouse Cloud#38103
sangeetashivaji wants to merge 7 commits into
masterfrom
sangeeta.shivajirao/clickhouse-single-endpoint-per-node-docs

Conversation

@sangeetashivaji

Copy link
Copy Markdown
Contributor

What does this PR do? What is the motivation?

Clarifies the behavior of single_endpoint_mode on the ClickHouse Cloud DBM setup page. When enabled, the Agent collects standard system-table metrics (system.events, system.metrics, system.asynchronous_metrics, system.errors) from every node using clusterAllReplicas() and tags each series with clickhouse_node, producing accurate per-node values for cumulative counters such as the one behind clickhouse.query.failed.count. Without this setting on a load-balanced multi-node service, consecutive collections can read independent per-node counters from different nodes, producing inaccurate metric values.

Documents that per-node collection requires Agent version 7.82.0 or later.

Merge instructions

Merge readiness:

  • Ready for merge

Additional notes

Per-node collection for the aggregated tables (system.parts, system.replicas, system.dictionaries) is intentionally not covered, as those queries use GROUP BY.

🤖 Generated with Claude Code

…House Cloud

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

sangeetashivaji and others added 3 commits July 10, 2026 13:55
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@sangeetashivaji sangeetashivaji marked this pull request as ready for review July 15, 2026 15:08
@sangeetashivaji sangeetashivaji requested review from a team as code owners July 15, 2026 15:08

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

This is a documentation-only PR combining related ClickHouse DBM changes across two files: per-node metric collection explanation, Agent version requirement (7.83.0), and Vale style fixes capitalizing "Boolean" in configuration tables. No regressions or factual errors introduced by the PR diff.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Datadog Autotest · Commit 660d8be · What is Autotest? · Any feedback? Reach out in #autotest

@domalessi domalessi self-assigned this Jul 15, 2026

@domalessi domalessi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some editorial suggestions. Let me know if that looks okay to you!

Comment on lines +152 to +154
`single_endpoint_mode: true` is required for ClickHouse Cloud. It enables `clusterAllReplicas()` queries to collect data across all nodes behind a single endpoint.

When enabled, the Agent also collects standard system-table metrics (from `system.events`, `system.metrics`, `system.asynchronous_metrics`, and `system.errors`) from every node and tags each series with `clickhouse_node`, which produces accurate per-node values. Each node maintains its own cumulative counters, such as the counter behind `clickhouse.query.failed.count`. Without `single_endpoint_mode`, consecutive collections through the load balancer can read these independent per-node counters from different nodes, producing inaccurate metric values. This per-node collection requires Agent version 7.83.0 or later.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This paragraph is a bit dense. Splitting up into two paragraphs and breaking up some of the longer sentences.

Suggested change
`single_endpoint_mode: true` is required for ClickHouse Cloud. It enables `clusterAllReplicas()` queries to collect data across all nodes behind a single endpoint.
When enabled, the Agent also collects standard system-table metrics (from `system.events`, `system.metrics`, `system.asynchronous_metrics`, and `system.errors`) from every node and tags each series with `clickhouse_node`, which produces accurate per-node values. Each node maintains its own cumulative counters, such as the counter behind `clickhouse.query.failed.count`. Without `single_endpoint_mode`, consecutive collections through the load balancer can read these independent per-node counters from different nodes, producing inaccurate metric values. This per-node collection requires Agent version 7.83.0 or later.
`single_endpoint_mode: true` is required for ClickHouse Cloud. It enables `clusterAllReplicas()` queries to collect data across all nodes behind a single endpoint, and helps ensure the Agent reports accurate per-node values for cumulative counters.
Each ClickHouse node maintains its own cumulative counters, such as the one behind `clickhouse.query.failed.count`. Without `single_endpoint_mode`, consecutive collections through the load balancer can read these counters from different nodes, producing inaccurate values. When `single_endpoint_mode` is enabled, the Agent also collects standard system-table metrics (`system.events`, `system.metrics`, `system.asynchronous_metrics`, `system.errors`) from every node and tags each series with `clickhouse_node` to produce accurate per-node values. This requires Agent version 7.83.0 or later.

Comment thread content/en/database_monitoring/setup_clickhouse/cloud.md Outdated
@domalessi

Copy link
Copy Markdown
Contributor

@sangeetashivaji Looks like you pushed a change after I started reviewing. Let me know if content is set, and I'll look again. Or you can see my suggestion and just apply that if it captures your intent.

Co-authored-by: domalessi <111786334+domalessi@users.noreply.github.com>
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

DataDog/documentation | build_preview   View in Datadog   GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ee67210 | Docs | Datadog PR Page | Give us feedback!

@domalessi domalessi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two minor suggestions that I'll apply directly. Otherwise, looks good.

Comment thread content/en/database_monitoring/setup_clickhouse/cloud.md Outdated
Comment thread content/en/database_monitoring/setup_clickhouse/cloud.md Outdated
Co-authored-by: domalessi <111786334+domalessi@users.noreply.github.com>
@sangeetashivaji

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Jul 15, 2026

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-07-15 20:01:06 UTC ℹ️ Start processing command /merge


2026-07-15 20:01:08 UTC ❌ MergeQueue

The merge queue is disabled for this repository/branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants