Skip to content

Emit untagged telemetry when OpenMetrics reaches its metric limit#24588

Draft
nubtron wants to merge 2 commits into
masterfrom
nubtron/ai-7012-untagged-telemetry
Draft

Emit untagged telemetry when OpenMetrics reaches its metric limit#24588
nubtron wants to merge 2 commits into
masterfrom
nubtron/ai-7012-untagged-telemetry

Conversation

@nubtron

@nubtron nubtron commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Emits the untagged openmetrics.max_returned_metrics_reached Agent telemetry counter once per OpenMetrics check run when the existing metric limiter reaches max_returned_metrics.

The producer checks the limiter's structured reached_limit state before reset, then uses the existing four-argument datadog_agent.emit_agent_telemetry API. OpenMetrics v1 and v2 opt in; generic AgentCheck limiters do not. Emission is best-effort and cannot alter warning, drop, or reset behavior.

The metric intentionally carries no tags, including no check_name.

Motivation

AI-7012 asks for a fleet-wide COAT signal for silent OpenMetrics drops after max_returned_metrics is reached. An untagged first version provides that signal while avoiding warning-text parsing, new rtloader callbacks, and a generic labeled telemetry bridge.

Validation included focused metric-limiter tests covering v1/v2 opt-in, once-per-run emission, generic-check exclusion, old-Agent compatibility, bridge failure behavior, and debug metric contexts. The complete datadog_checks_base format/lint target also passed.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

@nubtron nubtron added the qa/skip-qa Automatically skip this PR for the next QA label Jul 17, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Jul 17, 2026

Copy link
Copy Markdown

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry View in Datadog

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 88.45% (+0.11%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2231954 | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

base_package qa/skip-qa Automatically skip this PR for the next QA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant