[CONTINT-5389] Collect kubelet_containers_per_pod_count in the kubelet check#24600
[CONTINT-5389] Collect kubelet_containers_per_pod_count in the kubelet check#24600sehgal23 wants to merge 1 commit into
Conversation
Validation ReportAll 21 validations passed. Show details
|
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 1687a9b | Docs | Datadog PR Page | Give us feedback! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1687a9bc9c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| kubernetes.kubelet.pleg.relist_interval.count,gauge,,second,,The count of relisting pods in PLEG,0,kubernetes,k8s.pleg.relist_interv, | ||
| kubernetes.kubelet.pleg.relist_interval.sum,gauge,,,,The sum of interval in seconds between relisting in PLEG,0,kubernetes,k8s.pleg.relist_interv, | ||
| kubernetes.kubelet.containers_per_pod.count,gauge,,,,The number of pods observed when building the containers-per-pod distribution,0,kubernetes,k8s.containers_per_pod, | ||
| kubernetes.kubelet.containers_per_pod.sum,gauge,,,,The total number of containers across all pods on the node,0,kubernetes,k8s.containers_per_pod, |
There was a problem hiding this comment.
Describe the cumulative histogram sum accurately
When the kubelet has observed pods over time, this Prometheus histogram's _sum is the cumulative sum of all container-count observations since the kubelet process started; it does not decrease when pods leave. Describing it as the total across all pods currently on the node will cause users to interpret or dashboard it as an instantaneous container count. Describe it as the cumulative number of containers across observed pod samples instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
We faced the same confusion so it's worth clarifying per Codex's suggestion.
I think you should also adopt the metric's official description.
The number of containers per pod.
For updated descriptions:
The cumulative sum of the number of containers per pod.
and
The count of the number of containers per pod.
What does this PR do?
Motivation
https://datadoghq.atlassian.net/browse/CONTINT-5389
Review checklist (to be filled by reviewers)
qa/requiredif this PR needs QA validation, orqa/skip-qaif it does not. Exactly one of the two is required.backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged