feat: add more support for different workload types #1052
Conversation
sleepyfoodie
changed the title
There was a problem hiding this comment.
There is another non-standard workload rule missing for workloads you would see from the following query:
group by (owner_kind) (kube_pod_owner{owner_kind!="ReplicaSet", owner_kind!="DaemonSet", owner_kind!="StatefulSet", owner_kind!="Job", owner_kind!="Node", owner_kind!=""})
That should capture workload types that are directly controlled via CRDs like StrimziPodSet for example.
No join is required for this additional rule as the pods are created directly rather than via ReplicaSets.
Co-authored-by: Stephen Lang <skl@users.noreply.github.com>
Co-authored-by: Stephen Lang <skl@users.noreply.github.com>
Co-authored-by: Stephen Lang <skl@users.noreply.github.com>
Co-authored-by: Stephen Lang <skl@users.noreply.github.com>
Co-authored-by: Stephen Lang <skl@users.noreply.github.com>
rules/apps.libsonnet
Outdated
| { | ||
| record: 'namespace_workload_pod:kube_pod_owner:relabel', | ||
| expr: ||| | ||
| max by (%(clusterLabel)s, namespace, workload, pod) ( | ||
| group by(%(clusterLabel)s, namespace, workload, workload_type, pod) ( |
There was a problem hiding this comment.
workload_type is added as an external label so should not be present in the query, it also means the extra label_replace can be removed
| group by(%(clusterLabel)s, namespace, workload, workload_type, pod) ( | |
| group by(%(clusterLabel)s, namespace, workload, pod) ( |
| ) | ||
| * on (%(clusterLabel)s, namespace, job_name) group_left(owner_kind, owner_name) | ||
| group by (%(clusterLabel)s, namespace, job_name, owner_kind, owner_name) ( | ||
| kube_job_owner{%(kubeStateMetricsSelector)s, owner_kind!="Pod", owner_kind!=""} |
There was a problem hiding this comment.
Jobs with owner_kind="Pod" would always be excluded by both this rule and the job rule.
What should we do with kube_job_owner{owner_kind="Pod"} series so that they're not excluded? These appear to be jobs created by pods...
Update workload recording rules to support more workload type
New recording rules tested have good health status:
