Skip to content

Fix YAML mapper for helm2dda for Helm's configmap based configurations - #3373

Open
mrdoggopat wants to merge 4 commits into
mainfrom
mrmcpat-fix-yaml-mapper-otel-confd-mapping-for-helm2dda
Open

Fix YAML mapper for helm2dda for Helm's configmap based configurations#3373
mrdoggopat wants to merge 4 commits into
mainfrom
mrmcpat-fix-yaml-mapper-otel-confd-mapping-for-helm2dda

Conversation

@mrdoggopat

@mrdoggopat mrdoggopat commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes the following issues when the helm2dda flag is run for the Datadog Plugin for kubectl.

When using datadog.otelCollector.config in Helm like:

datadog:
  otelCollector:
    config: |
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317

As a result, the mappings come out with three duplicate entries which the latter two are invalid:

spec:
  features:
    otelCollector:
      conf:
        configData: |
          receivers:
            otlp:
              protocols:
                grpc:
                  endpoint: 0.0.0.0:4317
        configMap:
          items: |
            receivers:
              otlp:
                protocols:
                  grpc:
                    endpoint: 0.0.0.0:4317
          name: |
            receivers:
              otlp:
                protocols:
                  grpc:
                    endpoint: 0.0.0.0:4317

In addition, using clusterAgent.confd like:

clusterAgent:
  confd:
    mysql.yaml: |
      cluster_check: true
      instances:
        - host: 1.2.3.4
          port: 3306

As a result, the mappings for extraConfd are missing the configDataMap like:

  override:
    clusterAgent:
      extraConfd:
        mysql.yaml: |-
          cluster_check: true
          instances:
            - host: 1.2.3.4
              port: 3306
  • Of note datadog.confd is unaffected.

In addition, agents.customAgentConfig and clusterAgent.datadog_cluster_yaml does not render in the generated DatadogAgent at all and errors out with:

2026/08/19 16:08:37 ERROR source value key was not found in mapping key=agents.customAgentConfig.log_level
2026/08/19 16:08:37 ERROR source value key was not found in mapping key=agents.customAgentConfig.tags
2026/08/19 16:08:37 ERROR source value key was not found in mapping key=clusterAgent.datadog_cluster_yaml.log_level

Motivation

CONS-8504

Additional Notes

Some things to flag which was not covered in this PR:
clusterAgent.advancedConfd is also broken as it would render something like the following which isn't valid:

  override:
    clusterAgent:
      extraConfd:
        configData:
          mysql.d:
            1.yaml: |
              cluster_check: true
              instances:
                - host: 1.2.3.4
                  port: 3306
            2.yaml: |-
              cluster_check: true
              instances:
                - host: 5.6.7.8
                  port: 3306

Despite my changes rendering the DatadogAgent CR properly, it still produced these phantom errors (first error might be expected as there is no equivalent in the DatadogAgent):

026/08/19 16:18:00 ERROR DDA destination key not found sourceKey=agents.useConfigMap
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=clusterAgent.confd.mysql.yaml
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=agents.customAgentConfig.log_level
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=agents.customAgentConfig.tags
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=datadog.confd.mysql.yaml
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=clusterAgent.datadog_cluster_yaml.log_level
  • Furthermore, did not want to make further changes to the error handling as I don't want to potentially prevent the plugin from suppressing legitimate errors.

Describe your test plan

Tested changes locally with the following values.yaml:

datadog:
  apiKey: <your api key>
  otelCollector:
    enabled: true
    config: |
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: 0.0.0.0:4317
  operator:
    enabled: true
    migration:
      preview: true
  confd:
    mysql.yaml: |
      instances:
        - host: 1.2.3.4
          port: 3306
agents:
  useConfigMap: true
  customAgentConfig:
    log_level: debug
    tags: [foo:bar]
clusterAgent:
  confd:
    mysql.yaml: |
      cluster_check: true
      instances:
        - host: 1.2.3.4
          port: 3306
  datadog_cluster_yaml:
    log_level: debug

Then run the helm2dda flag with:

# run this in the root of the repo
make kubectl-datadog
export PATH="<path to datadog operator repo>/datadog-operator/bin:$PATH"

# then ran
kubectl datadog helm2dda \
  --sourcePath=values.yaml \
  --destPath=helm2dda.yaml \
  --namespace=datadog

Desired Output:

2026/08/19 16:18:00 ERROR DDA destination key not found sourceKey=agents.useConfigMap
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=clusterAgent.confd.mysql.yaml
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=agents.customAgentConfig.log_level
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=agents.customAgentConfig.tags
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=datadog.confd.mysql.yaml
2026/08/19 16:18:00 ERROR source value key was not found in mapping key=clusterAgent.datadog_cluster_yaml.log_level

Mapped DatadogAgent custom resource:

apiVersion: datadoghq.com/v2alpha1
kind: DatadogAgent
metadata:
  name: datadog
  namespace: datadog
spec:
  features:
    otelCollector:
      conf:
        configData: |
          receivers:
            otlp:
              protocols:
                grpc:
                  endpoint: 0.0.0.0:4317
      enabled: true
  global:
    credentials:
      apiKey: <api key>
  override:
    clusterAgent:
      customConfigurations:
        datadog-cluster.yaml:
          configData: |
            log_level: debug
      extraConfd:
        configDataMap:
          mysql.yaml: |
            cluster_check: true
            instances:
              - host: 1.2.3.4
                port: 3306
    nodeAgent:
      customConfigurations:
        datadog.yaml:
          configData: |
            log_level: debug
            tags:
            - foo:bar
      extraConfd:
        configDataMap:
          mysql.yaml: |
            instances:
              - host: 1.2.3.4
                port: 3306

2026/08/19 16:18:00 INFO YAML file successfully written path=helm2dda.yaml
2026/08/19 16:18:00 ERROR mapper run failed error="mapping completed with 6 error(s): the mapped DDA may contain misconfigurations"
  • The errors are expected as this PR does not fix those, see Additional Notes above.

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

@mrdoggopat mrdoggopat added the bug Something isn't working label Aug 19, 2026
@mrdoggopat mrdoggopat changed the title Fix YAML mapper for helm2dda for Helm's datadog.otelCollector.config and clusterAgent.confd Fix YAML mapper for helm2dda for Helm's configmap based configurations Aug 19, 2026
@mrdoggopat mrdoggopat changed the title Fix YAML mapper for helm2dda for Helm's configmap based configurations [CONS-8504] - Fix YAML mapper for helm2dda for Helm's configmap based configurations Aug 19, 2026
@mrdoggopat mrdoggopat changed the title [CONS-8504] - Fix YAML mapper for helm2dda for Helm's configmap based configurations Fix YAML mapper for helm2dda for Helm's configmap based configurations Aug 19, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Aug 19, 2026

Copy link
Copy Markdown

Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 96.00%
Overall Coverage: 49.67% (-0.43%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: c4401df | Docs | View more details | Give us feedback!

@mrdoggopat
mrdoggopat marked this pull request as ready for review August 19, 2026 20:37
@mrdoggopat
mrdoggopat requested a review from a team August 19, 2026 20:37
@mrdoggopat mrdoggopat changed the title Fix YAML mapper for helm2dda for Helm's configmap based configurations [CONS-8504] - Fix YAML mapper for helm2dda for Helm's configmap based configurations Aug 19, 2026
@mrdoggopat mrdoggopat changed the title [CONS-8504] - Fix YAML mapper for helm2dda for Helm's configmap based configurations Fix YAML mapper for helm2dda for Helm's configmap based configurations Aug 19, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1fed1d28e9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread cmd/yaml-mapper/mapper/mapper.go Outdated
mrdoggopat added a commit that referenced this pull request Aug 19, 2026
Allowing any multi-key table through the fallback (added to support
agents.customAgentConfig/clusterAgent.datadog_cluster_yaml) let it also
apply to plain string/list destinations like
agents.podSecurity.seLinuxContext, which have separate leaf mappings for
their sub-fields. That produced invalid duplicate/extra fields in the
output. Now the multi-key fallback only fires for mapFunc-based
destinations (e.g. mapCustomConfigFile), matching Codex review feedback
on PR #3373.

Co-authored-by: Cursor <cursoragent@cursor.com>
Allowing any multi-key table through the fallback (added to support
agents.customAgentConfig/clusterAgent.datadog_cluster_yaml) let it also
apply to plain string/list destinations like
agents.podSecurity.seLinuxContext, which have separate leaf mappings for
their sub-fields. That produced invalid duplicate/extra fields in the
output. Now the multi-key fallback only fires for mapFunc-based
destinations (e.g. mapCustomConfigFile), matching Codex review feedback
on PR #3373.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mrdoggopat
mrdoggopat force-pushed the mrmcpat-fix-yaml-mapper-otel-confd-mapping-for-helm2dda branch from 322168e to e47fbeb Compare August 19, 2026 21:04

@Mathew-Estafanous Mathew-Estafanous left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good based on my limited (zero) experience of helm2dda

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working team/container-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants