[kafka_consumer] Emit consumer_membership DSM events

[piochelepiotr]

What

The kafka_consumer integration now emits a consumer_membership data-streams-message per consumer group, containing kafka_cluster_id, group_id, and the sorted list of member_ids. Emitted from _collect_consumer_group_metadata where the member list is already gathered.

Why

DSM uses this to reconcile consumer-group membership: drop departed members and seed tagless rows for members the tracer hasn't reported, while tracer-sourced infra tags win.

Part of a cross-repo change

Feeds the same dsm-kafka-configs pipeline as the dd-trace-java tracer: → dsm-kafka-configs-writer → orgstore consumer_group_members → dsm-api.

Tests

ddev test kafka_consumer — tests/test_cluster_metadata.py all pass (54), including new test_consumer_membership_event_emitted.

🤖 Generated with Claude Code

[datadog-datadog-prod-us1]

 

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 92.59%
• Overall Coverage: 91.82% (+3.49%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7fac138 | Docs | Datadog PR Page | Give us feedback!}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 31861a8462

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Rename the test helper without a leading underscore

In this repo, AGENTS.md explicitly says that module-level helpers in test files should not use a leading underscore unless they are instance methods or reusable-library helpers with non-obvious side effects. This newly added test helper is a free function in a test module, so keeping the _consumer_membership_events name violates the repository convention; rename it without the privacy prefix and update the call site.

Useful? React with 👍 / 👎.

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run