Skip to content

[Bug]: health metrics never sent over a Unix DogStatsD socket — client uses SOCK_STREAM instead of SOCK_DGRAM #4100

Description

@NikitaCOEUR

Bug report

With DD_TRACE_HEALTH_METRICS_ENABLED=true and a Unix socket DogStatsD URL, no health metric is
ever delivered. Every attempt logs:

[ddtrace] [warning] Health metric 'datadog.tracer.heartbeat' failed to send: E_WRITE

139 occurrences on a single pod. Application traces are unaffected — they go through
/var/run/datadog/apm.socket and work fine. Only the tracer's own telemetry is lost.

Environment

  • dd-trace-php 1.23.3, PHP 8.5 ZTS under FrankenPHP
  • Datadog Agent 7.77.2, DogStatsD exposed over a Unix socket through the Datadog CSI driver
  • DD_DOGSTATSD_URL=unix:///var/run/datadog/dsd/dsd.socket (injected by the admission controller)
  • DD_TRACE_HEALTH_METRICS_ENABLED=true (injected as well)

Root cause

strace shows the connection being refused at the protocol level:

connect(24, {sa_family=AF_UNIX, sun_path="/var/run/datadog/dsd/dsd.socket"}, 110)
    = -1 EPROTOTYPE (Protocol wrong type for socket)

The agent's socket is a datagram socket, as expected for DogStatsD — from /proc/net/unix on the
host, the Type field is 0002 (SOCK_DGRAM):

0000000000000000: 00000002 00000000 00000000 0002 01 22282 /var/run/datadog/dsd.socket

But dd_alloc_unix_addr() builds a stream socket for the unix:// scheme —
tracer/dogstatsd_client.c#L19-L31,
the offending line being
L24:

static struct addrinfo *dd_alloc_unix_addr(char *path, size_t len) {
    struct addrinfo *addrs = malloc(sizeof(*addrs));
    addrs->ai_next = NULL;
    addrs->ai_family = PF_UNIX;
    addrs->ai_protocol = 0;
    addrs->ai_socktype = SOCK_STREAM;      // ← DogStatsD is datagram-based
    addrs->ai_addrlen = sizeof(struct sockaddr_un);
    …
}

SOCK_STREAM against a SOCK_DGRAM socket yields EPROTOTYPE every time, so the send can never
succeed.

Inconsistency within the product

libdatadog's own DogStatsD client handles the same unix:// scheme correctly —
libdd-dogstatsd-client/src/lib.rs#L208-L216:

fn create_client(endpoint: &Endpoint) -> anyhow::Result<StatsdClient> {
    match endpoint.url.scheme_str() {
        #[cfg(unix)]
        Some("unix") => {
            let socket = UnixDatagram::unbound()   // ← datagram

So the two implementations shipped in the same product disagree on the socket type for the same URL
scheme.

Still present on master

ai_socktype = SOCK_STREAM is unchanged on master as of 2026-08-06 (56a3b006b), so upgrading
does not help. We could not find an existing issue or PR covering it.

Suggested fix

SOCK_STREAMSOCK_DGRAM in dd_alloc_unix_addr().

Note on the impact

This is what makes the bug worth reporting despite its small footprint: health metrics are precisely
the signal that would warn about a tracer in trouble — dropped spans, send errors, saturated
queues. We hit an unrelated sidecar issue (separate report) where the tracer was failing for hours
with no way to surface it, because this channel was silently broken.

There is no user-side workaround that we can see: the code only accepts unix:// and udp://, so
unixgram:// is rejected, and the URL is injected automatically by the admission controller.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions