Add telemetry benchamark setup

[hossain-rayhan]

Add a pluggable local latency benchmark for DocumentDB

Builds on the shared gateway tracing demo (#5) to add a repeatable, driver-agnostic latency benchmark that measures client, gateway, and PostgreSQL time for the same workload — with and without tracing — and decomposes connected traces from Jaeger.

What this adds

• playgrounds/benchmarks/local-latency/ — shared orchestration that is not tied to any driver:
  • benchmark.sh — runs a baseline (untraced) and a traced phase, drives the shared telemetry stack, and writes durable artifacts per experiment.
  • analyze_jaeger.py — pulls connected traces from Jaeger, filters by benchmark.id, validates the app → gateway.request → postgres.execute parent chain, and computes per-hop percentiles.
  • adapters/mongoose.sh — the first driver adapter (process-invoked; no app-code imports).
  • schemas/adapter-result.schema.json — the contract every adapter must satisfy.
  • test_analyze_jaeger.py — unit tests for percentiles, interval-union math, and trace-completeness gating.
• playgrounds/mongoose/app/benchmark.js — the Mongoose-native workload (deterministic seed doc, bounded concurrency, monotonic timing, BSON-sized $in query, client spans with benchmark.* attributes).
• shared/telemetry/otel-collector-benchmark.yaml — a Collector profile without the debug exporter so high-volume runs aren't distorted; compose.yaml selects it via OTEL_COLLECTOR_CONFIG_FILE.
• Small, backward-compatible tweak to telemetry.js so client spans can carry benchmark attributes.

How it works

Each run measures the same operation twice and stores everything under results/<experiment-id>/:

File	Contents
baseline.json / traced.json	Client throughput and latency percentiles per phase
trace-analysis.json	Connected-trace counts, exclusions, and per-hop percentiles
trace-segments.csv	One row per complete connected trace
summary.md	Human-readable client + per-hop summary
experiment.json	Image identity and artifact manifest

Reported hops: full client operation, client/transport outside gateway, gateway total, PostgreSQL (sum and interval union), and gateway residual. The analyzer waits until every successful operation has a complete app → gateway → PostgreSQL trace and fails rather than reporting partial statistics.

Measurement caveats (documented in the README)

• Exact one-way client→gateway latency is not available — client and gateway wall clocks aren't synchronized. client_outside_gateway is the clock-robust combined measure (driver, pool wait, serialization, TLS, both network directions, response decoding).
• Nested-span subtraction is not CPU profiling.
• Results are diagnostic on local Docker networking, not production capacity claims.

Usage

BENCHMARK_ADAPTER=mongoose ./playgrounds/benchmarks/local-latency/benchmark.sh

Defaults to the published tracing image (ghcr.io/documentdb/documentdb/documentdb-local:trace-4fbbfcb8). Override operations, warm-up, concurrency, and query size via BENCHMARK_* env vars. Adding another driver only requires a new adapters/<name>.sh plus a driver-native workload that emits client spans and injects W3C context into the command comment.

Testing

• Analyzer unit tests: 5/5 passing.
• End-to-end run against the published image: 100/100 connected traces, 0 exclusions, all artifacts generated.
• Mongoose CRUD compatibility suite: 16/16 passing.
• Benchmark Collector profile confirmed to omit the debug exporter.

Notes

• Generated results/ are git-ignored.
• No changes to the existing app/test paths beyond the optional benchmark hooks.

[udsmicrosoft]

benchmark.sh does not load the shared telemetry .env. up.sh applies custom ports and credentials only in its child process, while the adapter and analyzer retain defaults. Please source TELEMETRY_ENV_FILE before deriving the endpoints.

[udsmicrosoft]

The analyzer verifies the application-to-gateway link, but then includes every postgres.execute span in the trace. Retries or multi-command operations can mix branches and skew the metrics. Please restrict these spans to descendants of the selected gateway request.