Skip to content

feat(clickstack): surface ClickHouse table TTL as configurable values #250

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ZeynelKoca wants to merge 1 commit into
base: main
Choose a base branch
from
+39 −0
Open

feat(clickstack): surface ClickHouse table TTL as configurable values #250

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/expose-table-ttl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
"helm-charts": minor
---

feat(clickstack): surface ClickHouse table TTL as configurable values

Defaults and documents `HYPERDX_OTEL_EXPORTER_TABLES_TTL` (720h) in `hyperdx.config`, and documents the per-signal overrides (`HYPERDX_OTEL_EXPORTER_LOGS_TTL` / `_TRACES_TTL` / `_METRICS_TTL` / `_SESSIONS_TTL`) plus `HYPERDX_OTEL_EXPORTER_RECONCILE_TABLE_TTL`. Operators can now set ClickHouse data retention per signal — e.g. keep logs and traces for 6 months for compliance while metrics stay short — without hand-crafting collector env vars. The global TTL works today; the per-signal overrides and reconcile require a collector image that includes hyperdxio/hyperdx#2709.
20 changes: 20 additions & 0 deletions charts/clickstack/tests/app-configmap_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,23 @@ tests:
- matchRegex:
path: data.OTEL_EXPORTER_OTLP_ENDPOINT
pattern: "http://.*-otel-collector:4318"

- it: should render the default ClickHouse table TTL
asserts:
- equal:
path: data.HYPERDX_OTEL_EXPORTER_TABLES_TTL
value: "720h"

- it: should render per-signal TTL and reconcile overrides from config
set:
hyperdx:
config:
HYPERDX_OTEL_EXPORTER_LOGS_TTL: "180d"
HYPERDX_OTEL_EXPORTER_RECONCILE_TABLE_TTL: "true"
asserts:
- equal:
path: data.HYPERDX_OTEL_EXPORTER_LOGS_TTL
value: "180d"
- equal:
path: data.HYPERDX_OTEL_EXPORTER_RECONCILE_TABLE_TTL
value: "true"
12 changes: 12 additions & 0 deletions charts/clickstack/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,18 @@ hyperdx:
HYPERDX_OTEL_EXPORTER_CLICKHOUSE_DATABASE: "default"
CLICKHOUSE_USER: "otelcollector"
RUN_SCHEDULED_TASKS_EXTERNALLY: "false"
# ClickHouse table retention (TTL). Go-duration string; "d" suffix allowed (e.g. "180d").
# This global default applies to every signal.
HYPERDX_OTEL_EXPORTER_TABLES_TTL: "720h"
# Per-signal overrides (each falls back to HYPERDX_OTEL_EXPORTER_TABLES_TTL when unset),
# e.g. keep logs & traces longer for compliance while metrics stay short:
# HYPERDX_OTEL_EXPORTER_LOGS_TTL: "180d"
# HYPERDX_OTEL_EXPORTER_TRACES_TTL: "180d"
# HYPERDX_OTEL_EXPORTER_METRICS_TTL: "30d"
# HYPERDX_OTEL_EXPORTER_SESSIONS_TTL: "30d"
# Apply TTL changes to tables that already exist, not just newly created ones (default false):
# HYPERDX_OTEL_EXPORTER_RECONCILE_TABLE_TTL: "true"
# Per-signal TTLs and reconcile require a collector image that includes hyperdxio/hyperdx#2709.
# Service endpoint defaults -- override to use external instances
FRONTEND_URL: "http://localhost:3000"
MONGO_URI: 'mongodb://hyperdx:{{ .Values.hyperdx.secrets.MONGODB_PASSWORD }}@{{ include "clickstack.mongodb.svc" . }}:27017/hyperdx?authSource=hyperdx'
Expand Down
Loading