From bcbb61bce0e522537ff74f799dd33e966e0a88a5 Mon Sep 17 00:00:00 2001 From: Peter Pathirana Date: Mon, 17 Aug 2026 01:29:17 +0000 Subject: [PATCH] feat: name workspace Deployment for Prometheus workload attribution Name the workspace Deployment coder-workspace-- instead of coder-. The cluster's Prometheus already resolves pod -> ReplicaSet -> Deployment via a kube_pod_owner recording rule and exposes it as a workload label, so this is what workspace CPU/memory/PSI/OOM metrics get attributed to in Grafana/PromQL, at zero added series and no join. Owner is included because workspace names are unique per-owner, not cluster-wide. The coder-workspace- prefix (already used as app.kubernetes.io/part-of) distinguishes workspaces from both the coder control-plane Deployment and other coder-prefixed infra sharing the namespace (verified live: a coder-db- CloudNativePG cluster exists today, so a bare coder- prefix is not sufficient). Reuses the existing hostname sanitization idiom instead of a new one, and documents the rationale in DESIGN.md/CLAUDE.md since the name is now load-bearing for observability, not just a Kubernetes identifier. --- CLAUDE.md | 1 + DESIGN.md | 6 ++++++ .../homelab-workspace/deployment.tf | 4 ++-- .../kubernetes/homelab-workspace/main.tf | 21 +++++++++++++++++++ 4 files changed, 30 insertions(+), 2 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8b5d6760..63023529 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -80,6 +80,7 @@ Things that look arbitrary in the code but are load-bearing (full reasoning in [ - The Dockerfile writes shared env vars to `/etc/environment` rather than using `ENV`, because `PATH` needs to be extended by a script running after the image is built, not fixed at build time. - `parameters.tf`'s `local.validated_*` regex allowlist is the only thing stopping `system_packages`/`preferred_nodes` from injecting shell metacharacters into the init container — any new list-type parameter must go through the same decode-then-validate step. - Adding a package/tool has three possible homes, and picking the wrong one is a real mistake, not a style choice — route by the rule in [DESIGN.md](DESIGN.md#where-the-workspace-environment-comes-from): universal + stable → image (`Dockerfile`); occasionally-needed + apt-only + too heavy to bake in → the template's `system_packages` parameter; personal, fast-moving, or not an apt package → the operator's dotfiles (a *different* repo — see below), never this one. +- `deployment.tf`'s Deployment `metadata.name` (`local.workload_name` in `main.tf`) is not cosmetic: the cluster's Prometheus resolves pod → ReplicaSet → Deployment via an existing `kube_pod_owner` recording rule and exposes the result as a `workload` label with no other join needed, so whatever this Deployment is named *is* the identity CPU/memory/PSI/OOM metrics get attributed to. Don't revert it to an opaque identifier (e.g. the workspace UUID) without re-breaking that attribution — see [DESIGN.md](DESIGN.md#design-tensions-and-decisions). ## Neighbouring repos diff --git a/DESIGN.md b/DESIGN.md index 4a384204..f50763a3 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -63,6 +63,12 @@ The rule that ties the layers together: a package or tool belongs in the *lowest **Unprivileged by default.** The workspace itself runs as an unprivileged, non-root, fixed-identity container. Anything that genuinely needs elevated privilege (installing packages, preparing shared volume state) is scoped to a narrow, short-lived setup step that runs before the workspace shell exists, not to something the workspace user can reach into. +**The Deployment name is a Prometheus identity, not just a Kubernetes identifier.** `deployment.tf` names the workspace Deployment `coder-workspace--` (`local.workload_name` in `main.tf`) rather than the workspace UUID it used before. cAdvisor's `container_*` series carry no Kubernetes labels — they come from the cgroup filesystem, with no API-server connection — so per-workspace CPU/memory/PSI/OOM can only be attributed to a human-readable identity through the cluster's existing `namespace_workload_pod:kube_pod_owner:relabel` recording rule, which resolves pod → ReplicaSet → Deployment into a `workload` label. That rule already runs for free; naming the Deployment meaningfully is the only lever this repo has to make its output meaningful, at zero added Prometheus series and no PromQL join. Three things shape the exact scheme: + +- *Owner is included* even though this is a single-operator homelab today, because Coder workspace names are unique per-owner, not cluster-wide — two owners could otherwise pick the same workspace name and collide. Cheap to include now; expensive to retrofit after a second migration. +- *The prefix is `coder-workspace-`, not just `coder-`*, matching the `app.kubernetes.io/part-of` value already used in `main.tf`'s `common_labels`. A bare `coder-` prefix isn't enough to unambiguously mean "workspace": the same Kubernetes namespace also holds the `coder` control-plane Deployment itself and other `coder`-prefixed infra (e.g. a CloudNativePG cluster named `coder-db-`) that a naive `workload=~"coder-.+"` match would also catch. +- *Renaming a workspace already relocates its home directory* (the `home` volume's `sub_path` is `data.coder_workspace.me.name`), so coupling the Deployment name to the workspace name too doesn't introduce a new class of rename hazard — it's already priced in. A rename recreates the Deployment (the pod restarts anyway) and needs a fresh `coder-workspace--` home subdirectory, exactly as it needed a fresh `sub_path` before this change. + ## Outcomes targeted - One operator can keep dependencies current and ship template/image changes at low ongoing effort, without a fleet of environments to maintain. diff --git a/templates/kubernetes/homelab-workspace/deployment.tf b/templates/kubernetes/homelab-workspace/deployment.tf index f4e22cd3..48236f9a 100644 --- a/templates/kubernetes/homelab-workspace/deployment.tf +++ b/templates/kubernetes/homelab-workspace/deployment.tf @@ -2,7 +2,7 @@ resource "kubernetes_deployment_v1" "deployment" { count = data.coder_workspace.me.start_count metadata { - name = "coder-${data.coder_workspace.me.id}" + name = local.workload_name namespace = "coder" labels = merge(local.common_labels, local.pod_labels) annotations = { @@ -153,7 +153,7 @@ resource "kubernetes_deployment_v1" "deployment" { } } enable_service_links = false - hostname = lower(replace(data.coder_workspace.me.name, "/[^a-zA-Z0-9]/", "-")) + hostname = local.sanitized_workspace_name node_selector = { "kubernetes.io/os" = "linux" "kubernetes.io/arch" = "amd64" diff --git a/templates/kubernetes/homelab-workspace/main.tf b/templates/kubernetes/homelab-workspace/main.tf index 4777be6d..efff7191 100644 --- a/templates/kubernetes/homelab-workspace/main.tf +++ b/templates/kubernetes/homelab-workspace/main.tf @@ -21,4 +21,25 @@ locals { home_directory = "/home/coder" homebrew_directory = "/home/linuxbrew/.linuxbrew" + + # Kubernetes object names must be a lowercase RFC 1123 label/subdomain. + # Coder's own name validation (NameValid in coder/coder's codersdk) already + # restricts workspace and owner names to `^[a-zA-Z0-9]+(?:-[a-zA-Z0-9]+)*$`, + # <= 32 chars, so lower() is the only transformation strictly required; + # replace() mirrors the pre-existing hostname sanitization in deployment.tf + # so there's one normalization idiom instead of two. + sanitized_owner_name = lower(replace(data.coder_workspace_owner.me.name, "/[^a-zA-Z0-9]/", "-")) + sanitized_workspace_name = lower(replace(data.coder_workspace.me.name, "/[^a-zA-Z0-9]/", "-")) + + # Deployment name (see deployment.tf). Owner is included because workspace + # names are unique per-owner, not cluster-wide - two owners could otherwise + # pick the same workspace name and collide. The "coder-workspace-" prefix + # distinguishes this from both the coder control-plane Deployment ("coder") + # and other "coder-*" infra objects that live in the same namespace (e.g. + # a CloudNativePG cluster named "coder-db-") - a bare "coder-" prefix + # is not enough to tell those apart. This name becomes the `workload` + # Prometheus label via the cluster's existing kube_pod_owner recording + # rule, so it's what workspace CPU/memory/PSI/OOM metrics get attributed + # to in Grafana/PromQL - see DESIGN.md. + workload_name = "coder-workspace-${local.sanitized_owner_name}-${local.sanitized_workspace_name}" }