Skip to content

feat: name workspace Deployment for Prometheus workload attribution - #861

Merged
ppat merged 1 commit into
mainfrom
feat/workload-rename
Aug 17, 2026
Merged

feat: name workspace Deployment for Prometheus workload attribution#861
ppat merged 1 commit into
mainfrom
feat/workload-rename

Conversation

@ppat

@ppat ppat commented Aug 17, 2026

Copy link
Copy Markdown
Owner

What this changes

Names the workspace kubernetes_deployment_v1 (deployment.tf) coder-workspace-<owner>-<workspace-name> instead of coder-<workspace-uuid>. local.workload_name and its sanitization helpers live in main.tf; deployment.tf's pre-existing hostname sanitization now reuses the same local instead of duplicating the regex.

Why

cAdvisor's container_* series carry no Kubernetes pod labels — they come from the cgroup filesystem, with no API-server connection, so they structurally can't. The cluster's Prometheus already has a namespace_workload_pod:kube_pod_owner:relabel recording rule that resolves pod → ReplicaSet → Deployment and exposes a workload label — verified live, this already gives workload="coder" for the control-plane pod and workload="coder-<uuid>" for a workspace pod. That machinery already works; it only produces a UUID because that's what the Deployment is named. Renaming the Deployment makes workload human-readable at zero added Prometheus series and no PromQL join. An alternative (kube-state-metrics metricLabelsAllowlist + group_left) was evaluated separately and rejected as more expensive/fragile for this purpose.

Naming scheme and rationale

coder-workspace-<owner>-<workspace-name>, e.g. a workspace named dev owned by a user alice becomes coder-workspace-alice-dev.

  • Owner is included even though this is a single-operator homelab today. Coder workspace names are unique per-owner, not cluster-wide, so without the owner segment two owners choosing the same workspace name would collide on terraform apply (Deployment names must be unique within the coder namespace). The owner mentioned possibly giving separate users to different AI agents later, each creating workspaces dynamically — cheap to fold in now, expensive to retrofit as a second migration later.
  • Prefix is coder-workspace-, not just coder-. A bare coder- prefix isn't sufficient to unambiguously mean "workspace" in this namespace: verified live, the coder namespace today also contains a coder-db-<date>-named CloudNativePG cluster (unrelated to workspaces) alongside the coder control-plane Deployment itself. coder-workspace- matches the app.kubernetes.io/part-of: coder-workspace value the template's common_labels already use, so it's an existing convention, not a new one — and it can't collide with the control-plane Deployment (exact-match workload="coder") or with other coder-* infra sharing the namespace.
  • Sanitization reuses the existing idiom (lower(replace(x, "/[^a-zA-Z0-9]/", "-")), previously only used for the pod hostname) rather than inventing a new one. In practice Coder's own server-side name validation (NameValid in coder/coder's codersdk) already restricts workspace and owner names to ^[a-zA-Z0-9]+(?:-[a-zA-Z0-9]+)*$, ≤ 32 chars — so lower() is the only transformation strictly required; the replace() is defense-in-depth consistent with the pre-existing pattern.

What references the Deployment name — checked across the whole template

  • Only deployment.tf's own metadata.name and hostname field. Nothing else in the template (agent config, coder_env, coder_agent metadata scripts, the ConfigMap, the release workflow) reads or constructs the Deployment name.
  • The kubernetes_config_map_v1.workspace_scripts resource (configmap.tf) is named init-scripts-<workspace-uuid> — unrelated to this change (not in the pod→ReplicaSet→Deployment ownership chain Prometheus uses) and left as-is, out of scope.
  • Confirmed the Coder control plane addresses agents by the CODER_AGENT_TOKEN bearer token (coder_agent.main.token, deployment.tf), not by any Kubernetes object name — this template has no coder_app/exec-based access path that would depend on a parseable pod/Deployment name.

Length limits — verified against Kubernetes source, not assumed

  • Deployment/ReplicaSet metadata.name are validated as IsDNS1123Subdomain (253 chars) in k8s.io/apimachinery. At Coder's own per-field max (32-char owner + 32-char workspace name), the Deployment name is 81 chars and the ReplicaSet name (+ "-" + <10-char pod-template-hash>) is 92 — both comfortably under 253.
  • Pods, however, are created via GenerateName, which truncates its input to 63 - 5 = 58 chars before appending a random suffix (k8s.io/apiserver/pkg/storage/names). At the 81-char worst case, the Pod's own object name gets silently truncated and randomized past char 58. This is cosmetic only: the workload label is read from the Deployment object via the ownership chain, not parsed from the Pod's own name, so attribution is unaffected regardless. Realistic names in this homelab (short owner/workspace names) don't come close to the threshold anyway.

Why the UUID was chosen, and why it no longer applies

No commit message or PR description spells this out directly (the UUID naming was present from the initial Docker→Kubernetes migration commit, 5c4c3a3/PR #361, with an empty PR body and no discussion), but the repo owner supplied the rationale from memory, and it's corroborated by what the history does show:

  • The reason it was chosen: to keep the workspace's storage identity stable across template/parameter changes, so persistent storage wouldn't get destroyed and recreated whenever a workspace parameter changed. A name derived from mutable inputs (workspace name, which is user-renameable) would have churned that identity along with it.
  • Verified in the Docker-based template that preceded this one (templates/docker/homelab-workspace/volumes.tf, removed by the same migration commit): it defined a template-managed docker_volume resource named coder-${data.coder_workspace.me.id}-${each.key} — keyed on the immutable workspace ID, not the mutable workspace name — with lifecycle { ignore_changes = all } on top, i.e. exactly the "don't let a mutable input churn volume identity" mechanism described.
  • Why it no longer applies, and precisely when it stopped applying: in the Kubernetes-based template, the home volume is a PersistentVolumeClaim referenced by a fixed, external name (claim_name = "coder-workspace-home") — not a resource this template creates or names itself. Checked the entire git history of this repo across all branches: no kubernetes_persistent_volume_claim resource has ever existed in it, in this file or any other. Checked live: coder-workspace-home in the coder namespace is Flux-managed (kustomize.toolkit.fluxcd.io labels/annotations), bound and provisioned entirely outside this template. So the PVC has been externally defined for the entire life of the Kubernetes template — since the same commit (5c4c3a3) that introduced the UUID-based Deployment name. The constraint that justified the UUID (Deployment name ↔ volume identity) had therefore already stopped applying by the time that commit landed; the UUID naming looks to have been carried forward from the Docker template's convention rather than re-derived against the new (already-external) storage model.
  • The dependency this rename now has: this naming choice is safe because the PVC is defined externally to the template — the Deployment's name has no bearing on volume identity or lifetime today. If workspace storage were ever brought back under the template's own lifecycle (a kubernetes_persistent_volume_claim resource managed here again), this naming scheme — or any scheme deriving a resource identity from the mutable workspace name — would need revisiting for the same reason the original UUID choice existed.

Rename behavior — unchanged in kind, not newly introduced

deployment.tf already coupled to the workspace name before this change: the home volume's sub_path is data.coder_workspace.me.name, so renaming a workspace already relocates (makes inaccessible from the new path) its home directory today. Making the Deployment name also track the workspace name doesn't add a new category of rename hazard — it's already priced into the existing design. A rename now additionally recreates the Deployment under the new coder-workspace-<owner>-<new-name> name, which is consistent with the pod restarting anyway on any apply.

Live verification

  • Confirmed live (before this change): workload="coder" for the control-plane pod, workload="coder-<uuid>" for workspace pods — the kube_pod_owner recording rule already resolves the Deployment name into workload with no gaps.
  • Confirmed live: the coder namespace contains a coder-db-<date>-named non-workspace resource today, which is why a bare coder- prefix was rejected in favor of coder-workspace-.
  • This PR's dry-run release pipeline run (release workflow, publish-template job) passed: it pushes this exact template against the real Coder deployment and Kubernetes API (redirected to a disposable test template per TESTING.md), confirming the new locals/resource are accepted by the live provider/schema — not just terraform validate.
  • Not verified pre-merge: that a real, started workspace pod ends up with workload="coder-workspace-<owner>-<name>". kubernetes_deployment_v1.deployment's count is gated on start_count, and a template push (what dry-run does) doesn't start a workspace, so the Deployment resource itself is never actually applied during CI — only planned/schema-validated. Confirming the live workload value requires either terraform apply (out of scope per instructions) or starting a real workspace from the merged template. What is verified: the naming expression by hand for realistic and worst-case (32/32-char) inputs, and the pre-change baseline (workload="coder-<uuid>" today, exact mechanism confirmed live).

Existing workspaces

Confirmed acceptable (per repo owner): applying this recreates existing workspace Deployments (pods restart anyway on apply; the PVC/home-volume data is unaffected since it's addressed by workspace name via sub_path, unchanged by this PR).

Interaction with PR #859

PR #859 (feat/memory-watchdog) is open and touches deployment.tf/configmap.tf/coder-agent.tf/etc., but its deployment.tf hunk only adds a new volume_mount block (for /memory-watchdog.sh) inside the workspace container — nowhere near the metadata.name (line 5) or hostname (previously line 156) lines this PR touches. No line-level overlap; expected to merge cleanly in either order. Not otherwise coordinated with that PR.

Docs updated

CLAUDE.md's "Implementation gotchas" and DESIGN.md's "Design tensions and decisions" both now explain that the Deployment name is a Prometheus identity, not just a Kubernetes identifier — so a future edit doesn't silently regress attribution back to an opaque name.

Related cleanup in a different repo (not part of this PR)

homelab-ops-kubernetes-apps's apps/subsystems/coder/prometheusrule-coder.yaml has a coder:coderd_pod recording rule that hand-rolls the same pod→ReplicaSet→Deployment join namespace_workload_pod:kube_pod_owner:relabel already does, and could collapse to a workload="coder" match. Checked all 15 open PRs in that repo — none touch that file, so this is not currently in flight elsewhere. Left undone here since it's a different repository and needs its own PR.

Testing

  • terraform fmt -check, terraform validate, tflint --config=../../../.tflint.hcl in the template dir: all pass.
  • pre-commit run --all-files: passes except one pre-existing, unrelated failure already present on main (hadolint SC3037/DL3066 on the Dockerfile) — not touched by this change.
  • Not exercised via test_mode/a live terraform apply — see "Live verification" above for what is and isn't proven pre-merge.

Branch base

#862 (the end-of-file-fixer fix this branch previously depended on) has landed on main. This branch is now rebased directly onto current main and carries none of that change itself.

@ppat
ppat force-pushed the feat/workload-rename branch from d6d7cd8 to d3a2ea3 Compare August 17, 2026 02:00
Name the workspace Deployment coder-workspace-<owner>-<workspace-name>
instead of coder-<workspace-uuid>. The cluster's Prometheus already
resolves pod -> ReplicaSet -> Deployment via a kube_pod_owner recording
rule and exposes it as a workload label, so this is what workspace
CPU/memory/PSI/OOM metrics get attributed to in Grafana/PromQL, at zero
added series and no join. Owner is included because workspace names are
unique per-owner, not cluster-wide. The coder-workspace- prefix (already
used as app.kubernetes.io/part-of) distinguishes workspaces from both the
coder control-plane Deployment and other coder-prefixed infra sharing the
namespace (verified live: a coder-db-<date> CloudNativePG cluster exists
today, so a bare coder- prefix is not sufficient).

Reuses the existing hostname sanitization idiom instead of a new one, and
documents the rationale in DESIGN.md/CLAUDE.md since the name is now
load-bearing for observability, not just a Kubernetes identifier.
@ppat
ppat force-pushed the feat/workload-rename branch from d3a2ea3 to bcbb61b Compare August 17, 2026 02:14
@ppat
ppat merged commit 281326a into main Aug 17, 2026
18 of 19 checks passed
@ppat
ppat deleted the feat/workload-rename branch August 17, 2026 02:26
@homelab-workflows-bot

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 2.25.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant