Remove referencingWorkloads/referenceCount from config CRD statuses

[ChrisJBurns]

Summary

Removes the status.referencingWorkloads and status.referenceCount fields (and the References printer column) from the six config CRDs — MCPOIDCConfig, MCPAuthzConfig, MCPExternalAuthConfig, MCPTelemetryConfig, MCPWebhookConfig, MCPToolConfig — and deletes the watch/rebuild machinery that maintained them.

After the field-indexer rollout (#5599) and stale-scan removal (#5626), that denormalized list was kept fresh only to populate a kubectl printer column. It no longer backs deletion protection: the finalizer recomputes referrers live (findReferencingWorkloads) before allowing deletion. So the stored list was pure overhead — every workload change reconciled the config purely to refresh a count — and this removes it.

Completes the cleanup tracked in #5607.

⚠️ Breaking change

status.referencingWorkloads and status.referenceCount are removed from all six config CRDs. Anyone relying on these status fields for automated workflows — scripts, GitOps assertions, dashboards, or the kubectl get "References" column — will no longer find them. There is no replacement field; if you need the set of referrers, list the workloads and filter by their config-ref. For example:

# workloads referencing an MCPOIDCConfig named "my-oidc" in namespace ns:
kubectl -n ns get mcpservers,mcpremoteproxies,virtualmcpservers -o json \
  | jq -r '.items[] | select(.spec.oidcConfigRef.name=="my-oidc" or .spec.incomingAuth.oidcConfigRef.name=="my-oidc") | "\(.kind)/\(.metadata.name)"'
``` **Deletion protection is unchanged** (a config still cannot be deleted while a workload references it; it sets the `DeletionBlocked` condition).

## Large PR Justification

Large, mostly-deletion refactor that must land atomically: removing a CRD status field requires the type change, the controller changes, regenerated CRD manifests/deepcopy/docs, and the test updates to all move together or nothing compiles. It is **net −3810 lines** (140 / 3950); the size is duplicated tracking code and tests being deleted across six controllers and four envtest suites at once, not added complexity. Splitting per-config-type would produce six interdependent PRs that each leave the `controllers` package uncompilable in isolation.

<details>
<summary><strong>What changed</strong></summary>

- **API types** (`api/v1beta1/*`, `api/v1alpha1/types.go`): dropped both fields + the `+kubebuilder:printcolumn` "References" markers; regenerated `zz_generated.deepcopy.go`. Kept the `WorkloadReference` type (still returned by the deletion-check).
- **Controllers**: removed the `Reconcile` status-population paths, the workload `Watches` + their map handlers (they existed only to refresh the list), and the `workloadReferenceCount` helper. **Kept** `findReferencingWorkloads` + the field indexes for the on-demand finalizer deletion check, and the `DeletionBlocked` condition/message.
- **Comments**: updated the now-inaccurate "ReferencingWorkloads is maintained by the config controller" notes in the MCPServer/VirtualMCPServer/MCPRemoteProxy controllers.
- **Tests**: deleted unit + envtest specs that asserted the removed fields (tracking/prune/watch-handler); retained deletion-protection specs, rewritten to assert via the `DeletionBlocked` condition + object existence.
- **Generated**: regenerated the six config CRDs (`deploy/charts/operator-crds/...`) and `docs/operator/crd-api.md`.

</details>

## Type of change

- [ ] Bug fix
- [ ] New feature
- [x] Breaking change
- [x] Refactoring
- [ ] Documentation

## Test plan

- [x] `go build ./cmd/thv-operator/...` passes
- [x] `task operator-test` (full operator unit suite) passes
- [x] Config integration suites pass at the spec level, run individually under envtest: MCPOIDCConfig 29/29, MCPToolConfig, MCPExternalAuth, MCPTelemetryConfig 5/5 (the only failures were the known envtest `kube-apiserver` AfterSuite teardown flake under parallel load — 0 spec failures; clean on isolated re-run)
- [x] `golangci-lint` clean on changed packages (only the pre-existing `mcpserver_controller.go:1645` finding remains)
- [x] CRD manifests / deepcopy / `crd-api.md` regenerated and contain no `referencingWorkloads`/`referenceCount`/`References`

## Does this introduce a user-facing change?

Yes — see the breaking-change note above: two status fields and the `References` printer column are removed from the six config CRDs.

Generated with [Claude Code](https://claude.com/claude-code)


## Review response

A multi-agent review identified five findings, all addressed in follow-up commits (see the PR comment for detail):

- **`WorkloadRefsEqual` dead code (MEDIUM)** — deleted the helper + its test; it was the orphaned "did the ref list change?" guard whose callers this PR removed (symmetric with the already-deleted `workloadReferenceCount`).
- **Stale arch docs (MEDIUM)** — updated `docs/arch/09-operator-architecture.md` (3 sentences) to drop `referencingWorkloads` and note referrers are recomputed on demand at deletion.
- **No block→unblock envtest for ExternalAuth (LOW)** — added a create-ref → blocked (`DeletionBlocked` condition) → deref → deleted transition spec to the ExternalAuth suite (35/35 green; 45s timeout to cover the 30s-requeue-driven unblock). MCPAuthzConfig has no integration suite — its `handleDeletion` is unit-covered and relies on shared framework behavior proven by other suites.
- **Empty `Context` blocks (LOW)** — verified non-issue: the prior commit removed whole Contexts cleanly.
- **Lost `kubectl` "References" column (LOW)** — accepted (restoring even a count requires the full removed watch apparatus); field-filter migration example added to the description above.

## State for the next session

- Branch: `chris/drop-referencing-workloads-status` (6 commits, all pushed)
- CI is running; the envtest `kube-apiserver` AfterSuite teardown flake may surface — all four config suites pass with 0 failed specs when run individually
- Deletion protection works; the unblock now rides the controller's 30s requeue (proven by the ExternalAuth envtest spec)
- No known outstanding issues; ready for re-review

[codecov]

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 70.32%. Comparing base (e7934ea) to head (b837630).

Files with missing lines	Patch %	Lines
...perator/controllers/mcpwebhookconfig_controller.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5631      +/-   ##
==========================================
+ Coverage   70.27%   70.32%   +0.04%     
==========================================
  Files         648      647       -1     
  Lines       66011    65738     -273     
==========================================
- Hits        46391    46231     -160     
+ Misses      16279    16159     -120     
- Partials     3341     3348       +7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ChrisJBurns]

Addressed the review findings (commits on top of the original three):

1. WorkloadRefsEqual dead code (MEDIUM) — removed the helper + its test; it was the orphaned guard whose callers this PR deleted (symmetric with the already-removed workloadReferenceCount).
2. Stale arch docs (MEDIUM) — updated the three docs/arch/09-operator-architecture.md references to drop referencingWorkloads and note referrers are recomputed on demand at deletion (finalizer + field-indexed lookup).
3. Missing block→unblock envtest (LOW) — added a create-ref → deletion-blocked (DeletionBlocked condition) → remove-ref → deleted transition spec to the MCPExternalAuthConfig suite (passes 35/35; allows up to 45s for the now-30s-requeue-driven unblock). Skipped MCPAuthzConfig — it has no integration suite and standing one up for this shared behavior is disproportionate; its handleDeletion blocking is unit-tested.
4. Empty Context blocks (LOW) — verified non-issue: the earlier commit removed whole Contexts (with setup), leaving no orphaned empty containers.
5. Lost kubectl "References" column (LOW) — accepted (keeping even a count would require restoring the removed watch apparatus); added a concrete kubectl ... | jq field-filter alternative to the PR description.

[JAORMX]

Re-reviewed the current head (b837630) since my July approval. The core change is sound — I verified the invariant holds across all six controllers: handleDeletion still recomputes referrers live via the field indexers, fails closed on query error, blocks with DeletionBlocked=True + 30s requeue, and only removes the finalizer at zero refs. The removed Watches(...) calls were purely status-list maintenance; the indexers survive and are still queried; RBAC markers unchanged; CRD YAMLs (both files/crds and templates) and docs/operator/crd-api.md are consistently regenerated; existing objects with populated status fields drop them silently on read (fields were optional, conversion None, v1alpha1 embeds v1beta1 status). Net −3688 lines of genuinely dead machinery.

Two things to address before merge:

1. Stale chainsaw test asserts a removed status field (will fail e2e).
test/e2e/chainsaw/operator/single-tenancy/test-scenarios/mcpwebhookconfig/chainsaw-test.yaml:127-129 still asserts:

status:
  referencingWorkloads:
    - kind: MCPServer
      name: missing-webhook-server

v1alpha1 MCPWebhookConfig embeds v1beta1.MCPWebhookConfigStatus, from which this field was removed — this scenario will fail against the new schema. It's the only remaining reference in test/.

2. api-break-allowed label not applied.
This PR removes fields from served v1beta1 CRD status — exactly what CONTRIBUTING.md's API-stability section prohibits without the escape hatch, and the api-compat CI workflow will fail without the api-break-allowed label. The PR body already carries the required break + migration notes, so it just needs the label (and the jq replacement one-liner would be worth preserving in docs/operator/ — it'll be lost once this merges, since there's no upgrade-guide entry for it).

Non-blocking observations from the panel:

• Pre-existing gap worth a follow-up issue: MCPExternalAuthConfigReconciler.findReferencingWorkloads only checks MCPServer + MCPRemoteProxy — a VirtualMCPServer referencing an MCPExternalAuthConfig via outgoingAuth.backends[].externalAuthConfigRef isn't counted, so the finalizer won't block that deletion. Pre-existing on main, not a regression here, but this PR touches every findReferencingWorkloads so it's the right moment to notice.
• Residual watch RBAC verbs on workload CRDs for the config controllers that no longer watch anything — cosmetic, could be dropped in a follow-up.
• DeletionBlocked clearing is asymmetric: the webhook controller clears it on unblock, the other five don't (object vanishes with a lingering True condition). Inherited, low-impact.
• Behavior note: unblock is now up to ~30s slower (requeue-driven rather than watch-driven) — the new integration test already accounts for this with the 45s timeout. Reasonable trade for the deleted machinery.

[ChrisJBurns]

Not sure whether to close this or leave open. The fix is sound but it is a breaking change on the status. objects and given we're in beta now, I'm hesitant on making breaking changes - unless @JAORMX you think this should be fine? I accept I may be a bit too cautious here so happy to fix this up and merge if you think its not the end of the world (given its only a status. object breakage)

[JAORMX]

I think we can still fix this @ChrisJBurns