From 67a4fbeeca37cd8285cf968d2226e3e891157e2e Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Wed, 23 Sep 2026 00:07:08 +0300 Subject: [PATCH 1/4] fix(operations): replace the component removal recipe that undoes itself Removing an installed component was documented as `kubectl delete hr -n `. The Package that emitted the release owns it through a controller ownerReference, and the operator watches HelmReleases with Owns(), so the delete fires the reconcile that recreates it. The command reports success, nothing logs an error, and the component is running again afterwards. That is not the harmless no-op it looks like. The operator sets no uninstall policy on the HelmReleases it builds, so Flux's default applies and an unsuspended release is uninstalled when the object goes away. Anything the uninstall destroys is gone before the reinstall brings the workloads back, and the restored release hides that it happened. The page now says so, for the benefit of anyone who already ran the old recipe. Deleting the cluster-scoped Package is what cascades into the uninstall deliberately. It needs the disabledPackages entry to land first, or the next platform upgrade renders the Package again and the removal undoes itself one level up. The reader confirms that by reading the disabledPackages list back off the platform HelmRelease, which is the only observation here that cannot go green early. A condition wait returns instantly against the Ready left standing by the previous reconcile; re-reading the Package after deleting it returns NotFound unconditionally; and comparing observedGeneration against generation on that HelmRelease is equal in exactly the window that matters, because the operator has not yet copied the edit across from the Package the reader changed. What the uninstall takes is stated above the command rather than after it, in a warning callout. Anything the chart rendered as an ordinary template and did not annotate resource-policy: keep goes with the release, CRDs and namespaces alike, and the apiserver cascades every custom resource of those CRD kinds. Removing cozystack.metallb takes the MetalLB CRDs and the objects of those kinds cluster-wide; removing cozystack.cozystack-basics takes the cozy-public namespace, which that chart renders as a plain template and which is not its install namespace. The one exception is the namespace a component installs into: the operator applies that itself, outside the release, so the uninstall never had it. The v1.1 and v1.2 pages qualify the step count in place, next to the instruction it governs, because the behaviour changed inside those lines rather than between them. The keep annotation on Packages arrived in v1.1.5 and v1.2.1 as backports, so v1.1.0 through v1.1.4 and v1.2.0 have no annotation and adding the name to disabledPackages removes the component by itself, putting the destruction on that step instead of the delete. The clauses name their range rather than saying "and earlier", which would be false across lines: v1.1.5 through v1.1.7 are lower-versioned than v1.2.0 and do carry the annotation. The same pages offered `kubectl get package` as the way to find the names for both lists, and it answers for one. An optional package is rendered under `and (has $name $enabled) (not (has $name $disabled))`, so it has no Package object until its name is already in enabledPackages. PackageSources are rendered unconditionally from sources/*.yaml, so `kubectl get packagesource` answers for both. Applied to next and v1.1 through v1.6. Left out v1.0, whose line never received the annotation at all, so that page needs the single-step correction rather than this one. v0 predates Package objects: its platform chart renders HelmReleases directly, so deleting the HelmRelease there names the object the chart creates. The v1.1 disabledPackages examples listed bare names, which match nothing against the fully-qualified names the platform chart feeds the helper. Both copies are fixed: the one in the components page, and the one in the Hetzner install guide that page links to as its worked example. Leaving the second would have had v1.1 teach the qualified form on one page and hand out the broken form on the page it cites, on the one provider where swapping MetalLB for RobotLB is mandatory. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- .../operations/configuration/components.md | 27 +++++++++++----- .../en/docs/v1.1/install/providers/hetzner.md | 4 +-- .../operations/configuration/components.md | 30 +++++++++++++++--- .../operations/configuration/components.md | 31 ++++++++++++++----- .../operations/configuration/components.md | 27 +++++++++++----- .../operations/configuration/components.md | 27 +++++++++++----- .../operations/configuration/components.md | 27 +++++++++++----- .../operations/configuration/components.md | 27 +++++++++++----- 8 files changed, 150 insertions(+), 50 deletions(-) diff --git a/content/en/docs/next/operations/configuration/components.md b/content/en/docs/next/operations/configuration/components.md index 53c04ea7..00d06444 100644 --- a/content/en/docs/next/operations/configuration/components.md +++ b/content/en/docs/next/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/next/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -70,8 +66,25 @@ spec: Disabling components must be done before installing Cozystack. Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Removing one that is already installed takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.1/install/providers/hetzner.md b/content/en/docs/v1.1/install/providers/hetzner.md index ded13a7b..17948983 100644 --- a/content/en/docs/v1.1/install/providers/hetzner.md +++ b/content/en/docs/v1.1/install/providers/hetzner.md @@ -344,9 +344,9 @@ The final stage of deploying a Cozystack cluster on Hetzner is to install Cozyst values: bundles: disabledPackages: - - metallb + - cozystack.metallb enabledPackages: - - hetzner-robotlb + - cozystack.hetzner-robotlb publishing: host: "example.org" apiServerEndpoint: "https://api.example.org:443" diff --git a/content/en/docs/v1.1/operations/configuration/components.md b/content/en/docs/v1.1/operations/configuration/components.md index 88bc17b7..5c5682c4 100644 --- a/content/en/docs/v1.1/operations/configuration/components.md +++ b/content/en/docs/v1.1/operations/configuration/components.md @@ -41,6 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix — run `kubectl get packagesource` to see the exact names on your cluster. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.1/install/providers/hetzner" %}}) requires swapping default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -56,16 +57,35 @@ spec: values: bundles: disabledPackages: - - metallb + - cozystack.metallb enabledPackages: - - hetzner-robotlb + - cozystack.hetzner-robotlb # rest of the config ``` Disabling components must be done before installing Cozystack. -Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +From v1.1.5 onward, applying updated configuration with `disabledPackages` will not remove components that are already installed. +On v1.1.0 through v1.1.4 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. + +From v1.1.5 onward, removing an installed component takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.2/operations/configuration/components.md b/content/en/docs/v1.2/operations/configuration/components.md index 3cdc9b9f..a9c0041e 100644 --- a/content/en/docs/v1.2/operations/configuration/components.md +++ b/content/en/docs/v1.2/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.2/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -69,9 +65,28 @@ spec: ``` Disabling components must be done before installing Cozystack. -Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +From v1.2.1 onward, applying updated configuration with `disabledPackages` will not remove components that are already installed. +On v1.2.0 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. + +From v1.2.1 onward, removing an installed component takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.3/operations/configuration/components.md b/content/en/docs/v1.3/operations/configuration/components.md index c9fb3eb3..3754b78d 100644 --- a/content/en/docs/v1.3/operations/configuration/components.md +++ b/content/en/docs/v1.3/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.3/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -70,8 +66,25 @@ spec: Disabling components must be done before installing Cozystack. Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Removing one that is already installed takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.4/operations/configuration/components.md b/content/en/docs/v1.4/operations/configuration/components.md index f3ce3c92..18036347 100644 --- a/content/en/docs/v1.4/operations/configuration/components.md +++ b/content/en/docs/v1.4/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.4/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -70,8 +66,25 @@ spec: Disabling components must be done before installing Cozystack. Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Removing one that is already installed takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.5/operations/configuration/components.md b/content/en/docs/v1.5/operations/configuration/components.md index 3d799682..e04ccaab 100644 --- a/content/en/docs/v1.5/operations/configuration/components.md +++ b/content/en/docs/v1.5/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.5/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -70,8 +66,25 @@ spec: Disabling components must be done before installing Cozystack. Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Removing one that is already installed takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.6/operations/configuration/components.md b/content/en/docs/v1.6/operations/configuration/components.md index 7224274c..9b431562 100644 --- a/content/en/docs/v1.6/operations/configuration/components.md +++ b/content/en/docs/v1.6/operations/configuration/components.md @@ -41,11 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. -Every entry in those lists is a fully-qualified Package name — the same name you see with -`kubectl get package`. All platform packages live under the `cozystack.` prefix (for example, -`cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run -`kubectl get package` to see the exact names available on your cluster before editing -the Platform Package. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix (for example, `cozystack.metallb`, `cozystack.hetzner-robotlb`, `cozystack.nfs-driver`). Run `kubectl get packagesource` to see the exact names available on your cluster before editing the Platform Package. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.6/install/providers/hetzner" %}}) requires swapping the default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -70,8 +66,25 @@ spec: Disabling components must be done before installing Cozystack. Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Removing one that is already installed takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: ```bash -kubectl delete hr -n +kubectl get helmrelease cozystack-platform --namespace cozy-system \ + --output jsonpath='{.spec.values.bundles.disabledPackages}' ``` + +Then delete the Package object. + +{{% alert title="Warning" color="warning" %}} +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +```bash +kubectl delete package.cozystack.io +``` + +Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. + +`kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. From 34825aaaf1e3d2bb98899888564583bff6a3818f Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Wed, 23 Sep 2026 00:07:12 +0300 Subject: [PATCH 2/4] docs(operations): widen the uninstall warning and give it a done signal The named CRD kinds in the warning cover the top-level MetalLB chart only; a subchart such as frr-k8s brings its own group, so the list undercounts what an uninstall takes. Point at the chart instead of enumerating kinds, which also stops the list needing maintenance per component. Package carries no finalizer, so deleting it returns before the Helm uninstall that follows it has done anything. Add the wait that tells an operator the destructive part is over. That wait needs a HelmRelease name and a namespace, and neither follows from the Package name: both come from the PackageSource install block, and one Package can render several releases. The operator labels every HelmRelease it builds with the Package that owns it, so listing by that label before the delete produces the values and enumerates the releases. The same listing prints spec.suspend, because helm-controller skips the uninstall for a suspended HelmRelease and only drops its finalizer, which would leave the component's objects behind with nothing managing them. `kubectl wait --for=delete` also exits 0 for a name that never existed, so the page says to take the values from the listing rather than guess them. Through v1.4 the cozystack-basics chart also renders the tenant-root namespace as a plain template with no keep annotation, so uninstalling it takes the whole root tenant, not just cozy-public. From v1.5 that namespace carries the no-delete label the platform's admission policy enforces, so the later pages keep the shorter example. The annotation is not the whole survival rule on those pages either: the policy denies DELETE on any labelled object, so the v1.5, v1.6 and next warnings name the label beside the annotation and give the way to drop it. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- .../operations/configuration/components.md | 19 ++++++++++++++++- .../operations/configuration/components.md | 21 +++++++++++++++++-- .../operations/configuration/components.md | 21 +++++++++++++++++-- .../operations/configuration/components.md | 19 ++++++++++++++++- .../operations/configuration/components.md | 19 ++++++++++++++++- .../operations/configuration/components.md | 19 ++++++++++++++++- .../operations/configuration/components.md | 19 ++++++++++++++++- 7 files changed, 128 insertions(+), 9 deletions(-) diff --git a/content/en/docs/next/operations/configuration/components.md b/content/en/docs/next/operations/configuration/components.md index 00d06444..80ba830b 100644 --- a/content/en/docs/next/operations/configuration/components.md +++ b/content/en/docs/next/operations/configuration/components.md @@ -76,15 +76,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. The annotation is not the only thing that keeps an object alive. From v1.5.0 the platform installs a ValidatingAdmissionPolicy that denies DELETE on anything labelled `platform.cozystack.io/no-delete: "true"`, and several component charts render objects that carry it; `kubectl get --all-namespaces --selector platform.cozystack.io/no-delete=true` lists them for a given kind. One denial fails the whole uninstall: Helm deletes what it can and then errors out, and the controller keeps its finalizer and retries, so the HelmRelease sits in deletion and the wait below runs to its timeout. Taking the label off hands the object to the uninstall, which is the whole point of the guard, and it has to come off every labelled object in the release: the `cert-manager-issuers` release of `cozystack.cert-manager` labels three ClusterIssuers, and unlabelling one of them still leaves the other two to fail the uninstall. The command is `kubectl label --namespace platform.cozystack.io/no-delete-`, without `--namespace` for cluster-scoped kinds. Weigh what that costs before doing it: `cozystack.cozystack-basics` labels two objects, the `tenant-root` Namespace and the `tenant-root` HelmRelease, and only the Namespace is one the uninstall would delete, since the HelmRelease also carries the keep annotation. Unlabelling that Namespace means the uninstall takes the root tenant and every application in it. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.1/operations/configuration/components.md b/content/en/docs/v1.1/operations/configuration/components.md index 5c5682c4..55d0b121 100644 --- a/content/en/docs/v1.1/operations/configuration/components.md +++ b/content/en/docs/v1.1/operations/configuration/components.md @@ -65,7 +65,7 @@ spec: Disabling components must be done before installing Cozystack. From v1.1.5 onward, applying updated configuration with `disabledPackages` will not remove components that are already installed. -On v1.1.0 through v1.1.4 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. +On v1.1.0 through v1.1.4 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. Take the listing below before making the edit and wait on the same releases afterwards: the Package its selector needs goes away with the component. From v1.1.5 onward, removing an installed component takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: @@ -77,15 +77,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` and `tenant-root` namespaces and everything stored in them, which is every application in the root tenant. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.2/operations/configuration/components.md b/content/en/docs/v1.2/operations/configuration/components.md index a9c0041e..5a6a4e4d 100644 --- a/content/en/docs/v1.2/operations/configuration/components.md +++ b/content/en/docs/v1.2/operations/configuration/components.md @@ -66,7 +66,7 @@ spec: Disabling components must be done before installing Cozystack. From v1.2.1 onward, applying updated configuration with `disabledPackages` will not remove components that are already installed. -On v1.2.0 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. +On v1.2.0 the platform does not annotate the Package with `helm.sh/resource-policy: keep`, so adding the name to `disabledPackages` does remove an installed component: the destruction described below happens at that point, and there is no second command to run. Back up anything you still need before making that edit. Take the listing below before making the edit and wait on the same releases afterwards: the Package its selector needs goes away with the component. From v1.2.1 onward, removing an installed component takes two steps. Add its name to `disabledPackages` in the Platform Package above, then wait for the operator to carry that edit across. The name appears in this output once it has: @@ -78,15 +78,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` and `tenant-root` namespaces and everything stored in them, which is every application in the root tenant. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.3/operations/configuration/components.md b/content/en/docs/v1.3/operations/configuration/components.md index 3754b78d..a6875999 100644 --- a/content/en/docs/v1.3/operations/configuration/components.md +++ b/content/en/docs/v1.3/operations/configuration/components.md @@ -76,15 +76,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` and `tenant-root` namespaces and everything stored in them, which is every application in the root tenant. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.4/operations/configuration/components.md b/content/en/docs/v1.4/operations/configuration/components.md index 18036347..8cebed47 100644 --- a/content/en/docs/v1.4/operations/configuration/components.md +++ b/content/en/docs/v1.4/operations/configuration/components.md @@ -76,15 +76,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` and `tenant-root` namespaces and everything stored in them, which is every application in the root tenant. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.5/operations/configuration/components.md b/content/en/docs/v1.5/operations/configuration/components.md index e04ccaab..cf9e12be 100644 --- a/content/en/docs/v1.5/operations/configuration/components.md +++ b/content/en/docs/v1.5/operations/configuration/components.md @@ -76,15 +76,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. The annotation is not the only thing that keeps an object alive. From v1.5.0 the platform installs a ValidatingAdmissionPolicy that denies DELETE on anything labelled `platform.cozystack.io/no-delete: "true"`, and several component charts render objects that carry it; `kubectl get --all-namespaces --selector platform.cozystack.io/no-delete=true` lists them for a given kind. One denial fails the whole uninstall: Helm deletes what it can and then errors out, and the controller keeps its finalizer and retries, so the HelmRelease sits in deletion and the wait below runs to its timeout. Taking the label off hands the object to the uninstall, which is the whole point of the guard, and it has to come off every labelled object in the release: the `cert-manager-issuers` release of `cozystack.cert-manager` labels three ClusterIssuers, and unlabelling one of them still leaves the other two to fail the uninstall. The command is `kubectl label --namespace platform.cozystack.io/no-delete-`, without `--namespace` for cluster-scoped kinds. Weigh what that costs before doing it: `cozystack.cozystack-basics` labels two objects, the `tenant-root` Namespace and the `tenant-root` HelmRelease, and only the Namespace is one the uninstall would delete, since the HelmRelease also carries the keep annotation. Unlabelling that Namespace means the uninstall takes the root tenant and every application in it. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. diff --git a/content/en/docs/v1.6/operations/configuration/components.md b/content/en/docs/v1.6/operations/configuration/components.md index 9b431562..b05286bb 100644 --- a/content/en/docs/v1.6/operations/configuration/components.md +++ b/content/en/docs/v1.6/operations/configuration/components.md @@ -76,15 +76,32 @@ kubectl get helmrelease cozystack-platform --namespace cozy-system \ Then delete the Package object. {{% alert title="Warning" color="warning" %}} -Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes the MetalLB CRDs and with them every IPAddressPool, L2Advertisement, BGPPeer and the rest of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. +Deleting the Package uninstalls the component's Helm release, and that destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. The annotation is not the only thing that keeps an object alive. From v1.5.0 the platform installs a ValidatingAdmissionPolicy that denies DELETE on anything labelled `platform.cozystack.io/no-delete: "true"`, and several component charts render objects that carry it; `kubectl get --all-namespaces --selector platform.cozystack.io/no-delete=true` lists them for a given kind. One denial fails the whole uninstall: Helm deletes what it can and then errors out, and the controller keeps its finalizer and retries, so the HelmRelease sits in deletion and the wait below runs to its timeout. Taking the label off hands the object to the uninstall, which is the whole point of the guard, and it has to come off every labelled object in the release: the `cert-manager-issuers` release of `cozystack.cert-manager` labels three ClusterIssuers, and unlabelling one of them still leaves the other two to fail the uninstall. The command is `kubectl label --namespace platform.cozystack.io/no-delete-`, without `--namespace` for cluster-scoped kinds. Weigh what that costs before doing it: `cozystack.cozystack-basics` labels two objects, the `tenant-root` Namespace and the `tenant-root` HelmRelease, and only the Namespace is one the uninstall would delete, since the HelmRelease also carries the keep annotation. Unlabelling that Namespace means the uninstall takes the root tenant and every application in it. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` namespace and everything stored in it. Back up anything you still need first. {{% /alert %}} The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. +List the releases the Package owns before deleting it. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: + +```bash +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' +``` + +Clear `spec.suspend` on any release that shows `true` before going on. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + ```bash kubectl delete package.cozystack.io ``` +Nothing holds a finalizer on the Package, so this command returns as soon as the object is gone and the uninstall it triggers runs afterwards. Wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + Deleting the Package while the platform values still render it means the next platform upgrade brings it back, undoing the removal one level up. Nothing reports this at the time: the delete succeeds either way and the Package reappears whenever that upgrade happens to run. `kubectl delete hr` is not a lighter-weight version of this. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. From 0e431ebd91a21ca6bf6d3e97d893935932a2faf3 Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Wed, 23 Sep 2026 00:07:12 +0300 Subject: [PATCH 3/4] docs(operations): correct the component removal path on v1.0 v1.0 documented `kubectl delete hr` as the way to remove an installed component, with no warning about what an uninstall takes with it. That command destroys the CRDs and every custom resource of those kinds, then the Package recreates the HelmRelease and the chart reinstalls, so the workloads come back and the data does not. v1.0 needs a different correction from later versions rather than a copy of it: the platform does not annotate rendered Packages with `helm.sh/resource-policy: keep`, so `disabledPackages` alone already removes an installed component there, in one step and with no confirmation. Because the removal starts when the operator picks that edit up, the completion signal has to be set up before the edit: the label selector that finds the component's HelmReleases needs the Package name, and the Package goes away with the component. The page puts the listing ahead of the edit for that reason, and keeps the same suspend check and delete wait the later versions use. Also qualify the package names in the Hetzner and components examples. `disabledPackages` and `enabledPackages` are matched against the full `cozystack.` form, so the bare names did nothing. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- .../en/docs/v1.0/install/providers/hetzner.md | 4 +-- .../operations/configuration/components.md | 34 +++++++++++++++---- 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/content/en/docs/v1.0/install/providers/hetzner.md b/content/en/docs/v1.0/install/providers/hetzner.md index 22eca2db..4cfe2ace 100644 --- a/content/en/docs/v1.0/install/providers/hetzner.md +++ b/content/en/docs/v1.0/install/providers/hetzner.md @@ -344,9 +344,9 @@ The final stage of deploying a Cozystack cluster on Hetzner is to install Cozyst values: bundles: disabledPackages: - - metallb + - cozystack.metallb enabledPackages: - - hetzner-robotlb + - cozystack.hetzner-robotlb publishing: host: "example.org" apiServerEndpoint: "https://api.example.org:443" diff --git a/content/en/docs/v1.0/operations/configuration/components.md b/content/en/docs/v1.0/operations/configuration/components.md index 1f04669a..03aa0963 100644 --- a/content/en/docs/v1.0/operations/configuration/components.md +++ b/content/en/docs/v1.0/operations/configuration/components.md @@ -41,6 +41,7 @@ Bundles have optional components that need to be explicitly enabled (included) i Regular bundle components can, on the other hand, be disabled (excluded) from the installation, when you don't need them. Use `bundles.enabledPackages` and `bundles.disabledPackages` in the Platform Package values. +Every entry in those lists is a fully-qualified name under the `cozystack.` prefix — run `kubectl get packagesource` to see the exact names on your cluster. `kubectl get package` answers only for `disabledPackages`, because an optional component has no Package object until its name is already in `enabledPackages`. For example, [installing Cozystack in Hetzner]({{% ref "/docs/v1.0/install/providers/hetzner" %}}) requires swapping default load balancer, MetalLB, with one made specifically for Hetzner, called RobotLB: @@ -56,16 +57,37 @@ spec: values: bundles: disabledPackages: - - metallb + - cozystack.metallb enabledPackages: - - hetzner-robotlb + - cozystack.hetzner-robotlb # rest of the config ``` -Disabling components must be done before installing Cozystack. -Applying updated configuration with `disabledPackages` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +Disabling a component before installing Cozystack keeps it out of the installation entirely. + +On v1.0 the platform does not annotate the Packages it renders with `helm.sh/resource-policy: keep`, so adding a name to `disabledPackages` also removes the component when it is already installed. The next platform reconcile drops the Package, the operator's ownerReference takes the component's HelmRelease with it, and Flux uninstalls the release. There is no second command and no confirmation step, so back up anything you still need before making that edit. + +{{% alert title="Warning" color="warning" %}} +Uninstalling the component's Helm release destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `cozystack.metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide; removing `cozystack.cozystack-basics` takes the `cozy-public` and `tenant-root` namespaces and everything stored in them, which is every application in the root tenant. Back up anything you still need first. +{{% /alert %}} + +The namespace a component installs into is the exception: the operator applies that one itself, outside the component's release and with no ownerReference, so the uninstall never had it to remove. + +The removal starts when the operator picks the edit up, so take the component's releases first — the Package the selector needs goes away with the component. The operator labels every HelmRelease it renders with the name of the Package that produced it, and one Package can own several: ```bash -kubectl delete hr -n +kubectl get helmrelease --all-namespaces --selector cozystack.io/package= \ + --output custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name,SUSPENDED:.spec.suspend' ``` + +Clear `spec.suspend` on any release that shows `true` before making the edit. Flux skips the uninstall for a suspended HelmRelease and only drops its own finalizer, so that release disappears with everything it installed left behind and nothing left managing it. + +Then wait on each release from the listing to know the destructive part has finished: + +```bash +kubectl wait --for=delete helmrelease/ --namespace --timeout=10m +``` + +`kubectl wait --for=delete` exits 0 for a name that was never there, silently and with nothing to tell it apart from a deletion it watched, so take both values from the listing rather than guessing them. A returned wait says the HelmRelease is gone, not that the uninstall ran. + +`kubectl delete hr` is not a lighter-weight way to do the same thing. Flux uninstalls the release when an unsuspended HelmRelease goes away, so it destroys the same CRDs and custom resources, and then the Package recreates the HelmRelease and the chart reinstalls. The workloads come back, the custom resources do not. If you have run it before, those custom resources are already gone and have to be recreated from your own manifests or a backup. From f621e84d766500e3c2dd394144c6c679c593c58d Mon Sep 17 00:00:00 2001 From: Aleksei Sviridkin Date: Tue, 22 Sep 2026 23:59:23 +0300 Subject: [PATCH 4/4] docs(operations): warn what removing a v0 component destroys The v0 removal recipe deletes the component's Helm release and says nothing about what the uninstall takes with it. The MetalLB chart renders its CRDs as ordinary subchart templates with no helm.sh/resource-policy annotation on every 0.x line, so the uninstall drops them and the apiserver cascades every custom resource of those kinds cluster-wide. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin --- content/en/docs/v0/operations/configuration/components.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/content/en/docs/v0/operations/configuration/components.md b/content/en/docs/v0/operations/configuration/components.md index 1ed47bb5..33757643 100644 --- a/content/en/docs/v0/operations/configuration/components.md +++ b/content/en/docs/v0/operations/configuration/components.md @@ -60,7 +60,11 @@ data: Disabling components must be done before installing Cozystack. Applying updated configuration with `bundle-disable` will not remove components that are already installed. -To remove already installed components, delete the Helm release manually using this command: +To remove already installed components, delete the Helm release manually. + +{{% alert title="Warning" color="warning" %}} +Uninstalling the component's Helm release destroys more than the workloads. Anything the chart rendered as an ordinary template without `helm.sh/resource-policy: keep` goes with the release, CRDs and namespaces included, and Kubernetes deletes every custom resource of those CRD kinds along with them. Removing `metallb` takes every CRD the MetalLB chart bundles, subcharts included, and with them every custom resource of those kinds cluster-wide. Back up anything you still need first. +{{% /alert %}} ```bash kubectl delete hr -n