ManagedChart generation is X, but latest observed generation is Y
Article Number: 000021469
Environment
- Rancher v2.6 - v2.11
- A Rancher-managed RKE2 or K3s cluster
Situation
After upgrading the Kubernetes version of a downstream RKE2 cluster, an error message of the following format is displayed for the managed-system-upgrade-controller ManagedChart:
ManagedChart generation is X, but latest observed generation is Y
Cause
In Rancher v2.6 - v2.11, the Rancher-deployed system-upgrade-controller in Rancher-managed RKE2 and K3s clusters is managed via a ManagedChart resource. This issue occurs if a user deletes the managed-system-upgrade-controller Fleet bundle that is created by this ManagedChart.
Deleting the managed-system-upgrade-controller bundle for the cluster within the fleet-default workspace resets the observedGeneration of the managed-system-upgrade-controller ManagedChart.
This leads to a mismatch with the generation, and an error of the format "ManagedChart generation is X, but latest observed generation is Y".
Resolution
To resolve the issue:
- Delete the managed-system-upgrade-controller ManagedChart resource for the affected downstream cluster from the Rancher local cluster (replacing <cluster-name> with the downstream cluster name):
kubectl -n fleet-default delete ManagedChart <cluster-name>-managed-system-upgrade-controller