HPA shown as "Missing" in Rancher UI due to API version mismatch
Article Number: 000022208
Environment
Rancher Prime
Kubernetes 1.23+
Situation
In the Apps interface in the Rancher UI, Horizontal Pod Autoscalers (HPAs) for some installed apps display a "Missing" state, even though the HPAs are active and functioning correctly.
Cause
In Kubernetes 1.23 and later, the autoscaling API group has a preferred version of v2.
When a autoscaling resource (such as a HPA) is deployed using the autoscaling/v1 API, the Kubernetes API server accepts the request and silently migrates this resource to autoscaling/v2.
Rancher tracks resources associated with an App using the app.catalog.cattle.io custom resource. Because the Helm chart specified v1 during deployment, the app.catalog.cattle.io object records the HPA resource version as autoscaling/v1.
Rancher's cluster cache in newer versions watches for autoscaling/v2 resources, due to the preferred apiVersion. The UI attempts to look up the autoscaling/v1 object defined in the App metadata, but since the cache contains only autoscaling/v2 objects, the lookup fails, resulting in the "Missing" status.
Resolution
Temporary workaround
- Edit the
app.catalog.cattle.ioobject and setapiVersionfor the HPA underspec.resourcestoautoscaling/v2:
kubectl edit app.catalog.cattle.io <app-name> -n <namespace>
- Locate the
spec.resourcessection. Find the entry corresponding to the HorizontalPodAutoscaler
- apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
name: <hpa-name>
namespace: <namespace>
- Change the
apiVersiontoautoscaling/v2
- apiVersion: autoscaling/v2 # Updated from v1
kind: HorizontalPodAutoscaler
name: <hpa-name>
namespace: <namespace>
- Save and exit the editor
- Refresh the Rancher UI
The HPA should now transition to an "Active" state in the Apps view.
Note: This change will be reverted if the Helm release is upgraded or rolled back
Permanent Solution
The underlying Helm chart must be updated by the chart maintainer.
Update the Helm chart templates to use autoscaling/v2 for HorizontalPodAutoscaler resources.
Publish the updated chart.
Upgrade the App in Rancher using the new chart version, this ensures the app.catalog.cattle.io resource is created with the correct v2 metadata automatically.