Skip to content

Incorrect kubectl image used by system-upgrade-controller in Rancher Prime v2.13.1

Article Number: 000022284

Environment

  • Rancher Prime v2.13.1
  • A Rancher-provisioned or imported RKE2 or K3s cluster

Situation

The System-Upgrade-Controller (SUC) uses the wrong kubectl container image while applying plans to a downstream RKE2 or K3s cluster. This causes the apply-job pod launched by the SUC to fail to initialize with the following error:

ImagePullBackOff (Back-off pulling image "registry.rancher.com/rancher/kubectl:v6.0.0": ErrImagePull: rpc error: code = NotFound desc = failed to pull and unpack image "registry.rancher.com/rancher/kubectl:v6.0.0": failed to resolve reference "registry.rancher.com/rancher/kubectl:v6.0.0": registry.rancher.com/rancher/kubectl:v6.0.0: not found)

Cause

The rancher/kuberlr-kubectl image has been introduced to replace rancher/kubectl in the SUC; however, the ConfigMap has not been updated accordingly.

Resolution

The ConfigMap associated with the system-upgrade-controller deployment must be patched to reference the correct image.

On the downstream cluster:

kubectl patch configmap -n cattle-system system-upgrade-controller-config --type='json' -p '[{"op": "replace", "path": "/data/SYSTEM_UPGRADE_JOB_KUBECTL_IMAGE", "value": "registry.rancher.com/rancher/kuberlr-kubectl:v6.0.0"}]'

Then, perform a rollout restart of the deployment for the changes to take effect:

kubectl rollout restart deployment system-upgrade-controller -n cattle-system