Recommendation against the use of the --atomic flag for Rancher helm chart upgrades

Article Number: 000021304

Environment

A Rancher v2.x instance installed via Helm chart

Situation

Rancher upgrades using the Helm CLI

Cause

The helm --atomic flag, causes helm to rollback any changes made in case of a failed helm chart upgrade.

Hypothetically, if the Rancher Deployment portion of the upgrade succeeds and new Rancher Pods start running, and making configuration state changes to the Rancher custom resources, but the helm upgrade then fails on another resource within the chart for any reason, a helm rollback would be automatically performed if the --atomic flag is used.

Rancher Helm chart version rollbacks, without a restore of Rancher state, are not supported and this may result in a broken state, with partially upgraded Rancher state, but with the older Rancher version running. 

For Rancher rollbacks in the case of a failed Rancher upgrade you should refer to the Rancher rollback documentation, using the Rancher backup operator, or a cluster-level restore, using the appropriate restore method for your local Rancher cluster (per the respective RKE, RKE2 and K3s documentation).

Resolution

Do not use the --atomic flag for Rancher upgrades using Helm