Skip to content

Recovering cluster.yml and cluster.rkestate files from kubeconfig - RKE clusters

This document (000021114) is provided subject to the disclaimer at the end of this document.

Environment

Important: This script is specifically intended for use with clusters created using the RKE CLI. Do not utilize it against any other clusters (e.g. a Rancher-created downstream cluster).

Situation

During the installation of an RKE Kubernetes cluster, two essential files are created locally in the working directory where you invoke the RKE CLI:

  • cluster.yml: also recognized as the Cluster Configuration File, this file is referenced by RKE to determine what nodes will be in the cluster and how to deploy Kubernetes.
  • cluster.rkestate: the Kubernetes Cluster State file, which contains the credentials for full access to the cluster.

These files are needed to maintain, troubleshoot and upgrade your cluster and, therefore, should always be preserved in a secure location. However, if something unforeseen happens, and these files are lost, it is possible to recover them from the cluster itself, per the steps below.

Resolution

To recover these two files, it's possible to use the following script. Please note that you will need to fulfill these prerequisites:

  • Access to the kubectl command line tool, with the kubeconfig file correctly configured to access the cluster.
  • jq command-line JSON processor installed.
  • yq command-line YAML, JSON, and XML processor installed.

For RKE binary versions < 1.4.19 , 1.5.10 or 1.6.0:

On RKE versions prior to 1.4.19, 1.5.10 or 1.6.10, the Kubernetes Cluster state file (cluster.rkestate) and cluster.yml files are stored as a configmap, under the kube-system namespace.

#!/bin/bash
echo "Building cluster_recovery.yml..."
echo "Working on Nodes..."
echo 'nodes:' > cluster_recovery.yml
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .desiredState.rkeConfig.nodes | yq -P | sed 's/^/  /' | \
sed -e 's/internalAddress/internal_address/g' | \
sed -e 's/hostnameOverride/hostname_override/g' | \
sed -e 's/sshKeyPath/ssh_key_path/g' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on services..."
echo 'services:' >> cluster_recovery.yml
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .desiredState.rkeConfig.services | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on network..."
echo 'network:' >> cluster_recovery.yml
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .desiredState.rkeConfig.network | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on authentication..."
echo 'authentication:' >> cluster_recovery.yml
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .desiredState.rkeConfig.authentication | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on systemImages..."
echo 'system_images:' >> cluster_recovery.yml
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .desiredState.rkeConfig.systemImages | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Building cluster_recovery.rkestate..."
kubectl -n kube-system get configmap full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r . > cluster_recovery.rkestate

For RKE binary versions  1.4.19+ , 1.5.10+ or 1.6.0+:

On RKE 1.4.19, 1.5.10 and 1.6.0 or higher, the Kubernetes Cluster state file (cluster.rkestate) and cluster.yml files are stored as a secret, under the kube-system namespace.

#!/bin/bash
echo "Building cluster_recovery.yml..."
echo "Working on Nodes..."
echo 'nodes:' > cluster_recovery.yml
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .desiredState.rkeConfig.nodes | yq -P | sed 's/^/  /' | \
sed -e 's/internalAddress/internal_address/g' | \
sed -e 's/hostnameOverride/hostname_override/g' | \
sed -e 's/sshKeyPath/ssh_key_path/g' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on services..."
echo 'services:' >> cluster_recovery.yml
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .desiredState.rkeConfig.services | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on network..."
echo 'network:' >> cluster_recovery.yml
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .desiredState.rkeConfig.network | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on authentication..."
echo 'authentication:' >> cluster_recovery.yml
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .desiredState.rkeConfig.authentication | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Working on systemImages..."
echo 'system_images:' >> cluster_recovery.yml
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .desiredState.rkeConfig.systemImages | yq -P | sed 's/^/  /' >> cluster_recovery.yml
echo "" >> cluster_recovery.yml

echo "Building cluster_recovery.rkestate..."
kubectl -n kube-system get secret full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r . > cluster_recovery.rkestate

Once the execution is completed, you will find two files in the folder of execution: cluster_recovery.yml and cluster_recovery.rkestate, which correspond to a copy of the Cluster Configuration File and the Kubernetes Cluster State file.

After the recovery is done, please back up these files in a secure location to avoid future loss.

Cause

This change has been implemented to mitigate the CVE-2023-32191.  More information is available at https://github.com/rancher/rke/security/advisories/GHSA-6gr4-52w6-vmqx

Additional Information

Original source:

https://gist.github.com/mattmattox/d32b3fea4820075c08c6cc2f6d736702

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.