Skip to content

How to configure cpu-manager-policy in an RKE2 or K3s cluster

Article Number: 000022055

Environment

  • RKE2
  • K3s Cluster

Procedure

To configure CPU Manager Policy:

For RKE2 or K3s Rancher-provisioned cluster:

a.1 Configure the CPU Manager Policy at cluster creation time

Create the new cluster from Cluster Management, fill in with the common set of configurations (kubernetes version, CNI version, Node Os Image, Node Os, sizing,Private registry, etc…), and  then select  “Edit as YAML” on the bottom section of the page:

Add the following configuration snippet under the section spec → rkeConfig → MachineGlobalConfig:

    machineGlobalConfig:
      kubelet-arg:
        - cpu-manager-policy=static
        - system-reserved=cpu=1,memory=1548Mi,ephemeral-storage=30Gi

a.2 Configure the CPU Manager Policy on a provisioned-cluster

To configure the CPU Manager Policy on existing rancher-provisioned clusters, go to Cluster Management -> Select your desired Downstream cluster, Edit Cluster as YAML:

Add the configuration:

Add the following configuration snippet under the section spec → rkeConfig → MachineGlobalConfig:

    machineGlobalConfig:
      kubelet-arg:
        - cpu-manager-policy=static
        - system-reserved=cpu=1,memory=1548Mi,ephemeral-storage=30Gi

The cluster will transition to Updating status, and the Cluster nodes will show "waiting for the Kubelet to start"

Remove the old CPU Manager State file:

ssh into each node, and execute the following command:

# rm -rf /var/lib/kubelet/cpu_manager_state

Then, the cluster will transition to "Ready" status.

b) For RKE2 or K3s standalone clusters:

b.1 Configure CPU Manager Policy at cluster creation time:

RKE2: The CPU Manager Policy can be configured in the  /etc/rancher/k3s/config.yaml file, adding the lines:

kubelet-arg:
  - cpu-manager-policy=static
  - system-reserved=cpu=1,memory=1548Mi,ephemeral-storage=30Gi

Then, continue with the standard bootstrap of the RKE2 cluster. 

K3s: The CPU Manager Policy can be configured  in the  /etc/rancher/k3s/config.yaml, adding the lines:

kubelet-arg:
  - cpu-manager-policy=static
  - system-reserved=cpu=1,memory=1548Mi,ephemeral-storage=30Gi

Then, proceed with the standard bootstrap of the K3s cluster 

b.2 Configure CPU Manager Policy on already-provisioned RKE2/k3s clusters:

Perform the same steps as in case b.1, and additionally,  ssh into each node to remove the CPU Manager state file:

# rm -rf /var/lib/kubelet/cpu_manager_state

After that, perform the following steps:

RKE2:

Server nodes:

systemctl stop rke2-server
rke2-killall.sh
systemctl start rke2-server

Agent nodes:

k3s:

Server nodes:

systemctl stop k3s
k3s-killall.sh
systemctl start k3s

Agent nodes:

systemctl stop k3s-agent
k3s-killall.sh
systemctl start k3s-agent

Additional docs:

[1] - Changing the CPU Policy Manager