Configure quota-backend-bytes to extend etcd keyspace limit in RKE2

This document (000021764) is provided subject to the disclaimer at the end of this document.

Environment

Suse Rancher

RKE2

Situation

In a Kubernetes cluster, etcd serves as the primary data store, maintaining the cluster state, configuration, and other critical metadata.

Over time, as workloads grow and more resources are created, the etcd database can reach its default size limit, leading to performance degradation or even failures in cluster operations.

Resolution

For a standalone RKE2 cluster, you can increase the etcd database size by modifying the RKE2 configuration file /etc/rancher/rke2/config.yaml.

etcd-arg:
  - "quota-backend-bytes=858993459"

and restart rke2-server

systemctl restart rke2-server

For a cluster managed by Rancher, go to Rancher, Cluster Management -> select the cluster and edit the yaml ( which you will get from the 3 dots).

Add quota-backend-bytes under machineSelectorConfig.config.etcd-arg.  The snapshot below shows an example.

Increasing the etcd database size can help mitigate etcd storage consumption issues and ensure the cluster remains stable and responsive.

Cause

The etcd database size is primarily influenced by the number of objects stored, frequent updates, and high write activity. If the database size limit is reached, you may experience issues such as slow API responses, failed leader elections, or cluster instability. Additionally, excessive fragmentation due to outdated entries and stale snapshots can contribute to rapid storage consumption.

Status

Top Issue

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.