How to update your etcd space alerts for better etcd monitoring
Article Number: 000020187
Situation
Task
An alert you may have seen is Database usage close to the quota 500M in your Rancher 2.x cluster.
This is a default etcd alert built into Rancher, you can find more info on the default alerts here: Rancher v2.x Default Alerts
Upon further examination of the alert, you see the description of
A warning alert is triggered when the size of etcd exceeds 500M.
This alert is somewhat misleading as the default etcd size is 2GB. This alert is also at a severity level of Warning. Below, we suggest configuring your etcd alerts to better utilize the alert thresholds and to avoid the default alert constantly going off.
Pre-requisites
- Running Rancher v2.x
Resolution
You can access the alerts by going to Tools -> Alerts at the cluster level. From here, clone the default alert three times. Once the 3 clones are created, disable the default alert so you always retain a clean reference alert.
Alert for 1.5GB usage - WARNING
Alert for 1.75GB usage - CRITICAL
