Skip to content

How to enable debug level logging for the Rancher Cluster/Project Alerting Alertmanager instance, in a Rancher v2.x managed cluster?

This document (000020144) is provided subject to the disclaimer at the end of this document.

Situation

Task

This article details how to enable debug level logging on the Alertmanager instance in a Rancher v2.x managed Kubernetes cluster, which may assist when troubleshooting cluster or project alerting.

Pre-requisites

Resolution

  1. Within the Rancher UI navigate to the System Project of the relevant cluster and click on the Apps view.
  2. Click 'Upgrade' on the cluster-alerting app.
  3. In the Answers section click 'Add Answer' and add the variable alertmanager.logLevel with a value of debug.
  4. Click upgrade to save the change and update the Alertmanager instance with the debug log level.
  5. Navigate to the cattle-prometheus namespace within the System Project for the cluster, and view the logs of the alertmanager-cluster-alerting-0 Pod running for the alertmanager-cluster-alerting StatefulSet. You should see level=debug log messages, such as in the following example, confirming debug level logging has been successfully configured:

plaintext level=debug ts=2019-07-09T15:03:37.511451301Z caller=dispatch.go:104 component=dispatcher msg="Received alert" alert=[433a194][active] level=debug ts=2019-07-09T15:03:38.511774835Z caller=dispatch.go:430 component=dispatcher aggrGroup="{}/{group_id=\"c-5h85q:event-alert\"}/{rule_id=\"c-5h85q:event-alert_deployment-event-alert\"}:{event_message=\"Scaled up replica set mynginx2-7994cd84ff to 1\", resource_kind=\"Deployment\", rule_id=\"c-5h85q:event-alert_deployment-event-alert\", target_name=\"mynginx2\", target_namespace=\"default\"}" msg=flushing alerts=[[433a194][active]]

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.