Skip to content

How to monitor Canal or Calico in RKE2 Downstream Clusters

This document (000021547) is provided subject to the disclaimer at the end of this document.

Environment

  • Rancher v2.x.
  • A Rancher-managed RKE2 cluster with Calico or Canal CNI.
  • Rancher-monitoring installed.

Situation

Canal and Calico offer metrics that can be exposed and consumed by rancher-monitoring. This article shows how to do it.

Resolution

Prerequisite for Calico

Calico's RKE2 addon does not export Prometheus metrics by default.

To enable this option in the rke2-calico addon, you can go to Cluster Management > Select the cluster > Edit Config > Add-on: Calico. And add the following options in the corresponding "felixConfiguration" and "installation" sections:

felixConfiguration:
  prometheusMetricsEnabled: true
installation:
  typhaMetricsPort: 9093

If your cluster's CNI is Canal, it already exports Prometheus metrics by default, as the following option is enabled in the rke2-canal addon:

    calico:
      felixPrometheusMetricsEnabled: true

After this prerequisite is fulfilled, there are two steps involved to gather these metrics in Prometheus:

Create the Services

  • Canal:

Only one service is needed to be able to extract the metrics from the canal pods through the 9091 metrics port:

apiVersion: v1
kind: Service
metadata:
  name: calico-felix-metrics
  namespace: kube-system
  labels:
    k8s-app: calico-felix
spec:
  clusterIP: None
  ports:
  - port: 9091
    protocol: TCP
    name: metrics-port
  selector:
    k8s-app: canal
  • Calico:

Two services are needed. One to export the calico-felix metrics and another one for the calico-typha metrics. The headless service to export metrics from the calico-kube-controllers is there by default so there is no need to create it.

apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: calico-felix
  name: calico-felix-metrics
  namespace: calico-system
spec:
  clusterIP: None
  ports:
  - name: metrics-port
    port: 9091
  selector:
    k8s-app: calico-node

apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: calico-typha
  name: calico-typha-metrics-headless-svc
  namespace: calico-system
spec:
  clusterIP: None
  ports:
  - name: metrics-port
    port: 9093
  selector:
    k8s-app: calico-typha

After the service YAMLs are applied, you can proceed to the next step:

Create the ServiceMonitors

  • Canal:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
    name: rancher-monitoring-canal
    namespace: kube-system
spec:
    selector:
      matchLabels:
        k8s-app: calico-felix
    endpoints:
  - port: metrics-port
    relabelings:
    - sourceLabels:
      - __meta_kubernetes_endpoint_node_name
      targetLabel: instance
  • Calico:

apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: rancher-monitoring-calico namespace: calico-system spec: endpoints: - port: metrics-port relabelings: - sourceLabels: - __meta_kubernetes_endpoint_node_name targetLabel: instance selector: matchExpressions: - key: k8s-app operator: In values: [ "calico-felix", "calico-typha","calico-kube-controllers" ]

After both the Service and ServiceMonitors are applied, the target will appear in Prometheus, and the metrics will be scraped. Be aware that it might take a minute for Prometheus to see the new target and scrape the metrics.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.