How to scale fluentd on rancher-logging

Article Number: 000022018

Environment

rancher-logging version 101.0.0+up3.17.7 or superior.

Procedure

By default, fluentd in rancher-logging is a statefulset deployed with only one replica. However, you might need to scale it up for different reasons, the biggest being performance issues. The log forwarder in rancher-logging is fluentbit, which is a daemonset, so it is deployed on every node on your cluster. This means that fluentd can have an issue keeping up with all the connections from the different fluentbit pods [1].

In order to scale the rancher-logging-root-fluentd statefulset, you can go to Apps -> Installed Apps -> Click the three dots on the "rancher-logging" App -> Edit/Upgrade -> Next -> Edit YAML.

In this page you have the values file of the rancher-logging HelmChart. You will need to go to the fluentd section and include the replicas parameter (it is not included by default):

fluentd:
  replicas: 2

*The example above is for scaling the statefulset to 2 replicas. Make sure to change the number of replicas to the ones desired.

References:
[1] https://ranchermanager.docs.rancher.com/integrations-in-rancher/logging/logging-architecture