How to filter Rancher API audit log events with rancher-logging

Article Number: 000022372

Environment

Rancher v2.5+
Rancher API audit log enabled
rancher-logging

Procedure

When the Rancher API audit log is enabled, the volume of logs generated can be significant. Many of these entries - such as read-only GET requests, health checks (/healthz), or internal service account activity - may be considered "noise" depending on your compliance requirements. Filtering these events at the source helps reduce storage costs and improves the signal-to-noise ratio in your log analytics platform.

To filter Rancher API audit logs, utilize the Flow resource within the rancher-logging stack. This allows you to match specific containers and apply filters before the data is sent to an Output.

Follow these steps to configure the filter:

Create or Edit a Flow:
Define a Flow in the cattle-system Namespace of the Rancher local cluster.
Configure Matching:
The Flow must specifically select the rancher-audit-log container of the Rancher pods within the cattle-system namespace. Use the following matching criteria:
Namespace: cattle-system
Labels: app: rancher
Container Name: rancher-audit-log
Add Filters:
To filter based on the content of the audit log, you must first parse the raw log message.
Parser: Use the parser filter plugin, with type JSON, to transform the log string into structured fields.
Grep Filter: Use the grep filter plugin to exclude (drop) specific events based on patterns.
Nested Fields: To access nested JSON data (e.g., name within the user object), use record_accessor syntax such as $.user.name .

Example Configuration

The following example Flow parses the audit logs and excludes:

All GET requests (Read-only operations).
Health check requests to the /healthz endpoint.
Requests from the internal Rancher service account.

apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: rancher-audit-filter
  namespace: cattle-system 
spec:
  # 1. MATCH: Target the rancher-audit-log container in the Rancher Pods
  match:
    - select:
        namespaces:
          - cattle-system
        labels:
          app: rancher
        container_names:
          - rancher-audit-log

  # 2. FILTERS: Parse and Drop Noise
  filters:
    # A. Parse the log message as JSON so we can access specific fields
    - parser:
        parse:
          type: json

    # B. Filter out the specific Audit Events
    - grep:
        exclude:
          # Exclude "GET" requests.
          - key: method
            pattern: ^(GET)$
          # Exclude health checks
          - key: requestURI
            pattern: ^(/healthz)$
          # Exclude requests from one system serviceaccount
          - key: $.user.name
            pattern: ^(system:serviceaccount:cattle-system:rancher)$

  # 3. OUTPUT: Send the remaining (Write) logs to your storage
  localOutputRefs:
    - audit-output