Skip to content

Configuring multus CNI for rancher RKE2 clusters

Article Number: 000022072

Environment

RKE2

Procedure

Overview

By default, every pod in Kubernetes is assigned a single network interface connected to the cluster network. However, certain workloads require additional network interfaces to meet specific requirements:

  • Database pods may need separate networks for backup traffic to avoid impacting production performance
  • Network functions (such as firewalls or routers) require multiple networks to route traffic between different network segments
  • Security-sensitive workloads benefit from traffic isolation for compliance or performance optimisation

What is Multus CNI?

Multus CNI is a Container Network Interface (CNI) plugin that enables attaching multiple network interfaces to Kubernetes pods. Rather than replacing existing CNI plugins, Multus functions as a CNI plugin multiplexer, coordinating multiple network attachments.

Multus is particularly valuable for network-intensive workloads that require additional interfaces supporting dataplane acceleration techniques such as SR-IOV.

For comprehensive information about Multus, refer to the official Multus CNI documentation.

Important Prerequisites

Multus cannot be deployed as a standalone solution. It always requires at least one conventional CNI plugin that fulfills Kubernetes cluster network requirements. This CNI plugin becomes the default for Multus and provides the primary network interface for all pods.

Installation and Configuration

Option 1: Rancher Downstream Cluster

When creating a downstream cluster through the Rancher UI, select the appropriate Multus CNI option during cluster creation:

  1. Navigate to the cluster creation workflow
  2. Under networking options, select both Multus and your primary CNI (e.g., Calico)
  3. Complete cluster creation

 

Option 2: Standalone RKE2 Cluster

For standalone RKE2 clusters, configure Multus by modifying the RKE2 configuration file:

# /etc/rancher/rke2/config.yaml
cni:
  - multus
  - canal

Note: The configuration uses lowercase cni (not Cni). The primary CNI plugin (in this example, canal) handles the default pod network, while Multus enables additional network attachments.

For additional network configuration options, refer to the RKE2 Network Options documentation.

Verification

Verify Multus DaemonSet

After installation, confirm that the Multus DaemonSet is running correctly:

kubectl get pods --all-namespaces | grep -i multus

Expected output should show Multus pods running on each cluster node.

Understanding the Multus DaemonSet

The Multus DaemonSet performs the following operations:

  1. Binary deployment: Places the Multus CNI binary in /opt/cni/bin on each node

  2. Configuration generation:

  3. Reads the lexicographically (alphabetically) first configuration file in /etc/cni/net.d

  4. Generates a new Multus configuration file at /etc/cni/net.d/00-multus.conf based on the default network configuration
  5. API authentication: Creates /etc/cni/net.d/multus.d directory containing credentials for Multus to access the Kubernetes API

Additional Verification

Validate the installation by checking the CNI configuration directory:

ls -la /etc/cni/net.d/

Ensure that the auto-generated /etc/cni/net.d/00-multus.conf file exists and corresponds to your default CNI plugin configuration.

For detailed configuration options, refer to the Multus Configuration Reference.

Creating NetworkAttachmentDefinitions

What is a NetworkAttachmentDefinition?

A NetworkAttachmentDefinition is a Kubernetes Custom Resource Definition (CRD) that defines how pods connect to additional networks beyond the default cluster network. It is the primary mechanism for configuring secondary network interfaces when using Multus CNI.

Key characteristics:

  • Describes network configurations using JSON specifications
  • Configures parameters such as IP ranges, gateways, network types (macvlan, ipvlan, SR-IOV, bridge), and plugin-specific properties
  • Enables pods to request additional network attachments through annotations in their YAML definitions

NetworkAttachmentDefinition Structure

Use the following template as a starting point:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: <network-attachment-name>
  namespace: <target-namespace>
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "type": "<plugin-type>",
      "name": "<network-name>",
      <plugin-specific-configuration>
    }

Important notes:
  • The config block must contain valid JSON matching the schema required by your chosen CNI plugin
  • IP Address Management (IPAM) parameters (subnet, gateway, rangeStart, rangeEnd) are configured within the IPAM section when required by the plugin
  • The configuration syntax varies depending on the selected CNI plugin type

Configuration Considerations

When designing a NetworkAttachmentDefinition, consider the following factors:

FactorConsiderationsCNI Plugin SelectionChoose an appropriate plugin: macvlan, ipvlan, bridge, SR-IOV, etc.Network InterfaceEnsure the specified interface name exists on all target nodesNetwork ModeSelect mode based on plugin capabilities (bridge/L2/L3)IP ManagementDetermine IPAM strategy: DHCP, static assignment, host-local, etc.Namespace IsolationPlan for namespace-level security and network segregationPerformance RequirementsUse SR-IOV for high-speed requirements; overlays for flexibilityCompatibilityVerify compatibility with your Kubernetes/RKE2 version and infrastructure

Available CNI Plugins

For a complete list of officially supported CNI plugins and their configuration options, refer to the CNI Plugin Reference documentation.

Example NetworkAttachmentDefinition

Here's a practical example using macvlan:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
  namespace: default
spec:
  config: |
    {
      "cniVersion": "0.3.1",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.1.0/24",
        "rangeStart": "192.168.1.200",
        "rangeEnd": "192.168.1.250",
        "gateway": "192.168.1.1"
      }
    }

Using NetworkAttachmentDefinitions in Pods

To attach additional networks to a pod, add the k8s.v1.cni.cncf.io/networks annotation:

apiVersion: v1
kind: Pod
metadata:
  name: samplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
  containers:
  - name: samplecontainer
    image: nginx

For multiple network attachments, use a comma-separated list:

annotations:
  k8s.v1.cni.cncf.io/networks: network1,network2,network3

Additional Resources