How to enable IPVS proxy mode for kube-proxy

This document (000020035) is provided subject to the disclaimer at the end of this document.

Environment

A cluster managed using Rancher

Or

A cluster managed using Rancher Kubernetes Engine (RKE) CLI

Situation

The default proxy mode for kube-proxy in Kubernetes and clusters is iptables, and this is also the case for clusters created with Rancher 2.x and the Rancher Kubernetes Engine (RKE) CLI.

This article aims to provide all the needed steps and configuration to deploy or update a cluster to use IPVS proxy mode.

Please note, IPVS provides load balancing functionality, with this in mind it does not cover all of the traffic handling maintained by kube-proxy. Some scenarios will still utilise iptables, such as services that require NAT, like NodePort and LoadBalancer services.

Resolution

The --proxy-mode flag for kube-proxy is used to override the default iptables mode, using the below steps for Rancher or RKE the --proxy-mode flag can be provided to enable IPVS.

Note: Enabling IPVS is best done when creating a cluster, the process to update an existing cluster does include some follow-up steps at the end of this article, please ensure to read these beforehand, and complete these when migrating to IPVS on an existing cluster.

Rancher v2.x

Log into the Rancher UI:

From the Global view click on the cluster
Click the Edit Cluster button, and Edit as YAML
Locate or create the services.kubeproxy field under rancher_kubernetes_engine_config

Add extra_args under kubeproxy to apply the IPVS changes to the kube-proxy component when it is started as a container on all nodes.

This example uses the lc (least connection) load balancing algorithm, rr (round-robin) is the default.

    kubeproxy:
      extra_args:
        ipvs-scheduler: lc
        proxy-mode: ipvs

Click Save, the above changes will be applied to the cluster

Note: Ensure that the necessary kernel modules (such as ip_vs_lc) are loaded when using the lc (least connection) load balancing algorithm Rancher Kubernetes Engine (RKE) CLI

Edit the cluster.yaml configuration file for your cluster:

Locate or create the services.kubeproxy field

Add extra_args under kubeproxy to apply the IPVS changes to the kube-proxy component when it is started as a container on all nodes.

This example uses the lc (least connection) load balancing algorithm, rr (round-robin) is the default.

    kubeproxy:
      extra_args:
        ipvs-scheduler: lc
        proxy-mode: ipvs

Use the rke up command to apply the changes to the cluster

Migrating to IPVS on an existing cluster

In recent Kubernetes versions when a proxy-mode is changed the managed iptables rules are not cleaned. To avoid inconsistency and unpredictable outcomes it is recommended to restart nodes that are in an existing cluster to ensure all service connectivity is accurate.

If using using an immutable approach in your environment, replacing each node is also an option instead of restarting.

Once the cluster has applied the above arguments to kube-proxy successfully and returned to the Active state, plan to drain, restart and/or replace each node during a maintenance period.

This can be done on one node initially, and performed on one or more nodes at a time once tested.

Additional Information

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.