How to enable SSL passthrough on the nginx-ingress controller in Rancher Kubernetes Engine (RKE) CLI, RKE2 standalone or Rancher v2.x provisioned Kubernetes clusters

This document (000020147) is provided subject to the disclaimer at the end of this document.

Environment

A Kubernetes cluster provisioned by the Rancher Kubernetes Engine (RKE) CLI, RKE2 or Rancher v2.x
For RKE provisioned clusters, you will require the RKE binary and access to the cluster configuration YAML, rkestate file and kubectl access with the kubeconfig for the cluster sourced.
For RKE2 standalone clusters (provisioned outside of Rancher), you will require access to the RKE2 Configuration file, located under /etc/rancher/rke2/config.yaml
For Rancher v.2x provisioned clusters, you will require cluster owner or global admin permissions in Rancher

Situation

This article details how to enable SSL passthrough on the Nginx-ingress controller in Rancher Kubernetes Engine (RKE) CLI, RKE2 or Rancher v2.x provisioned Kubernetes clusters.

Resolution

Configuration for RKE provisioned clusters:

Edit the cluster configuration YAML file to include the enable-ssl-passthrough: true option for the ingress, as follows:

ingress:
     provider: nginx
     extra_args:
       enable-ssl-passthrough: true

Apply the changes to the cluster, by invoking rke up:

rke up --config <cluster configuration yaml file>

Recycle the nginx pods in-order to pick up new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name);
      do kubectl delete $pod -n ingress-nginx; echo "Sleeping for 5 seconds"; sleep 5;
      done

Verify the new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name | awk -F '/' '{print $2}'); do echo -n "Checking $pod .... "; kubectl -n ingress-nginx exec "$pod" -- bash -c "ps aux | grep -v grep | grep enable-ssl-passthrough=true" > /dev/null 2>&1 && echo "Good" || echo "Bad"; done

Edit the ingress to include the new annotations:

kubectl -n default edit ingress hello-world-lb

Example:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
     annotations:
       nginx.ingress.kubernetes.io/ssl-passthrough: "true"
     name: hello-world-lb
     namespace: default

Configuration for RKE2 standalone provisioned clusters:

Create the file /var/lib/rancher/rke2/server/manifest/rke2-ingress-nginx-config.yaml with the desired HelmChartConfig:

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
     name: rke2-ingress-nginx
     namespace: kube-system
spec:
     valuesContent:  |-
       controller:
         extraArgs:
           enable-ssl-passthrough: true

After it's been correctly created, the embedded RKE2 Helm-controller will create a new helm job pod, performing the Helm chart upgrade.

Configuration for Rancher RKE provisioned clusters:

Login into the Rancher UI.
Go to Global -> Clusters -> <>.
From the Cluster Dashboard edit the cluster by Clicking on "⋮" then select Edit.
Click "Edit as YAML".

Include the enable-ssl-passthrough: true option for the ingress, as follows:

ingress:
  provider: nginx
  extra_args:
    enable-ssl-passthrough: true

Click "Save" at the bottom of the page.
Wait for the cluster to finish upgrading.
Go back to the Cluster Dashboard and click "Launch kubectl".

Run the following inside the kubectl CLI to recycle the nginx pods in-order to pick up new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name); do kubectl delete $pod -n ingress-nginx; echo "Sleeping for 5 seconds"; sleep 5; done

Run the following inside the kubectl CLI to verify the new argument:

for pod in $(kubectl get pods -l app=ingress-nginx -n ingress-nginx --no-headers -o name | awk -F '/' '{print $2}'); do echo -n "Checking $pod .... "; kubectl -n ingress-nginx exec "$pod" -- bash -c "ps aux | grep -v grep | grep enable-ssl-passthrough=true" > /dev/null 2>&1 && echo "Good" || echo "Bad"; done

Browse to the ingress and click edit.
Expand "Labels & Annotations".
Click "Add annotation" and add nginx.ingress.kubernetes.io/ssl-passthrough=true under "Annotations".
Click "Save"

Configuration for Rancher RKE2 provisioned clusters:

Login into the Rancher UI.
Go to Global -> Clusters -> <>.
From the Cluster Dashboard edit the cluster by Clicking on "⋮" then select Edit.
Click "Edit Config". Under the Cluster Configuration Section , select "additional Manifests"
Include the following configuration, then click Save.

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
     name: rke2-ingress-nginx
     namespace: kube-system
spec:
     valuesContent: |-
        extraArgs:
          enable-ssl-passthrough: true

Additional Information

Verification Steps

Run the following command to verify new certificate:

curl --insecure -v https://<<APP URL>> 2>&1 | awk 'BEGIN { cert=0 } /^\* SSL connection/ { cert=1 } /^\*/ { if (cert) print }'

Example output:

* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: OU=Domain Control Validated; CN=*.rancher.tools
*  start date: Jul  2 00:42:01 2019 GMT
*  expire date: May  2 00:19:41 2020 GMT
*  issuer: C=BE; O=GlobalSign nv-sa; CN=AlphaSSL CA - SHA256 - G2
*  SSL certificate verify ok.
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
* Connection #0 to host lab.rancher.tools left intact

N.B. Some browsers will cache the certificate, as a result you might need to close and re-open the browser in order to get the new certificate. How to clear the SSL state in a browser.

Disclaimer

This Support Knowledgebase provides a valuable tool for SUSE customers and parties interested in our products and solutions to acquire information, ideas and learn from one another. Materials are provided for informational, personal or non-commercial use within your organization and are presented "AS IS" WITHOUT WARRANTY OF ANY KIND.