FailedCreatePodSandBox "cannot allocate new block due to per host block limit" events in an RKE2 cluster with the Calico CNI

Article Number: 000022137

Environment

A Rancher-provisioned or standalone RKE2 cluster with the Calico CNI

Situation

Creating new Pods is failing with a FailedCreatePodSandBox event containing a message of the following format:

failed to setup network for sandbox "<hash>": plugin type="calico" failed (add): cannot allocate new block due to per host block limit 

Cause

This issue occurs when the Calico IP Address Management (IPAM) cannot allocate an IP address for the pod, as there are no available addresses in the block(s) assigned to the node, and the node cannot be assigned a new block, due to the maxBlocksPerHost setting (default 20). This is indicative of an IPAM resource leak.

Over time, cluster events such as improper node shutdowns or failed pod deletions can lead to "leaked" IP addresses. These addresses remain marked as "allocated" in the Calico datastore but are not associated with any active workload or node, eventually exhausting the available pool.

Resolution

Check for and release any leaked IP addresses in the cluster by following the procedure in How to use calicoctl to query for and release leaked addresses in an RKE2 cluster.