Lesson 6: Configuring Canal and Network Policies

Canal was installed automatically when RKE2 started in Lesson 5, but it needs verification and hardening before we build on top of it in later lessons. This lesson verifies that dual-stack networking is working, enables WireGuard encryption for inter-node traffic, and configures Calico network policies to secure pod communication.

This is the 6th lesson of the guide Migrating from k3s to RKE2. Make sure you have completed the previous lesson before continuing here. The full list of lessons in the guide can be found in the overview.

Understanding Canal#

Architecture#

Canal is a composite CNI that combines two well-established projects. Flannel handles inter-node traffic by creating a VXLAN overlay network, while Calico manages intra-node routing and enforces network policies. This separation of concerns gives Canal the simplicity of Flannel’s overlay networking with the power of Calico’s policy engine.

Each Canal pod runs both a Flannel and a Calico container as a DaemonSet, ensuring every node in the cluster participates in both the overlay network and the policy engine. Canal uses the traditional iptables/nftables data plane, which works well for most clusters and requires no special kernel dependencies. RKE2 also bundles Cilium and Calico with eBPF support as alternative CNIs, but switching requires rebuilding the cluster.

Overlay and Encryption#

VXLAN (Virtual Extensible LAN) is a network encapsulation protocol that creates a virtual Layer 2 network on top of an existing Layer 3 infrastructure. It works by wrapping each original Ethernet frame inside a UDP packet with a VXLAN header, effectively creating a tunnel between two endpoints. The outer UDP packet is routable across any IP network, while the inner frame carries the original pod-to-pod traffic unchanged.

Canal uses VXLAN as its default overlay for inter-node traffic. When a pod on Node A sends a packet to a pod on Node B, Flannel wraps the packet in a VXLAN header addressed to Node B’s IP, sends it across the vSwitch, and Flannel on Node B unwraps it and delivers the original packet to the destination pod.

The diagram shows how the original pod-to-pod packet is nested inside VXLAN encapsulation for transit across the vSwitch. The underlying infrastructure only needs to route between node IPs and never sees the pod CIDRs directly.

The trade-off is a small overhead per packet (approximately 50 bytes for the VXLAN + UDP + outer IP headers) and the fact that encapsulated traffic is unencrypted by default. On a private vSwitch this is generally secure enough, but for defense in depth we enable WireGuard encryption later in this lesson.

Verification#

Lesson 5 confirmed the node is Ready and the cluster CIDRs are configured, but those checks only prove that RKE2 started successfully. Canal is responsible for actually assigning pod IPs and routing traffic, so we need to verify that the CNI is functioning before we change its backend to WireGuard.

Canal Pod Status#

Canal runs as a DaemonSet with two containers per pod: one for Flannel (overlay networking) and one for Calico (routing and policy enforcement). If either container fails to start, pods will not receive IP addresses and no traffic can flow.

$ kubectl get pods -n kube-system -l k8s-app=canal -o wide
NAME               READY   STATUS    RESTARTS   AGE   IP         NODE
rke2-canal-xxxxx   2/2     Running   0          30m   10.1.0.14   node4

Both containers must show 2/2 in the READY column. A single-node cluster shows one Canal pod; this grows to one per node as additional nodes join.

Dual-Stack Pod Test#

The cluster CIDRs are configured, but we have not yet confirmed that Canal actually assigns addresses from those ranges and that Calico routes traffic correctly for both address families. This test deploys a pod and verifies both IPAM and intra-node routing.

$ kubectl run dual-stack-test --image=busybox:1.36 --restart=Never -- sleep 3600
pod/dual-stack-test created
$ kubectl wait --for=condition=Ready pod/dual-stack-test --timeout=60s
pod/dual-stack-test condition met
$ kubectl get pod dual-stack-test -o jsonpath='{.status.podIPs}' | jq .
[
  {
    "ip": "10.42.0.10"
  },
  {
    "ip": "fd00:42::a"
  }
]

The pod should have one address from 10.42.0.0/16 and one from fd00:42::/56. If either address is missing, Canal did not detect the dual-stack CIDRs from the cluster configuration.

Ping the node’s vSwitch addresses from inside the pod to confirm Calico’s routing rules are in place for both IPv4 and IPv6:

$ kubectl exec dual-stack-test -- ping -c 2 10.1.0.14
PING 10.1.0.14 (10.1.0.14): 56 data bytes
64 bytes from 10.1.0.14: seq=0 ttl=64 time=0.105 ms
64 bytes from 10.1.0.14: seq=1 ttl=64 time=0.069 ms

--- 10.1.0.14 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.069/0.087/0.105 ms

$ kubectl exec dual-stack-test -- ping6 -c 2 fd00::14
PING fd00::14 (fd00::14): 56 data bytes
64 bytes from fd00::14: seq=0 ttl=64 time=0.148 ms
64 bytes from fd00::14: seq=1 ttl=64 time=0.108 ms

--- fd00::14 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.108/0.128/0.148 ms

These pings traverse Calico’s veth and iptables rules on the local node, confirming that intra-node pod routing works for both address families. Cross-node traffic through the VXLAN overlay cannot be tested until additional nodes join the cluster in Lesson 11.

Clean up the test pod:

$ kubectl delete pod dual-stack-test

Enabling WireGuard Encryption#

Why Encrypt Overlay Traffic#

VXLAN encapsulation carries pod traffic in cleartext between nodes. On a shared physical network like Hetzner’s vSwitch, where VLAN tagging provides logical isolation but not encryption, a compromised adjacent server could theoretically capture inter-node packets.

WireGuard adds an encryption layer around the VXLAN tunnel, so the packet on the wire is fully encrypted:

The diagram extends the earlier VXLAN flow with WireGuard wrapping the entire VXLAN packet in an encrypted tunnel before it hits the wire. Each node establishes a WireGuard tunnel to every other node, and all overlay traffic flows through these tunnels transparently.

MTU and Encapsulation Overhead#

Every layer of encapsulation adds headers to each packet, reducing the maximum payload that fits within the physical network’s MTU. Our Hetzner vSwitch interface has a standard MTU of 1500 bytes, and each overlay technology subtracts its header size from that budget:

When Flannel uses VXLAN, it creates a flannel.1 interface at MTU 1450 and sets the pod veth to match. When Flannel switches to the WireGuard backend, it creates flannel-wg interfaces at MTU 1420, but Canal’s default veth_mtu remains at 1450 unless explicitly overridden.

This mismatch is critical: if pod interfaces have a higher MTU than the WireGuard tunnel, packets between 1421 and 1450 bytes that cross nodes will exceed the tunnel capacity. TCP relies on Path MTU Discovery (PMTUD) to detect this and reduce the segment size, but PMTUD depends on ICMP “Packet Too Big” messages reaching the sender, which can fail when packets traverse multiple encapsulation layers. The result is intermittent connection stalls and timeouts that are difficult to diagnose because small requests succeed while larger transfers hang.

The theoretical WireGuard overhead is ~80 bytes, giving a tunnel MTU of 1420. In practice, we found that setting the pod veth MTU to 1420 still produced occasional packet drops on Hetzner’s vSwitch, likely due to additional encapsulation overhead from VLAN tagging or internal switching that is not accounted for in the standard calculation. Reducing the veth MTU further to 1360 eliminated these drops entirely. The tradeoff is a slightly lower effective throughput for bulk transfers, since each packet carries less payload and more overhead relative to the data. For typical API and web workloads the difference is negligible, and the reliability gain is worth the minor bandwidth cost.

Applying the Configuration#

With the overlay, encryption, and MTU concepts covered, we can now configure Canal to use all three. WireGuard requires kernel module support, so before writing the manifest, verify the module loads correctly:

$ modprobe wireguard
$ lsmod | grep wireguard
wireguard             118784  0

If the module fails to load, the kernel may need the WireGuard package installed, though Rocky Linux 10 includes it in the default kernel.

RKE2 bundles Canal as a Helm chart and customizes it through a HelmChartConfig resource. Placing this resource in the auto-deploy manifests directory tells RKE2 to merge our values into the chart the next time it reconciles:

$ cat <<'EOF' > /var/lib/rancher/rke2/server/manifests/rke2-canal-config.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-canal
  namespace: kube-system
spec:
  valuesContent: |-
    flannel:
      backend: "wireguard"
      regexIface: "\\.4000$"
    calico:
      vethuMTU: 1360
EOF

The manifest contains three settings, each addressing a specific aspect of our Hetzner environment.

flannel.regexIface controls which network interface Flannel uses for inter-node tunnel endpoints. When this value is empty, Flannel follows the default route to discover the outbound interface, which on Hetzner dedicated servers is the public network interface. WireGuard tunnels would then use public IPs as their endpoints, forcing inter-node pod traffic onto the public internet and requiring an additional firewall rule for UDP port 51820.

Setting regexIface to \\.4000$ matches the VLAN-tagged vSwitch interface (e.g., enp35s0.4000 or enp195s0.4000) on every node, regardless of the underlying NIC name.

Flannel then uses the vSwitch IP (10.1.0.x) as the WireGuard endpoint, keeping all tunnel traffic on the private network where the Hetzner firewall’s vSwitch rule already permits it. The Helm chart maps this value to the FLANNELD_IFACE_REGEX environment variable in the flannel container. See the Flannel configuration documentation for details on interface selection behavior.

Configuring flannel.backend: "wireguard" switches the overlay from VXLAN to WireGuard, as described in the encryption section above.

calico.vethuMTU: 1360 sets the pod veth MTU below the WireGuard tunnel capacity, as described in the MTU section above. Without this, Canal defaults to a veth MTU of 1450 (the Helm chart default), which exceeds the tunnel capacity and causes intermittent packet loss for cross-node traffic.

Once the manifest is written, RKE2 detects it and upgrades the Canal Helm release automatically. The chart values update in place, but the Canal pods themselves need a restart to switch from the VXLAN backend to WireGuard:

$ kubectl rollout restart ds rke2-canal -n kube-system
daemonset.apps/rke2-canal restarted

$ kubectl rollout status ds rke2-canal -n kube-system --timeout=120s
Waiting for daemon set "rke2-canal" rollout to finish: 0 of 1 updated pods are available...
daemon set "rke2-canal" successfully rolled out

After the rollout completes, verify the ConfigMap reflects the new MTU value. This confirms that RKE2 merged our HelmChartConfig values into the Canal chart correctly:

$ kubectl get configmap -n kube-system rke2-canal-config -o jsonpath='{.data.veth_mtu}'
1360

Canal applies the new veth MTU only when it creates a new veth pair, so existing pods retain whatever MTU they started with. To ensure every workload uses the correct 1360 MTU, restart all Deployments, StatefulSets, and DaemonSets across the cluster:

$ kubectl get deployments --all-namespaces \
    --no-headers \
    -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name | \
    while read ns name; do kubectl rollout restart deployment "$name" -n "$ns"; done

$ kubectl get statefulsets --all-namespaces \
    --no-headers \
    -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name | \
    while read ns name; do kubectl rollout restart statefulset "$name" -n "$ns"; done

$ kubectl get daemonsets --all-namespaces \
    --no-headers \
    -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name | \
    while read ns name; do kubectl rollout restart daemonset "$name" -n "$ns"; done

Standalone pods (such as CronJob-created pods) will pick up the new MTU when they are next recreated.

Verifying the Interface#

While the WireGuard kernel module is built into Rocky Linux 10’s default kernel, the wg userspace tool for inspecting tunnels is a separate package. Install it now so it it can be used to inspect available interfaces and verify the WireGuard tunnel is active:

$ sudo dnf install -y wireguard-tools

On a single-node cluster, WireGuard tunnels have no peers to connect to, but we can verify the interface was created and the kernel module is active:

$ ip link show | grep flannel-wg
87: flannel-wg: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default
88: flannel-wg-v6: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default

$ wg show flannel-wg
interface: flannel-wg
  public key: <node-public-key>
  private key: (hidden)
  listening port: 51820

The flannel-wg and flannel-wg-v6 interfaces confirm that Canal switched from VXLAN to WireGuard. Both interfaces should show mtu 1420, which is Flannel’s default tunnel MTU (distinct from the 1360 pod veth MTU we set via calico.vethuMTU). The wg show output lists the interface with a public key and listening port, but no peers yet since we only have one node. Peers appear automatically as additional nodes join the cluster in Lesson 11.

Confirm that Flannel selected the vSwitch interface by checking the node annotation:

$ kubectl get node $(hostname) -o jsonpath='{.metadata.annotations.flannel\.alpha\.coreos\.com/public-ip}'
10.1.0.14

The public-ip annotation should show the vSwitch address, not the server’s public IP. If it shows a public IP, the regexIface pattern did not match the vSwitch interface. Verify the interface name with ip -o addr show | grep '10.1.0' and adjust the regex accordingly.

Verify that the pod veth MTU also matches the WireGuard tunnel MTU by deploying a test pod:

$ kubectl run mtu-test --image=busybox:1.36 --restart=Never -- sleep 60
pod/mtu-test created
$ kubectl exec mtu-test -- cat /sys/class/net/eth0/mtu
1360

$ kubectl delete pod mtu-test
pod "mtu-test" deleted

The pod’s eth0 interface must report 1360. If it shows 1450 instead, the calico.vethuMTU: 1360 setting in the HelmChartConfig was not applied. Verify the manifest contents and restart the Canal DaemonSet.

TCP MSS Clamping#

Matching the pod veth MTU to the WireGuard tunnel prevents pods from sending oversized packets, but it does not protect against Path MTU Discovery (PMTUD) failures on the return path. Although the pod correctly advertises a reduced Maximum Segment Size (MSS) during the TCP handshake, PMTUD must still function across the entire network path for the remote side to respect the effective MTU.

In environments where ICMP Fragmentation Needed (IPv4) or Packet Too Big (IPv6) messages are filtered, cloud overlays, and WireGuard encapsulation and large return packets may be silently dropped. The result is connections that work fine for small requests but stall or time out when transferring larger payloads.

The standard fix is TCP MSS clamping: an iptables rule in the mangle table that rewrites the MSS value in TCP SYN and SYN-ACK packets to match the path MTU. With --clamp-mss-to-pmtu, the kernel automatically calculates the correct MSS from the outgoing interface’s MTU, ensuring that neither side of the connection ever sends segments too large for the tunnel.

Calico Felix manages MTU on tunnel interfaces and veth devices but does not insert MSS clamping rules. We deploy the rule ourselves using a privileged DaemonSet, the same pattern we use for Canal, Longhorn, and Traefik. A DaemonSet runs on every node automatically, including nodes that join the cluster later, so no per-node manual setup is needed.

Create the manifest at /var/lib/rancher/rke2/server/manifests/mss-clamp.yaml:

# /var/lib/rancher/rke2/server/manifests/mss-clamp.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: mss-clamp
  namespace: kube-system
  labels:
    k8s-app: mss-clamp
spec:
  selector:
    matchLabels:
      k8s-app: mss-clamp
  template:
    metadata:
      labels:
        k8s-app: mss-clamp
    spec:
      hostNetwork: true
      tolerations:
        - operator: Exists
      initContainers:
        - name: mss-clamp
          image: rancher/hardened-calico:v3.31.3-build20260206
          command:
            - /bin/sh
            - -c
            - |
                iptables -t mangle -C FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu 2>/dev/null \
                  || iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
                ip6tables -t mangle -C FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu 2>/dev/null \
                  || ip6tables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
          securityContext:
            privileged: true
            capabilities:
              add: ["NET_ADMIN", "NET_RAW"]
      containers:
        - name: pause
          image: rancher/mirrored-pause:3.6
          resources:
            requests:
              cpu: 1m
              memory: 4Mi
            limits:
              cpu: 10m
              memory: 16Mi

The DaemonSet uses an init container to apply the iptables rules on the host’s network namespace (hostNetwork: true), then idles with a minimal pause container. The tolerations with operator: Exists ensures it schedules on every node, including control plane nodes, matching the Traefik DaemonSet configuration from Lesson 8. We use the rancher/hardened-calico image that Canal already pulls on every node, so no additional image download is required.

The -C flag checks whether the rule already exists before -A appends it, preventing duplicate entries if the pod restarts.

RKE2 detects the new manifest and deploys the DaemonSet automatically. Verify the rules are in place on the node:

$ sudo iptables -t mangle -L FORWARD -v -n
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 TCPMSS     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 TCPMSS clamp to PMTU

The rule matches TCP packets with the SYN flag set and rewrites the MSS option to fit the outgoing interface’s MTU.

Network Policies#

Understanding Network Policies#

By default, Kubernetes allows all pod-to-pod communication across all namespaces. A NetworkPolicy resource changes this by defining explicit ingress and egress rules for pods matching a selector. Once any NetworkPolicy selects a pod, all traffic not explicitly allowed by a policy is denied.

Canal enforces these policies through Calico’s policy engine, which supports standard Kubernetes NetworkPolicy resources at L3-L4 (IP addresses, ports, and protocols).

Unlike Cilium, Canal does not provide host-level network policies; the Hetzner firewall configured in Lesson 4 serves that role.

Default Deny per Namespace#

A common security pattern is to deny all ingress traffic by default and then allow specific communication paths. This policy selects all pods in a namespace and permits only traffic from within the same namespace. Pods also need to reach CoreDNS (in kube-system) to resolve service names, so a companion egress policy must allow DNS traffic; without it, pods cannot look up any service addresses.

We place both policies in the RKE2 auto-deploy manifests directory so they are applied on every cluster start and survive node rebuilds, consistent with how we deployed the Canal HelmChartConfig.

Create a file at /var/lib/rancher/rke2/server/manifests/default-network-policies.yaml with the following content:

# /var/lib/rancher/rke2/server/manifests/default-network-policies.yaml

# Default deny ingress: only allow traffic from within the same namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ingress
  namespace: default
spec:
  podSelector: {}
  policyTypes:
    - Ingress
  ingress:
    - from:
        - podSelector: {}
---
# Allow DNS egress to CoreDNS in kube-system
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
  namespace: default
spec:
  podSelector: {}
  policyTypes:
    - Egress
  egress:
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - port: 53
          protocol: TCP
        - port: 53
          protocol: UDP

RKE2 picks up the manifest automatically within a few seconds. The default-deny-ingress policy restricts pods in the default namespace to only accept traffic from other pods in the same namespace, while allow-dns ensures DNS resolution continues to work.

Verifying Network Policies#

Check that both policies are applied:

$ kubectl get networkpolicies -A
NAMESPACE   NAME                   POD-SELECTOR   AGE
default     allow-dns              <none>         26s
default     default-deny-ingress   <none>         26s

Test from within a pod that DNS works but cross-namespace traffic is blocked:

$ kubectl run policy-test --image=busybox:1.36 -n default --restart=Never -- sleep 3600
pod/policy-test created

$ kubectl wait --for=condition=Ready pod/policy-test -n default --timeout=60s
pod/policy-test condition met

# DNS should work
$ kubectl exec -n default policy-test -- nslookup kubernetes.default.svc.cluster.local
Server:         10.43.0.10
Address:        10.43.0.10:53


Name:   kubernetes.default.svc.cluster.local
Address: 10.43.0.1

# Cross-namespace traffic should be blocked (will timeout)
$ kubectl exec -n default policy-test -- wget -qO- --timeout=3 http://rke2-metrics-server.kube-system.svc:443 2>&1 || echo "Blocked as expected"
wget: download timed out
command terminated with exit code 1
Blocked as expected

$ kubectl delete pod policy-test -n default
pod "policy-test" deleted from default namespace

Troubleshooting#

For Canal pod startup failures and dual-stack IPv6 issues, refer to the troubleshooting section in Lesson 5. The sections below cover issues we encountered specific to this lesson’s configuration.

Network Policy Not Blocking Traffic#

Calico applies network policies asynchronously. After creating a policy, allow a few seconds for Felix to program the iptables rules. Verify that the policy is recognized:

$ kubectl get networkpolicies -n <namespace>

If traffic is still flowing despite a deny policy, check that no other policy in the namespace is allowing it. Kubernetes network policies are additive, meaning any policy that allows traffic takes precedence.

Intermittent Timeouts After Enabling WireGuard#

If pods experience sporadic connection timeouts, especially for large downloads or HTTP requests that succeed for small payloads but hang on larger ones, the most likely cause is an MTU mismatch between the pod veth and the WireGuard tunnel.

Check the pod veth MTU and the WireGuard tunnel MTU:

# Pod veth MTU (should be 1360)
$ kubectl run mtu-check --image=busybox:1.36 --restart=Never -- sleep 60
pod/mtu-check created

$ kubectl exec mtu-check -- cat /sys/class/net/eth0/mtu
1360

# WireGuard tunnel MTU (should be 1420)
$ ip link show flannel-wg | grep mtu
862: flannel-wg: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default

If the pod veth shows 1450 while flannel-wg shows 1420, add calico.vethuMTU: 1360 to the Canal HelmChartConfig as described in the Applying the Configuration section above, restart the Canal DaemonSet, and then restart all workload pods so they pick up the new MTU.

If the MTU values match but timeouts persist, verify that MSS clamping is active on the node handling the traffic:

$ sudo iptables -t mangle -L FORWARD -v -n | grep TCPMSS
    0     0 TCPMSS     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 TCPMSS clamp to PMTU

If no TCPMSS rule appears, follow the TCP MSS Clamping section above to install it. Without MSS clamping, external servers may send TCP segments sized for the node’s 1500-byte physical MTU rather than the 1420-byte tunnel MTU, causing large responses to stall.

DNS Failures and Fragmented UDP Packets#

If pods intermittently fail to resolve external domains (returning getaddrinfo errors or timing out), but short lookups like nslookup kubernetes.default succeed, the likely cause is large DNS responses being fragmented and then silently dropped inside the cluster.

DNS uses UDP by default, and modern resolvers advertise EDNS0 buffer sizes of up to 4096 bytes so that large responses (DNSSEC signatures, dual-stack records, big TXT records) can arrive in a single UDP datagram without falling back to TCP. When one of these large replies travels through the WireGuard tunnel, the 1420-byte tunnel MTU forces it to be split into IP fragments. The problem is that the Linux conntrack subsystem (nf_conntrack) tries to reassemble those fragments before processing them. When reassembly fails, conntrack marks the individual fragments as INVALID, and both kube-proxy and Calico have iptables rules that drop INVALID packets:

The net effect is that small DNS queries work (the response fits in one packet), but queries that produce large responses silently fail. TCP traffic is not affected when MSS clamping is in place, because the clamped segment size prevents packets from exceeding the tunnel capacity.

To confirm this is the cause, check the kernel’s reassembly counters:

$ cat /proc/net/snmp | grep "^Ip:"
Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates OutTransmits
Ip: 1          64         2294121351 1           50320        1145307131    46              0          1148759561 1285546787  9           3666        56           72         0        56         0       2496      0           2430147649

A non-zero ReasmFails with ReasmOKs at zero means every reassembly attempt is failing. You can also verify the drop rules exist:

$ sudo iptables-save | grep "INVALID"
-A KUBE-FORWARD -m conntrack --ctstate INVALID -j DROP
-A cali-fw-cali0463b04862f ... --ctstate INVALID -j DROP
...

These rules cannot be removed: kube-proxy adds KUBE-FORWARD and Calico’s Felix regenerates the per-pod rules automatically. The fix is to prevent fragmentation from occurring in the first place. The CoreDNS HelmChartConfig in Lesson 5 includes the bufsize plugin set to 1232 bytes, which tells upstream DNS servers to keep responses small enough to fit within the tunnel MTU without fragmenting. You can verify it by checking if bufsize 1232 appears in the Corefile:

$ kubectl get configmap -n kube-system rke2-coredns-rke2-coredns -o jsonpath='{.data.Corefile}' | grep bufsize
bufsize 1232

Lesson Conclusion

Congratulations! Canal is providing encrypted dual-stack pod networking with namespace-level network policies, and Node 4 is Ready to accept additional nodes into Cluster B. You have completed this lesson and you can now continue with the next one.