I have recently started using OpenShift for work and thought, why not create my own cluster at home? I have a bunch of Raspberry Pis (and Orange Pi) lying around, and it would be nice if I can have a playground cluster available.
After some digging, I have decided to use K3s as it seems more appropriate for IoT. I am using the latest Raspberry Pi OS image (2020-08-20-raspios-buster-armhf-lite.img) available. Installation went smoothly. I was able to run pods, have the pods communicate via services etc. However, when attempting to create an ingress, it fails.
My deployment and service configuration:
apiVersion: apps/v1
kind: Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysite
labels:
app: mysite
spec:
replicas: 1
selector:
matchLabels:
app: mysite
template:
metadata:
labels:
app: mysite
spec:
containers:
- name: httpd
image: httpd:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: mysite-service
spec:
selector:
app: mysite
ports:
- protocol: TCP
port: 80
My ingress configuration:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: mysite-ingress
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- host: k8s.example.com
http:
paths:
- backend:
serviceName: mysite-service
servicePort: 80
As you can see, everything should be working fine, but it's not. The ingress was created, but it's acting as if it's not.
After days of digging, I was finally able to figure out the problem. As it turns out, Traefik, the default ingress controller of K3s, doesn't work with iptables-nft (for now). This can easily be resolved by switching iptables to legacy and rebooting:
sudo update-alternatives --set iptables /usr/sbin/iptables-legacy
sudo reboot -h now
Once booted up, you should be able to access your pod via the ingress.