diff --git a/docs/projects/posts/k3s-on-pi.md b/docs/projects/posts/k3s-on-pi.md new file mode 100644 index 0000000..8a2d5d9 --- /dev/null +++ b/docs/projects/posts/k3s-on-pi.md @@ -0,0 +1,97 @@ +--- +title: What to look out for when setting up a K3S cluster on PIs +date: + created: 2025-01-07 +tags: + - k8s + - k3s + - pi +--- + + + +## Configure the nodes +- static IP +- DNS: disable systemd-resolved, or it will conflict with the embeded DNS server of K3S +- storage: + - nfs-kernel-sever, nfs-common + - ZFS + +## Install K3S +1st master: +`curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET sh -s - server --cluster-init` + +Other masters: +`curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET sh -s - server --server https://:6443` + +Workers: +`curl -sfL https://get.k3s.io | K3S_TOKEN=SECRET sh -s - agent --server https://:6443` + +## Access +The kubectl config is located at `/etc/rancher/k3s/k3s.yaml`. + +## Install basic services +- keepalived +- storage + - [NFS](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) + - [ZFS](https://github.com/ccremer/kubernetes-zfs-provisioner) +- cert-manager + ```bash + helm repo add jetstack https://charts.jetstack.io + helm repo update + + helm install \ + cert-manager jetstack/cert-manager \ + --namespace cert-manager \ + --create-namespace \ + --set installCRDs=true + ``` + +- DNS: coreDNS +- configure traefik + ```yaml + # /var/lib/rancher/k3s/server/manifests/traefik-config.yaml + apiVersion: helm.cattle.io/v1 + kind: HelmChartConfig + metadata: + name: traefik + namespace: kube-system + spec: + valuesContent: |- + additionalArguments: + - "--entryPoints.dnsudp.address=:53/udp" + - "--entryPoints.dnstcp.address=:53/tcp" + ... + ports: + dnsudp: + port: 53 + exposedPort: 53 + expose: + default: true + protocol: UDP + dnstcp: + port: 53 + exposedPort: 53 + expose: + default: true + protocol: TCP + ... + ``` +- install registry +- confgure registry + ```yaml + # /etc/rancher/k3s/registries.yaml + mirrors: + docker.io: + endpoint: + - https://registry-mirror.example.org/v2 + configs: + registry.example.org: + auth: + username: username + password: password + registry-mirror.example.org: + auth: + username: username + password: password + ```