Skip to content

K3s installation

Master node

First we need to install the master node. Therefore access your Raspberry Pi master node via SSH:

Install K3s:

Run the following command to install K3s on the master node:

curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode 644 --disable servicelb --token some_random_password --node-taint CriticalAddonsOnly=true:NoExecute --bind-address 192.168.86.100 --disable-cloud-controller --disable local-storage

  • --write-kubeconfig-mode 644: This option sets the file mode for the Kubeconfig file to 644, allowing read access for the owner and group. It ensures that the Kubeconfig file is secure but can be accessed by authorized users.

  • --disable servicelb: This flag disables the built-in service load balancer in K3s. Service load balancing can be handled by other external load balancers or methods.

  • --token some_random_password: Here, you specify a custom token, some_random_password, to be used for joining worker nodes to the cluster. This should be replaced with the actual token you've generated.

  • --node-taint CriticalAddonsOnly=true:NoExecute: This sets a node taint on the master node, which allows it to run pods with the "CriticalAddonsOnly" taint key. The "NoExecute" effect ensures that existing pods on the node are not terminated, but new pods are scheduled based on the taint.

  • --bind-address 192.168.0.10: It specifies the network interface IP address that K3s should bind to. This can be useful when you want to control which network interface K3s uses for communication.

  • --disable-cloud-controller: This option disables the built-in cloud controller, which is used for node registration. You would use this if you're running K3s on a non-cloud environment like bare-metal or on-premises servers.

  • --disable local-storage: This flag disables the local-storage provisioner, which manages local persistent volumes. This is useful when you want to use external storage solutions for your workloads.

OpenSSL random password

You can use the following command to generate a random token of 32 characters:

openssl rand -base64 32

This command will download and install K3s on the master node. Wait for the installation to complete; it may take a few minutes.

Retrieve the Kubeconfig

After the installation, you'll find the Kubeconfig file at /etc/rancher/k3s/k3s.yaml on the master node. Copy it to your local machine, so you can use kubectl from your local machine.

scp root@<master-node-ip>:/etc/rancher/k3s/k3s.yaml ~/.kube/config
Info

Replace <master-node-ip> with the IP address of your master node.

Set Permissions for Kubeconfig (Optional):

If you experience permission issues when using kubectl with the copied Kubeconfig, you may need to change the file's permissions. Run the following command on your local machine:

chmod 600 ~/.kube/config

This command ensures that only the owner of the file has read and write permissions.

Worker nodes

With K3s successfully installed on the master node, you can now use the Kubeconfig to manage your Kubernetes cluster. The next steps will involve joining the worker nodes to this master node to create a complete and functional Kubernetes cluster on your Raspberry Pi devices.

K3s node token

The K3s token is used to securely join worker nodes to the Kubernetes cluster managed by the K3s master node. The token is typically generated during the installation of K3s on the master node, and you can retrieve it from the master node.

To get the K3s token from the master node, follow these steps:

Retrieve the K3s Token:

Run the following command on the master node to retrieve the K3s token:

cat /var/lib/rancher/k3s/server/node-token

This command will display the token on your terminal. Use the part after ::server:

Once you have the K3s token, you can use it to join worker nodes to the Kubernetes cluster by providing it as part of the installation command on the worker nodes.

ansible workers -b -m shell -a "curl -sfL https://get.k3s.io | K3S_URL=https://<master-node-ip>:6443 K3S_TOKEN=<token> sh -s -"
  • Replace workers with the appropriate group or list of worker nodes in your inventory.
  • Replace <master-node-ip> with the IP address of your master node.
  • Replace <token> with the token used to join worker nodes to the cluster.

This Ansible command will remotely execute the K3s installation script on the specified worker nodes, and they will join your Kubernetes cluster controlled by the master node.

Check the get nodes command of kubectl to see the nodes appear.

watch kubectl get nodes

Labelling nodes

For cosmetics i will label all worker nodes with the worker role.

ansible worker-nodes -i /path/to/your/inventory/file -b -m shell -a "kubectl label nodes --selector=node-role.kubernetes.io/worker='' node-role.kubernetes.io/worker=true"

cluster-2        Ready    worker                 224d    v1.27.5+k3s1
cluster-1        Ready    worker                 224d    v1.27.5+k3s1
cluster-master   Ready    control-plane,master   224d    v1.27.5+k3s1
cluster-3        Ready    worker                 224d    v1.27.5+k3s1
cluster-4        Ready    worker                 4m17s   v1.29.3+k3s1