tutorials

Developing with Docker and Kubernetes locally - and avoid the errors that "Hello World" examples do not show

View on GitHub

Tutorial: Kubernetes Multipass Setup

After using docker-for-mac a lot for working with Kubernetes I decided to search for an replacement that fulfills the following requirements:

Some experiments and several distributions later I found MicroK8s, K0s and K3s.

MicroK8s

I followed this installation guide.

To be honest, I did not experiment a lot with MicroK8s, because of this issue. It worked when creating the old version manually (select an old snap channel), but the CPU load seemed to be higher than k3s when doing nothing (no pods deployed). That may differ for you, so please check out for yourself.

For documentation reasons I keep the documentation in place - but I deleted those instances after playing with them.

multipass launch -c 4 -n k8shost

Installing Channel 1.18, because of bug in 1.19 that does not start

ubuntu@k8shost:~$ sudo snap install microk8s --classic --channel=1.18/stable
microk8s (1.18/stable) v1.18.16 from Canonical✓ installed
ubuntu@k8shost:~$ sudo usermod -a -G microk8s $USER
ubuntu@k8shost:~$ sudo chown -f -R $USER ~/.kube
ubuntu@k8shost:~$ sudo su - $USER
ubuntu@k8shost:~$ microk8s status --wait-ready

After that step the installation was done and the cluster was accessible using

microk8s kubectl get nodes

It is obvious that an alias can be used to map kubectl="microk8s kubectl" to ease access within the virtual machine.

K0s

The distribution K0s looks fairly promising and has in my tests the same ressource consumption as K3s. Because I like traeffik more I did not follow the K0s path. See here for a working installation guide.

The K0s can be installed in multipass the same way as in the script below. It is fairly easy to script and install if you like this distribution more.

K3s

Again, I followed the available installation guide.

Installation consists of one command:

ubuntu@k3s-master:~$ curl -sfL https://get.k3s.io | sh -

Get the token for this master node with the following command:

ubuntu@k3s-master:~$ sudo cat /var/lib/rancher/k3s/server/node-token

And register each worker node with the following command using the given token:

ubuntu@k3s-worker:~$ curl -sfL https://get.k3s.io | K3S_URL="https://$IP:6443" K3S_TOKEN="$TOKEN" sh -

You can get access to the cluster using the following command:

ubuntu@k3s-master:~$ sudo cat /etc/rancher/k3s/k3s.yaml > k3s.yaml
export KUBECONFIG=${PWD}/k3s.yaml

I use .envrc (direnv) to work with different clusters.

Automated provisioning

The whole process is automated in the shell script createCluster.sh using virtual multipass instances. The cluster can be given an argument indicating the number of nodes the cluster should have. Default value is “1” node. The kubeconfig file is saved to the local directory with name k3s.yaml.

The cluster can be created in multipass in a matter of minutes, can be started and stopped in seconds and can be simply deleted by deleting (and purging) its virtual multipass nodes.

Different Kubernetes Contexts

You can work with several Kubernetes contexts in different ways.

One way is to define each context in its own file and set the according environment variable KUBECONFIG.

A second way is to merge two different configuration files into one using the command:

KUBECONFIG=~/.kube/config.backup:k3s.yaml kubectl config view --flatten > ~/.kube/config

Be aware that you should backup a working config before merging and that you should not overwrite an active configuration. Because of that I switched to working with different Kubernetes config files using direnv.

Contact

You can contact me using Twitter (my profile) or if you have comments regarding this tutorial, visit me on GitHub and file an issue or create a pull requests.

References