Raspberry Pi Kubernetes Cluster: Getting Started - Part#1

Raspberry Pi is a series of small single-board computers developed in the United Kingdom by the Raspberry Pi Foundation in association with Broadcom.

The Raspberry Pi, 4 Model B, is the latest version (as at the time of writing) of the low-cost Raspberry Pi computer. The Pi isn’t like your typical device; in its cheapest form it doesn’t have a case, and is simply a credit-card sized electronic board, similar to the type you might find inside a PC or laptop, but much smaller.

I had a few websites and woo-commerce projects hosted by a managed web hosting company, it recently occurred to me that I could host my projects from home after reading about the Raspberry Pi 4’s impressive specification. I did the maths; it worked out cheaper to host my sites (including the WordPress woo-commerce site) from home, so I decided to build a Kubernetes cluster using Raspberry Pi boards. That said, the cost-saving part of the decision process was only an excuse to justify getting myself some Pi toys. I derive a lot of fun tinkering with programmable boards.

Furthermore, some of my sites require databases, so there was a need to set up persistent storage for the database pods. The idea of a NAS server crawled into the picture at this point. As well as all the benefits that a regular NAS server provides, I configured mine to host and manage the Kubernetes NFS volume - this lets me mount NFS volumes on multiple pods.

I will use the Raspberry Pi blog series to document and share my experience, and the wrong/right decisions I made whilst tinkering with the Raspberry Pi.

In the end, I ended up building a Pi Project that does the following:

K3s Kubernetes Cluster

I containerised all the sites so I can use Kubernetes to orchestrate the deployments. The Rancher K3s Kubernetes cluster is managed via Ansible playbooks. I used K3s because it’s lightweight and super-efficient on the Pi.
Network Attached Storage

The NAS server serves two purposes:

a) It manages the NFS share and raid replication that is used for the WordPress MySQL (Kubernetes) persistent storage.

b) It is used to sync photos, files, videos etc. from our phones and laptops. We also stream contents from the NAS server to TV. It’s a private cloud for the family.

I used NextCloud open source software to manage the NAS server. It has a really good app for Android, iPhones.

The result of my setup is shown in the pictures below:

A cased up Pi booting from an SSD drive (using a SATA III USB 3 converter).

Stack ‘em up!

stack

I stacked the cased Raspberry Pis up using glue dots (they hold very well), and a rubber ring to hold the SSD drives together. I will explain why I did not use the regular Pi cluster casings in consequent blog posts.

stack2

Components

These are some of the components that I used:

One 8GB RAM and two 4GB RAM Raspberry Pi 4 boards

The 8GB RAM board is used as a NAS server, NFS persistent storage, and a Kubernetes worker node.

The first 4GB RAM board is used as the Kubernetes master node. I also configured it as a worker node; however, I ensured to use NodeAffinity to selectively run lightweight workloads on this nodes.

The second 4GB RAM board is a Kubernetes worker node - again, I used NodeAffinity to schedule heavier workloads on this node.

All three nodes run the Raspberry Pi OS.

The 8GB RAM node is still heavily underutilised - it’s roughly under 3GB RAM utilisation on average. So it’s probably not worth the investment, the 4GB is just what is needed for my kind of workloads.

I bought all three Raspberry Pi boards from RS-Components.
Three SSD drives:

I used a 500GB repurposed SSD for the NAS server, and two 250GB SSD drives for the master and worker nodes. However, I bought these SATA III 2.5 inch enclosures.

A combination of SSD and USB 3 gives you speed, which is most needed by the Kubernetes master node.
Acrylic case with a fan and 4pcs Heat sinks - The famous Pi cluster case didn’t work for me, because of the SSDs. I had to return it. I also needed the flexibility to be able to take the boards apart if needed in the future. The fan keeps the board cool, but it can be a little bit noisy when used with the 5V GPIO pin. Amazon link
USB Power - You need a 5V and 3.1A constant power supply for each board. I bought an extension cable that can also be used as a three-pin plug. Amazon link
3 park of 1.5m USB C cables. Amazon link
Silicon Power 32GB 3D NAND High-Speed MicroSD Card. You will need at least one SSD card to boot the Raspberry Pis. Amazon link
SD Card Reader. Amazon link