Building a 32-Core Raspberry Pi Cluster From Scratch

A Raspberry Pi is a mini-computer board to which you can connect a monitor, mouse, and keyboard, and install a Linux-based operating system with a GUI. Or you can use it in “headless” mode with no GUI and run, for example, a database server. There are many usages you can give a Raspberry Pi—from building a Minecraft server to smart mirrors, the possibilities are endless.

Since I started to discover MariaDB and learned about database clusters, Distributed SQL, and Xpand, the idea of building a Raspberry Pi cluster has been in the back of my head. A cluster like this is a great way to experiment with distributed systems.

In this article, I’ll show you how to build a Raspberry Pi cluster with:

• 8 nodes

• 32 cores

• 64 GB of RAM

• 2TB of storage

What You Need

If you want to build your own cluster, you don’t have to buy 8 Raspberry Pi devices. Even one device is enough to learn stuff. The instructions in this article are useful even if you plan to configure only one Raspberry Pi. My advice, however, is to have at least three devices to build your cluster if you truly want to boost your skills in Linux administration, Ansible, Docker, Kubernetes, database clusters… you name it!

With that in mind, please replace 8 with whatever is your number. Do the same with the specifications of the Raspberry Pi devices if you have different models with less or more RAM or storage. Here’s what I used:

• 8 x Raspberry Pi 4 Model B (8 GB RAM)

• 1 x Anker 60W Desktop Charger (10-Ports)

• 8 x Short USB Type C cable (5A, 40W, 23cm)

• 8 x MicroSD card (256 GB)

• 2 x Raspberry Pi cluster case (4 layers with cooling fan)

Adapt the quantities to your setup. You can either get one or more Raspberry Pi power supplies if you have very few devices (say 1 or 2) or a multi USB charger. If you go with a multi USB charger, make sure that each port can deliver at least 2.4Am (5V). Pick microSD cards that fit your goals and budget as well. I recommend going with at least 32 GB SD cards. On the cables and case side, pick anything that accommodates your setup. You don’t necessarily need cooling fans, but I recommend them if you plan to leave your devices on for prolonged periods of time.

I also recommend getting all the ingredients before starting “cooking," especially if you want to use multiple Raspberry Pi devices.

Installing Raspberry Pi OS (Headless)

The operating system (OS) we’ll use is Raspberry Pi OS a Debian-based OS optimized for Raspberry Pi boards. Raspberry Pi OS is installed on the microSD cards, using your computer. You later connect these microSD cards to your Raspberry Pi devices and boot them.

Get all the microSD cards and Raspberry Pi boxes on your desk. You’ll need an SD card adapter (they normally come with microSD cards) or an SD card USB reader if your working computer doesn’t have a slot for SD cards. Bring a sharpie as well.

Download the Raspberry Pi Imager application and install it on your computer. Take a microSD card and connect it to your computer. Open the Raspberry Pi Imager application and click on CHOOSE OS. From the list, click on Raspberry Pi OS (other) and select Raspberry Pi OS Lite (64-bit) if your Raspberry Pi devices are 64-bit or Raspberry Pi OS Lite (32-bit) otherwise:

Click on CHOOSE STORAGE and select the microSD card. Double-check that you selected the correct drive, and click on the “gear” icon (advanced options). For hostname use rpi01, or something similar. You’ll be naming the devices as rpi01, rp02, rp03, etc.

Check the Enable SSH and Use password authentication options. Set a username (I’ll leave the default pi) and set a secure password.

Check the Configure wireless LAN option and enter the name and password of your WiFi connection. Configure your locale settings as well.

Since we have to do this once with each microSD card, make sure to set the Image customization options field to: to always use. This way the settings will be saved and the process will be easier for the next cards:

Now you can click SAVE and then WRITE to start the process. Once completed, eject the card and connect it to one of the Raspberry Pi devices, put it back into its box, and mark the box with the number 01. Repeat this process for all of the devices.

Assembling the Cluster

Depending on what kind of cluster case you have, the assembling process might vary. Follow the instructions that come with your case or look for photographs online to get a clear idea of how the devices should be placed in the layers.

I recommend looking ahead to decide which parts to assemble first. For example, with the case I picked, I had to install the cooling fans before mounting the Raspberry Pi devices on the layers.

Once you mount a Raspberry Pi, use a sharpie to write its number on the layer. This will help you in the future if you have to take a microSD card out for reconfiguration or any other administrative tasks that involve touching the hardware.

Once you have all the devices mounted, connect the USB cables to the Raspberry Pi devices and the multi-USB charger. You can optionally use Power over Ethernet and a network switch instead of the USB charger if you prefer, but I’ll live that for you to explore: definitely worth checking. You can either have the charger (or network switch if you go for it) next to your Raspberry Pi rack or attached to it. I used rubber bands to keep the charger attached to the side of the case. It worked fine.

With the USB cables plugged in, you are ready to start all those mini-computers! Connect the charger and switch it on. Depending on the Raspberry Pi models that you use, they might take some time to boot, so be patient. Meanwhile, enjoy the LEDs flashing and fans starting.

Connecting Through SSH

If everything went well, you should be able to reach the Raspberry Pi devices through SSH. Give it a try by running the following from your computer:

ssh pi@rpi01.local

Introduce your password and answer yes to add the device to the list of known hosts. Congrats! Your cluster is live now.

You may want to run commands to check the hardware information. For example:

• lscpu: Information about the CPU architecture

• df -H: File system disk space usage

• sudo fdisk -l: Partition information

• free -m: Amount of used, free and total amount of RAM

• cat /proc/version: Linux kernel information

Manual Configuration With raspi-config

If you have one Raspberry Pi, you can connect to it through SSH as described above and use the raspi-config utility program to configure various settings. You can modify the WiFi connection (S1), change the hostname (S4) and user password (S3), and expand the filesystem (A1), among many other things.

Automated Configuration With Ansible

If you have more than one Raspberry Pi in your cluster, you might want to automate the configuration process using a tool like Ansible. With Ansible, you can run a command on multiple machines without having to perform repetitive tasks. For example, let’s say we want to expand the filesystem on each Raspberry Pi. This can be done from the command line as follows:

sudo raspi-config --expand-rootfs

Without automation, you’d have to SSH to rpi01.local, run the command above, and end the SSH session. You’d have to repeat all these steps for rpi02.local, pr03.local, rp04.local, and so forth. Instead, you can simply tell Ansible to run the command for you on all the machines. Let’s see how to do this.

You have to install Ansible on a control node that is connected to your local network. It can be one of the Raspberry Pi devices, your working laptop, or any other machine as long as it runs a Unix-based operating system like Linux or macOS. I happen to have an old laptop that I repurposed as a dedicated database connected to my network so I used it as the control node (I’ll show how to install a database with replication in the Raspberry Pi devices in a future article). I won’t go through the instructions on how to install Ansible since the process is different on different operating systems. Check the official documentation and install it on your laptop or any device that you want to assign as a control node.

Before you start, I recommend generating an example configuration file that you can tweak later as you wish:

sudo su
ansible-config init --disabled -t all > /etc/ansible/ansible.cfg
exit

In this file, disable host key checking to simplify the process. Don’t do that in production environments! Change the following line:

;host_key_checking=True

to this:

host_key_checking=False

Next, you need to define an inventory. This is a list of machines that you want to control with Ansible. The inventory is defined in the /etc/ansible/hosts file. For example, you could add the following machines to the inventory using their IP addresses or hostnames:

[mymachines]
foo.example.com
bar.example.com
192.0.2.50
192.0.2.51

If the machines have some kind of pattern in their IP addresses or hostnames, you can alternatively use ranges. This is what you can do with your Raspberry Pi devices. Edit the /etc/ansible/hosts file by adding the following to the end:

[rpis]
rpi[01-08].local

Change the pattern to match your hostnames and numbers. This is equivalent to:

[rpis]
rpi01.local
rpi02.local
rpi03.local
rpi04.local
rpi05.local
rpi06.local
rpi07.local
rpi08.local

In this inventory rpis is an arbitrary name that you can use to refer to all the Raspberry Pi devices when running commands on them using Ansible.

You need to configure the SSH username that Ansible will use when connecting to the machines. Add the following to the /etc/ansible/hosts file:

[rpis:vars]
ansible_user=pi

Time to control de machines. A good start is to ping them:

ansible rpis -m ping --ask-pass

Type the SSH password. You should see in the output how the “ping” is answered by a “pong” for each machine (I’m showing only one here):

rpi01.local | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

Let’s back to the filesystem expansion (although, I think nowadays this is not required anymore, let me know in the comments if you can confirm this). To perform this action on all the machines, run:

ansible rpis -m shell -a "raspi-config --expand-rootfs" --become --ask-pass

And as simple as that, all your Raspberry Pi devices have an expanded filesystem. This is a good time to update the system on all the machines:

ansible rpis -m shell -a "apt update -y" --become --ask-pass
ansible rpis -m shell -a "apt upgrade -y" --become --ask-pass

You can rebook all the devices as follows:

ansible rpis -m shell -a "reboot" --become --ask-pass

And once you are done and want to safely turn all the machines off, just run:

ansible rpis -m shell -a "poweroff" --become --ask-pass

What's Next?

This is only the beginning. You now have a cluster of small computers ready to be controlled by Ansible. You can try setting up any kind of server software, including web servers or databases (see this example). There also is much more about Ansible. I merely scratched the surface here using ad-hoc commands, but you can for example create playbooks that contain the state in which you want your cluster to be. Explore the documentation and have fun!