Patricio Díaz

Posted on Nov 13

Setting up a local Kubernetes environment with Multipass and bridge networking

#kubernetes #ubuntu #virtualmachine #networking

Introduction

Preparing for the Certified Kubernetes Application Developer (CKAD) certification requires a reliable environment for hands-on practice. While cloud providers are often recommended, setting up a local environment can be more cost-effective and provide valuable learning opportunities. In this guide, I'll walk you through creating a robust local Kubernetes environment using Ubuntu's Multipass and bridge networking.

Why Local Setup?

When I started preparing for my CKAD certification, I wanted an environment that would:

Provide consistent networking between nodes
Allow full control over the infrastructure
Minimize costs compared to cloud solutions
Offer flexibility for experimentation

Hardware Requirements

Before getting started, ensure your system meets these minimum requirements:

Minimum Requirements:

CPU: 4 cores (8 threads recommended)
RAM: 16GB minimum, 32GB recommended
Storage: 100GB free space (SSD recommended)

My Test Environment:

CPU: AMD Ryzen 5 3600 (6 cores, 12 threads)
RAM: 64GB DDR4
Storage: 1TB SSD + 1TB HDD + 750GB HDD

Note: While you can run this setup on lower specifications, you might experience performance issues when running multiple containers and services.

System Compatibility

This guide has been tested on:

Ubuntu 24.10 (host system)
Ubuntu 22.04 LTS (Jammy) for VMs
Multipass 1.14.1
LXD 5.21.2 LTS

While the steps should work on other Ubuntu versions (20.04+), you might encounter slight differences in network interface names or default configurations. If you're using Ubuntu 22.04 or earlier, you may need to modify the networkd configuration syntax slightly.

Network Architecture

Before we dive into the setup, let's understand the network architecture we'll be creating:

Our setup creates multiple network interfaces:

Bridge Network (br0): Primary network for Kubernetes communication
Host Network (enp37s0): Connection to external network
VM Networks: Each VM gets two interfaces
- enp5s0: Multipass default network
- enp6s0: Bridge network interface

Setting Up the Network Bridge

Our first task is creating a dedicated network bridge for our Kubernetes nodes. We'll use systemd-networkd for this purpose, which provides a robust and modern networking stack.

1. Preparing the Network Environment

First, we need to disable NetworkManager and enable systemd-networkd:

# Disable NetworkManager
sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager

# Enable systemd-networkd
sudo systemctl enable systemd-networkd 
sudo systemctl start systemd-networkd 

# Configure DNS resolution
sudo systemctl enable systemd-resolved
sudo systemctl start systemd-resolved
sudo rm /etc/resolv.conf 
sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

2. Configuring the Bridge

We'll create a bridge network with the following specifications:

Network: 172.19.180.0/24
Gateway: 172.19.180.254
DHCP Range: 172.19.180.1 - 172.19.180.253

Create two configuration files in /etc/systemd/network:

Bridge device definition (50-br0.netdev):

[NetDev]
Name=br0
Kind=bridge

Bridge network configuration (51-br0-bind.network):

[Match]
Name=br0

[Network]
Address=172.19.180.254/24
IPForward=yes
ConfigureWithoutCarrier=yes
DHCPServer=yes

[DHCPServer]
PoolSize=253
EmitRouter=yes
EmitDNS=yes
DNS=8.8.8.8

DefaultLeaseTimeSec=600
MaxLeaseTimeSec=7200

[DHCPServerStaticLease]
MACAddress=52:54:00:ab:cd:01
Address=172.19.180.1

[DHCPServerStaticLease]
MACAddress=52:54:00:ab:cd:02
Address=172.19.180.2

3. Enabling IP Forwarding

To allow communication between nodes and the internet:

# Enable IP forwarding
sudo sysctl -w net.ipv4.ip_forward=1

# Configure iptables to allow forwarding
sudo iptables -P FORWARD ACCEPT
# use either this
sudo iptables -t nat -A POSTROUTING -o <host-nic> -j MASQUERADE
# or this
sudo iptables -t nat -s 172.19.180.0/24 -A POSTROUTING -j MASQUERADE

Creating Virtual Machines with Multipass

Multipass provides a lightweight way to create Ubuntu VMs. We'll configure it to work with our bridge network.

1. Setting Up Multipass with LXD

snap install lxd
snap connect multipass:lxd lxd
multipass set local.driver=lxd

2. Launching Kubernetes Nodes

# Control plane node
multipass launch jammy \
    --name ckad-cp \
    --cpus 2 \
    --memory 8G \
    --disk 20G \
    --network name=br0,mode=auto,mac=52:54:00:ab:cd:01

# Worker node
multipass launch jammy \
    --name ckad-wn0 \
    --cpus 2 \
    --memory 8G \
    --disk 20G \
    --network name=br0,mode=auto,mac=52:54:00:ab:cd:02

3. Optimizing Network Configuration

To ensure Kubernetes uses the correct network interface, we need to adjust the interface priorities on each node. Here's a quick way to do it using yq:

# Install yq
sudo wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq \
&& sudo chmod +x /usr/bin/yq

# Update netplan configuration
sudo cp /etc/netplan/50-cloud-init.yaml /etc/netplan/50-cloud-init.yaml.old
sudo yq '.network.ethernets.default.match.macaddress as $default | 
        .network.ethernets.extra0.match.macaddress as $extra | 
        .network.ethernets.default.match.macaddress = $extra | 
        .network.ethernets.extra0.match.macaddress = $default' \
    /etc/netplan/50-cloud-init.yaml.old | 
    sudo tee /etc/netplan/50-cloud-init.yaml

# Restart the node
sudo reboot

Troubleshooting Guide

Common Issues and Solutions

Bridge Network Not Coming Up

   # Check bridge status
   ip link show br0
   # If DOWN, try:
   sudo ip link set br0 up

VMs Can't Reach Internet
- Verify IP forwarding:
```
 sysctl net.ipv4.ip_forward
```

Check iptables rules:
```
 sudo iptables -L -v -n
```
Verify DNS resolution:
```
 dig @8.8.8.8 google.com
```

DHCP Not Working on Bridge
- Check systemd-networkd logs:
```
 journalctl -u systemd-networkd
```

Verify bridge configuration:
```
 networkctl status br0
```

VM Network Priority Issues
- Verify interface metrics:
```
 ip route show
```

Check netplan configuration:
```
 cat /etc/netplan/50-cloud-init.yaml
```

Verification Steps

After setup, verify your environment:

Network Connectivity

   # From VMs
   ping 172.19.180.254  # Should reach bridge
   ping 8.8.8.8         # Should reach internet

Interface Priority

   # Should show bridge interface as priority
   ip route | grep default

DNS Resolution

   # Should resolve correctly
   nslookup kubernetes.io

Performance Optimization Tips

VM Resource Allocation
- Don't overcommit CPU cores
- Leave at least 4GB RAM for host system
- Use SSD for VM storage when possible
Network Performance
- Enable jumbo frames if your network supports it:
```
 sudo ip link set br0 mtu 9000
```

Adjust network buffer sizes:

 sudo sysctl -w net.core.rmem_max=16777216
 sudo sysctl -w net.core.wmem_max=16777216

Conclusion

With this setup, you now have a robust local environment for CKAD preparation. The bridge network configuration ensures reliable communication between nodes, while Multipass provides easy VM management. This environment strikes a balance between functionality and resource efficiency, making it perfect for learning Kubernetes administration.

Remember to monitor system resources during heavy testing, and consider adjusting VM resources based on your workload requirements. While this setup requires more initial configuration than cloud-based alternatives, it provides valuable hands-on experience with networking and system administration concepts that will serve you well in your Kubernetes journey.

Next steps would be to install Kubernetes components and configure your cluster following the provided documentation by the course.

Happy practicing!

DEV Community