The ultimate goal for a Cloud Engineer and a Sysadmin who is interested in DevOps is to setup a Kubernetes cluster on their servers to manage applications cycles.
On bare-metal infrastructure (and homelabs), this can be quite a challenge, as we do not have any public cloud service to help us.
Using Load Balancers inside of our Kubernetes cluster seems impossible without a proper infrastructure. We do not have any BGP router, nor do we have a physical Load Balancer to guarantee failover.
In this article, I will be trying to give you a correct solution to expose your Kubernetes cluster to the Internet safely and for free.
The problem
Every person that has tried to setup a Kubernetes cluster on a bare-metal infrastructure has encountered the <pending> status under the External-IP field on their LoadBalancer services... and the headaches that followed.
Actually, on Public Clouds, the Load Balancing products (Cloud Load Balancing on Google Cloud) assign IP addresses to your Kubernetes services. Without one, Kubernetes doesn't actually know which IP address to assign to a specific Service.
The solution
Fortunately, the Kubernetes ecosystem is quite vast and a solution to this specific problem has been developed, it is called MetalLB.
Note that the L2 mode of MetalLB doesn't guarantee a zero downtime failover, because of the ARP protocol. While the ARP cache is not refreshed, the users would hit a specific node that is no longer up. It can sometimes last many minutes.
To install MetalLB, please refer to the documentation here : MetalLB, bare metal load-balancer for Kubernetes
MetalLB (L2 mode) uses Address Resolution Protocol (ARP) to associate a Service to a specific IP address. This allows Kubernetes to balance the load using the internal kube-proxy component that establishes a connection to a specific Pod.
In practice, we tell MetalLB to use a specific pool of IP addresses, 10.0.0.100-10.0.0.200, to expose services. The nginx service would have its External IP set to 10.0.0.100.
Precision : An ARP request is broadcast (given that we don't know who is at the specific IP address), but the previous diagram has been simplified. The concerned MetalLB Operator (not all MetalLB operators) responds to the ARP request by giving the MAC address of the Service and the Pod of the application. The user can now access the application. Although, the user and the Kubernetes cluster must be located in the same broadcast domain in order for it to work.
Read more about MetalLB L2 (and its limitations) here.
To indicate an IP Address pool to MetalLB, we can use the following manifest :
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: home-pool
namespace: metallb-system
spec:
addresses:
- 192.168.1.100-192.168.2.200
We must also tell MetalLB to advertise this pool using Layer2 (and not BGP) :
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: home
namespace: metallb-system
spec:
ipAddressPools:
- home-pool
The given IP addresses must not be in use. It would lead to a conflict, and thus, connectivity issues. To avoid any problem, I recommend you to exclude this pool from your DHCP range.
Now, MetalLB will assign an IP address to your service, which you can get with kubectl get svc :
Your service is then exposed on your local network. You can access it directly, but you might want to expose it on your domaine name.
You can also install an Ingress Controller to route your different domains and subdomains !
Expose your service to the Internet
Historically, a simple solution was to setup a PAT, which means to bind a port of your public IP address to a specific port of a private IP address.
This solution has different limitations :
- You would be exposing your home network
- Your public IP address might change (depending on your ISP)
- You would not always be able to bind a port to a virtual IP address
A possible solution to avoid these limitations would be Cloudflare Tunnels. It is an agent that you can install on a machine of your home network, which will establish a connection between your network and Cloudflare, like an IPSec VPN for instance.
This method has many advantages :
- Cloudflare would be acting as a reverse proxy, thus, would not expose your home network
- Cloudflare would be managing your public TLS certificates (no need to renew them every 3 months !)
- No configuration is required, the agent install being quite easy
To set this service up, you would need to configure the NS fields of your domain name to the Cloudflare servers. Follow the steps on Cloudflare Dash. You can then go to the "Zero Trust" category and choose a plan (the free one preferably). Then go to the Tunnels page under the Access category.
Click on the Create a tunnel button, give a name to your tunnel (for example: home), and choose an install method for the agent. I recommend using Docker and to host it outside of your Kubernetes cluster, so that, if your cluster goes down, the tunnel remains active. Please note that if the agent is down, so is your website.
When the tunnel status becomes HEALTHY, you can associate a subdomain (or your main domain) to a private IP address (and even to a specific port if necessary). Cloudflare will host a proxy to your local address via a subdomain, then configure a CNAME field to expose it to your (sub)domain.
If you use an Ingress Controller, you would need to set the value of the Host HTTP header manually, under the HTTP Settings category. Your controller will then know which service you try to access.
Alternative solution
Normally, I would have talked about a free and open-source solution to resolve this specific problem. However, given the current Internet state and the difficulties to get IPv4 addresses, the easiest way of solving it was to rely on a 3rd-party provider. Here, Cloudflare. I do not have any specific relation with them, other than being a customer.
If you do not want to rely on a "Giant tech company", you could use the dynamic DNS principle to assign, even if it changes, your public IP address to a DNS A field, using PAT. This does not cancel the other limitations.
Note that you would always rely on a provider for your domain names. This is only a question of opinions. Cloudflare being an omnipresent company on the Internet, and defending freedom of expression, I feel okay using their services.
But that's your choice !
Top comments (0)