Globally Replicated Services for the Rest of Us

After reading endless serverless horror stories, you decided to deploy your application on the internet's current darling, Hetzner. But now, your server is stuck in one location while your users are spread all over the world, suffering from terrible latency. What do you do? Let me introduce you to geolocation DNS-based routing!

The Problem: Global Latency

When hosting applications in a single region, users from other parts of the world experience higher latency. For example, if your server is in Germany, users from Australia might experience delays of 300ms or more. This latency can significantly impact user experience, especially for interactive applications. Of course, you could just put more servers in more regions, but how would a user know which server to connect to? You dont want eu.your-domain.com, us.your-domain.com, au.your-domain.com, right? Just your-domain.com that redirects to the closest server.

Anycast vs DNS based Routing

If you want to serve your application globally, you have two options, anycast and DNS based routing.

Anycast

One potential way to solve global routing is using Anycast IP addresses. Usually, an IP address is used by only one device. With Anycast, an IP address can be used by multiple devices in multiple locations. The magic here is done by BGP, which is a routing protocol that will usually just tell you what route to take in order to get to your destination IP. In the case of anycast addresses there are multiple possible routes and BGP will pick the route with the lowest hop count (which translates to the closest device!).

Anycast is used by large CDNs like Cloudflare, AWS, Google Cloud and also the newer providers like Fly.io. While Anycast is a super cool technology, it has one big downside: you need to own your own AS and IP addresses. This is usually not a problem for large companies, but for most startups this is not an option.

So, assuming you don't have your own AS, what else can you do? Let's take a look at DNS based routing.

DNS based Routing

DNS based routing works by returning different IP addresses during DNS lookups based on the user's location. Amazon Route53 supports this through the EDNS0 (Extended DNS) protocol extension, specifically using the edns-client-subnet extension. Here's how it works:

1. When a DNS resolver supports edns-client-subnet:
   • The resolver sends Route53 a truncated version of the user's IP address
   • Route53 uses this truncated IP to determine the user's location
   • Route53 returns the IP address of the server closest to the user's actual location
2. When a DNS resolver does not support edns-client-subnet:
   • Route53 falls back to using the DNS resolver's IP address
   • The user gets routed based on the resolver's location, not their own
   • This may be less accurate, especially if using a distant DNS resolver

For most users, this provides good enough accuracy - the average user will be routed to the closest server. However, there are some limitations which are discussed in detail in Christian Elsen's article. The TL;DR is that the accuracy is not perfect and can sometimes route users to sub-optimal locations. We are only talking about some added latency here though, so usually this isn't a dealbreaker.

The Solution: AWS Route53 Geolocation routing

We'll solve this by deploying our application to multiple Hetzner locations and using Amazon Route53's Geolocation based DNS feature to route users to their nearest server. This approach gives us:

• Lower latency for all users worldwide
• Better reliability through redundancy
• Cost-effective global presence (Starts at $0.5 / month)
• Easy implementation (No AS + BGP) needed

Implementation Guide

To implement this we will need to setup a few Hetzner servers that serve a basic http hello world and then configure Route53 to route requests to the nearest server. That means you will need to have a domain name, a hetzner account and an AWS account. Hetzner is optional here, you can of course do the exact same with any other VPS provider. DigitalOcean, Linode, Vultr, OVH, AWS, etc. They all do the same:)

If you want to give Hetzner a try, feel free to use my referral link for 20 Euro in free credits.

1. Set Up Hetzner Servers

First, deploy your application to multiple Hetzner locations. In this example, we'll use:

• Falkenstein (eu-central)
• Ashburn (us-east)
• Singapore (ap-southeast)

In this example I just create the cheapest server with IPv4 addresses. If you would want to do this in production you probably want to use a floating IP instead (more about that later)

Create an init.sh file that we will run on every server:

#!/bin/bash

set -e

sudo apt-get update -y
sudo apt-get install nginx -y

sudo rm -f /etc/nginx/sites-enabled/default
sudo rm -f /etc/nginx/sites-available/default

sudo bash -c 'cat > /etc/nginx/conf.d/hostname.conf <<EOF
server {
    listen 0.0.0.0:80 default_server;
    server_name _;

    location / {
        return 200 "\$hostname\n";
        add_header Content-Type text/plain;
    }
}
EOF'

sudo nginx -t
sudo systemctl reload nginx
echo "Server is ready"

And then create 3 servers, 1 per location. Of course you can also do fewer or more. If you don't have hcloud installed, just use the web interface!

hcloud server create --image ubuntu-24.04 --name eu-central-1 --type cpx11 --location fsn1 --user-data-from-file init.sh
hcloud server create --image ubuntu-24.04 --name us-east --type cpx11 --location ash --user-data-from-file init.sh
hcloud server create --image ubuntu-24.04 --name ap-southeast --type cpx11 --location sin --user-data-from-file init.sh

Then listing our servers we will see the IPV4s:

$ hcloud server list
ID         NAME           STATUS    IPV4             IPV6                      PRIVATE NET   DATACENTER   AGE
55560647   eu-central     running   116.203.41.15    2a01:4f8:1c1b:dbf2::/64   -             nbg1-dc3     1d
55560663   us-east        running   178.156.130.48   2a01:4ff:f0:3c8e::/64     -             ash-dc1      1d
55560674   ap-southeast   running   5.223.50.246     2a01:4ff:2f0:3a21::/64    -             sin-dc1      1d

Now you should be able to curl each of the IP addresses and get something back like "Hello, from eu-central". (or us-east, or ap-southeast)

curl http://116.203.41.15

These are really just 3 basic nginx servers that we will use to test our setup, you can do that with any other VPS provider as well. I just like Hetzner :)

2. Configure Route53

To configure Route53 we really just need to create a hosted zone and add geolocation records for each region. I usually do that with Terraform or AWS CDK, but in this case we are just doing it manually in the console.

First, let's start by creating a hosted Zone for our domain:

1. Create a hosted zone in Route53

After creating the hosted zone you will get ~4 Nameserver addresses that you need to setup in your domain registrars dashboard. Most registrars have a field "Custom Nameservers", thats where they belong. That basically means that any DNS request that belongs to your domain will be routed to the nameservers of AWS.

After creating the hosted zone and setting the nameservers in our registrars dashboard, we can create records. In this example I want to use the domain hetzner.example.com. So I create an A record for each region with the corresponding IP address, all records share the record name. As routing policy you will select geolocation routing, and then the region that corresponds to the server.

Repeat that for each region, and one additional record for the default region. Then you will end up with something like this:

That's it, we are ready to test!

Testing

Since we are working with DNS here, you might want to wait a few minutes before you test it out, just to make sure that the DNS changes propagated.

To test it out you will either need some friends in different regions or a VPN. Then you can simply curl the domain name and see which IP address you get redirected to.

$ curl http://hetzner.example.com # request from somewhere in Germany
eu-central-1
$ curl http://hetzner.example.com # request from somewhere in the US
us-east
$ curl http://hetzner.example.com # request from somewhere in Hong Kong
ap-southeast

At the time of writing I was in Germany, so I got redirected to the Falkenstein server. I then turned on an US VPN and got redirected to the Ashburn server. Then I turned on a Hong Kong VPN and got redirected to the Singapore server. And that's it :D

PS: If you do a traceroute you will see that the request will not go through the AWS network, but directly to the selected server.

What about cost?

The cost are (obviously) just a combination of your Hetzner compute + AWS Route53 costs. This is napkin math, your mileage may vary.

One hosted zone in Route53 costs $0.5 per month. Then, in addition to that you have the cost of your queries. For 1 million requests this costs $0.70. How many requests you get is very dependant on the type of service you have, really the only way to figure out what you will pay is by trying it out.

Since you pay by queries, reducing the number of queries will reduce your costs. You can do this by using a longer TTL for your DNS records, which is the time a DNS record is cached by the DNS resolver. By increasing the TTL you will also increase the time it takes to propagate the changes to all your users. If you need to change your IP address in a hurry (maybe because you accidentally deleted the server?) its possible that your users will experience some downtime. To combat this you can use a floating IP, which will cost you roughly $3 per region. If your volume is very high (far above 13 million DNS queries), it might make sense to increase the TTL and add a floating IP instead.

Limitations and Considerations

DNS based routing is not perfect, here are some things to keep in mind:

1. DNS TTL: Changes take time to propagate
2. VPN Users: May get routed to unexpected locations
3. Mobile Users: Carrier DNS servers can affect routing
4. Replication: You need to make sure that your application is properly replicated across regions, if data is involved thats not always easy.

As previously noted, Christian Elsen has written a great article about the limitations of DNS based routing here.

Anyway, I think the limitations are manageable and the benefits are worth it!

Conclusion

There is no free lunch. Globally replicated services come with costs other than just your increased Hetzner Bill. Most applications have data that they need to serve to their users, and now you have to serve that data from multiple locations. This means that you will have to deal with replication and consistency, or you still have one global data store (which will then be your single region bottleneck again). Topic for another article!

Using GeoDNS with Hetzner provides a cost-effective way to serve global users with low latency. While not as sophisticated as anycast solutions used by major CDNs, it's a practical approach for the rest of us :)

Interesting Links

During the research for this article I found a few interesting links that I want to share with you:

• Limitations of GeoDNS
• Anycast Wikipedia
• Hetzner Locations
• Route53 Geolocation Routing
• EDNS0 RFC

I hope you enjoyed this article and learned something new! If you have any questions or feedback, please let me know on Twitter @JonasScholz19, or at jonas@sliplane.io.