What I learned from automating millions of web site deploys

Back in 2018 I decided to demonstrate how it had become feasible to automate the deployment of a web site thanks to modern web tooling and hosting services. I had been immersed in the world of enterprise web projects where deployments involved code and content freezes, a deployment team, holding your breath, and hoping for the best. the idea was triggered by this throw-away tweet from Zach Leatherman.

Free side project idea: HTML-only static site generated clock that deploys a new version to @netlify every minute
— Zach Leatherman

So I made setyourwatchby.netlify.app, a site which displayed the time. No client-side JavaScript to get the correct time for each visitor, just HTML containing the time that the site was generated as part of an automated build and deployment (and localised pages served up thanks to some CDN routing).

I've described this in some detail in the past, but let's move along.

The point I was making was that the friction and risk in deploying updates to a site had been so dramatically reduced, that it was now safe to perform unattended deployments automatically every single minute of the day.

From 2018 to 2022 the site has been automatically rebuilt and redeployed millions of times.

Time and technology moves on

As my millions of site deploy logs on Netlify will illustrate, time moves on. Reliable build automation has become readily available for developers, even in the enterprise space. But deploying every minute is silly, really. It's fine for making a point as part of a demo, but we can be far more efficient and responsible than that.

I needed to retire the demo site, but also wanted to make sure that you, dear reader, still had a way to tell the time! setyourwatchby.netlify.app still had to function even after I turned off the scheduled builds.

Edge Functions to the rescue

The arrival of Edge Functions saved the day. Netlify's Edge Functions are a low latency serverless runtime which can handle HTTP requests at locations close to the user and modify the HTTP response being returned. They provide a way to dynamically service every HTTP request without the need to manage and maintain a server. They're JavaScript, so I could just write a single function and be done.

That's what you'll find if you visit setyourwatchby.netlify.app now. Your request will be intercepted by an Edge Function running on Netlify's CDN which will determine your locale based on your IP address, and insert your current time into the HTML that it is sending to you.

No more regular redeployments for this little demo site.

But what did I learn?

Over the course of millions of unattended deployments, I did learn a few things.

1. Don't let your mistakes "leave the kitchen"

Borrowing this notion from the catering trade, it's fine for mistakes to happen in the kitchen, but they must never reach the diner. Automating deployment on this (or any) scale is possible thanks to the model where builds which fail do not result in a deployment being published.

My logs show that I had a few handfuls of builds which failed over the course of the millions of deployments since 2018, but they never worried me. The previous good deploy would always be visible until replaced by a new one, so at worst the time would be a wrong for one minute if a build broke or was skipped.

This level of confidence in automation is key to its success. And these days we include all sorts of tests to our automated builds to check for accessibility, performance, and other integrity before they can "leave the kitchen" and take their final journey out to the hosting infrastructure.

2. Knowledge is power

A model of immutable deployments, where each new build becomes an instance of the site which can live on in perpetuity, is incredibly powerful and liberating. By retaining every "known state" of a site over the course of its life, you gain the ability to redeploy any previous version of the site in milliseconds. Or to address any version of the site via a unique URL. Risk plummets when you know that you're not disrupting the integrity of your site, and can always revert.

For posterity, here is what the site looked like the very last time it was automatically redeployed.

3. Automating got easier

In 2018, I used a third party service to trigger a build of the site every minute. You can create webhooks on Netlify which trigger a build when called so this worked nicely.

Since then, Netlify released Scheduled Functions which give a cron-like interface for executing a serverless function. If I were to need to redeploy a site regularly these days, I'd use these to reduce the number of external services in play and make my life a little simpler.

Try Edge Functions yourself

If you want to learn more about Netlify Edge Functions you can explore some examples here, or visit the docs. Scheduled Functions are also very handy and simple to use, and you can find their docs here.

Lastly, if you want to know the time, you can still visit setyourwatchby.netlify.app, but I suspect you might have some other resources to help you with that :)