Your gift:
- the fastest and safest Kubernetes (k8s) download experience you can possibly imagine.
- warm fuzzy feelings, because you’ll be saving the k8s infra Special Interest Group (SIG) headaches and cash money.
And all we need is a trivial amount of your time to do update any hardcoded references of the Kubernetes soon-to-be-retired GCS bucket’s hostname to their new download endpoint:
That's cdn.dl.k8s.io, for the Ctrl+C, Ctrl+V crowd. It's me, I'm crowd.
Let me explain.
I lead Fast Forward, how Fastly helps support and empower everyone to build the open internet together, and we proudly accepted Kubernetes to our Fast Forward program earlier this year. It was an absolute no-brainer for us. Kubernetes is an outstanding example of open source at its best, with lived values that both our organizations share like inclusivity, trustworthiness, openness, and leadership. And of course, like most globally available and distributed software deployments running at massive scale, we use Kubernetes throughout our stack.
After we accepted Kubernetes to the program, the K8s Infrastructure SIG began the process of deploying Fastly in front of the GCS bucket hosting their binaries. They notified their userbase to prepare for the change by adding Fastly’s IP space to their allowlists, and got started working on their Fastly configurations. Then came the time to instantaneously send over 5 PETABYTES of monthly traffic, roughly 115 gigabytes a minute, to Fastly’s network. So Arnaud Meukam (a chair of the Infra SIG) and Ben Elder (a technical lead) convened with Fastly’s Mission Control team to get it done. Our Mission Control team is like Air Traffic Control for Fastly’s network. They monitor the network during huge live digital events like the Super Bowl, merch drops, or ticket sales to make sure nothing goes sideways.
Once we were gathered, Arnaud deployed the DNS changes necessary to begin routing traffic through Fastly’s caches. Everything was going very smoothly — traffic graphs were going up, binaries were being cached — but then Arnaud and Ben noticed something:
Watch 👀
Growing up in an unplanned way.
By way of a brief history, the Kubernetes project was started at Google around 10 years ago. Google engineers started building Kubernetes in 2013, and it was first released in June 2014. In 2015, Google and the Linux Foundation worked together to create the Cloud Native Computing Foundation (CNCF), with Kubernetes becoming its seminal project in 2016.
In the years since Kubernetes usage has exploded. According to a report from the CNCF, Kubernetes “is the second largest open source project in the world after Linux and is the primary container orchestration tool for 71% of Fortune 100 companies.” There are more than 74,680 devs who contribute to Kubernetes, and 5.6 million developers who use it (that’s 31% of all backend developers).
But as the community around the project grew, the way the project hosted and served its binaries to their global user base remained the same. The binaries stayed in a Google-owned bucket, to which few in the community had any visibility, much less access. Now, Kubernetes is planning to migrate to a new, community-owned bucket. Deploying Fastly is the first step in accomplishing that. Once as much traffic as possible is captured, they’ll spin down the old Google bucket and migrate over to the new one. With Fastly in front that transition will be invisible, as long as you’re using the abstracted domain. So update your references, or your stuff just might break!
How we can help the internet’s plumbers.
If you’ve spent any time working on or near the internet’s plumbing — maybe you’ve worked on building the pipes, or maybe you direct stuff through the pipes — you know how inscrutable and difficult this work can be. Especially when you’ve reached the size of Kubernetes, and even for scale and backend experts like the Infrastructure SIG working on Kubernetes. So any hindrance to accessibility or visibility? Scary.
When I asked Arnaud how he and the team felt managing this crucial piece of infrastructure, on which so much of the internet relies, he (graciously) said, “Serving 5 petabytes is a measure of success, regardless of how we want to present it. We’re struggling through that success. We want to make sure we guarantee the sustainability of our infrastructure over time, and it’s really difficult to maintain it as an open source project. We don’t pay people to do that.”
Elsewhere, he joked:
So I guess you could say it’s a mixed bag.
The immediate and majorly impactful thing you can do to help the K8s Infrastructure SIG is update your CI/CD pipelines, project build workflows, or any OSS repos you have that reference Kubernetes’ GCS bucket hostname to cdn.dl.k8s.io. If you don’t have a project that references the bucket’s hostname, maybe you could submit some PRs to a few of your favorite projects!
For Arnaud and the team, “having access and the capability to use a CDN, to be able to distribute binaries, without needing to worry about what’s happening to the network… it’s a good thing.” It occurs to me that while Fastly’s gift of bandwidth to the project is a big upgrade to their downloads infrastructure in terms of speed, security, and observability, perhaps the best gift of all is merely abstracting away the workload. It's the fact that there's one less thing to worry about, and when things go wrong, there are a lot more people worrying about it along with them. Because let's be real, it's a very fine line between stuff just working and chaos.
(Or less dramatically, a lot of stress and sleepless nights.)
So another thing we can do to support the internet plumbers is roll up our sleeves and help. Sometimes that may be code, or sometimes that may be care — in the form of a thank you, or coffee, or ko-fi. Whatever form your contribution takes, it is meaningful and important. And besides, isn’t this exactly what makes free and open source software, and this community, so special? It’s all of us, from all over the world, building together and helping each other. I think it’s a pretty unique thing, and it's a very special way to be in this world.
As the Infrastructure SIG’s community outreach to get references switched over to the new cdn.dl.k8s.io endpoint continues, you can watch their traffic tick up in real-time on our Developer Hub thanks to our Edge Observer dashboard. Binaries go brr!!
—
Oh! And if you know (or are!) a maintainer who needs help with scale, send them our way. We're here to help.
Now let’s go build the good internet — together ⏩
Top comments (0)