What is this about?
Docker is continuously moving towards commercialising their services, which is in a way correct thing to do to keep the ship sailing. One such change they brought in November 2020 is to rate limit the API calls being made to Docker Hub for Docker image pulls.
How we got impacted?
At our organisation, we run our CI system at fairly large scale, where there are thousands of developers actively consume the central CI system for their code commits. All the repos are Docker compatible hence most of them have rely on DockerHub for official images like centos, java, node, python, etc.
So, when this rate limiting was introduced it intermittently impacted our CI system where builds started failing with exceptions from Docker Hub like below :-
ERROR: toomanyrequests: Too Many Requests.
You have reached your pull rate limit.
You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits.
You must authenticate your pull requests.
Solution: Sonatype Nexus — Docker Registry
In my opinion Sonatype Nexus is an amazing, must have tool in a large scale CI system. We started using Nexus for proxying Java, NodeJS and also hosting private packages too.
So it was an obvious choice to use Nexus for solving DockerHub rate limiting issue. By proxying DockerHub and reducing dependency on DockerHub.
Docker Hub is the common registry used by all image creators and consumers. To reduce duplicate downloads and improve download speeds for your developers and CI servers, you should proxy Docker Hub and any other registry you use for Docker images.
We created a proxy repo in Nexus for DockerHub and added a small configuration to mirror the registry in our CI servers (which are autoscaled, so it was a change in the AMI).
How it works?
The first time you request an image from your local registry mirror, it pulls the image from the public Docker registry and stores it in Nexus before handing it back to the CI servers. On subsequent requests, the Nexus registry mirror is able to serve the image from its own storage.
/etc/docker/daemon.json
{
"registry-mirrors": ["http://nexus.domain.org:18000"]
}
Simple and effective !
For detailed information on configuration do check the official documentation of Nexus here https://help.sonatype.com/repomanager3/formats/docker-registry/proxy-repository-for-docker
Looking for More…!!
Pls do follow. I am planning to post more such blogs which are practical and scalable solutions around Docker, AWS, Python.
In case any help is required for setup or approach pls don’t think twice, before asking me.
Cheers!
Top comments (0)