Docker Engine - Deep Dive into Docker.

In my previous post I gave an Overview about Docker if you haven't read that then stop whatever you are doing right now and go have a look at the previous post I guarantee you won't regret it.
Here is a link to my previous post:
Docker - Basic Introduction

Now, let's Move on with our topic but wait before going to the engine itself I would like to clear some of the terminologies first that are used in Docker Engine.

1. Linux Namespaces: This is nothing new and has been in Linux for years now. Then, what is this namespace and how it is used in Docker??
   Namespaces are basically used to provide isolation to our containers. It limits the access to resouces for our container. Like a single Docker Engine can have multiple Docker Containers running on it and all of these containers are completely isolated from one another. Each Container thinks that it is the only container running on the host. If you ask a container about the other containers it will be confused as it thinks it is the only container running on this host and doesn't know anything about the other containers because containers are completely isolated.
   Even now if you didn't get it then think of it in this way namespaces basically restrict your permissions like it restricts how much amount of data you can view/modify/edit or delete. Also the processes running within a container are completely isolated from those services running on the host machine.

2. Control Groups : As from the name itself you can guess that this is something used to control stuff and your guess is right. Control Groups are basically used to limit the amount of resouces( CPU,Memeory, Networking.....) that a container can use like we don't want a single container taking up all of the underlying OS resources and leaving none for the other containers or processes of the Host machine.

3. Daemons: ooh they sound like evil monsters don't worry they are not something that your are thinking of. Daemons in computers are basically software/processes that run in background without intervening the user and performing certain tasks periodiaclly or in response to certain commands. They are basically processes that run in background and perform tasks for us.

4. Shim : Shims are nothing but programs that let newer API's to work on older hardware and older API's to work on newer hardware. It basically intercepts API calls and makes them compatible to use.

5. OCI : Open Conatiner Initiative started by GNU which gives an industry standards specifications for creating and running containers.

So in short Namespace are used for isolating and control groups are used to control the resouces being allocated to a container.

Now these things getting out of our way let's move further with the Docker Engine. But before we do that I just want to bring an important point to your notice which I missed Earlier, A Docker engine set up on a particular OS can run only those types of containers. Now what do I mean by that, it basically means that if I am running docker on top of Linux I will only be able run Linux Based Containers and not windows or Mac Based Containers but things are a little bit different on Windows since we got virtualisation support in windows 10 and with the use of wsl (windows subsystem for Linux) we can easily run linux on top of our native windows (if you want to know more about wsl or how did they achieved it then feel free to google about it.) So in windows you can literally run windows as well as linux based containers.

ooh that was a lot to digest but just bare with me all this will help you in understanding the core of Docker.

Docker Engine :

So now let's see how docker works under the hood.
First we have client that is us the end users whenever we type in commands in the CLI for docker, docker daemon get's activated remember what I said about daemons they run in background and responsd to certain commands or execute commands periodically. Now docker uses REST API to send commands to the docker daemon and in turn docker daeomon asks containerd. Now what is containerd - It is an industry standards runtime for containers. It's main responsibility is to maintain the container's lifecycle. (create/update/stop/restart or delete) Now how does containerd start a container it asks runc to do it. Now what is runc - It is nothing but runtime specification given by OCI (remembers OCI) for running containers.

Containerd basically communicates with the underlying kernel inorder to create containers using Liunx Namespaces and Control Groups as discussed earlier.
Just a point to be noted Runc is initialised when we need to create a container and after the creation of that container runc exits and only shim remains throughout the lifecycle of the container.

Think of it in this way, we send commands to docker daemon through the CLI(REST API) then the daemon asks containerd and if something needs to be created or updated it asks runc to co-ordinate with the kernel and complete the tasks assigned like creating a container.

Also, somethings are different in case of windows docker engine, in windows we don't have containerd or runc instead we have compute services which is microsoft's equivalence to containerd and does the same job as that of containerd. Also windows provides additional isolation methods as compared to linux known as hyperv isolation as I explained earlier we now have wsl in windows that means whenever we need to run linux containers on windows it fires up a lightweight vm in the background on top of which we run our docker engine and are able to run linux based containers on windows.
Also if you are wondering why we have different engines for each Operating Systems you see docker is unlike a Virtual machine it uses the hosts kernel and a windows Kernel is completely different than that of Linux Or Mac's Kernel.

So that was it for Docker Engine, I hope by now you are familiar with the docker engine and are aware of how docker works under the hood.

The next blogs will be on Containers, Images and Docker Hub and then we'll apply all of these concepts to make our first docker image and start a container from it so stay tuned for that :)