Docker Volume Management

Introduction

We are aware of the fact that Docker images are Read-only. When you run a container in Docker, it creates a writable layer on top of the image file system. This layer is used to store any changes made to the container during its lifetime, such as installed software or modified files. When a container is terminated, the writable layer is removed, and any data or settings that were stored in the layer are lost.

To solve this, we can use Docker volumes. They allow us to mount directories from the host machine or other containers into the container. By storing data and settings in a volume, they can be easily accessed and reused between container runs. There are three types of volume management:

• Host Volume
• Container Volume
• Docker Volume

👉 I will be going through each of them along with a short hands on examples

Host Volumes

Host volumes are directories on the host machine that are mounted into containers, allowing the container to read and write files directly to the host file system

✨ Useful for scenarios where we need to persist data outside of the container, or share data between the host and container

Simple Test

Creating a mysql_DB container

docker run -d \
> --name mysql_db \
> -e MYSQL_ROOT_PASSWORD=root \
> -e MYSQL_DATABASE=testDB \
> -v /docker_dir/con_volume_1:/var/lib/mysql \
> mysql:5.7

👉 The -e option lets us create root user's password and a test DB as well

👉 The line containing -v option means that any data written to the /var/lib/mysql directory within the container will be stored in the /docker_dir/con_volume_1 directory on the host, allowing the data to persist even if the container is deleted

💡 In Kubernetes, credentials are handled through a secret file

Checking docker process

docker ps

👉 We will be able to see the mysql5.7 container that we just created and started

In our host system,

ls -l /docker_dir/con_volume_1/
drwxr-x--- 2 polkitd input     4096 Mar  5 19:34 mysql
drwxr-x--- 2 polkitd input       20 Mar  5 19:34 testDB

The data is saved in our local host

We can also find the same data inside our container

docker exec -it mysql_db /bin/bash
bash-4.2# ls -l /var/lib/mysql
drwxr-x--- 2 mysql mysql     4096 Mar  5 10:34 mysql
drwxr-x--- 2 mysql mysql       20 Mar  5 10:34 testDB

Returning to the host and after deleting the container, we can still see the database data still remains

docker stop mysql_db
docker rm mysql_db
ls -l /docker_dir/con_volume_1
drwxr-x--- 2 polkitd input     4096 Mar  5 19:34 mysql
drwxr-x--- 2 polkitd input       20 Mar  5 19:34 testDB

😥 But using the -v option has a slight issue

Creating a new container with a different DB name

docker run -d \
> --name mysql_db \
> -e MYSQL_ROOT_PASSWORD=root \
> -e MYSQL_DATABASE=MYSQLDB \
> -v /docker_dir/con_volume_1:/var/lib/mysql \
> mysql:5.7

👉 The above should create a MYSQLDB but instead it shows the previous database data

From the host

ls -l /docker_dir/con_volume_1
drwxr-x--- 2 polkitd input     4096 Mar  5 19:34 mysql
drwxr-x--- 2 polkitd input       20 Mar  5 19:34 testDB

Inside the container

ls -l /var/lib/mysql
drwxr-x--- 2 mysql mysql     4096 Mar  5 10:34 mysql
drwxr-x--- 2 mysql mysql       20 Mar  5 10:34 testDB

👉 We can confirm that the container directory replicates the same data from the host directory. This means that the -v option is actually mounting the container's directory into the host directory.

Container Volume

Container volumes are volumes created and managed by Docker, and are associated with a specific container

✨ Useful when we need to isolate and share data between containers

Simple Test

Creating an index.html file inside the directory

cat > /docker_dir/con_volume_1/index.html
Nginx Main Page !!

Starting a container on port 8001

docker run -d --name Nginx_1 -p 8001:80 \
> -v /docker_dir/con_volume_1:/usr/share/nginx/html \
> nginx:latest

On port 8002

docker run -d --name Nginx_2 -p 8002:80 \ 
> --volumes-from Nginx_1 \
> nginx:latest    

On port 8003

docker run -d --name Nginx_3 -p 8003:80 \
> --volumes-from Nginx_1 \
> nginx:latest

👉 Here --volumes-from shares the host directory that is connected to the specified container with another container

Now from the host system

curl localhost:8001
Nginx Main Page !!

curl localhost:8002
Nginx Main Page !!

curl localhost:8003
Nginx Main Page !!

If we add something to the existing file,

cat >> /docker_dir/con_volume_1/index.html
Volumes From Test !!

curl localhost:8001
Nginx Main Page !!
Volumes From Test !!

curl localhost:8002
Nginx Main Page !!
Volumes From Test !!

curl localhost:8003
Nginx Main Page !!
Volumes From Test !!

👉 We can see that one volume directory is shared across several containers

Docker Volume

Docker volumes are volumes created and managed by Docker, and can be used by multiple containers, not just a single container.

✨ Useful for scenarios where we need to share data between multiple containers, or when we need to back up or restore volumes

Simple Test

Creating a docker volume

docker volume create myVolume_1
myVolume_1

To check current volumes

docker volume ls

DRIVER              VOLUME NAME
local               97e467c79d320138ef408f661ea6ce37d8294ce7e58524ae018d4225ad5ba914
local               db9eb4a8b3eb12c36056d398b65c6385315a28e3d3a43b7a7913afc94efc86aa
local               myVolume_1

👉 We can also use docker volume inspect myVolume_1 to inspect further details such as mountpoint of the volume

Now we can start a new container connected to the docker volume

docker run -d \
> --name mysql_DB \
> -e MYSQL_ROOT_PASSWORD=root \
> -e MYSQL_DATABASE=testDB \
> -v myVolume_1:/var/lib/mysql \
> mysql:5.7

Accessing the CLI inside the container

docker exec -it mysql_DB /bin/bash

Inside the container

mysql -u root -p testDB

mysql> create database testDB2;
Query OK, 1 row affected (0.00 sec)

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
| testDB             |
| testDB2            |
+--------------------+
6 rows in set (0.00 sec)

mysql> exit

👉 I created a testDB2 database to check results

From the host system

ls -l /var/lib/docker/volumes/myVolume_1/_data/
drwxr-x--- 2 polkitd input     4096  6월 23 11:53 mysql
drwxr-x--- 2 polkitd input       20  6월 23 11:53 testDB
drwxr-x--- 2 polkitd input       20  6월 23 11:55 testDB2

👉 We can confirm the database that we created inside the container can be seen from the host system

Read-only Docker Volume

Creating another docker volume

docker volume create myVolume_2
myVolume_2

Creating a new index.html inside this volume

cat > /var/lib/docker/volumes/myVolume_2/_data/index.html
My Nginx Web Site !!

ls -l /var/lib/docker/volumes/myVolume_2/_data/
-rw-r--r-- 1 root root 41  6월 23 12:06 index.html

Creating a new nginx container and checking

docker run -d --name Nginx_1 -p 80:80 \
> -v myVolume_2:/usr/share/nginx/html:ro \
> nginx:latest

curl localhost:80
My Nginx Web Site !!

👉 The ro is used to connect it as read-only

Now from inside this container if we try to edit the file

cat >> /usr/share/nginx/html/index.html
bash: /usr/share/nginx/html/index.html: Read-only file system

👉 As we mounted as read-only, we cannot write from inside the container

However, we can add contents from the host system

cat >> /var/lib/docker/volumes/myVolume_2/_data/index.html
Change Content !!

curl localhost:80
My Nginx Web Site !!
Change Content !!

Conclusion

In conclusion, Docker volumes provide a way to manage and persist data within containers. The three types of Docker volumes - host volumes, container volumes, and Docker volumes - offer different approaches to managing data, each with its own strengths and limitations ✔