Docker Volumes: Mastering Persistent Storage in Containers

Introduction

In this lab, you will learn how to manage data in Docker containers using volumes. Docker volumes are the preferred mechanism for persisting data generated by and used by Docker containers. This lab will guide you through various aspects of Docker volumes, including creation, management, data sharing, backup, and restoration. By the end of this lab, you will have a solid understanding of how to effectively manage data in Docker environments.

Understanding Docker Storage Options

Before we dive into Docker volumes, it's important to understand the different storage options available in Docker. Docker provides three main options for storing data:

1. Volumes: The preferred mechanism for persisting data in Docker.
2. Bind mounts: Connect a specific path of the host machine to a container.
3. tmpfs mounts: Store data temporarily in the host machine's memory.

In this lab, we'll focus primarily on volumes, as they are the most flexible and recommended option for managing data in Docker.

Let's start by listing the current volumes on your system:

docker volume ls

You'll see output similar to this:

DRIVER    VOLUME NAME
local     jenkins-data

This command lists all the Docker volumes on your system. The output shows the volume driver (usually "local") and the volume name. You might see some existing volumes, or the list might be empty if you haven't created any volumes yet.

Don't worry if you see different volume names or no volumes at all. This is normal and depends on what you've done with Docker previously on your system.

Creating and Managing Named Volumes

Now, let's create a new named volume. A named volume is a volume that you create explicitly and give a specific name. This makes it easier to reference and manage later.

Run this command to create a new volume:

docker volume create my_data

This command creates a new volume named my_data. Docker will handle all the details of where and how this volume is stored on your host system.

Let's verify that the volume was created:

docker volume ls

You should now see my_data in the list of volumes, along with any volumes that were there before.

To get more detailed information about the volume, we can use the inspect command:

docker volume inspect my_data

This will output something like:

[
  {
    "CreatedAt": "2024-08-22T14:31:09+08:00",
    "Driver": "local",
    "Labels": {},
    "Mountpoint": "/var/lib/docker/volumes/my_data/_data",
    "Name": "my_data",
    "Options": {},
    "Scope": "local"
  }
]

This output tells us several things about our volume:

• When it was created
• The driver it's using (local in this case)
• The mountpoint (where the data is actually stored on your host system)
• The name we gave it

Don't worry if you don't understand all of these details right now. The most important parts for us are the Name and Mountpoint.

Using Volumes with Containers

Now that we have a volume, let's use it with a container. We'll start a new container and mount our volume to it.

Run this command:

docker run -d --name my_container -v my_data:/app/data ubuntu:latest sleep infinity

Let's break down this command:

• docker run: This tells Docker to run a new container
• -d: This runs the container in detached mode (in the background)
• --name my_container: This gives our container a name, making it easier to refer to later
• -v my_data:/app/data: This mounts our my_data volume to the /app/data directory inside the container
• ubuntu:latest: This is the image we're using for our container
• sleep infinity: This is the command the container will run. It just keeps the container running indefinitely

Now our container is running with the volume mounted. Let's create some data in the volume:

docker exec my_container sh -c "echo 'Hello from Docker volume' > /app/data/test.txt"

This command does a few things:

• docker exec: This lets us execute a command in a running container
• my_container: This is the name of our container
• sh -c "...": This runs a shell command inside the container
• The actual command creates a file named test.txt in our volume with the content "Hello from Docker volume"

To verify the data was written, we can read the file:

docker exec my_container cat /app/data/test.txt

You should see the message "Hello from Docker volume" printed to the console.

Sharing Data Between Containers

One of the great benefits of Docker volumes is the ability to share data between containers. Let's create another container that uses the same volume:

docker run -d --name another_container -v my_data:/app/shared_data ubuntu:latest sleep infinity

This command is very similar to the one we used before, but we're giving the container a different name and mounting the volume to a different path inside the container.

Now, let's verify that this new container can access the data we created earlier:

docker exec another_container cat /app/shared_data/test.txt

You should see the same "Hello from Docker volume" message we wrote earlier. This shows that both containers are accessing the same data.

Let's add some more data from this new container:

docker exec another_container sh -c "echo 'Data from another container' >> /app/shared_data/test.txt"

This command appends a new line to our test.txt file.

Now, if we check the contents of the file from either container, we should see both lines:

docker exec my_container cat /app/data/test.txt

You should see both "Hello from Docker volume" and "Data from another container" in the output.

This demonstrates how Docker volumes can be used to share data between containers, which is incredibly useful for many applications.

Volume Backup and Restore

Backing up and restoring Docker volumes is crucial for data preservation. Let's walk through the process:

First, we need to stop and remove the containers that were using the volume. This is because we can't remove a volume while it's in use:

docker stop my_container another_container
docker rm my_container another_container

Now, let's create a backup of our volume:

docker run --rm -v my_data:/source:ro -v $(pwd):/backup ubuntu tar cvf /backup/my_data_backup.tar -C /source .

This command might look complex, so let's break it down:

• docker run --rm: Run a temporary container and remove it when it's done
• -v my_data:/source:ro: Mount our volume as read-only in the container
• -v $(pwd):/backup: Mount the current directory as /backup in the container
• ubuntu: Use the Ubuntu image
• tar cvf /backup/my_data_backup.tar -C /source .: Create a tar archive of the volume data

Now, let's remove our original volume:

docker volume rm my_data

To restore the data, we'll create a new volume and extract the backup into it:

docker volume create my_restored_data
docker run --rm -v my_restored_data:/dest -v $(pwd):/backup ubuntu bash -c "tar xvf /backup/my_data_backup.tar -C /dest"

This creates a new volume and extracts our backup into it.

Let's verify that the data has been restored:

docker run --rm -v my_restored_data:/app/data ubuntu cat /app/data/test.txt

You should see the content of the file we created earlier.

Summary

In this lab, you've learned how to work with Docker volumes. You've created and managed named volumes, used volumes with containers, shared data between containers, and performed backup and restore operations. These skills are essential for effective data management in Docker environments.

Key takeaways:

• Docker volumes provide a flexible and efficient way to manage persistent data in containers.
• Volumes can be easily created, inspected, and removed using Docker CLI commands.
• Data in volumes can be shared between multiple containers.
• Backing up and restoring volumes is crucial for data preservation and can be achieved using standard Linux commands.

As you continue working with Docker, remember to consider data persistence and management as a crucial aspect of your containerized applications. Always plan for data backup and recovery, especially in production environments.

🚀 Practice Now: Working with Docker Volumes

Want to Learn More?

• 🌳 Learn the latest Docker Skill Trees
• 📖 Read More Docker Tutorials
• 💬 Join our Discord or tweet us @WeAreLabEx