Containers have been around for a long time starting with Linux Containers (LXC). In 2013, Docker revolutionized how engineers can use containers. Essentially, they made the idea of containers mainstream because they became far more straightforward to use.
With the growth of containers over years, they’ve went from being a toy to being relevant to being in just about every production environment.
Now that the wave of AI and ML is here, engineers need a way to run those workloads in a small form factor the same way that engineers needed to run application stacks in a more efficient way, which is what led them to containers in the first place.
In this quickstart, you’ll learn how to get a container up and running and connected to a GPU in 5 minutes.
Prerequisite
The first thing that you want to do is confirm that that your server/instance has a Nvidia GPU. If it doesn’t, the rest of this article won’t work on your system.
To confirm that a GPU is available, run the following:
lspci | grep -i nvidia
Install Docker
Without a container engine, you won’t be able to use a Nvidia GPU within a container.
To install Docker on Ubuntu, run the following:
sudo apt install docker.io
You can also use CRI-O or Containerd for the container engine.
Drivers
As with all hardware, Device Drivers are needed so the hardware can properly communicate with the Operating System. With containers and GPUs, it’s no different, so you will need to intall the Nvidia Device Drivers.
On Ubuntu, you can use the following package.
sudo apt install ubuntu-drivers-common -y
Once installed, use the package to install the automatic configuration for Nvidia drivers. You can also specify the GPU model, but the automatic option typically makes more sense as you may not know the specific GPU model or it may change.
sudo ubuntu-drivers install
CUDA Toolkit
The Compute United Device Architecture (CUDA) is, by definition per Nvidia docs: “a parallel computing platform and programming model that allows developers to use a graphics processing unit (GPU) for general-purpose processing”
To install the CUDA toolkit, run the following:
# Download the package
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
# Put the package in the package manager
sudo dpkg -i cuda-keyring_1.1-1_all.deb
# Update the system
sudo apt-get update
# Install CUDA
sudo apt-get -y install cuda-toolkit-12-6
Nvidia Container Toolkit
The last step is the Nvidia Container Toolkit, which gives you the ability to, as the name suggests, use a GPU within containers.
First, bring down the package.
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
&& curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
Next, add the package to the package manager.
sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
Update the system.
sudo apt-get update
Install the toolkit.
sudo apt-get install -y nvidia-container-toolkit
Configure the Ncidia Container toolkit to work with the Docker engine.
sudo nvidia-ctk runtime configure --runtime=docker
Restart Docker.
sudo systemctl restart docker
Test
Now that the Drivers, CUDA Toolkit, and Container Toolkit are installed, it’s time to test and see if everything is working as expected.
To test, you can run the following command:
nvidia-smi
You’ll see an output similar to the below specifying the Nvidia GPU.
Mon Oct 14 23:53:12 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.256.02 Driver Version: 470.256.02 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:00:1E.0 Off | 0 |
| N/A 46C P0 61W / 149W | 0MiB / 11441MiB | 93% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
You can now run a Docker Container and configure it to use GPUs like in the example below.
docker run --rm -it --gpus all nvcr.io/nvidia/pytorch:22.03-py3
Top comments (0)