DEV Community

Open Sauce Projects
Open Sauce Projects

Posted on

Access Nvidia gpu inside docker container in Arch linux

Why

  • If you need gpu acceleration in a container or just have too much free time on your hands, you can run all kinds of things. For example Large Language Models or GUI apps.
  • You can even create your own gpu accelerated docker image by using Nvidia ubuntu images

How

  • We have to install the NVIDIA Container Toolkit
  • The package can be installed from the Extras repository

    sudo pacman -S nvidia-container-toolkit
    
  1. Configure the container runtime by using the nvidia-ctk command. The nvidia-ctk command modifies the /etc/docker/daemon.json file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.

    sudo nvidia-ctk runtime configure --runtime=docker
    
  2. For the change to take effect we have to restart the docker daemon.

    sudo systemctl restart docker
    
  3. You can add the gpu using docker compose:

    service_name:
      image: ....
      deploy:
        resources:
          reservations:
            devices:
              - driver: nvidia
                count: 1
                capabilities: [gpu]
    

    If you use the docker run command, you can do:

    sudo docker run --runtime=nvidia --gpus all ....
    

Resources

Top comments (0)