Enabling gpu support in docker

Some workloads like video processing or machine learning tasks heavily benefit from having access to gpus. Docker containers are mostly popular for their isolated nature, but they can also provide gpu resources from the host inside containers - if configured correctly.

Installation for nvidia graphics cards

The host running the docker daemon will need to be prepared in order to expose gpus to containers by installing the nvidia container toolkit.

For debian based distros, this might look like:

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.18.0-1
sudo apt-get update && sudo apt-get install -y --no-install-recommends \
   curl \
   gnupg2
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y \
    nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
    libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

These instructions are for v1.18; check the toolkit link above for the current version and adjust the variable at the top accordingly if you want a newer version.

Once complete, you can test if your installation worked by running the nvidia-smi command inside a container that was passed the --gpus all flag:

docker run --rm -it --gpus all debian nvidia-smi

If you get no error, your host can now run gpu accelerated containers.

Note that you do not need a special container image; running any container image with --gpus all with automatically make the nvidia-smi command available inside it

Installation for amd graphics cards

Graphics cards from amd do not require special software installation as long as you have the amdgpu kernel module loaded and the card is supported by the Radeon Open Compute platform (ROCm).

You can check if the required module is installed and active by running:

lsmod | grep ^amdgpu

If the command returns a line starting with amdgpu the module is running; if you get no output then you may need to install and load the amd graphics driver module first.

Since amd does not expose a container toolkit that docker can work with as of 2025, you instead pass the driver-related devices into the kernel directly and disable seccomp to allow full control of system calls. *(This also increases the attack surface; do not run containers with amd gpu support for workloads you do not trust!)*Verify your amd gpu is available in containers with

docker run --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined debian amd-smi

If you get no error message, you are ready to run gpu workloads in docker.

Enabling gpu support for containers and compose projects

If you have an nvidia graphics card, you can run a single container with gpu acceleration like:

docker run --rm -it --gpus all linuxserver/ffmpeg

or an entire compose project, for example ollama:

services:
  ollama:
    image: ollama/ollama:latest
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]

Special caveat: video tooling is disabled by default for nvidia cards; add video to the NVIDIA_DRIVER_CAPABILITIES environment variable to allow containers to use video processing features

For amd gpus, you instead need to pass the necessary devices into the container:

docker run --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined linuxserver/ffmpeg

this also works for compose projects:

services:
  ollama:
    image: ollama/ollama:latest
    devices:
      - /dev/kfd
      - /dev/dri
    security_opt:
      - seccomp=unconfined

Adding these few lines and options is all you need in order to provide gpu support to container workloads.

Common pitfalls for video processing inside containers

For amd graphics cards, there is only one issue users regularly run into, namely using an unprivileged user inside the container. Since the container user needs to interact with protected resources in /dev, they require access to those. Running as the root user inside the container avoids this problem entirely.

There are two common pitfalls with nvidia cards and containers:

The first one shows up when trying to run video processing workloads inside containers. Let's assume you want to test if your nvidia card is working inside a container with a command like

ffmpeg -f lavfi -i testsrc=duration=5:size=1280x720:rate=30 -c:v h264_nvenc -preset fast -y out.mp4

This should simply encode a blank frame as a video using nvidia's nvenc encoder (which only works if an nvidia graphics card is accessible).

You may get an error like this:

[h264_nvenc @ 0x555c732f18c0] Cannot load libnvidia-encode.so.1
[h264_nvenc @ 0x555c732f18c0] The minimum required Nvidia driver for nvenc is 470.57.02 or newer

You may be thrown off by the second error talking about version numbers, but if you see the first one complaining about a missing shared library, that is more likely your culprit. The error is caused by nvidia's default settings, which enabled all capabilities except video processing by default. Running the container with the specific capabilities added resolves the problem:

docker run --rm -it --gpus all -e NVIDIA_DRIVER_CAPABILITIES=compute,video,utility

The second common error does involve driver versions, for example error message like this:

[h264_nvenc @ 0x55aad6126700] Driver does not support the required nvenc API version. Required: 13.0 Found: 12.2
[h264_nvenc @ 0x55aad6126700] The minimum required Nvidia driver for nvenc is 570.0 or newer

tell you that the ffmpeg version is too new for your graphics cards driver. You can either try to update your graphics cards driver on the host, or use an older build of ffmpeg if that is not possible. In my cases, picking a base image that matches your host OS will avoid these issues entirely, e.g. if you run on debian 12 you should run the debian:bookworm image instead of just debian or debian:latest (which default to trixie, aka debian 13 now).