NVIDIA Container Runtime not recognized in Docker on Fedora

Ask Fedora
1

Hi,
I’m using Fedora 41 and trying to run GPU-enabled containers with nvidia-container-runtime and Docker. But Docker doesn’t seem to recognize the nvidia runtime.

System Info:

  • OS: Fedora 41 (x86_64)
  • Docker: installed and running
  • nvidia-container-runtime: present at /usr/bin/nvidia-container-runtime
  • Driver: nvidia-smi works fine on the host
  • daemon.json content:
{
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

What I’ve tried:

Confirmed nvidia-container-runtime is installed

Ran nvidia-ctk runtime configure --runtime=docker

Manually verified /etc/docker/daemon.json

Restarted Docker (sudo systemctl restart docker)

Running docker run --rm --gpus all …

fails because Docker doesn’t recognize the GPU runtime

docker run --rm --gpus all nvidia/cuda:12.3.0-base-ubuntu22.04 nvidia-smi

docker: Error response from daemon: could not select device driver “” with capabilities: [[gpu]]

What could be missing in this setup? Any help is appreciated!

2

Not a direct answer - Is Docker a requirement or will podman also work? Podman is working well for me with the nvidia-container-toolkit.

I use this as the source for the toolkit rpm: @ai-ml/nvidia-container-toolkit Copr