How to use Nvidia container-toolkit without disabling selinux?

Ask Fedora
, , , , ,
1

Hello, I am looking to use the ollama container run LLMs using podman. I have intalled nvidia container toolkit and it seems to only have access to the GPU if I add --security-opt=label=disable to the podman run command. Without it, I get the following error: Failed to initialize NVML: Insufficient Permissions.

How can I go about using my GPU with containers without disabling selinux? Any policy I have to install, or selinux configuration I need to change?

Not familiar with selinux, so I appreciate any guidance provided. Thank you! :pray:

2

Try to reproduce the problem and check the output:

journalctl --no-pager -b _AUDIT_TYPE_NAME=AVC
3

Hello I see the following entries:

AVC avc:  denied  { getattr } for  pid=4836 comm="nvidia-smi" path="/dev/nvidiactl" dev="devtmpfs" ino=909 scontext=system_u:system_r:container_t:s0:c227,c662 tcontext=system_u:object_r:xserver_misc_device_t:s0 tclass=chr_file permissive=0

AVC avc:  denied  { getattr } for  pid=4836 comm="nvidia-smi" path="/dev/nvidiactl" dev="devtmpfs" ino=909 scontext=system_u:system_r:container_t:s0:c227,c662 tcontext=system_u:object_r:xserver_misc_device_t:s0 tclass=chr_file permissive=0

AVC avc:  denied  { read } for  pid=4836 comm="nvidia-smi" name="nvidiactl" dev="devtmpfs" ino=909 scontext=system_u:system_r:container_t:s0:c227,c662 tcontext=system_u:object_r:xserver_misc_device_t:s0 tclass=chr_file permissive=0
1 Like
4

Temporarily switch SELinux to permissive mode to collect the complete denial log:

sudo setenforce 0

Then use the collected log to create and install a permissive module policy:
audit2allow | policycoreutils-python-utils Commands | Man Pages | ManKier

With the module policy installed you can switch SELinux to enforcing mode:

sudo setenforce 1
5

For those that found this question via a search engine there is a solution that works that worked for me.

This assumes that you use the rpmfusion nvidia packages.

These instructions provide a working solution: https://copr.fedorainfracloud.org/coprs/g/ai-ml/nvidia-container-toolkit/