I’m looking at adding support for the nvidia drivers on top of silverblue, using package layering.
As part of this I’ve modified kmodtool and akmods to build the driver during the %post script here:
I also have some local changes to the actual nvidia drivers which are enough to make them build and install with the above changes. However for some reason the driver isn’t loading even though it is there, so I’m not putting that up for public yet. Instead I want to talk about some general issues with this silverblue and this approach.
When building the kernel module we need the kernel-devel package, and it needs to match the kernel version that is installed in the ostree image. This causes two problems:
First of all, the kernel-devel dependencies are not strictly tied to the kernel package (i.e you can have kernel and kernel-devel installed of different versions, even multiple of them). So, every time you layer a kernel-devel on top of the base silverblue image you need to manually specify the right version.
Secondly, even if the rpm dependencies would pull in the right version, that version may not be the most recent one, which means it will not even be available in the dnf repository. So, you have to manually dig for the right build in koji and layer it as a local rpm.
This is far from ideal, especially the fact that the kernel isn’t even available which means you can’t even script the dependencies realiably. The kernel tends to rev pretty often, and the ABI is incompatible on changes, so I think the only reasonable way this could ever work is that we bundle the matching kernel-devel with the base image. That seems to be about 50 megs, which doesn’t strike me as prohibitive.
Then we have a problem with the other dependencies that get pulled in. There is not actually that many (I’ve listed them all at the end of this post), so size-wise its not really a problem. However some of them are -devel packages which are version-tied with the ones in the base image. For example, there is glibc-devel and elfutils-libelf-devel, and if either glibc or elfutils-libelf has been updated in the yum repo since the base image was created then the package layering will fail.
I think this is less of a problem, because these packages are few and don’t rev so often. But, in practice it means that layering a new package or otherwise re-deploying will fail sometimes, and you will then need to first rpm-ostree upgrade, and then do the layering.
So, is there anything we could do to help here?
This is the layered packages i get when the nvidia driver is layered:
akmod-nvidia-3:415.22-1.silverblue1.fc30.x86_64 akmods-0.5.6-17.silverblue.fc30.noarch annobin-8.65-1.fc30.x86_64 cpp-8.2.1-5.fc30.x86_64 dwz-0.12-9.fc29.x86_64 efi-srpm-macros-4-1.fc30.noarch egl-wayland-1.1.1-3.fc30.x86_64 elfutils-libelf-devel-0.175-2.fc30.x86_64 fakeroot-1.23-2.fc30.x86_64 fakeroot-libs-1.23-2.fc30.x86_64 fpc-srpm-macros-1.1-5.fc29.noarch gc-7.6.4-4.fc29.x86_64 gcc-8.2.1-5.fc30.x86_64 gdb-headless-18.104.22.16881130-11.fc30.x86_64 ghc-srpm-macros-1.4.2-8.fc29.noarch glibc-devel-2.28.9000-26.fc30.x86_64 glibc-headers-2.28.9000-26.fc30.x86_64 gnat-srpm-macros-4-4.fc30.noarch go-srpm-macros-2-18.fc29.noarch guile-5:2.0.14-12.fc29.x86_64 ima-evm-utils-1.1-4.fc29.x86_64 isl-0.16.1-7.fc29.x86_64 kernel-devel-4.20.0-0.rc6.git1.1.fc30.x86_64 kernel-headers-4.20.0-0.rc6.git0.1.fc30.x86_64 kmodtool-1-31.fc30.noarch libatomic_ops-7.6.6-1.fc29.x86_64 libbabeltrace-1.5.6-1.fc29.x86_64 libglvnd-opengl-1:1.1.0-2.fc30.x86_64 libipt-2.0-1.fc30.x86_64 libmpc-1.1.0-2.fc29.x86_64 libva-vdpau-driver-0.7.4-22.fc29.x86_64 libvdpau-1.1.1-10.fc29.x86_64 libxcrypt-devel-4.4.1-1.fc30.x86_64 make-1:4.2.1-10.fc29.x86_64 nim-srpm-macros-1-3.fc29.noarch nvidia-driver-3:415.22-1.silverblue1.fc30.x86_64 nvidia-driver-cuda-libs-3:415.22-1.silverblue1.fc30.x86_64 nvidia-driver-libs-3:415.22-1.silverblue1.fc30.x86_64 ocaml-srpm-macros-5-4.fc29.noarch openblas-srpm-macros-2-4.fc29.noarch perl-srpm-macros-1-28.fc29.noarch python-srpm-macros-3-39.fc30.noarch python3-rpm-22.214.171.124-3.fc30.x86_64 qt5-srpm-macros-5.11.3-1.fc30.noarch redhat-rpm-config-125-1.fc30.noarch rpm-build-126.96.36.199-3.fc30.x86_64 rpm-build-libs-188.8.131.52-3.fc30.x86_64 rpm-sign-libs-184.108.40.206-3.fc30.x86_64 rpmdevtools-8.10-7.fc30.noarch rust-srpm-macros-6-1.fc30.noarch xemacs-filesystem-21.5.34-31.20171230hg92757c2b8239.fc30.noarch zlib-devel-1.2.11-14.fc30.x86_64 zstd-1.3.6-1.fc30.x86_64