Talk: Third-party modules (e.g. Nvidia) are not rebuilt during system upgrade

This is a discussion topic for the following Common Issue:

You can discuss the problem and its solutions here, but please note that debugging and technical feedback should primarily go to the issue trackers (e.g. Bugzilla) linked in the Common Issue, because that’s the place that developers watch, not here.

If there are any updates/changes/amendments for the Common Issue description, which you believe should be performed, please post it here.

1 Like

Please see the Common Issue for solution/workarounds:

Is this necessary? We don’t have it included in the Common Issue description.
If akmods run the compile processes in background, is there a better way to ensure the processes have finished, rather than waiting some time and hoping it was enough? Grep the process list perhaps?

The script doesn’t seem to offer the functionality.

Possibly not. Once the akmods command ends (while running in the foreground) the kmod package has already been installed. I suggested that for additional insurance.

Note that on the initial install of akmod-nvidia the fact that akmods is still performing the background compile after the install is complete and a prompt reboot without waiting does interfere with the compile and install of the kmod-nvidia-$(uname -r) package.

The current understanding of the problem is that some (system upgrade) process kills the akmod task before it ends. The usual workaround is to run sudo depmod -a
(then modinfo nvidia will works).

See also 2011120

Note that this issue never occurs with dnf distro-sync --releasever=39 (some insight are explained on the bug).

Im running Fedora 40 with a Nvidia RTX 3050 and updated my MSI PRO B650M-P Bios

Since the update i get the message NVIDIA kernel module missing, falling back to nouveau
Rolling back to older bios does not work

I tried every method described in the article, the error message persists

Is there something i can do or am i following the wrong path?

             .',;::::;,'.                 shured@p200300eb97135f009a1621d5cb706ca8
         .';:cccccccccccc:;,.             ----------------------------------------
      .;cccccccccccccccccccccc;.          OS: Fedora Linux 40 (Workstation Edition) x86_64
    .:cccccccccccccccccccccccccc:.        Host: MS-7E27 (1.0)
  .;ccccccccccccc;.:dddl:.;ccccccc;.      Kernel: Linux 6.9.11-200.fc40.x86_64
 .:ccccccccccccc;OWMKOOXMWd;ccccccc:.     Uptime: 13 mins
.:ccccccccccccc;KMMc;cc;xMMc;ccccccc:.    Packages: 2536 (rpm), 8 (flatpak), 39 (brew)
,cccccccccccccc;MMM.;cc;;WW:;cccccccc,    Shell: fish 3.7.0
:cccccccccccccc;MMM.;cccccccccccccccc:    Display (LG ULTRAWIDE): 3440x1440 @ 50Hz [External]
:ccccccc;oxOOOo;MMM000k.;cccccccccccc:    DE: sway
cccccc;0MMKxdd:;MMMkddc.;cccccccccccc;    WM: Hyprland (Wayland)
ccccc;XMO';cccc;MMM.;cccccccccccccccc'    Cursor: Adwaita
ccccc;MMo;ccccc;MMW.;ccccccccccccccc;     Terminal: kitty 0.35.2
ccccc;0MNc.ccc.xMMd;ccccccccccccccc;      Terminal Font: FiraCode-Regular (10.5pt)
cccccc;dNMWXXXWM0:;cccccccccccccc:,       CPU: AMD Ryzen 5 7600X (12) @ 5.45 GHz
cccccccc;.:odl:.;cccccccccccccc:,.        GPU 1: AMD Raphael @ 0.60 GHz [Integrated]
ccccccccccccccccccccccccccccc:'.          GPU 2: NVIDIA GeForce RTX 3050 6GB [Discrete]
:ccccccccccccccccccccccc:;,..             Memory: 3.40 GiB / 30.45 GiB (11%)
 ':cccccccccccccccc::;,.                  Swap: 0 B / 8.00 GiB (0%)
                                          Disk (/): 155.67 GiB / 1.82 TiB (8%) - btrfs
                                          Local IP (enp12s0): 192.168.178.20/24 *
                                          Locale: en_US.UTF-8

       09:48:20
NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

Sorry my fault
After the BIOS flash, secure boot was active again
Secure boot prevents NVIDIA modules to be loaded as mentioned here:

Secure boot prevents the drivers from loading while the key has not been imported into the bios, and a reload of the bios does wipe out keys that may have been previously loaded.

Simply redo the mokutil --import /etc/pki/akmods/certs/public_key.der process that is described in the README that is noted in that post and it should solve the problem with secure boot.

2 Likes

For what it’s worth, I haven’t seen in any other thread the specific solution order that Jeff V put together here:

Which, expanded out to include the commands from the README file, would read something like the Proposed Addition below, I think.

This complete set and order of instructions worked for me when the individual steps posted in the current Common Issues thread, and elsewhere online, never did work correctly. Should these be added to the Common Issues thread in order to help it address the broadest possible scope of folks’ issues?

–Proposed Addition–

IF you have installed the Nvidia drivers from RPMFusion (and nowhere else), and are getting the “NVIDIA kernel module missing. Falling back to nouveau” message during startup, run the following commands from a terminal:

  1. sudo kmodgenca --auto --force - This creates the certificate for the Nvidia drivers that can then be enrolled in the UEFI, and ensures that they have been freshly created for the currently installed driver version
  2. sudo mokutil --import /etc/pki/akmods/certs/public_key.der - This places the certificate for the Nvidia drivers in the UEFI, ready to be “enrolled” (accepted) by you later - use a password that you’ll enter in a few minutes at the MOK Management prompt
  3. sudo dnf remove kmod-nvidia-$(uname -r) - This removes any Nvidia kernel modules that weren’t properly signed to match this new certificate
  4. sudo akmods --force - This rebuilds Nvidia kernel modules with the proper signature to match the new certificate
  5. Reboot the system
  6. Respond to the MOK Management prompt - usually involves pressing Enter, then using the Up and Down arrows to select to enroll the new key, and using the password that you entered in Step 2

1 is not required. The key is automatically generated with the first boot after akmods is installed. Your method creates a new key and replaces what may already be there. Updating the driver later then gets signed with this new key and if the original key was already imported into the bios now the modules will not load. The instructions say to use kmodgenca -a which will create the key if not already there but does not overwrite an already existing key. (too many keys imported into the bios can be detrimental since bios has a limited space to store the keys and there have been threads on that specific problem)

2 is following the instructions in the readme and on rpmfusion how-to

the README.secureboot file covers steps 1 & 2 & 6

3 is for a completely new install that installs akmods at the same time as installing akmod-nvidia. The module will be compiled as soon as the install is complete and before the key has been created. It then has to be recompiled with the key after the key is created. With changes that have been done to the akmods package and command that step is no longer required.

3 & 4 are combined now with the use of akmods --rebuild --force which both removes the old kmod-nvidia-$(uname -r) package and compiles and installs the newly signed package.

The full process is

  1. install the driver dnf install akmods-nvidia
  2. create and import the key using the steps in the README.secureboot file
    a. keygenca -a
    b. mokutil --import /etc/pki/akmods/certs/public_key.der
  3. recompile the driver with akmods --force --rebuild
  4. reboot (and complete the mokutil import during the boot)

Do not use the keygenca -a --force since that is 1) not required & 2) may cause problem if the key had previously been created and imported into bios. Once the key has been created there is really no reason to create a new key.

1 Like

Thanks, that is awesome info! It would be fantastic if that were embedded in, or linked to, somewhere in the RPMFusion guides, as I feel that those do a good job of separately explaining the concept of how akmods works for Secure Boot, and of explaining how to install the Nvidia drivers, but that the combination of the two is tricky to navigate (perhaps just my experience)

Experience has shown that if the user first installs akmods then does the above in a different order step 3 can be avoided, but all is still required.

The new order would be

  1. install akmods sudo dnf install akmods
  2. create and import the key using the steps in the README.secureboot file
    a. sudo keygenca -a
    b. sudo mokutil --import /etc/pki/akmods/certs/public_key.der
  3. install the driver sudo dnf install akmod-nvidia
  4. wait for the driver to be compiled & installed (in the background)
  5. reboot (and complete the mokutil import during the boot)

Note that this actually adds a user enforced delay in step 4 that is hidden in the previous listing by doing step 3 which requires the delay before it completes. It does, however, eliminate the need to perform the akmods --force step

Also note that this process is relevant to a new installation only and that the failure to properly build the modules during a system upgrade (this topic) may only require the
sudo akmods --force --rebuild command. It is assumed that the state of secure boot, creating the signing key, and the key import into the bios would have previously been completed and not altered. This is another case where using the --force option with the keygenca command would cause more headaches than cures.