How can I secure the Akmod Signing Key

Ask Fedora
1

When setting up Akmod for Nvidia, drivers, the RPMFusion instructions for setting up the key and adding it to my MOK include this line:

Because the Secure Boot key is available locally on your computer, (by default it’s in /etc/pki/akmods) you might need to consider encrypting your rootfs as appropriate in order to protect the key. Please consider this as a mandatory requirement, or consider to transfer the key to an external (and secure) location. or even use an hardware token

I already have LUKS setup, so my filesystem is encrypted. And this is what is suggested in responses to other questions, for example RPM fusion: What to do with the AKmods key stored in /etc/pki/akmods.

I find this answer unsatisfactory.

Here’s why. Starting from this comment: RPM fusion: What to do with the AKmods key stored in /etc/pki/akmods - #6 by computersavvy

Theoretically no one except root and akmods should be able to ever see the content of those files.

SecureBoot, as I understand it, will primarily protect me against an attacker with remote access. That is, even if an attacker gains root access, SecureBoot + Lockdown should prevent the attacker from loading unsigned kernel modules, and if they use their root access to modify the files (kernel, modules, etc.) in my /boot partition, SecureBoot should detect that manipulation. I am basing this on this Reddit comment.

For that to work, an attacker, even one that has found a way to gain escalated privileges, must not be able to provide a legitimate signature for a kernel module.

If the Akmod key is sitting on my disk, then even if its permissions are set to “only root + akmod” can read, an attacker who has achieved privilege escalation to manipulate my /boot (which is also only writeable by root) can presumably also read the Akmod key.

That seems like it would defeat the whole purpose of SecureBoot

LUKS doesn’t help here, because once my system has booted, the disk is decrypted, and Root can obviously read anything on the disk. The attack that “put the Akmod key in an encrypted partition” protects against would be something like:

  1. attacker gains physical access to the machine
  2. attacker uses a different system to bypass OS-level permissions in order to read the akmod key and install a signed malicious kernel module.
  3. attacker yields physical access and leaves the machine in my possession, and I continue to use it unaware of malware.

That just isn’t a threat I have any expectation of dealing with on my personal/consumer-level laptop. The threats I care about are either (1) malware/remote access achieves privilege esclation XOR (2) my laptop gets stolen.

In the case of (1) that’s where I want SecureBoot to prevent the malware from installing kernel/firmware level malware, and in the case of (2) that’s why I have LUKS, and SecureBoot isn’t doing anything anyway because the attacker can either just turn it off or move the drive to a machine that has SecureBoot disabled. (I have not been able to find instructions on enabling Measured Boot for Fedora, which I guess combines the two, and might protect more against a combined software/hardware attack, but again, a combined software/hardware attack is not something I really expect to deal with! Problem for another day.)

What can I do?

Let’s look at the options mentioned on Rpmfusion again:

Because the Secure Boot key is available locally on your computer, (by default it’s in /etc/pki/akmods) you might need to consider encrypting your rootfs as appropriate in order to protect the key. Please consider this as a mandatory requirement, or consider to transfer the key to an external (and secure) location. or even use an hardware token

We’ve already encrypted our rootfs, and also I’ve argued that doing so is useless at accomplishing the thing that I want SecureBoot to do for me.

Moving the key to an external location is not practical. Having to plug in an external hard drive or usb stick in order to perform system updates is a level of inconvenience I’m not willing to accept. Additionally, I don’t know how I would configure the akmod builder to use an externally stored key, except maybe by mounting the external drive over top of /etc/pki/akmods before performing any operation that might trigger an akmod rebuild.

The only other option mentioned is “use an hardware token”. I have a yubikey I use for some web logins, though I haven’t ever experimented with using it for key storage for SSH or GPG keys, and I have no idea how I would configure akmod to sign kernel modules using my yubikey instead of a key stored in /etc/pki/akmods.

The other option I can think of is to set up local encryption for just the /etc/pki/akmods folder. I can’t use fscrypt because I’m on btrfs and as best I can tell that’s not supported by btrfs yet, but eCryptfs is apparently deprecated/unsupported, so I don’t want to use that. I also don’t want to repartition just for this. Maybe I can create an ext4 image inside of a file and secure that with some other crypt setup? That just seems very complicated though and the more I have to do to mount /etc/pki/akmods the more it seems like it would be easier to just put the mount for /etc/pki/akmods on a flash drive…

Is there not a better way to deal with this? I really just want the private key in /etc/pki/akmods to be encrypted with a symmetric key like my GPG key is, and have the akmod module builder just ask me for the password whenever it rebuilds kernel modules through dnf update or akmods --force.

2

Is there not a better way to deal with this? I really just want the private key in /etc/pki/akmods to be encrypted with a symmetric key like my GPG key is, and have the akmod module builder just ask me for the password whenever it rebuilds kernel modules through dnf update or akmods --force.

A MOK can be generated with a password when it’s created manually with openssl tools (see this document from Debian, as Fedora’s went missing), so the secure boot key architecture can definitely prompt for it.

It’s just akamod’s current shape of wrapping / offering it what doesn’t consider it.

3

This is a way beyond reality concern.

It is known that if a malicious user has physical access to a system that system can be broken into relatively easily. Using disk encryption is the best way to prevent that.

The absolute best protection is to ensure that no unauthorized persons ever have physical access to the system.

If they are able to get past your firewall, your secure remote access methods, and make those changes remotely then you should reconsider your security over all.

Even if an attacker is able to access the system and make changes that include forcing an attempt to install another key that would allow loading a malicious kernel module, they would need to also load that previously unknown key into bios which would also require a reboot.

There are many steps and layers to the current model for security and concern about the security of a mok key is only at risk when routine security layers have already been breached. If an unauthorized someone has obtained root level access then the entire system is immediately at risk, not just a single kernel module.

4

Forget about physical access. My system is encrypted, all is well as far as physical access is concerned.

Even if an attacker is able to access the system and make changes that include forcing an attempt to install another key that would allow loading a malicious kernel module, they would need to also load that previously unknown key into bios which would also require a reboot.

This is not true. If I am using Akmod with key signing, then I will already have installed my akmod key as a MOK, so a malicious user or virus or whatever will not need to do a reboot to install a new MOK, they can just use my Akmod key, which is accessible to Root.

My issue is, if a malicious user has root access, they can do a lot to compromise my system – replacing system tools, etc, but they shouldn’t be able to modify my kernel or kernel modules, even with root, at least not without failing SecureBoot. But if root has access to the Akmod key, then root can install malicious kernel modules and thereby modify the kernel. My understanding (again, see here) is that kernel modification makes it much more possible to any malware/virus/etc to hide its own existence or worse install itself into the system firmware, at which point even a full system wipe and reinstall might not actually remove it.

5

The root user can do anything so if an attacker has gotten that much access then the kernel modules are the least of your worries.

Your concern should be preventing an attacker from gaining access, not worrying about a kernel module specifically.

If you are that concerned then consider using one of the atomic versions that does not even allow root to make changes to most system files

It is quite easy to move the keys to offline storage and restore them only when necessary for compiling new kernel modules yourself.

6

This is not true.

When the kernel is booted with SecureBoot, it boots in Lockdown Mode, which blocks access to a number of facilities which could be used to modify the kernel, for example by disabling /dev/mem and /dev/kmem entirely. That means that not even root is supposed to have access to modify the kernel.

7

I think that what @computersavvy is trying to point out, to which I agree, is that you must apply defense in depth and apply a set of measures that complement each other to get a resulting system that is secure enough.

Secure Boot is not meant to be an absolute and infallible way of protecting the kernel, but just the boot chain for most of the use cases. As such, it can’t defend itself if the adversary is already in the system. You need to protect the system so that no one else than you can get root access.

Still, I think that the ability of generating the akmods’ private part of the MOK with a password that would be prompted when akmods sign the modules would be a nice optional feature to have for those that share your concerns (I don’t, I prefer to take the above approach as my general behaviour).

What you can do to prove your thinking as a concept is the following:

  • generate a new MOK manually with a password. It is an openssl generated key pair as shown in the Debian documentation I linked above, so this is 100% doable and I did it in the past when I was manually signing my modules.
  • back up your current akmods’ key
  • replace / place your newly created MOK in place of the akmods’ one
  • try to force an RPM Fusion module upgrade, to see if their process is already password friendly

There’s a chance it will work, there’s many chances it won’t. In any case, you can pitch the functionality and current experience results to the akmods’ package maintainer.

In the meantime, you can skip completely the akmods’ signing process and, with the key generated as above, manually sign your modules. I had written a script precisely for that that supports a MOK secured with a password – check this repo: https://forja.gvisoc.com/gabriel/signmodules. I haven’t maintained it lately but it should give you the tools.

EDITS: to fix typpos. I mean tupos. I mean tyops. Typos.

8

Which is why I would like to use SecureBoot to help make it harder for an attacker to hijack the kernel or install malicious firmware, rather than stop at “just don’t let an attacker get root”.

If I get malware I’m probably not messing around with trying to remove it; I have backups of anything really important, so I’m probably just going to nuke the system and clean install just to be sure. But it seems to me like SecureBoot can slightly increase confidence that reformatting and reinstalling will actually clear out anything on the system, by making it harder for malicious code to execute in kernel mode or during early boot stages when firmware related things are running.

It’s also partly that, if I am going to use SecureBoot at all, I would like it to actually improve system security in some way. Leaving the Akmod accessible to root is perfectly fine if you just want signed kernel modules so you can dual-boot Windows without having to mess with turning SecureBoot on and off all the time, which is many people’s motivation for it.

I don’t dual boot Windows, I only have Fedora, but I like to turn on any security features that will actually increase security in some way, if their inconvenience doesn’t outweigh the benefit.

That’s partly why I’m so interested in adding this layer of security without an extra substantial burden to deal with manual signing or manually locking/unlocking partitions, etc. If the Akmod key is unencrypted, then it seems like there’s little point to enabling SecureBoot at all, since if you have the root access necessary to modify the /boot, then you also have access to the akmod key to sign anything you put in /boot. But if it’s burdensome to mount/decrypt/unlock/sign etc in order to use signed akmods without leaving the key exposed all the time, then it doesn’t seem like the extra burden is worth the trouble. So in either case it seems easier to just not bother with SecureBoot at all.

But if I could secure the key without a substantial extra burden (e.g. “just type one more password during dnf update”), then it would be easy enough to be worth the tiny bit of extra security that enabling SecureBoot gains.

9

I think you’re ignoring the significant part of my answer (twice).

  1. Akmod’s key is not the only way to sign akmods’ modules for a computer with secure boot enabled. It’s optional.
  2. You can generate a MOK yourself with openssl. See below
  3. You can set a password to protect a MOK. It’s an openssl key pair.
  4. You can embed that in your update / upgrade workflow, and evolve the script I sent the way you need to evolve it so that you minimise your extra burden
$ openssl req -new -x509 -newkey rsa:2048 -keyout ./MOK.priv -outform DER -out ./MOK.der -days 36500 -subj "/CN=My Name/" -passout pass:foobar
$ sudo mokutil --import ./MOK.der
[sudo] password for gvisoc: ****
input password: [type here foobar...]

The same it prompts for the password for registering the MOK, it will for signing the modules.

When / if I get your attention on the above, I am happy to walk you through how your update process would look like without using Akmods’ key to sign akmods’ modules.

10

I will try this method to see if its within what I’m ok with dealing with for updates. I was just frustrated with responses that seemed to focus on ‘just don’t let malware get root’, which obviously I don’t intend to. Sorry.

Should I delete the keys in /etc/pki/akmods and put my new key there? Or should I store these somewhere else to avoid problems with akmod rebuilds and akmods-keygen@akmods-keygen.service? Looks like the keygen service won’t run if the keys exist, so I guess that won’t be an issue but I don’t want to break akmod building if it can’t support loading the password-protected MOK.

Does sign-file work if the module was previously already signed by akmods with a different key? E.g. if I don’t delete/replace the current content of /etc/pki/akmods?

Looks like I’ll need a few changes to that script; my akmod modules don’t appear to be compressed.

Why does the script try to restart failed units after signing?

11

Does it matter that the nvidia modules aren’t compress, I disabled compression to fix an issue.
Since then I addressed the issue in kmodtool but I decided to leave it disabled as it loaded quicker (0.5 second).

12

Not possible

Secure boot can ensure that the kernel and any modules loaded must be properly signed by a recognized key. Secure boot has absolutely no effect on protection other than during the boot process (Maybe also when manually loading a kernel module).

Unless you have already taken steps to prevent a malicious attack from all other sources, including intrusion prevention, it is wasted effort to worry about secure boot.

There are reasons that fedora has chosen to lock the root account during initial installation. This means that at least 2 things must be subverted before any malicious intrusion is possible.

  1. An existing user account must be accessed
  2. The use of sudo must be enabled for the compromised user account
    or some other method used to elevate privileges to root level.

For most home users the ISP router has its own firewall and the attached home LAN issues only private (nonroutable) IPs to the users devices. Many ISPs also have the users router on their own private network so breaking in adds the necessity for subverting the ISP internet gateway as well.

Corporate systems that have direct internet facing access usually have at least 2 levels of defense as well.
–internet gateway–webserver & honeypot servers-- router/LAN gateway–internal LAN (private network).

The LAN gateway is usually directly linked to only one of the servers in the intermediate lan and does not accept communications from any other source.

  1. Beyond the difficulty of breaking in from the internet through the intermediate routers and accessing a (usually) private IP address for your system, it has been reported many times that most intrusions are the result of compromised user accounts and social engineering.

Ensuring that user accounts have secure passwords, that no accounts are compromised by social engineering, that an IDS is in use, that the firewall is properly configured, and many more steps taken to prevent unauthorized access should all be of more concern than secure boot since once access is compromised the system can no longer be considered secure in any way.

13

I think it is technically possible, as the Linux Kernel has direct access to hardware interfaces which even root does not, since even if root is the king of userland, it is still in userland. So if you have kernel-mode access, you have more ability to do things like flash the uefi firmware or hard-drive firmware, I think. It’s a bit hard to research these things as search results are poor, but my understanding is that userland-root is not enough for some of these firmware attack vectors.

Those kind of firmware attacks, i.e. install malware into the uefi firmware or into the hard drive firmware, are what can persist even if you try to wipe the drive and reinstall the OS. Are they common? Probably not, but they are pretty nasty.

Yeah, SecureBoot ensures that the kernel and any loaded modules are properly signed … which means that I can trust that the Kernel isn’t running malware, which affects protection for as long as I have the computer on, not just during boot. The kernel’s lockdown mode then in-turn ensures that root doesn’t have access to kernel-only interfaces.

This is not helpful if e.g. an indirect dependency of a cargo crate/node module/NuGet Package/pip package I add to a project has a supplychain attack. From there it would be relatively trivial to e.g. insert alias sudo=... into my zshrc and hope I don’t see it before the next time I need to run a root command. So being sure that even root can’t run kernel-level code to access really low level firmware interfaces would be nice.

When I say remote access attacks, I am including viruses/worms/trojans/etc, not just an attacker trying to get direct access from the internet through whatever firewalls are in-between. That is why I keep saying things like ‘malware’ and ‘virus’, and I would appreciate it if you stop telling me to use a firewall and set a password, because I already have those kinds of things, thank you. This thread is about using SecureBoot in combination with Kernel Lockdown Mode to add an additional layer of security divider between userland-root and kernel-mode (which is the purpose of Kernel Lockdown Mode), without leaving the key to that extra security divider sitting around where userland-root can trivially access it. I do not need suggestions for other security steps I can take. If you don’t have suggestions for concrete ways to protect the key to kernel-mode against access by userland-root, then you are not helping.

14

If you want to start from scratch, I would recommend:

  1. Delete / uninstall the modules you’ve already signed with akmods’ key. That is to say, all akmods’ modules
  2. Unregister the key from Secure Boot with mokutils. You’ll have to do the One-Time password and reboot
  3. Uninstall akmods if you didn’t do it on step 1 with a wildcard
  4. Delete /etc/pki/akmods
  5. Following your own adventure with the method pointed out, from scratch.

When you install the first akmod-* module, it will pull akmods but, as not having a key in /etc/pki, it won’t try to sign. Then, you can try to sign it yourself.

I have no idea, hence my recommendation to uninstall everything to avoid any mess.

There’s a service that will appear as failed if there’s any kernel module unsigned in the last boot or right after installing the package. By restarting all the modules that are failed, you’ll restart the aforementioned service that will insert / initalise the modules with insmod, and then all services depending on a module being loaded (famously, VirtualBox’s), so you will most likely end up with a system that is fully usable without the need of rebooting.

15

You can trivially add an “if” to skip decompressing depending on the extension of the files found.

Feel free to send me a patch in any way, happy to merge.

16

Actually, the first reboot after installing the akmods package will automatically create the default keys under /etc/pki/akmods. This is done by the akmods.service if the keys do not already exist.

17

That’s good. I thought it had to be created by kmodgenca manually. In any case, you’d still have to register the key with mokutils, I remember the rest of the process working with no issues; therefore I guess that there’s no issue with one module being signed with more than one key.

18

If the system has not been rebooted since the akmods package was installed, then kmodgenca will create a key (if it does not already exist).
If the key already exists then kmodgenca can be used to overwrite the old key with a new one by forcing the action with an option to the command.

AFAIK a module may only be signed with a single key.