Collecting useful results from this thread
Basically that bug in that link indicates there were permissions issues with the systemd-logind.service accessing the swapfile. They recommended adding an override to the service to set a debug flag.
When I did that I found it just reported that access was denied to the swapfile, with no further details given.
While you could just disable all checks of the swapfile before hibernating, those checks are there for a reason so it’s probably better to fix it so they work than to just disable them.
What’s happening on my system, and almost certainly on yours, is that SELinux is denying our access to the swapfile we created manually. If you were to run sudo audit2allow -b you’ll see:
#============= systemd_logind_t ==============
allow systemd_logind_t unlabeled_t:dir search;
The fix isn’t the obvious one though, you shouldn’t just blindly add that to your policy. First, you should actually label your swapfile how it’s expected to be, with the swapfile_t label. The swapfile_t label needs to be added to the /var/swap folder and everything inside it, which we can make a permanent rule and then apply it to the current system with:
sudo semanage fcontext -a -t swapfile_t '/var/swap(/.*)?'
sudo restorecon -RF /var/swap
You can confirm your swapfile is now correctly labeled with ls -Z /var/swap/swapfile.
Now when we try to hibernate and it fails, audit2allow gives us a new policy recommendation:
#============= systemd_logind_t ==============
allow systemd_logind_t swapfile_t:dir search;
allow systemd_logind_t unlabeled_t:dir search;
The unlabled_t is leftover from the previous denials, and we don’t want that. Since it’s much easier to use audit2allow to generate the policy for us than it is to do it by hand, I recommend just rebooting to clear the audit.log, running only the hibernate command again, and then using audit2allow to create policy for only the one denial.
Reboot, then only run sudo systemctl hibernate after the boot, then the following to generate policy from the denial and load the policy:
sudo audit2allow -b
# check that only the swapfile_t policy recommendation is listed
sudo audit2allow -b -M systemd_hibernate
sudo semodule -i systemd_hibernate.pp
The system should now hibernate (or at least flicker and log you out if there are other hibernate related problems that cause it to immediately resume)
5 Likes
For my own benefit, when I find this later, the full set of steps for hibernation setup is the following.
1. Create the btrfs subvolume
sudo btrfs subvolume create /var/swap
2. Calculate the needed size of your swapfile
Use swapon to get the size of your current zram swap. The size you need for the swapfile (to be safe) is:
(ram_size - zram_size) + (zram_size * 2)
Or equivalently:
ram_size + zram_size
3. Create the swapfile
You no longer have to do all the steps by hand, btrfs has a subcommand to do them all for you;
sudo btrfs filesystem mkswapfile --size 40g --uuid clear /var/swap/swapfile
I have 32G or RAM and 8G of zram, so I’m creating a 40 G(i)B swapfile in this case.
4. Find the UUID of the swapfile
sudo findmnt -no UUID -T /var/swap/swapfile
Looks something like ffe8a473-6525-4a46-a7e1-e7fea5bdf20e
5. Find the offset on the disk
Btrfs has a native command for this too now, and it doesn’t require any math to convert it.
sudo btrfs inspect-internal map-swapfile -r /var/swap/swapfile
Looks something like 6148378
6. Add the kernel parameters for the swapfile
sudo grubby --args="resume=UUID=ffe8a473-6525-4a46-a7e1-e7fea5bdf20e resume_offset=6148378" --update-kernel=ALL
Replace the ffe8a473-6525-4a46-a7e1-e7fea5bdf20e and 6148378 with the values you found in the prior steps
7. Add the swapfile to your fstab with low priority
Add a line like the following to the end of your /etc/fstab. This sets the priority of the swapfile as low as it can possibly be so it won’t get used unless it has to:
# add swapfile, but set priority as low as possible so it only gets used if zram (pri=100) can't be
/var/swap/swapfile none swap defaults,pri=0 0 0
8. Enable swapfile
sudo swapon /var/swap/swapfile
9. Add SELinux labeling
Add the permanent rule for the label, and apply the rule to the files.
sudo semanage fcontext -a -t swapfile_t '/var/swap(/.*)?'
sudo restorecon -RF /var/swap
10. Reboot
You want to have a clean audit.log for the next step, and the best way to do that is to reboot and do absolutely nothing else except the steps that follow.
11. Try to Hibernate (it won’t work)
After you login, try to hibernate from the command-line. It won’t work, but it generates audit events we can use for the next steps.
sudo systemctl hibernate
# it won't work, but that's expected
12. Generate SELinux policy
Check to make sure you don’t have other random things in your policy.
sudo audit2allow -b
You should only get back:
#============= systemd_logind_t ==============
allow systemd_logind_t swapfile_t:dir search;
If you get back anything else, you’ll need to manually edit the generated policy, and recompile it by hand.
Generate the policy from the audit2allow rules, which gives you both a *.te file and a compiled *.pp file. If you need to do manual changes, you’ll use the .te file and overwrite the .pp file from this.
cd /tmp
sudo audit2allow -b -M systemd_hibernate
13. (Optional) Manually edit and re-compile the policy
If you had extraneous results in your audit2allow, those are now in the policy it generated for you. Luckily it provides you the source file (.te) as well as the compiled policy file (.pp), so you can edit and recompile.
Edit the policy file and fix it up. Editing SELinux policy files is beyond the scope of this, but in simple cases might not be too hard. Make sure to remove unnecessary requires as well as the polcies.
sudo nano systemd_hibernate.te
Then re-compile it into a .pp file.
sudo checkmodule -M -m -o systemd_hibernate.mod systemd_hibernate.te
sudo semodule_package -o systemd_hibernate.pp -m systemd_hibernate.mod
14. Load the new SELinux policy
Load the new policy into the permanent database.
sudo semodule -i systemd_hibernate.pp
15. Try to Hibernate (again)
This time it should actually work. It’s possible you have other problems though. One common case is the screen only going black for a second and then coming back to the login screen, which can be caused by a few different things.
Troubleshooting
Adding an override to the systemd-logind.service so it include more debug data in the journalctl should almost always be your first step.
- Add the override folder and file
The normal systemctl edit method doesn’t seem to work for me for some reason.
sudo mkdir -p /etc/systemd/system/systemd-logind.service.d/
sudo nano /etc/systemd/system/systemd-logind.service.d/override.conf
You can undo this change later by just removing the file and folder that were created, and rebooting.
- Populate the override file
Add this to the file and save it:
[Service]
Environment=SYSTEMD_LOG_LEVEL=debug
- Reboot your system.
Restarting the service logs you out and causes weird bugs in the service, so just reboot.
- View the logs for the service
sudo journalctl -u systemd-logind.service
# scroll to the bottom with 'GG' to see the most recent logs
6 Likes
These are the services installed for nvidia GPUs when a user installs the proprietary drivers from rpmfusion. The specific ones I mentioned are the suspend, hibernate, and resume services.
$ ls /lib/systemd/system/nvidia-*
/lib/systemd/system/nvidia-fallback.service /lib/systemd/system/nvidia-powerd.service
/lib/systemd/system/nvidia-hibernate.service /lib/systemd/system/nvidia-resume.service
/lib/systemd/system/nvidia-persistenced.service /lib/systemd/system/nvidia-suspend.service
Digging into what actually happens (I do not use suspend or hibernate) I see that the procedure creates and saves the gpu ram data to a file and restores from the same file so it is not in the swap space. The data is saved, just not to swap.
1 Like
Yes I know how to scroll but it was just a lot of mumbo jumbo…
But this was smart:
I only have 8GB physical RAM, yet this is the output:
$ swapon
NAME TYPE SIZE USED PRIO
/dev/zram0 partition 7,4G 0B 100
/var/swap/swapfile file 12G 0B 0
No idea why zram0 is not 50% of my RAM. It’s much bigger! So swapfile will need to be 16GB.
I will go for 20GB instead just to test.
Edit:
I did swapoff /var/swap/swapfile and
sudo rm /var/swap/swapfile
then followed the steps to create a 20g swapfile, (steps 3 to 6) and rebooted. Unfortunately the behaviour did not change.
1 Like
So I removed /etc/systemd/system/systemd-logind.service.d/override.conf and /etc/systemd/system/systemd-hibernate.service.d/override.conf, rebooted, cleaned up journalctl, ran sudo systemctl hibernate. There was no warning about swap size 
But it didn’t work: screen went black for 1 sec, then login screen. I notice the following messages in the log:
Sleep mode "disk" is supported by the kernel.
Disk sleep mode "platform" is supported by the kernel.
/dev/zram0: ignoring zram swap
/var/swap/swapfile: detection of swap file offset on Btrfs is not supported
/var/swap/swapfile: device matches configured resume settings.
Hibernation will attempt to use swap entry with path: /var/swap/swapfile, device: 259:3, offset: 8332544, priority: 0
Enough swap for hibernation, Active(anon)=1632748 kB, size=20971516 kB, used=0 kB, threshold=98%
But also lots of these (with different destinations), not sure if related?
Got message type=signal sender=:1.16 destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=UnitRemoved cookie=2662 reply_cookie=0 signature=so error-name=n/a error-message=n/a
This is the full output of sudo journalctl -u systemd-logind.service from the moment I run the hibernation command, with debug enabled:
This is readable if you hit download and open in a text editor with word wrap disabled.
I can’t find anything specific in there, any moment when it fails to hibernate.
FYI: In my bios (HP Spectre x360, i5-1135G7), TPM STATE is enabled, TPM DEVICE is available.
Should I disable TPM STATE? Or would that require me to re-install Silverblue?
I barely use my laptop outside of my own house so I am not sure if TPM is really important for security in my case.
1 Like
This may be the issue. The swapfile used for hibernation should not be on btrfs since that would require loading the kernel and drivers for btrfs before that file could be read.
Place the swapfile on an ext4 file system or use a separate swap partition and it should work.
Hibernate always requires a fixed on disk location to function and always has. The kernel is loading the swap content as a binary blob from the physical disk before file system drivers are loaded when it resumes from hibernation.
SSDs certainly prefer to move content around and distribute writes across a range of internal physical hardware bits (like eMMC in an SD Card), and modern file system drivers try to assist with that by distributing churn across a dynamic range of disk interface addresses. But SSDs aren’t anywhere near as brittle as they were 15 years ago when they first started getting popular. If you’re not using it your SSD for high churn server content 24 hours a day for year on end, you’re unlikely to run into these types of issues anymore. They are still kit as robust as platter disks, but for an average consumer it’s basically an invisible difference now.
As an example Optane disks for are specifically designed to have a high speed SSD as a semi-hidden hibernation and sleep cache, exactly like this swapfile, attached to an HDD. The drivers for them are largely Windows-only, and hit have some extra logic, but still.
2 Likes
fix hibernate on btrfs swapfile (F40) by MartinNowak · Pull Request #2076 · fedora-selinux/selinux-policy · GitHub should be in selinux-policy-targeted-40.17-1.fc40.noarch
2 Likes
Summarized from some posts on the thread these were pulled from.
TL;DR
Is Secure Boot important? Yes.
Should it be made better? Yes.
Do you benefit from it in its current form (likely until >2028)? Almost certainly no.
Can Secure Boot and Hibernate work together? Not without disabling even more important security features, or the kernel adding a feature it requires but hasn’t yet even designed.
Why Secure Boot has to be disabled, but that’s OK.
Everyone (including me) rejects the idea out of hand of disabling Secure Boot to get Hibernate working. However, Secure Boot isn’t currently (likely until >2028) what you think it is.
Fedora, like a number of more secure distros, includes and enables the Lockdown kernel driver. This driver enforces certain minimum security standards from the driver level to make sure you, as a somewhat ignorant/lazy/pragmatic/realistic end-user, don’t accidentally do something to break the security of your system. Part of what it includes however is a conditional switch based on whether Secure Boot is enabled in the UEFI/BIOS or not.
If Secure Boot is enabled, the Lockdown driver requires hibernation to use a kernel-specific implementation of “encrypted hibernate swap”, otherwise the driver will block hibernate. The standard LUKS encrypted swap partition/file however doesn’t qualify, and in fact the only thing that does qualify is a kernel implementation that doesn’t exist and hasn’t even been designed yet. So effectively you can’t do hibernation with the Lockdown driver present and Secure Boot enabled. Lockdown still has a lot of security benefits though, so you don’t want to disable it.
The way Secure Boot works in practice currently (probably >2028) isn’t really very secure. It’s promoted in concept by a lot of people and organizations, rightfully so, but the current implementation for desktop systems doesn’t really add much security.
The security mitigation of Secure Boot is mostly supposed to be against physically-present attackers (i.e. Evil Maid), attacking a system that is off, to bypass other existing security mechanisms, in order to add malicious software or access protected data. Additionally, unless you’ve taken very extreme manual steps in setting up Secure Boot, signing keys from Microsoft, your hardware vendor, Fedora, and any signing keys you manually added (e.g. dkms/akmods driver signing keys) can all be used to sign things for secure boot. Microsoft has already had a few cases where malware was found to be signed with their keys just recently, and most hardware vendors are so terrible at security that they just use the same keys for almost everything and their private signing keys get leaked often.
So current Secure Boot implementation really only gives you protection in the very niche case of:
- You have other security protections against tampering/access (e.g full disk encryption)
- Your system is off
- The attacker has physical access to the system
- The attacker doesn’t have Microsoft or hardware vendor signed binaries
Most people rarely turn off their laptops anymore, usually just suspending/sleeping it instead, so requirement 2 usually isn’t met if requirement 3 is. Requirement 2 and 3 together means it’s not just a drive-by-hack in a coffee shop but a dedicated attacker that’s probably targeting you specifically, so they’re likely to have the tools that make requirement 4 invalid.
So keeping Secure Boot enabled directly blocks the use of hibernate in Fedora systems, a major necessary feature for most laptop users, in order to provide only an incomplete part of the mitigation of an incredibly niche threat.
Side note: As of Linux kernel 6.2 , only properly functioning TPM hardware is supported anymore. Unfortunately as of now, most TPM hardware is NOT properly functioning and has hardware design defects that make them non-compliant with the TPM standards and unsupported by Linux >=6.2. If you had a working pre-6.2 setup with Secure Boot enabled but one of these bad TPMs, and then upgrade to post-6.2, your system simply won’t boot past the initial grub menu while Secure Boot remains enabled. There is no fix except to either disable Secure Boot or replace the hardware (it’s a hardware design issue though, not a manufacturing issue).
So you’ll probably be forced to turn off Secure Boot on many laptops anyway, just so they can function at all.
3 Likes
This is a very thorough commentary, but it lacks addressing self-signed binaries & builds of bootloaders for Secure Boot, which is often a better practice rather than relying on Microsoft signed binaries.
Most of the UEFI firmwares allow addition of user-generated signatures to firmware bootloader memory in one way or another (via built-in or third-party tools).
One of the existing solutions is to use Secure Boot with self-signed self-compiled GRUB2 binary which can include built-in user-defined or locally generated LUKS paths & optionally decryption keys for convenience (but for security reasons its better not to do that), leaving this GRUB binary the only unencrypted file on the drive.
This process is well documented on different resources (e.g. Arch Wiki), but is yet to be implemented in mainstream distributions.
1 Like
You’re correct that the steps are documented. The removal of existing pre-enrolled keys, adding of your own custom key, and then setting up install time local self-signing is exactly what I was referring to when I mention complex extra manual setup steps to make Secure Boot secure.
But absent that removal of existing keys, which can be very dangerous, is rarely ever recommended, and usually only done by the most skilled users, it can probably be circumvented by compromised pre-enrolled keys. Which is the case probably 99% of users are in.
1 Like
Which means there is a need for a well-designed user-friendly way of doing these steps automatically. But for some reason no one has made it yet.
1 Like
Ah, you are correct!
After a reboot:
First hibernation > grub menu
Second hibernation > no grub menu, proper wake after hibernation
Third and every following hibernation > grub menu
I tried to compare log from the “good” hibernation and the “bad”, but couldn’t spot any differences.
I cleared out my journal and did a hibernation, got grub, here is the full log (it’s not huge). Using Pastebin because it has syntax highlighting and line numbers:
Fedora Silverblue 40 - hibernation shows grub after waking up - Pastebin.com
Note:
- hibernation process starts almost immediately at the beginning of the log
- lines 1204 and 1215 confirms enough space for hibernation
- line 1220: “The system will hibernate now!”
- storing memory seems to happen between 1370 and 1409
What I can’t find is the moment I wake up the system again, it happens somewhere in that log after 1409. What also confuses me is the message " Detected enough swap for hibernation" after 1409, showing twice. That is weird, because I just expect the system to boot, because I did not start hibernation a second time. But perhaps it does this to enable hibernation, not start it.
My conclusion is that hibernation happens successfully, maybe something goes wrong during wake up?
Or I am missing something essential that should happen at the very end of the hibernation process, before the system goes off?
1 Like
Adding to my previous post: Hibernation DOES work! Even when you see grubmenu.
Because all my apps, even never-saved new documents that I created, are still open after waking from hibernation!
So I think the real issue is boot_success not being set after successful wakeup. This explains exactly the behaviour I described here. This Ubuntu-related post mentions it as well but specifically for Ubuntu.
The issue for Fedora lies within this file /boot/grub2/grub.cfg:
### BEGIN /etc/grub.d/10_reset_boot_success ###
# Hiding the menu is ok if last boot was ok or if this is a first boot attempt to boot the entry
if [ "${boot_success}" = "1" -o "${boot_indeterminate}" = "1" ]; then
set menu_hide_ok=1
else
set menu_hide_ok=0
fi
# Reset boot_indeterminate after a successful boot
if [ "${boot_success}" = "1" ] ; then
set boot_indeterminate=0
# Avoid boot_indeterminate causing the menu to be hidden more than once
elif [ "${boot_indeterminate}" = "1" ]; then
set boot_indeterminate=2
fi
# Reset boot_success for current boot
set boot_success=0
save_env boot_success boot_indeterminate
### END /etc/grub.d/10_reset_boot_success ###
To test I did the following, after each wake up:
sudo /usr/sbin/grub2-set-bootflag boot_success
And indeed! I do not see the grub menu anymore 
I tried to create a systemd service to execute this command after hibernation:
[Unit]
Description=Unset recordfail in grubenv after hibernation.
After=hibernate.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/grub2-set-bootflag boot_success
[Install]
WantedBy=hibernate.target hybrid-sleep.target
but it fails::
sudo systemctl start /etc/systemd/system/grub-hibernate-setbootsucess
Failed to start etc-systemd-system-grub\x2dhibernate\x2dsetbootsucess.mount: Unit etc-systemd-system-grub\x2dhibernate\x2dsetbootsucess.mount not found.
2 Likes
Can you publish the whole compiled process of debugging & setting up hibernation somewhere like GitHub / GitLab / CodeBerg?
I would like to try making a script or even a hibernation config wizard yad GUI util & contributing it to your research results.
You keep liking my messages but I am trying to find help here with an actual issue.
@boredsquirrel the split into 2 topics has made it even messier… the post with the original guide (#2 in this topic) is still the unedited old one, for a full explanation of the additional required steps you need post #63 (in the prev topic) as well.
1 Like
I updated my post with the final remaining issue, I managed to solve it but need a little help to make the fix permanent. But at least:
- Hibernation works, confirmed
- The issue of grub menu is now fully understood, all we need is an expert to help create a proper service to make the fix permanent.
1 Like
We can probably use systemd-sleep(8) — Arch manual pages
More details described here:
Power management/Suspend and hibernate - ArchWiki
5.2 shows how to use /usr/lib/systemd/system-sleep/
Here is my file I am using to get around a bug in the wifi driver on my Framework 16:
# cat /usr/lib/systemd/system-sleep/hibernate-pre-post.sh
!/bin/bash
WiFiModule=mt7921e
case "$1 $2" in
"pre hibernate" | "pre suspend-then-hibernate")
modprobe -r $WiFiModule
;;
"post hibernate" | "post suspend-then-hibernate")
modprobe $WiFiModule
;;
*)
:
;;
esac
I just enabled secure boot on my device because I want to try the kernel patch and do FDE and get hibernate working with that so I can’t try this until I get that working.
2 Likes
Systemd 256 will get another one for hibernation I think