Hi,
Since arround 09/06/2024, I can’t upgrade my Fedora Silverblue 40 server.
The logs I have told me that : 2024 06 24 11 51 06 742 — Postimages
Does some of you have an idea?
If I try to boot the 09th june version, everythings works but if I try to boot a more recent one, it fails to boot.
I’m of course able to provide any more informations or logs.
Thanks you!
From the picture, I can see an error Timed out waiting for device dev-disk-by\x2uuid-....
And the fact that you can boot an older version successfully, seems to point in the direction of a kernel problem.
I would compare the versions of the kernel between the two deployments (rpm-ostree status -v) and try filing a bug with the Fedora kernel.
Or search for problems with the newer version of the kernel and your specific hardware, disk drives, etc. There might be a bug report/problem report already out there.
Hi,
Thanks for your help!
So, the two UUID are related to my secondary sata SSDs and to my HDDs USB.
They are all encrypted and auto-decrypted with an keyfile, do you think it’s related?
I really don’t understand, the sata ssds are directly connected to the motherboards, so they should work at least, no?
Thanks you for your time, I will continue to inquiry.
So, I have tried to add “nofail” option to crypttab and fstab, and it achieve to boot.
Then, it seems I’m able to manually unlock and mount the drives, does someone know why autodecrypt or automount doesn’t work anymore?
Thanks you
That’s a good hint at what is wrong, but I don’t know off-hand what is the trouble.
I would try to examine the package changes between the two deployments (working and not-working) to see if something jumps out. Maybe the kernel, maybe cryptsetup or the like?
Thanks you a lot, do you know how to obtain the differences in package between two deployments?
I have tried rpm-ostree status -v but it doesn’t tell that.
Thanks!
I think I have found it, it’s here : PrivateBin
Do you find something related, except kernel of course?
Thanks again!
I see:
cryptsetup 2.7.2-1.fc40 -> 2.7.3-1.fc40
cryptsetup-libs 2.7.2-1.fc40 -> 2.7.3-1.fc40
And the kernel, as you pointed out.
Those feel like the most likely suspects.
You may have to do some searching around to see if some behavior has changed in cryptsetup or the kernel around the automatic unlock of LUKS devices.
Thanks, I will open a ticket on their github I think.
Hi again,
With the latest version of Fedora Silverblue 39, everything work as expected.
So, the problem seems to came from either cryptsetup or the kernel.
Does someone have an idea?
Thanks you
Added f40, rpm-ostree and removed server
Hey, I rotated the image and added it in, also changed tags. “server” is only meant for Fedora Server.
Unrelated, did you know Fedora IOT is basically Silverblue but for servers?
To help you, we need more info I think.
Your system cant boot after you upgraded? So the upgrade worked but you cannot boot anymore?
To reproduce exactly, you could download the old working cryptsetup RPM or kernel RPM.
That RPM can be layered with rpm-ostree
rpm-ostree install /path/to/package.rpm
That way it will not update. Then try the update again, if it still fails, it is more complex.
Also please use
rpm -qa | grep cryptsetup
rpm -qa | grep kernel
rpm-ostree status
To get the relevant info.
Added cryptsetup, kernel, luks2
Thanks you, I will try all that right now, I knowed fedora coreos but it was too hard to install, fedora IOT could work for a server?
I will keep you updated asap, thanks you again!
So, I have tried to type rpm-ostree replace cryptsetup.rpm, but it tell me :
error: Could not depsolve transaction; 1 problem detected:
Problem: conflicting requests
- nothing provides cryptsetup-libs = 2.6.1-3.fc39 needed by cryptsetup-2.6.1-3.fc39.x86_64 from @commandline
Do you have another idea?
Thanks you
Hi again,
So, I have narrowed down the issue to systemd-cryptsetup-generator, which seems to generate non valid config file.
The log tell me :
systemd-cryptsetup Service has no ExecStart=, ExecStop=, or SuccessAction=. Refusing
Does someone have an idea?
Thanks you
Hi again,
I begin to really feel lost, I have posted the issue on the systemd bug tracker, and Poettering seems to think that the problem is not related to systemd but with SELinux.
Does someone know selinux well enough to help me with that?
If I boot with selinux=0, how could I relabel all the system after that (to reenable it)?
There is another error message that I haven’t seen :
Jul 08 12:00:42 homeserver systemd-cryptsetup-generator[1773]: Failed to generate keydev mount unit: Permission denied
Jul 08 12:00:42 homeserver (sd-exec-[1767]: /usr/lib/systemd/system-generators/systemd-cryptsetup-generator failed with exit status 1.
I’m of course able to give any more information on my setup, and will be very very grateful if someone could help me with that, I can give 50-100€ worth of monero or ethereum with a huge pleasure if someone could spare me a day or two reinstalling the whole server.
Thanks you a lot!
Could you link to the issue you’ve posted?
Yes, of course, it’s here : Unable to auto-decrypt my secondary drives since systemd 256 · Issue #33652 · systemd/systemd · GitHub
Thanks again for your help!
Added bug-reported