Fedora Kinoite: systemd degraded upon installation

Ask Fedora
, ,
1

For the first time, I have setup a Fedora Kinoite (F43) on native hardware, outside VM etc. However, systemd is degraded on installation: I have seen that before on VMs (there it was actually two degradations), and thought this is an issue related to VM. However, now I have it on a native installation too.

The system works fine other than the error logs. Unfortunately, on the very system I have no possibility to get data out at the moment, so I can provide only a photo of the degradation:

kinoite-systemd-degraded
kinoite-systemd-degraded3806Ă—1324 515 KB

→ The system works on a laptop with current coreboot efi, Intel Core Ultra 9 285H, 64G DDR5. Installation worked out fine and Fedora booted immediately out of the box after installation.

Given the error log, I am not even sure that “error” is intended?

2

what’s the content of your fstab - do you have an entry in there for the root partition?

This is the second time I’ve seen this error in 24 hours…

2 Likes
3

I don’t think it is the same. Actually, I see no correlation between the case you refer to and mine. Neither errors nor behavior correlate. The issue of systemd-remoount-fs, which is the degrading factor, is the only I have.

My issue also does not increase the time to boot: it’s actually quite quick. If the error causes an increase in booting time, it is below my perception threshold :classic_smiley:

yes.

Just to be on the same page Steve : you also use Kinoite 43, and your output of systemctl status | grep State: contains State: running ?

I wonder if additional partitions in /var/* can cause that (I have a few). Does someone else have additional partitions in /var/* (e.g., /var/abc or so) and no degraded systemd?

4

Same remount failure in post 8

I don’t use Kinoite unfortunately, don’t have any atomic systems which is why I was wondering what the remounter was trying to do and failing.

1 Like
5

I use Kinoite and have this in /var:

ls /var/
total 84K
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 adm
drwxr-xr-x. 17 root root 4,0K Jan 30 16:10 cache
drwxr-xr-x.  3 root root 4,0K Jan 30 13:43 db
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 empty
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 ftp
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 games
drwxr-xr-x.  4 root root 4,0K Jan 30 13:40 home
drwxr-xr-x.  3 root root 4,0K Jan 30 13:43 kerberos
drwxr-xr-x. 55 root root 4,0K Jan 31 16:19 lib
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 local
lrwxrwxrwx.  1 root root   11 Jan 30 13:43 lock -> ../run/lock
drwxr-xr-x. 18 root root 4,0K Feb 13 07:31 log
lrwxrwxrwx.  1 root root   10 Jan 30 13:43 mail -> spool/mail
drwxr-xr-x.  4 root root 4,0K Jan 31 09:32 mnt
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 nis
drwxr-xr-x.  2 root root 4,0K Jan 30 13:37 opt
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 preserve
drwx------.  4 root root 4,0K Jan 30 13:55 roothome
lrwxrwxrwx.  1 root root    6 Jan 30 13:43 run -> ../run
drwxr-xr-x.  6 root root 4,0K Jan 30 13:37 spool
drwxr-xr-x.  2 root root 4,0K Jan 30 13:37 srv
drwxrwxrwt. 15 root root 4,0K Feb 13 19:10 tmp
drwxr-xr-x. 11 root root 4,0K Jan 30 13:43 usrlocal
drwxr-xr-x.  2 root root 4,0K Jan 30 13:43 yp

This is my fstab file:

UUID=1e7f0a57-349a-4fff-aad0-08ab70c70749 / ext4 defaults,ro 1 1
UUID=4f6c436e-c14c-4259-9516-52f785d0fbf0 /boot ext4 defaults 1 2
UUID=4E70-84B6                            /boot/efi vfat umask=0077,shortname=winnt 0 2
UUID=81b199c1-76dd-4fc9-92ab-a66400885560 /home ext4 defaults 1 2

So I only have /home as a partition, all other folders in /var are just folders.
I use manual partition to keep the /home partition when I re-install Kinoite or move over to Fedora KDE. When you have multiple partitions in /var you probably use the automatic partitioning scheme, right?

You asked about this:

systemctl status | grep State:
    State: running
                 │ │   └─4877 grep --color=auto State:

When I do systemctl status --failed I get nothing, just the command, also when using sudo.

1 Like
6

I wonder if anyone with additional data partitions has a running state in systemctl status :thinking:

Another possibility is encrypted devices: my devices are all encrypted.

I have system partitions/mountpoints /, /var/, /var/home/ (all 3 one btrfs), /boot/ (default ext4), /boot/efi (default efi), plus the data partitions below /var/* (containing one more subvolume of the btrfs, plus two further file systems xfs)

I added some partitions manually to /var. But I mounted all of them on installation, so through anaconda. So they should have proper defaults in fstab & crypttab I would guess. All system partitions have been formatted/created by anaconda (boot-efi, boot, root, home). But the error also does not really look like something that is related to something like that :confused: Though I cannot exclude it of course

7

Just saw the comment from Hristo, referencing this workaround from Timothee: Root mount options are ignored in Fedora Atomic Desktops 42

Having limited experience with Kinoite and being likely prone to now-no-longer-applicable experience, I will try that out later and see if that was the issue.

1 Like
8

Anaconda has indeed added the root partition to /etc/fstab, and removing it solves the issue → systemd is no longer degraded.

After reading Timothee’s workaround, it makes sense of course regarding the immutable root. But I would have never linked the root entry in fstab, on itself, to an issue :smiley: It feels like being against nature :classic_smiley: But I’ll get used to it.

Thanks @hricky for the reference to Timothee’s workaround, and thanks @anothermindbomb for the reference to the reference :smiley:

1 Like
9

I wonder how it is possible that my Kinoite setup does not have that issue, in my /etc/fstab file I do have an entry for /. But as written in post 5 I use manual partitioning and maybe using that is different from the automatic way using btrfs.

systemctl status --failed shows nothing

Regarding this issue can there stil be something wrong in my setup?

I read this:
Manual Partitioning in Fedora Kinoite is not fully supported and comes with known issues, as highlighted in the official documentation and community discussions.

Only specific mount points can be manually defined: /boot/efi , /boot , /var , /var/home , /var/log , /var/containers , and the root / filesystem.

Since I only have /, /boot, /boot/efi and /home as partitions I think it should be okay.

One question about this: does it make a difference when I mount /home instead of /var/home? I know they are linked.

10

If your state in systemctl status is running, things should be fine. However, are u sure the / entry is enabled? If u want u can show your cat /etc/fstab to have a look. Now Im curious too and want to get a feeling for the immutables :slight_smile:

Concerning the home ↔ var/home, i post something later as i talked about this to siosm once. But Im travelling and phone-only right now :smiley:

11

These are the four lines in my fstab file.

How is it even possible to have it disabled, it is the basis directory to which all is connected. Without the system can not run, can it?

Enjoy your trip. This can wait.

12

Effectively no. I was wondering too, as /home should be not possible and fail. I asked Timothee about it when testing Kinoite/Silverblue for the first time, and I think to remember that he was not aware of this before as well: /home is done by anaconda that way, but generally it should be /var/home, not /home. However, systemd resolves /home automatically into /var/home, which means it does not “hurt”. However, it is an unnecessary complexity that can cause confusion, and I assume over time, this will be changed. But there is a long way for anaconda to go before the immutables can become default: I just opened a bug report as anaconda keeps crashing with an error when a user by accident mounts something else outside /var, rather than just informing the user that this cannot work. It took me some time to get my error in reasoning that resulted out of old habits (although my notes were correct, I intuitively mounted my main data partition in /<dir> rather than /var/<var>) :smiley: The other thing is obviously that anaconda happily creates a / mount in fstab → I might add that to the report to make aware (though I tend to wait for having an evaluation of your case as you indeed have / enabled :open_mouth: ). Anyway, we still have time to make anaconda immutable-proof :classic_smiley:

The same for me. Though anaconda allows the use of two tools to implement this, and their results are not necessarily identical. Unfortunately, I don’t have time to engage in much testing, comparing, reproducing, etc atm. The other thing is of course that there are further differences in how people customize their partitioning.

Keep in mind that the fstab is stored on root: this means, root needs to be mounted in order to access fstab. Therefore, the dependency you assume would result in a chicken & the egg problem :smiley: Effectively, the root mount on boot is done after grub (see the config files of your grub entries). Concerning details of how all things are done to boot an immutable and where differences are, I remain hesitant though, as I am myself new to the immutables.

Off the cuff, the major difference I see is that you have ext4 on your root. However, as long as there are systems that successfully work with that /, I will not add a post about this to the bug reports/feature requests. The existing workaround indicates that the developers know about the general issue anyway.

Just to be sure “explicitly” that we talk of the same thing: when you do systemctl status | grep State, the first line of output is State: running, right?

13

Hi Chris, thank you for your extensive answer.
Yes, this is the result of the command:

systemctl status | grep State
    State: running
                 │ │   └─6797 grep --color=auto State

 User jan @ Server Fedora-Kinoite in Folder /var/home/jan : Sat Feb 14 - 20:01:44

I “invented” one partition scheme for both Fedora KDE and Kinoite and until now it works. I can keep all my data in the home folder alive between re-installs, whereby I can jump from KDE to Kinoite and back again. In the KDE live installer I first delete everything outside Bin, Documents, Downloads, Music and Videos so all the dot directories and files in there are new.

So the /home and /var/home issue is also something Timothee is working on, or at least is thinking about. Let’s see what the outcome is, I use /home and it works, no idea if /var/home also works.

Ah yes, when / is not mounted, fstab is not reachable. Makes sense.

Yes, I always EXT4 cause I had one terrible issue with btrfs: I got the message disk full and since then I have not used automatic partitioning nor btrfs again. This was not on fedora but I used openSUSE Tumbleweed at that time.

I see a solution for this thread but don’t understand how this works. You removed the / partition from your fstab file and now it works? How does the system find the / partition? Is this just something in a system with automatic btrfs partitoning, as Fedora would like you to use, or is this also something when manual partitioning, with other file systems is used?

I do have one plea for the @anaconda team and /or the @packaged team: please leave manual partitioning in tact. It works great.
Thank you.

14

In the two variants I have seen so far (as setup by some anaconda setups), it’s the same for you as user: systemd ensures that /home works, as it resolves /home → /var/home. But if home is directly mounted to /var/home, it would “remove one step” and thus formally simplify. /var/home would work, as this is effectively the outcome in both cases.

Depending on what exactly you mean, I think that’s more a plea to the anaconda team or the storaged team :classic_smiley:

15

Oops, my mistake. Thank you for telling me.

1 Like
16

I have done another installation on the very device, this time 100% default (automatic partitioning): it stll adds / and thus the systemd is degraded :open_mouth: I’ll report to anaconda in the existing bug ticket so that they can verify. But it remains interesting that there are cases in which / doesn’t cause a degrading.

17

It’s ugly as I formally bring together two things in one report, but I lack time to invest too much time in this and anaconda will not need much details from me to investigate+solve this but only a hint to become aware. I link for the record (relevant is only the comment 3) → 2439682 – Anaconda no longer supports pre-created encrypted disks since web-ui (necessary if no aes-ni is available as anaconda does not allow to customize encryption itself)