Boot stuck at Job dev-disk-by-uuid

Ask Fedora
,
1

Hello all,

I have issues with new installed F39 on my Thinkpad T480. I have installed it via everything iso with custom partition layout.

I have two unencrypted partitions (EFI and boot) and then encrypted LVM volume group named vg0 and then two logical volumes - root and home.

Here is my fstab:

UUID=a35096a3-e391-40d3-b5b0-59be29e06cef /                       ext4    defaults,x-systemd.device-timeout=0 1 1
UUID=6112e213-a6e2-42f4-bf09-fe85b786c8b8 /boot                   ext4    defaults        1 2
UUID=FC65-C916          /boot/efi               vfat    umask=0077,shortname=winnt 0 2
UUID=c4e59c03-21e9-4eb4-a5b1-9215fef44030 /home                   ext4    defaults,x-systemd.device-timeout=0 1 2

The first boot after installation I had no issues. But after some setups (nothing fancy mostly gnome extensions) I have rebooted the machine and then things happen.

I see boot menu then prompt for decryption password and after that booting hang waiting for Job dev-disk-by-uuid-c4e59c03-21e9-4eb4-a5b1-9215fef44030.device/start running, which is my home lv.

When system enters in emergency mode (I have changed timeout in fstab from 0 to 30sec.) I see that only root partition is available.

When I boot live USB and open partition with cryptsetup luksOpen I see both logical volumes.

Any idea what can cause the issue?

2

Just a guess since I do not use encrypted LVM.
You specified that the VG is encrypted, and I must assume that both / and /home are LVs within that VG.
On my system that is not encrypted but does use LVM the fstab is different.

/dev/mapper/fedora_raptor-root /                       ext4    defaults        1 1
UUID=ff3a62b9-6fd2-42f1-aa2d-7a32ea82a3ff /boot        ext4    defaults        1 2
UUID=D335-468C          /boot/efi               vfat    umask=0077,shortname=winnt 0 2
/dev/mapper/fedora_raid-home /home                     ext4    defaults        1 2

Note that by default it uses the /dev/mapper path to the LVs for mounting and does not use the UUIDs for those devices. Maybe that is the issue. After all, once the VG has been unlocked then the content of /etc/fstab should be available and should match in both the initramfs image and in the actual file system /etc/fstab.

This is a system where I did a new clean install of F39 when I installed a new SSD for the root file system and retained the VG for /home so the config in /etc/fstab is the way it was done default when installing Workstation with custom partitioning.

Check what you see with lsblk -f and verify the uuid for both / and /home match the results from /etc/fstab. You should see a separate UUID for the VG as well as for each LV within that VG
Something similar to this.

nvme0n1                                                                                                                  
├─nvme0n1p1     vfat         FAT32                                 D335-468C                                 1.2G     2% /boot/efi
├─nvme0n1p2     ext4         1.0                                   ff3a62b9-6fd2-42f1-aa2d-7a32ea82a3ff    534.5M    38% /boot
└─nvme0n1p3     LVM2_member  LVM2 00                               eUNYL5-Bz9V-5E4s-FFzN-Odxs-uCOf-JmsTC4                                 **VG UUID**
  ├─fedora_raptor-root
  │             ext4         1.0                                   2c687ebd-e50c-4234-91b7-8e411c0cc168    397.1G     8% /                **LV UUID**
  └─fedora_raptor-home
                ext4         1.0                                   de6e8251-109c-4064-933c-3b627ff60337                                   **LV UUID**

Note that I have a separate VG used for /home so I do not have the displayed LV mounted, but rather the one shown in fstab above. I added the indications at the end of those lines with the identify of each UUID.

3

I use full disk encryption and my fstab uses GUIDs, which is expected.

Given you can use luksOpen and see the partitions I wonder if the boot is waiting for you to enter the unlock passphrase.

Did you enter the passphrase to unlock the disks?

4

Thanks for the answers. Fstab was generated automatically by anaconda. And yes, I enter passphrase to unlock partition. However, I noticed that lsblk -f in emergency mode is missing home partition for some reasons.

NAME                                          FSTYPE      FSVER    LABEL       UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
sda
└─sda1                                        vfat        FAT32    FEDORA-WS-L 363C-3CB7
sdb
zram0                                                                                                                                [SWAP]
nvme1n1
nvme0n1
├─nvme0n1p1                                   vfat        FAT32    EFI         FC65-C916                               591.4M     1% /boot/efi
├─nvme0n1p2                                   ext4        1.0      BOOT        6112e213-a6e2-42f4-bf09-fe85b786c8b8    680.7M    23% /boot
└─nvme0n1p3                                   crypto_LUKS 2                    52a8c847-d05b-4d92-8732-dbb419544826
  └─luks-52a8c847-d05b-4d92-8732-dbb419544826 LVM2_member LVM2 001             rqNDZM-WXlx-vvmG-71iF-6uiP-T7iY-BsuIXl
    └─vg0-root                                ext4        1.0      ROOT        a35096a3-e391-40d3-b5b0-59be29e06cef    119.6G     7% /

But when I start system from liveUSB and manually open encrypted partition with cryptsetup luksOpen /dev/nvme1n1p3 myluks home partition with right UUID is there:

NAME           FSTYPE          FSVER    LABEL       UUID                                   FSAVAIL FSUSE% MOUNTPOINTS
loop0          squashfs        4.0
loop1          ext4            1.0      Anaconda    be85788b-98c7-47df-afaf-45dc8fdefb15
├─live-rw      ext4            1.0      Anaconda    be85788b-98c7-47df-afaf-45dc8fdefb15      1.6G    78% /
└─live-base    ext4            1.0      Anaconda    be85788b-98c7-47df-afaf-45dc8fdefb15
loop2          DM_snapshot_cow
└─live-rw      ext4            1.0      Anaconda    be85788b-98c7-47df-afaf-45dc8fdefb15      1.6G    78% /
sda
└─sda1         vfat            FAT32    FEDORA-WS-L 363C-3CB7                                26.7G     7% /run/initramfs/live
sdb
zram0                                                                                                     [SWAP]
nvme0n1
nvme1n1
├─nvme1n1p1    vfat            FAT32    EFI         FC65-C916
├─nvme1n1p2    ext4            1.0      BOOT        6112e213-a6e2-42f4-bf09-fe85b786c8b8
└─nvme1n1p3    crypto_LUKS     2                    52a8c847-d05b-4d92-8732-dbb419544826
  └─myluks     LVM2_member     LVM2 001             rqNDZM-WXlx-vvmG-71iF-6uiP-T7iY-BsuIXl
    ├─vg0-home ext4            1.0      HOME        c4e59c03-21e9-4eb4-a5b1-9215fef44030
    └─vg0-root ext4            1.0      ROOT        a35096a3-e391-40d3-b5b0-59be29e06cef

And here is part of journalctl -b that seems to be related to home partition:

Mar 21 14:16:59 T480 systemd[1]: dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.device: Job dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.device/start timed out.
Mar 21 14:16:59 T480 systemd[1]: Timed out waiting for device dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.device - /dev/disk/by-uuid/c4e59c03-21e9-4eb4-a5b1-9215fef44030.
Mar 21 14:16:59 T480 systemd[1]: Dependency failed for systemd-fsck@dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.service - File System Check on /dev/disk/by-uuid/c4e59c03-21e9-4eb4-a5b1-9215fef44030.
Mar 21 14:16:59 T480 systemd[1]: Dependency failed for home.mount - /home.
Mar 21 14:16:59 T480 systemd[1]: Dependency failed for local-fs.target - Local File Systems.
Mar 21 14:16:59 T480 systemd[1]: Dependency failed for selinux-autorelabel-mark.service - Mark the need to relabel after reboot.
Mar 21 14:16:59 T480 systemd[1]: selinux-autorelabel-mark.service: Job selinux-autorelabel-mark.service/start failed with result 'dependency'.
Mar 21 14:16:59 T480 systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
Mar 21 14:16:59 T480 systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
Mar 21 14:16:59 T480 systemd[1]: home.mount: Job home.mount/start failed with result 'dependency'.
Mar 21 14:16:59 T480 systemd[1]: systemd-fsck@dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.service: Job systemd-fsck@dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.service/start failed with result 'dependency'.
Mar 21 14:16:59 T480 systemd[1]: dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.device: Job dev-disk-by\x2duuid-c4e59c03\x2d21e9\x2d4eb4\x2da5b1\x2d9215fef44030.device/start failed with result 'timeout'.
5

That log appears to show that the system is attempting to perform an fsck on /home LV and fails so it cannot activate and mount the file system.

Have you tried running an fsck on that file system when booted from the live usb?

If not then I suggest that you do so and see if there is any difference after that completes. Linux is picky and if a file system shows as dirty it will automatically perform the fsck before mounting. If fsck cannot complete it will not mount the (dirty) file system.

6

Decided to format disk and install Fedora again.
No issues since that.

1 Like