System is unbootable after the update

Today I got a message from the Software Center that there are important updates and I need to install them. As I had some time for this, I just went for it.

  1. The system didn’t reboot normally, there was a message about hard lock on cpu 10.
  2. I shut it down manually.
  3. The update flow on boot up got stuck on 31%.
  4. I waited for about 20 mins and decided to shut it down, because the amount of updates wasn’t that massive and shouldn’t have taken up too much time.
  5. It booted normally and did not tell me anything about the updates.
  6. I used dnf to update all manually. It went well.
  7. I rebooted and got into this.

After I press Enter in tries to load system manager config and throws the same message after some time.

I would simply reinstall the OS. But it’s a pain. Can anything be done here? Can I revive it via recovery mode?

I have latest Fedora 38. The laptop is HP Dev One.

Are you able to boot to a previous state (prior update) of the system by selecting other entry in the Fedora’s boot menu?

No, any kernel and rescue option lead to the same result.

The two images show different errors - first image shows a failure of local-fs.targetwhile second image shows success mounting /home. Did the second boot hang at the last line shown or did it continue on to “emergency mode”?

Emergency mode in linux boot error gives a reasonable overview, but needs updating. Your boot drive may not be sda1, and SystemRescueCd changed its name to SystemRescue (since few people boot CD’s anymore).

If you need more help, please tell us if you can you boot the Fedora Live USB or the SystemRescue[Cd] tool, whether you are dual booting another OS, and whether you are comfortable using the bash shell in a terminal. This will help us determine what tools are available and how much detail to provide.

Here are the errors with the rescue option.

I think I can boot any usb image here. And I’m ok with using terminal, but I’ll need some kind of a guide so I don’t mess anything up.

I suggest booting the Fedora Live USB. You can start with the Disks GUI. This will show the partition structure of your system drive with:

Size NN GB – NN GB free
Contents: fstype
Device: /dev/... with partition ID suffix: /dev/nvme0n1p1 or /dev/sda1
UUID:
Partition Type: (usually EFI or Linux filesystem will be the ones used by Fedora)

You should record this information where you can refer to when the system is down (paper!).

You should also scan the S.M.A.R.T Data and tests in case your system drive has failed (for many users, updates exercise disks harder than normal workloads, so a disk that is reaching End-of-Life will choose to fail during updates).

You can use Disks to mount each partition in turn to see if they can be mounted without errors.

Once you have done that, use the terminal to get a plain text inventory of the partitions with sudo fdisk -l <Device name> and post that (as plain text using the </> button).

Here’s what I have.

Disk /dev/nvme01np3: 952.28 GiB, 1022505254912 bytes, 1997080576 sectors
Units: sectors of 1* 512 = 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/luks-5804e086-d555-98c0-794db93975c5: 952.27 GiB, 1022488477696 bytes, 1997047808 sectors
Units: sectors of 1* 512 = 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I mounted my partition with no errors.

The command fdisk is intended to be used on a device, not a partition.
Thus your command sudo fdisk -l /dev/nvme0n1p3 should have been formed as
sudo fdisk -l /dev/nvme0n1 so it would show the data from the device.

After all, fdisk is a partition table formatting tool and not a file system tool.

1 Like

Here:

Disk /dev/nvme0n1: 953.87 GiB, 1024209543168 bytes, 2000409264 sectors
Disk model: SK hynix PC711 HFS011TDE9X073N
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 0A53E2D4-5A47-46B2-8290-2B8316BFB471

Device           Start        End    Sectors   Size Type
/dev/nvme0n1p1    2048    1230847    1228800   600M EFI System
/dev/nvme0n1p2 1230848    3327999    2097152     1G Linux filesystem
/dev/nvme0n1p3 3328000 2000408575 1997080576 952.3G Linux filesystem

I chrooted into my broken partition and did these commands

dnf history list last
dnf history info 167

and got this


sudo dnf update fails, because it can’t resolve urls for the installed packages. I can update /etc/hosts with relevant IP addresses, but there are just too many and I’m not sure whether it’s going to result in fixing the issue.

Please post terminal text as text using the </> button. This will make the text visible to internet searches and could help other with the same problem find this thread.

You didn’t mention luks which affects the name of the system partition. All three partitions are needed to boot, so please check that partitions 1 and 2 can be mounted without errors and have some free space.

If your system is from Dell, there were firmware updates for SK Hynix SSD drives used in their models.

All three partitions can be mounted without errors and have free space.
My laptop is from Hewlett-Packard.

HP participates in Linux Vendor Firmware Service, so if they support linux on your model you may find updates using fwupdmgr [OPTION…]. To see available options, run fwupdmgr --help. If HP does not support linux on your model you may find updates on the HP site, but installation may require Windows.

The dnf history showing a problem with nodejs requiring module(platform:f37) and error loading shared libraries are the sort of issues you can expect after an interrupted update, but don’t tell us why the update stalled. After making reasonable efforts to ensure that firmware updates have been applied, I would proceed to trying to repair the failed update. Start with dnf check and try tofix problems. Once dnf check is not reporting any issues, you can try to remove 3rd party packages (e.g., google-chrome-stable) that can be reinstalled after the system is booting properly.
You can try dnf distro-sync and dnf update – if the proposed changes don’t seem reasonable, you can “just say no” and nothing will be changed.

Here are the results:

[liveuser@localhost-live ~]$ fwupdmgr update
Devices with no available firmware updates: 
 • PC711 HFS001TDE9X073N
 • System Firmware
 • HP HD Camera
No updatable devices

I already did a dnf update excluding the pipewire packages. It went well, but the system is still unbootable.

[root@localhost-live /]# dnf check
Modular dependency problem:

 Problem: conflicting requests
  - nothing provides module(platform:f37) needed by module nodejs:18:3720220818062143:9e842022.x86_64 from @modulefailsafe
systemd-253.5-1.fc38.x86_64 is a duplicate with systemd-253.7-1.fc38.x86_64
systemd-libs-253.5-1.fc38.x86_64 is a duplicate with systemd-libs-253.7-1.fc38.x86_64
systemd-networkd-253.5-1.fc38.x86_64 is a duplicate with systemd-networkd-253.7-1.fc38.x86_64
systemd-pam-253.5-1.fc38.x86_64 is a duplicate with systemd-pam-253.7-1.fc38.x86_64
systemd-resolved-253.5-1.fc38.x86_64 is a duplicate with systemd-resolved-253.7-1.fc38.x86_64
Error: Check discovered 5 problem(s)

[root@localhost-live /]# dnf update
google-chrome                                                                                                                                                                                                 5.5 kB/s | 1.3 kB     00:00    
Modular dependency problem:

 Problem: conflicting requests
  - nothing provides module(platform:f37) needed by module nodejs:18:3720220818062143:9e842022.x86_64 from @modulefailsafe
Dependencies resolved.

 Problem 1: package pipewire-codec-aptx-0.3.74-1.fc38.x86_64 from @System requires pipewire = 0.3.74, but none of the providers can be installed
  - cannot install both pipewire-0.3.75-1.fc38.x86_64 from updates and pipewire-0.3.74-1.fc38.x86_64 from @System
  - cannot install the best update candidate for package pipewire-codec-aptx-0.3.74-1.fc38.x86_64
  - cannot install the best update candidate for package pipewire-0.3.74-1.fc38.x86_64
 Problem 2: problem with installed package pipewire-codec-aptx-0.3.74-1.fc38.x86_64
  - package pipewire-codec-aptx-0.3.74-1.fc38.x86_64 from @System requires pipewire = 0.3.74, but none of the providers can be installed
  - package pipewire-codec-aptx-0.3.74-1.fc38.x86_64 from rpmfusion-free-updates requires pipewire = 0.3.74, but none of the providers can be installed
  - package pipewire-0.3.74-1.fc38.x86_64 from @System requires pipewire-libs(x86-64) = 0.3.74-1.fc38, but none of the providers can be installed
  - cannot install both pipewire-libs-0.3.75-1.fc38.x86_64 from updates and pipewire-libs-0.3.74-1.fc38.x86_64 from @System
  - cannot install the best update candidate for package pipewire-libs-0.3.74-1.fc38.x86_64
==============================================================================================================================================================================================================================================
 Package                                                      Architecture                                          Version                                                      Repository                                              Size
==============================================================================================================================================================================================================================================
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 pipewire                                                     x86_64                                                0.3.75-1.fc38                                                updates                                                108 k
 pipewire-libs                                                x86_64                                                0.3.75-1.fc38                                                updates                                                1.8 M

Transaction Summary
==============================================================================================================================================================================================================================================
Skip  2 Packages

Nothing to do.
Complete!

The pipewire-codec-aptx-0.3.74-1.fc38.x86_64 from rpmfusion-free-updates issue may be due the time it takes for rpmfusion to package new versions. I would remove rpmfusion packages (keep a list so you an reinstall later) and focus on the Fedora updates.

Sorry, what do you mean by focus on Fedora updates? Except for pipewire all the packages are up to date. Should I resolve problems that are noted in dnf check? And how do I do this?

The problems are probably due to new packages being installed without removing
the old packages when updates stalled. Now you have a mix of old and new packages,
hence “duplicates”. The system is unlikely to be usable until the problems are resoved.
You can either attempt a repair or reinstall Fedora. Which option is best depends on
your situation. If the system has been running Fedora over many updates there may
be things you no longer use. A clean install means you are closer to what others have
and may help troubleshooting in the future, but if you have put a lot of effort into
customization you may want to try repairing the system.

Use dnf check --duplicates to see just these issues.

I did sudo dnf remove --duplicates, then sudo dnf distro-sync. After that dnf check showed only one issue with some module for nodejs.

Now on boot I get the message [!!!!!!] Failed to mount API filesystems.

I managed to boot by disabling the selinux enforcing parameter.
Now it boots only when it’s disabled. But I guess it’s another issue.

It is important to provide the dnf output. Prevous report had:

 Problem: conflicting requests
  - nothing provides module(platform:f37) needed by module nodejs:18:3720220818062143:9e842022.x86_64 from @modulefailsafe
% koji list-pkgs --package=nodejs | grep 18-3720220818062143-9e842022
nodejs                  module-nodejs-18-3720220818062143-9e842022                  ppisar
nodejs                  module-nodejs-18-3720220818062143-9e842022-build                  ppisar

You see @modulefailsafe when an installed package doesn’t come from an active repository, perhaps installed by rpm. nodejs should not be needed to boot the system (I don’t have nodejs installed on this system), but given earlier messages indicating a fedora 37 requirement, it would be best to remove it.

In the rescue environment you can try running btrfs check /dev/mapper/luks-[...] where the [...] is the system btrfs partiton. Note the warnings about trying to repair btrfs filesystems.