Gnome Software crashes, doesn't open after update to Fedora 38

Is there a broken /root/system-update symlink like what happened to Matias? If so, does deleting it fix the boot problem?

To be honest I’m not sure if I knew how to check it at this point.

All in all finally I decided to make a clean install from USB I didn’t miss too much data or settings of apps as most of that is inside services, clouds.

Compiling the schemas did fix the gnome-software crash issue for me. But in the next “restart & update” It stuck on boot again. Yes, there was this broken link on the root but I decided to reinstall 38 from scratch from a live USB. This broken link may be a symptom of a broken upgrade. I don´t know.

It definitely seems like something went wrong on upgrade for you both, but as you’ve both reinstalled, it’s almost impossible to tell what exactly it was (it rather sounds like the upgrade transaction actually failed somehow before reaching the schema recompilation or wiping the /root/system-update file) or what caused it :frowning:

Is anyone left who hit this bug and has not reinstalled the system yet? If so, we really need to see the logs from the upgrade attempt: you should be able to find them with journalctl, using the -b-N arg to limit messages to a single boot (journalctl -b-1 gives you messages from one boot ago, journalctl -b-2 gives messages from two boots ago, etc.) or the --since and --until args to specify a specific date/time range if you remember when the upgrade happened (e.g. journalctl --since "2023-04-10 18:00:00" --until "2023-04-11 11:00:00" or whatever). It should be relatively obvious when you find the right logs - there’ll be a bunch of stuff about updating from fc37 packages to fc38 ones.

Alternatively, if either of you can remember any particularly unusual packages etc. you may both have had installed, that might help figure it out. The most likely cause, I guess, is some kind of bad scriptlet bug in some package you both had installed.

1 Like

I’m rather a noobie linux user who copies a lot of stuff from the web until I get the result I’m satisfied with. I don’t know what is the hardware Matias uses but the most obscure things I had installed on my laptop were maybe:

  1. This thing to get fingerprint reader working:
    GitHub - uunicorn/python-validity: Validity fingerprint sensor prototype
    with a special fix based on this issue: Fix for "Fingerprint not working after waking up from suspend" doesn't work on Fedora 36 · Issue #127 · uunicorn/python-validity · GitHub

  2. I can’t remember the exact way I got my nvidia mx150 graphics card to work last time but from what I recall it was a painful process of trial and error. I’m actually unable to make it work now even with all the resources online.

  3. I was also using this thing for battery life: GitHub - AdnanHodzic/auto-cpufreq: Automatic CPU speed & power optimizer for Linux

  4. Don’t really know what else. Had some aliases in bashrc to combine commands, some nerd fonts. Many Gnome extensions. However Gnome Software did not work even when extensions were turned off.

I manage the computers for my family and one of the computers was experiencing this exact problem. I managed to fix gnome-software crashing on startup by running

sudo glib-compile-schemas /usr/share/glib-2.0/schemas/

But after clicking restart and upgrade in gnome software, the computer gets stuck on boot every time. Removing the /root/system-update symlink using a live ISO now lets the computer boot again.

I also found the logs of the upgrade attempt. I’m new to Fedora Discussion and Discourse in general. In what format should I share the logs?

After a quick look at the logs, it looks like PackageKit experienced a segmentation vault and generated a core dump at about 84% of the upgrade process. You probably need the core dump too, right? Again, my question is in what format should I share it?

Kind Regards,
Lyvendia

3 Likes

Here in discourse you can use “Preformated Text” see menu in editor. This makes a scroll-able box and not uses too much space while scrolling to the topic.

2 Likes

Wow, that’s fantastic, that’s exactly what we need indeed. Thanks a lot! The ideal would be if you can backtrace the coredump, but that’s a bit tricky if you haven’t done it before. Possibly if we’re lucky coredumpctl will be able to do it: you can run coredumpctl list, and if you see an entry that looks like it matches (the exe and the time), you can run coredumpctl gdb <pid>, where <pid> is the PID for that crash. (If you don’t have gdb installed you may need to dosudo dnf install gdb). If that gets to gdb and seems to be doing stuff, you can try doing set logging on then thread apply all bt full, hit ‘c’ when it prompts you, then hit ‘ctrl-d’ to exit and upload gdb.txt.

If that doesn’t work, if you can at least upload the coredump file somewhere, we can try and get a backtrace out of it. If it shows in coredumpctl list, then coredumpctl info <pid> should tell you where the file is (on the line Storage:). Just upload the file to any kind of file sharing site you have access to and link it here.

In either case, if you can post the ~100 lines or so up to the segfault here (as “preformatted text”, as @ilikelinux says), that might help figure out what it was doing when it crashed.

Thanks a lot!

2 Likes

Thanks for the help Adam and ilikelinux!

It looks to me like the segmentation fault occoured in libdnf. Here are the full logs of the upgrade attempt:

...

Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        unicode-ucd-15.0.0-1.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        libpinyin-data-2.6.2-2.fc37.x86_64 (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        vim-data-2:9.0.1440-1.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        lv2-1.18.8-2.fc37.x86_64 (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        catatonit-0.1.7-10.fc37.x86_64 (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        qt6-qttools-common-6.4.3-1.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        words-3.0-40.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        fwupd-plugin-uefi-capsule-data-1.8.14-1.fc37.x86_64 (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        mypaint-brushes-1.3.1-6.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: percentage 84%
Apr 22 06:42:30 Papaputer pk-offline-update[879]: sent msg to plymouth 'Systemaktualisierung wird installiert - 84%'
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        samba-dc-provision-2:4.17.7-0.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        zd1211-firmware-1.5-11.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        rootfiles-8.1-32.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        opencl-headers-3.0-12.20220510gitdef8be9.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        microcode_ctl-2:2.1-53.fc37.x86_64 (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        gnome-user-docs-43.0-1.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        fedora-bookmarks-28-25.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        atmel-firmware-1.3-27.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        alsa-sof-firmware-2.2.4-2.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer pk-offline-update[879]: package cleanup        NetworkManager-config-connectivity-fedora-1:1.40.10-1.fc37.noarch (installed)
Apr 22 06:42:30 Papaputer kernel: show_signal_msg: 13 callbacks suppressed
Apr 22 06:42:30 Papaputer kernel: PK-Backend[955]: segfault at 38 ip 00007fd2cf7298fd sp 00007fd2ce2264b0 error 6 in libdnf.so.2 (deleted)[7fd2cf6ba000+f0000] likely on CPU 1 (core 1, socket 0)
Apr 22 06:42:30 Papaputer kernel: Code: 8b 4c 24 10 48 89 de 4c 89 e2 e8 2e 79 f9 ff 48 8b 44 24 08 48 8b 7b 40 48 8b 5b 48 48 83 40 50 01 48 85 db 0f 85 39 ff ff ff <c7> 47 38 01 00 00 00 e8 37 b6 f9 ff 48 83 c4 28 5b 5d 41 5c 41 5d
Apr 22 06:42:30 Papaputer kernel: audit: type=1701 audit(1682138550.583:513): auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:rpm_t:s0 pid=910 comm="PK-Backend" exe=2F7573722F6C6962657865632F7061636B6167656B697464202864656C6574656429 sig=11 res=1
Apr 22 06:42:30 Papaputer audit[910]: ANOM_ABEND auid=4294967295 uid=0 gid=0 ses=4294967295 subj=system_u:system_r:rpm_t:s0 pid=910 comm="PK-Backend" exe=2F7573722F6C6962657865632F7061636B6167656B697464202864656C6574656429 sig=11 res=1
Apr 22 06:42:30 Papaputer audit[24767]: AVC avc:  denied  { read } for  pid=24767 comm="false" path="pipe:[97790]" dev="pipefs" ino=97790 scontext=system_u:system_r:kernel_generic_helper_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=fifo_file permissive=0
Apr 22 06:42:30 Papaputer kernel: audit: type=1400 audit(1682138550.583:514): avc:  denied  { read } for  pid=24767 comm="false" path="pipe:[97790]" dev="pipefs" ino=97790 scontext=system_u:system_r:kernel_generic_helper_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=fifo_file permissive=0
Apr 22 06:42:30 Papaputer pk-offline-update[879]: status finished
Apr 22 06:42:30 Papaputer pk-offline-update[879]: writing failed results
Apr 22 06:42:30 Papaputer pk-offline-update[879]: failed to upgrade system: PackageKit daemon disappeared
Apr 22 06:42:30 Papaputer systemd[1]: packagekit.service: Main process exited, code=dumped, status=11/SEGV
Apr 22 06:42:30 Papaputer systemd[1]: packagekit.service: Failed with result 'core-dump'.
Apr 22 06:42:30 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Apr 22 06:42:30 Papaputer systemd[1]: packagekit.service: Consumed 6min 47.078s CPU time.
Apr 22 06:42:30 Papaputer kernel: audit: type=1131 audit(1682138550.615:515): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Apr 22 06:42:40 Papaputer pk-offline-update[879]: rebooting
Apr 22 06:42:40 Papaputer pk-offline-update[879]: sent mode to plymouth 'shutdown'
Apr 22 06:42:40 Papaputer pk-offline-update[879]: sent msg to plymouth 'Nach dem Installieren der Aktualisierungen wird das System neu gestartet …'
Apr 22 06:42:40 Papaputer systemd[1]: Removed slice system-modprobe.slice - Slice /system/modprobe.
Apr 22 06:42:40 Papaputer systemd[1]: systemd-ask-password-plymouth.path: Deactivated successfully.
Apr 22 06:42:40 Papaputer systemd[1]: Stopped systemd-ask-password-plymouth.path - Forward Password Requests to Plymouth Directory Watch.
Apr 22 06:42:40 Papaputer systemd[1]: Stopped target bluetooth.target - Bluetooth Support.
Apr 22 06:42:40 Papaputer systemd[1]: Stopped target network-online.target - Network is Online.
Apr 22 06:42:40 Papaputer systemd[1]: Stopped target nss-lookup.target - Host and Network Name Lookups.
Apr 22 06:42:40 Papaputer systemd[1]: Stopped target sound.target - Sound Card.
Apr 22 06:42:40 Papaputer systemd[1]: lvm2-lvmpolld.socket: Deactivated successfully.
Apr 22 06:42:40 Papaputer systemd[1]: Closed lvm2-lvmpolld.socket - LVM2 poll daemon socket.
Apr 22 06:42:40 Papaputer systemd[1]: systemd-rfkill.socket: Deactivated successfully.
Apr 22 06:42:41 Papaputer systemd[1]: Closed systemd-rfkill.socket - Load/Save RF Kill Switch Status /dev/rfkill Watch.
Apr 22 06:42:41 Papaputer systemd[1]: Unmounting mnt-1267d749\x2d9cc2\x2d4c55\x2dbb16\x2d427c4c3e36b9.mount - /mnt/1267d749-9cc2-4c55-bb16-427c4c3e36b9...
Apr 22 06:42:41 Papaputer systemd[1]: NetworkManager-wait-online.service: Deactivated successfully.
Apr 22 06:42:41 Papaputer systemd[1]: Stopped NetworkManager-wait-online.service - Network Manager Wait Online.
Apr 22 06:42:41 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-wait-online comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer kernel: audit: type=1131 audit(1682138561.594:516): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-wait-online comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer systemd[1]: Stopping alsa-state.service - Manage Sound Card State (restore and store)...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping bluetooth.service - Bluetooth service...
Apr 22 06:42:41 Papaputer bluetoothd[873]: Terminating
Apr 22 06:42:41 Papaputer systemd[1]: Stopping bolt.service - Thunderbolt system service...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping colord.service - Manage, Install and Generate Color Profiles...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping cups.service - CUPS Scheduler...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping dracut-shutdown.service - Restore /run/initramfs on shutdown...
Apr 22 06:42:41 Papaputer systemd[1]: grub2-systemd-integration.service - Grub2 systemctl reboot --boot-loader-menu=... support was skipped because of an unmet condition check (ConditionPathExists=/run/systemd/reboot-to-boot-loader-menu).
Apr 22 06:42:41 Papaputer alsactl[872]: alsactl daemon stopped
Apr 22 06:42:41 Papaputer systemd[1]: Starting plymouth-reboot.service - Show Plymouth Reboot Screen...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping systemd-logind.service - User Login Management...
Apr 22 06:42:41 Papaputer systemd[1]: Stopping systemd-random-seed.service - Load/Save OS Random Seed...
Apr 22 06:42:41 Papaputer systemd[1]: packagekit-offline-update.service: Deactivated successfully.
Apr 22 06:42:41 Papaputer bluetoothd[873]: Stopping SDP server
Apr 22 06:42:41 Papaputer bluetoothd[873]: Exit
Apr 22 06:42:41 Papaputer systemd[1]: Stopped packagekit-offline-update.service - Update the operating system whilst offline.
Apr 22 06:42:41 Papaputer audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer systemd[1]: packagekit-offline-update.service: Consumed 1.449s CPU time.
Apr 22 06:42:41 Papaputer kernel: audit: type=1130 audit(1682138561.718:517): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer kernel: audit: type=1131 audit(1682138561.718:518): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=packagekit-offline-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:41 Papaputer systemd[1]: cups.service: Deactivated successfully.
Apr 22 06:42:42 Papaputer systemd[1]: Stopped cups.service - CUPS Scheduler.
Apr 22 06:42:42 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=cups comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: colord.service: Deactivated successfully.
Apr 22 06:42:42 Papaputer kernel: audit: type=1131 audit(1682138562.038:519): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=cups comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: Stopped colord.service - Manage, Install and Generate Color Profiles.
Apr 22 06:42:42 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=colord comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer kernel: audit: type=1131 audit(1682138562.105:520): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=colord comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: bolt.service: Deactivated successfully.
Apr 22 06:42:42 Papaputer systemd[1]: Stopped bolt.service - Thunderbolt system service.
Apr 22 06:42:42 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=bolt comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: bluetooth.service: Deactivated successfully.
Apr 22 06:42:42 Papaputer kernel: audit: type=1131 audit(1682138562.341:521): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=bolt comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: Stopped bluetooth.service - Bluetooth service.
Apr 22 06:42:42 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=bluetooth comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: systemd-logind.service: Deactivated successfully.
Apr 22 06:42:42 Papaputer kernel: audit: type=1131 audit(1682138562.554:522): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=bluetooth comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: Stopped systemd-logind.service - User Login Management.
Apr 22 06:42:42 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-logind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer kernel: audit: type=1131 audit(1682138562.701:523): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-logind comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:42 Papaputer systemd[1]: mnt-1267d749\x2d9cc2\x2d4c55\x2dbb16\x2d427c4c3e36b9.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounted mnt-1267d749\x2d9cc2\x2d4c55\x2dbb16\x2d427c4c3e36b9.mount - /mnt/1267d749-9cc2-4c55-bb16-427c4c3e36b9.
Apr 22 06:42:43 Papaputer systemd[1]: alsa-state.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped alsa-state.service - Manage Sound Card State (restore and store).
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=alsa-state comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: dracut-shutdown.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer kernel: audit: type=1131 audit(1682138563.142:524): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=alsa-state comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dracut-shutdown comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Stopped dracut-shutdown.service - Restore /run/initramfs on shutdown.
Apr 22 06:42:43 Papaputer kernel: audit: type=1131 audit(1682138563.273:525): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dracut-shutdown comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: systemd-random-seed.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-random-seed.service - Load/Save OS Random Seed.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-random-seed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer audit: BPF prog-id=200 op=UNLOAD
Apr 22 06:42:43 Papaputer systemd[1]: Removed slice user.slice - User and Session Slice.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target network.target - Network.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target system-update-pre.target - Offline System Update (Pre).
Apr 22 06:42:43 Papaputer systemd[1]: cups.socket: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Closed cups.socket - CUPS Scheduler.
Apr 22 06:42:43 Papaputer systemd[1]: Stopping NetworkManager.service - Network Manager...
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3267] caught SIGTERM, shutting down normally.
Apr 22 06:42:43 Papaputer systemd[1]: Stopping polkit.service - Authorization Manager...
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3286] device (br0): bridge port eno1 was detached
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3294] device (wlp0s20f3): state change: unavailable -> unmanaged (reason 'unmanaged', sys-iface-state: 'managed')
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3300] device (wlp0s20f3): set-hw-addr: reset MAC address to BC:17:B8:83:FB:DA (unmanage)
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3425] dhcp4 (br0): canceled DHCP transaction
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3427] dhcp4 (br0): activation: beginning transaction (timeout in 45 seconds)
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3427] dhcp4 (br0): state changed no lease
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3431] dhcp6 (br0): canceled DHCP transaction
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=plymouth-reboot comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3431] dhcp6 (br0): activation: beginning transaction (timeout in 45 seconds)
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3431] dhcp6 (br0): state changed no lease
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3436] manager: NetworkManager state is now CONNECTED_SITE
Apr 22 06:42:43 Papaputer systemd[1]: Started plymouth-reboot.service - Show Plymouth Reboot Screen.
Apr 22 06:42:43 Papaputer systemd[1]: polkit.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped polkit.service - Authorization Manager.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=polkit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Requested transaction contradicts existing jobs: Transaction for NetworkManager-dispatcher.service/start is destructive (cryptsetup.target has 'stop' job queued, but 'start' is included in transaction).
Apr 22 06:42:43 Papaputer systemd[1]: plymouth-switch-root-initramfs.service - Tell Plymouth To Jump To initramfs was skipped because of an unmet condition check (ConditionPathExists=/run/initramfs/bin/sh).
Apr 22 06:42:43 Papaputer NetworkManager[909]: <info>  [1682138563.3719] exiting (success)
Apr 22 06:42:43 Papaputer systemd[1]: NetworkManager.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped NetworkManager.service - Network Manager.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target network-pre.target - Preparation for Network.
Apr 22 06:42:43 Papaputer dbus-broker[893]: Dispatched 16171 messages @ 6(±12)μs / message.
Apr 22 06:42:43 Papaputer systemd[1]: Stopping dbus-broker.service - D-Bus System Message Bus...
Apr 22 06:42:43 Papaputer systemd[1]: systemd-network-generator.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-network-generator.service - Generate network units from Kernel command line.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-network-generator comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: dbus-broker.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped dbus-broker.service - D-Bus System Message Bus.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dbus-broker comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: dbus-broker.service: Consumed 2.085s CPU time.
Apr 22 06:42:43 Papaputer systemd[1]: dbus.socket: Deactivated successfully.
Apr 22 06:42:43 Papaputer audit: BPF prog-id=203 op=UNLOAD
Apr 22 06:42:43 Papaputer systemd[1]: Closed dbus.socket - D-Bus System Message Bus Socket.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target sysinit.target - System Initialization.
Apr 22 06:42:43 Papaputer systemd[1]: proc-sys-fs-binfmt_misc.automount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unset automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target cryptsetup.target - Local Encrypted Volumes.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target integritysetup.target - Local Integrity Protected Volumes.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target veritysetup.target - Local Verity Protected Volumes.
Apr 22 06:42:43 Papaputer systemd[1]: systemd-boot-update.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-boot-update.service - Automatic Boot Loader Update.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-boot-update comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Stopping systemd-resolved.service - Network Name Resolution...
Apr 22 06:42:43 Papaputer systemd[1]: systemd-sysctl.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-sysctl.service - Apply Kernel Variables.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: systemd-modules-load.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-modules-load.service - Load Kernel Modules.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-modules-load comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Stopping systemd-update-utmp.service - Record System Boot/Shutdown in UTMP...
Apr 22 06:42:43 Papaputer systemd[1]: proc-sys-fs-binfmt_misc.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer audit[24801]: SYSTEM_SHUTDOWN pid=24801 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="systemd-update-utmp" exe="/usr/lib/systemd/systemd-update-utmp" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: systemd-resolved.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-resolved.service - Network Name Resolution.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-resolved comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: systemd-update-utmp.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer audit: BPF prog-id=194 op=UNLOAD
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-update-utmp.service - Record System Boot/Shutdown in UTMP.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-update-utmp comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-tmpfiles-setup.service - Create Volatile Files and Directories.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: import-state.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped import-state.service - Import network configuration from initramfs.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=import-state comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target local-fs.target - Local File Systems.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounting boot-efi.mount - /boot/efi...
Apr 22 06:42:43 Papaputer systemd[1]: Unmounting home.mount - /home...
Apr 22 06:42:43 Papaputer systemd[1]: Unmounting tmp.mount - Temporary Directory /tmp...
Apr 22 06:42:43 Papaputer systemd[1]: home.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounted home.mount - /home.
Apr 22 06:42:43 Papaputer systemd[1]: boot-efi.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounted boot-efi.mount - /boot/efi.
Apr 22 06:42:43 Papaputer systemd[1]: tmp.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounted tmp.mount - Temporary Directory /tmp.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target swap.target - Swaps.
Apr 22 06:42:43 Papaputer systemd[1]: Deactivating swap dev-zram0.swap - Compressed Swap on /dev/zram0...
Apr 22 06:42:43 Papaputer systemd[1]: Unmounting boot.mount - /boot...
Apr 22 06:42:43 Papaputer systemd[1]: systemd-fsck@dev-disk-by\x2duuid-DF17\x2d2362.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer kernel: EXT4-fs (nvme0n1p2): unmounting filesystem bdcc1c07-3966-4ffa-b5fe-fd3e43ff6c2b.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-fsck@dev-disk-by\x2duuid-DF17\x2d2362.service - File System Check on /dev/disk/by-uuid/DF17-2362.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-disk-by\x2duuid-DF17\x2d2362 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: dev-zram0.swap: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Deactivated swap dev-zram0.swap - Compressed Swap on /dev/zram0.
Apr 22 06:42:43 Papaputer systemd[1]: boot.mount: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Unmounted boot.mount - /boot.
Apr 22 06:42:43 Papaputer systemd[1]: Reached target umount.target - Unmount All Filesystems.
Apr 22 06:42:43 Papaputer systemd[1]: systemd-fsck@dev-disk-by\x2duuid-bdcc1c07\x2d3966\x2d4ffa\x2db5fe\x2dfd3e43ff6c2b.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-fsck@dev-disk-by\x2duuid-bdcc1c07\x2d3966\x2d4ffa\x2db5fe\x2dfd3e43ff6c2b.service - File System Check on /dev/disk/by-uuid/bdcc1c07-3966-4ffa-b5fe-fd3e43ff6c2b.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-fsck@dev-disk-by\x2duuid-bdcc1c07\x2d3966\x2d4ffa\x2db5fe\x2dfd3e43ff6c2b comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Apr 22 06:42:43 Papaputer systemd[1]: Removed slice system-systemd\x2dfsck.slice - Slice /system/systemd-fsck.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped target local-fs-pre.target - Preparation for Local File Systems.
Apr 22 06:42:43 Papaputer systemd[1]: Stopping lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Apr 22 06:42:43 Papaputer systemd[1]: systemd-remount-fs.service: Deactivated successfully.
Apr 22 06:42:43 Papaputer systemd[1]: Stopped systemd-remount-fs.service - Remount Root and Kernel File Systems.
Apr 22 06:42:43 Papaputer audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-remount-fs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

...

Apr 22 06:42:43 Papaputer systemd[1]: Reached target reboot.target - System Reboot.
Apr 22 06:42:43 Papaputer systemd[1]: Shutting down.
Apr 22 06:42:43 Papaputer audit: BPF prog-id=197 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=196 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=195 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=199 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=198 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=202 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=201 op=UNLOAD
Apr 22 06:42:43 Papaputer audit: BPF prog-id=131 op=UNLOAD
Apr 22 06:42:43 Papaputer systemd[1]: Using hardware watchdog 'iTCO_wdt', version 0, device /dev/watchdog0
Apr 22 06:42:43 Papaputer systemd[1]: Watchdog running with a timeout of 10min.
Apr 22 06:42:43 Papaputer kernel: watchdog: watchdog0: watchdog did not stop!
Apr 22 06:42:43 Papaputer systemd-shutdown[1]: Using hardware watchdog 'iTCO_wdt', version 0, device /dev/watchdog0
Apr 22 06:42:43 Papaputer systemd-shutdown[1]: Watchdog running with a timeout of 10min.
Apr 22 06:42:43 Papaputer systemd-shutdown[1]: Syncing filesystems and block devices.
Apr 22 06:42:44 Papaputer systemd-shutdown[1]: Sending SIGTERM to remaining processes...
Apr 22 06:42:44 Papaputer systemd-journald[649]: Journal stopped

Unfortunately, the coredump seems to be missing. It does not appear in coredumpctl list. Is there some other place I could look?

[Edit] P.S. Could this system be repaired without reinstalling, or would you recommend reinstalling?

1 Like

Thanks again, this is really helpful. ugh, if coredumpctl doesn’t show the coredump, it’s probably gone or was never stored :frowning: I’ll forward it to the bug report, but if it’s not too much trouble, could you post a longer log extract so we can see all the packages you had installed, then we can try an upgrade with the same package set and see if it triggers the bug again? It’s always easier to fix a problem once we can reliably reproduce it - that lets us do whatever we need to do to actually save a coredump and backtrace it, and so on.

edit: as for recovery - um. I’m not sure offhand exactly how far along in the process “package cleanup” is, unfortunately. I’ll have to work that out and see what might have been missed. There’s no straightforward way to recover completely from a crash during a package transaction, unfortunately - it’s kind of a case-by-case thing, you need to know how far along it got so you can figure out what might have been missed and try to do things to trigger the missed operations. You can just keep using the system and deal with any weirdnesses as they arise, that’s one option.

This is one area where immutable systems (Silverblue, Kinoite etc.) definitely have an advantage.

2 Likes

Here you can find the complete upgrade logs.

And a list of currently installed packages, obtained using dnf list installed.

(Best to download the text documents as they are only available for a limited time)

Ok, as my family member needs this computer back up and running, I’m going to reinstall this system tomorrow. Let me know if you need anything else or if I can go ahead and wipe this system.

Kind regards
Lyvendia

1 Like

Perhaps this is related to the following issue:
DNF operations use large amount of RAM and may fail in low-memory environments

You can usually complete the interrupted upgrade like this:
F38upgrade error; System Is full of duplicate packages after update - #17 by vgaetera

1 Like

The system on which this error occurred has 32GB of RAM. And thanks for the tips on completing the interrupted upgrade! I will give it a try before trying to reinstall.

1 Like

This isn’t the RAM thing, the errors would be different in that case. That approach to recovery might help, I’m not sure there will be any duplicate packages in this case; I think if something’s been missed it’s likely to be %posttrans operations, which won’t necessarily be the same in a distro-sync, but it probably can’t hurt.

1 Like

For me this problem was only a symptom of a deeper problem with package duplicates that had remained from fedora 37. I would highly recommend looking into the thread mentioned by @vgaetera. His solution was also quite helpful in my eventual resolution of the problem. A simple marker to see if you are having the same issue is to run the following command and if the result is a number greater than 0, you most likely are.

sudo dnf repoquery --duplicates | wc -l

To be fair, duplicates appear when the package manager terminates abnormally during package upgrades possibly due to a crash and leaves unfinished transactions.

1 Like

The crash during upgrade is what can cause duplicate package entries in the RPM database. Any time dnf/PackageKit/rpm crashes during a package transaction, there are various consequences, depending on exactly where it crashes. Some of the scriptlets that would usually be run as part of the transaction do not get run. Duplicate entries can appear in the package database; this is because the upgrade process is a complex multi-step process where the entries for the newer package versions are added to the database at one point in the process, and the entries for the older versions are removed at a later point in the process. If the process crashes between those points, you get duplicates.

In this specific case, folks are running into at least two problems caused by the crash: the scriptlet that should update the gsettings schema would have happened after the crash (so it didn’t happen), and the /root/system-update file should be removed at the very end of the process (but because of the crash, that didn’t happen). There could be other problems lurking; because of the nature of this kind of issue it’s hard to be sure.

The duplicates don’t cause the crash, it’s the other way around.

2 Likes

Did you have any warning when doing the upgrade, for example that some packages won´t be available?

It happened to me with pipewire-(something-I-cant-remember) and Steam (I see they are connected with rpmfusion-free repos).

That’s likely not relevant - those kinds of warnings aren’t unusual, it’s basically Software warning you it’s doing the equivalent of what happens if you use the --allowerasing arg for dnf: if it can’t manage to include a package in the upgrade because of dependency problems, it’ll remove it. This can happen with more uncommon packages we haven’t tested upgrades of, or things from third-party repos. It shouldn’t be the cause of the crash.