Broken SDDM/Plasma after upgrading Fedora 36 to 37

I had this Fedora KDE installation which hasn’t been upgraded in quite some time. It was still on 36 (primarily because Discover didn’t have the ability to perform distro upgrades in that version yet). A few days ago I tried to finally get it fully upgraded.

I followed the official guide. The relevant dnf commands being:

dnf dsync
# ↑ not sure why I did this one, maybe I've seen it mentioned somewhere else
dnf upgrade --refresh
dnf install dnf-plugin-system-upgrade
dnf system-upgrade download --releasever=39^C
# the guide mentions that upgrades across >2 releases are not tested
# (it was only starting to download stuff)
dnf system-upgrade download --releasever=37
dnf system-upgrade reboot

After the system was done with the upgrade and it rebooted again, I was presented with a barebones SDDM login screen and starting a Plasma session was failing (no matter if Wayland or X.org was selected). The following error message was also shown below the login dialog:

The current theme cannot be loaded due to the errors below, please select another theme.

file:///usr/share/sddm/themes/01-breeze-fedora/Main.qml:12:1: plugin cannot be loaded for module "org.kde.plasma.core": Cannot load library /usr/lib64/qt5/gml/org/dke/plasma/core/libcorebindingsplugin.so: (/lib64/libKF5WidgetsAddons.so.5: undefined symbol: _ZN11QToolButton13checkStateSetEv, version Qt+5)

I tried to take a screenshot, but running DISPLAY=… spectacle --background … from a tty results in a lookup error about the same symbol.

Here are some logs from sddm.service:

journalctl -u sddm.service -b=…

A session before the upgrade:

sty 05 18:13:13 faltuel systemd[1]: Started sddm.service - Simple Desktop Display Manager.
sty 05 18:13:14 faltuel sddm-helper[1255]: pam_unix(sddm-greeter:session): session opened for user sddm(uid=988) by (uid=0)
sty 05 18:13:14 faltuel sddm-helper[1255]: Starting X11 session: "" "/usr/bin/sddm-greeter --socket /tmp/sddm-:0-XQJiTs --theme /usr/share/sddm/themes/01-breeze-fedora"
sty 05 18:13:22 faltuel sddm-helper[1299]: gkr-pam: unable to locate daemon control file
sty 05 18:13:22 faltuel sddm-helper[1299]: gkr-pam: stashed password to try later in open session
sty 05 18:13:22 faltuel sddm-helper[1299]: pam_kwallet5(sddm:auth): pam_kwallet5: pam_sm_authenticate
sty 05 18:13:22 faltuel sddm-helper[1299]: pam_kwallet5(sddm:setcred): pam_kwallet5: pam_sm_setcred
sty 05 18:13:22 faltuel sddm-helper[1299]: pam_unix(sddm:session): session opened for user $user(uid=1000) by (uid=0)
sty 05 18:13:22 faltuel sddm-helper[1299]: gkr-pam: gnome-keyring-daemon started properly and unlocked keyring
sty 05 18:13:22 faltuel sddm-helper[1299]: pam_kwallet5(sddm:session): pam_kwallet5: pam_sm_open_session
sty 05 18:13:22 faltuel sddm-helper[1299]: Starting Wayland user session: "/etc/sddm/wayland-session" "/usr/libexec/plasma-dbus-run-session-if-needed /usr/bin/startplasma-wayland"
sty 06 19:25:25 faltuel sddm[1236]: Authentication error: SDDM::Auth::ERROR_INTERNAL "Process crashed"
sty 06 19:25:25 faltuel sddm[1236]: Auth: sddm-helper (--socket /tmp/sddm-authc99b9245-1577-46b9-bdc0-22fdd82511a6 --id 1 --start /usr/libexec/plasma-dbus-run-session-if-needed /usr/bin/startplasma-wayland --user $user) crashed (exit code 1)
sty 06 19:25:25 faltuel sddm[1236]: Authentication error: SDDM::Auth::ERROR_INTERNAL "Process crashed"
sty 06 19:25:25 faltuel sddm[1236]: Auth: sddm-helper exited with 1
sty 06 19:25:25 faltuel sddm[1236]: Signal received: SIGTERM
sty 06 19:25:25 faltuel systemd[1]: Stopping sddm.service - Simple Desktop Display Manager...
sty 06 19:25:25 faltuel systemd[1]: sddm.service: Deactivated successfully.
sty 06 19:25:25 faltuel systemd[1]: Stopped sddm.service - Simple Desktop Display Manager.
sty 06 19:25:25 faltuel systemd[1]: sddm.service: Consumed 1.017s CPU time.

A session afterwards:

lut 12 11:59:43 faltuel systemd[1]: Started sddm.service - Simple Desktop Display Manager.
lut 12 11:59:44 faltuel sddm-helper[1533]: pam_unix(sddm-greeter:session): session opened for user sddm(uid=988) by (uid=0)
lut 12 11:59:44 faltuel sddm-helper[1533]: Starting X11 session: "" "/usr/bin/sddm-greeter --socket /tmp/sddm-:0-xnAHTI --theme /usr/share/sddm/themes/01-breeze-fedora"
lut 12 12:00:29 faltuel sddm-helper[1636]: gkr-pam: unable to locate daemon control file
lut 12 12:00:29 faltuel sddm-helper[1636]: gkr-pam: stashed password to try later in open session
lut 12 12:00:29 faltuel sddm-helper[1636]: pam_kwallet5(sddm:auth): pam_kwallet5: pam_sm_authenticate
lut 12 12:00:29 faltuel sddm-helper[1636]: pam_kwallet5(sddm:setcred): pam_kwallet5: pam_sm_setcred
lut 12 12:00:30 faltuel sddm-helper[1636]: pam_unix(sddm:session): session opened for user $user(uid=1000) by (uid=0)
lut 12 12:00:30 faltuel sddm-helper[1636]: gkr-pam: gnome-keyring-daemon started properly and unlocked keyring
lut 12 12:00:30 faltuel sddm-helper[1636]: pam_kwallet5(sddm:session): pam_kwallet5: pam_sm_open_session
lut 12 12:00:30 faltuel sddm-helper[1636]: Starting Wayland user session: "/etc/sddm/wayland-session" "/usr/libexec/plasma-dbus-run-session-if-needed /usr/bin/startplasma-wayland"
lut 12 12:00:30 faltuel sddm-helper[1636]: pam_unix(sddm:session): session closed for user $user
lut 12 12:00:30 faltuel sddm-helper[1636]: pam_kwallet5(sddm:session): pam_kwallet5: pam_sm_close_session
lut 12 12:00:30 faltuel sddm-helper[1636]: pam_kwallet5(sddm:setcred): pam_kwallet5: pam_sm_setcred
lut 12 12:00:30 faltuel sddm[1513]: Auth: sddm-helper exited with 127
lut 12 12:00:31 faltuel sddm-helper[1729]: pam_unix(sddm-greeter:session): session opened for user sddm(uid=988) by (uid=0)
lut 12 12:00:31 faltuel sddm-helper[1729]: Starting X11 session: "" "/usr/bin/sddm-greeter --socket /tmp/sddm-:0-DRfgFH --theme /usr/share/sddm/themes/01-breeze-fedora"
lut 12 12:00:41 faltuel sddm-helper[1749]: gkr-pam: unable to locate daemon control file
lut 12 12:00:41 faltuel sddm-helper[1749]: gkr-pam: stashed password to try later in open session
lut 12 12:00:41 faltuel sddm-helper[1749]: pam_kwallet5(sddm:auth): pam_kwallet5: pam_sm_authenticate
lut 12 12:00:41 faltuel sddm-helper[1749]: pam_kwallet5(sddm:setcred): pam_kwallet5: pam_sm_setcred
lut 12 12:00:42 faltuel sddm-helper[1749]: pam_unix(sddm:session): session opened for user $user(uid=1000) by (uid=0)
lut 12 12:00:42 faltuel sddm-helper[1749]: gkr-pam: gnome-keyring-daemon started properly and unlocked keyring
lut 12 12:00:42 faltuel sddm-helper[1749]: pam_kwallet5(sddm:session): pam_kwallet5: pam_sm_open_session
lut 12 12:00:42 faltuel sddm-helper[1749]: Starting Wayland user session: "/etc/sddm/wayland-session" "/usr/libexec/plasma-dbus-run-session-if-needed /usr/bin/startplasma-wayland"
lut 12 12:00:42 faltuel sddm-helper[1749]: pam_unix(sddm:session): session closed for user $user
lut 12 12:00:42 faltuel sddm-helper[1749]: pam_kwallet5(sddm:session): pam_kwallet5: pam_sm_close_session
lut 12 12:00:42 faltuel sddm-helper[1749]: pam_kwallet5(sddm:setcred): pam_kwallet5: pam_sm_setcred
lut 12 12:00:42 faltuel sddm[1513]: Auth: sddm-helper exited with 127
lut 12 12:00:42 faltuel sddm-helper[1837]: pam_unix(sddm-greeter:session): session opened for user sddm(uid=988) by (uid=0)
lut 12 12:00:42 faltuel sddm-helper[1837]: Starting X11 session: "" "/usr/bin/sddm-greeter --socket /tmp/sddm-:0-DFptsq --theme /usr/share/sddm/themes/01-breeze-fedora"
lut 12 12:00:44 faltuel sddm-helper[1856]: pam_unix(sddm:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=  user=$user
lut 12 12:00:44 faltuel sddm-helper[1856]: gkr-pam: unable to locate daemon control file
lut 12 12:00:44 faltuel sddm-helper[1856]: gkr-pam: stashed password to try later in open session
lut 12 12:00:44 faltuel sddm-helper[1856]: pam_kwallet5(sddm:auth): pam_kwallet5: pam_sm_authenticate
lut 12 12:00:44 faltuel sddm-helper[1856]: pam_kwallet5(sddm:auth): pam_kwallet5: Empty or missing password, doing nothing
lut 12 12:00:46 faltuel sddm-helper[1856]: [PAM] authenticate: Authentication failure
lut 12 12:00:46 faltuel sddm[1513]: Authentication error: SDDM::Auth::ERROR_AUTHENTICATION "Authentication failure"
lut 12 12:00:46 faltuel sddm-helper[1856]: [PAM] Asked to close the session but it wasn't previously open
lut 12 12:00:46 faltuel sddm[1513]: Auth: sddm-helper exited with 1
lut 12 12:01:29 faltuel sddm-helper[1876]: pam_unix(sddm:auth): authentication failure; logname= uid=0 euid=0 tty= ruser= rhost=  user=$user
lut 12 12:01:29 faltuel sddm-helper[1876]: gkr-pam: unable to locate daemon control file
lut 12 12:01:29 faltuel sddm-helper[1876]: gkr-pam: stashed password to try later in open session
lut 12 12:01:29 faltuel sddm-helper[1876]: pam_kwallet5(sddm:auth): pam_kwallet5: pam_sm_authenticate
lut 12 12:01:31 faltuel sddm-helper[1876]: [PAM] authenticate: Authentication failure
lut 12 12:01:31 faltuel sddm[1513]: Authentication error: SDDM::Auth::ERROR_AUTHENTICATION "Authentication failure"
lut 12 12:01:31 faltuel sddm-helper[1876]: [PAM] Asked to close the session but it wasn't previously open
lut 12 12:01:31 faltuel sddm[1513]: Auth: sddm-helper exited with 1
lut 12 12:02:21 faltuel sddm[1513]: Error from greeter session: "Process crashed"
lut 12 12:02:21 faltuel sddm[1513]: Auth: sddm-helper (--socket /tmp/sddm-auth-07bec49d-5b83-4970-835f-49ccf8ad0229 --id 6 --start /usr/bin/sddm-greeter --socket /tmp/sddm-:0-DFptsq --theme /usr/share/sddm/themes/01-breeze-fedora --user sddm --greeter) crashed (exit code 1)
lut 12 12:02:21 faltuel sddm[1513]: Error from greeter session: "Process crashed"
lut 12 12:02:21 faltuel sddm[1513]: Auth: sddm-helper exited with 1
lut 12 12:02:21 faltuel systemd[1]: Stopping sddm.service - Simple Desktop Display Manager...
lut 12 12:02:21 faltuel sddm[1513]: Signal received: SIGTERM
lut 12 12:02:21 faltuel systemd[1]: sddm.service: Deactivated successfully.
lut 12 12:02:21 faltuel systemd[1]: Stopped sddm.service - Simple Desktop Display Manager.
lut 12 12:02:21 faltuel systemd[1]: sddm.service: Consumed 3.373s CPU time.

It might also be relevant that the system had rpmfusion set up.

At this point, a clean install of Fedora 39 seems to be the easiest option. However, maybe someone here would be interested in figuring out what exactly went wrong. Then it could be fixed, so that it doesn’t happen to anyone else (or I can get better at using dnf hehe).

Usunięto f39

Usunięto gnome

Usunięto wayland

In general, I’d advise against skipping even one release version. It takes longer, but going one release at a time is much more reliable than skipping versions. IIRC, the “you can skip a version” rule is only true when the version you are upgrading to is a relatively new release. Fedora Linux 37 (nor even Fedora Linux 39) is a “new” release at this time, so that upgrade path should be considered entirely untested.

As far as the missing files go, you can usually find and install whatever package provides those files with a command like dnf provides '<full-path-to-file>'.

Yes, my plan was to do it in three steps (36 → 37 → 38 → 39), but something broke already on the first one.

“something broke” is obviously much too vague to diagnose. But in the extremely rare cases that I’ve run into problems with dnf update --releasever=XX, the typical solution (for me) has been to just remove the “problematic” package with a command like rpm --nodeps -e <package-name> (or more often just adding --allowerasing to the dnf update command is enough). But the possibilities as to what could go wrong during an upgrade are quite numerous and that may not always work. In any case, having a snapshot of your rootfs to rollback to in case anything does go wrong is always advisable.

Both F36 and F37 are EOL (end of life) so the repositories are inf fact offline, or full of outdated packages.
I advise you, in this case, to download a F39 iso and do a fresh install. You can even reuse the old /home btrfs subvolume. (here is how its done QA:Testcase partitioning custom btrfs preserve home - Fedora Project Wiki )

1 Like

My goal here isn’t to “make it work again” – as I said a clean install is absolutely fine in this case. Instead, I would like to understand what broke. Have I done something wrong or is that a problem with the packages? (If it’s the first one, maybe we could improve the documentation. If the second, shouldn’t that be fixed?)

While reading up on dnf provides, I noticed there’s a dnf check as well. It doesn’t detect any issues though. I’ve double-checked and all these files and their packages are present. It really seems it’s just that symbol missing. Is there something like dnf provides that is able to track symbols as well? The manpage does not mention this so if not, is there anything else I could check?

Now that I read it again, I’m starting to doubt if I understood you correctly. Do you mean that upgrades between consecutive releases are possibly unsafe after enough time passes?

Hmm, I don’t think I got any errors at that stage. There was a lot of text about all the packages that will be installed, removed and so on so I might have missed something. But the docs state:

If some of your packages have unsatisfied dependencies, the upgrade will refuse to continue until you run it again with an extra --allowerasing option.

and nothing of that sort happened, so I’m lead to assume it’s all good in that regard.

Now that I’m fairly certain the true issue is with that missing symbol, I’ve tried to search around on the internet and found a few different posts about applications not starting due to this. All from fall last year. The first one is specifically about Fedora 37 and the last (while being about Rocky Linux) describes exactly the same symptoms.

It looks to me that what I hit is a Fedora 37 packaging bug, introduced some time after its release.

Are they the right versions of the packages though? The “symbol missing” error sounds a bit like it is looking for something within a library file that isn’t present for some reason (e.g., because it is an older library from an older package than the one the main/calling executable is expecting).

Use rpm -qf <path-to-file> to find out what packages the files belong to (in this case, that would be rpm -qf /lib64/libKF5WidgetsAddons.so.5 and rpm -qf /usr/lib64/qt5/gml/org/dke/plasma/core/libcorebindingsplugin.so).

Then, verify that those packages are intact/uncorrupted with rpm -V <package-name>.

sudo rpmreaper ~B is also a useful tool to see if you have any packages installed with missing dependencies.

Edit: I mis-typed that previous command initially – note that the character before the B is supposed to be a tilde (~).

This is the documentation I was thinking of (emphasis added):

Can I do a single upgrade across many releases (i.e. 30→34)?

Upgrades to the very next release (e.g. 38 to 39) as well as upgrades skipping one release (e.g. 37 to 39) are both supported. However, it is highly recommended to perform the upgrade before your release reaches End of Life (EOL). That happens roughly a month after N+2 release has been released (when you’re currently on release N). The Fedora Release Life Cycle is specifically designed to provide this approximate one month “grace period” to allow users the choice to upgrade their systems on a yearly basis, i.e. once every two releases. You can study Releases to see the current release status and schedule. Around a month after the new release comes out, the last-but-one release becomes End of Life (EOL). The upgrade is likely to work successfully after the release goes EOL, but the time period after the new release may be uncertain.

So it doesn’t specifically state how long after a new release the upgrade from an EOL version to a new release (e.g. 37 → 39) is “likely to work”. But the implication is that it isn’t guaranteed to work after the new release has been out for a while. Understand that the packages that make up a new release are in flux throughout the lifecycle of the release. Testing is only done initially for upgrades from 37 (EOL) to 39 (just released package set). As time progresses, the packages that make up Fedora Linux 39 change and upgrades from the Fedora Linux 37 package set (which is no longer changing because it is EOL) are not continually tested against the ever-changing Fedora Linux 39 package set. You are only “guaranteed” (though really that word is too strong) to be able to upgrade from 37 to 39 immediately after 39 comes out.

If I understood the error correctly it seems there is a theme that is preventing a clean boot.
Removing the faulty theme should allow booting properly.

It is quite common for themes to change or be removed with version updates so this could easily be a simple case of fedora 37 not supporting a theme that previously worked in F36.

Thank you for that link! Preserving the /home subvolume is an awesome feature to have.

The libKF5WidgetsAddons.so.5 indeed does not contain that symbol, but it does contain a very similar one: _ZN11QToolButton13checkStateSetEv@Qt_5. The package that provides it is kf5-widgetsaddons-5.108.0-2.fc37.x86_64 from the updates repo. There’s also one in fedora with version 5.99.0-1.fc37.x86_64, but that’s not installed. Maybe the library was reorganized on the KDE side and the symbols changed slightly, but the package metadata doesn’t indicate this incompatibility? After demangling, the symbol in question becomes QToolButton::checkStateSet(). I guess I can look for it in the source code.

Also, thanks for pointing out that rpmreaper ~B command. I’ll take a look at it.

It’s a bit more complex I believe. The error in SDDM occurs while loading the theme, but due to an incompatible dependency, and not a bug in the theme itself.

Also, the theme in question is Breeze, the default one. I highly doubt it’s gonna be removed any time soon.

I’m not seeing anything that provides /usr/lib64/qt5/gml/org/dke/plasma/core/libcorebindingsplugin.so.

dnf provides --repo=fedora --repo=updates '*/dke/plasma/core/libcorebindingsplugin.so*'
Last metadata expiration check: 0:21:20 ago on Thu 15 Feb 2024 11:02:33 PM CST.
Error: No matches found. If searching for a file, try specifying the full path or using a wildcard prefix ("*/") at the beginning.

If it is from a third-party repo, then anything is possible and your best bet is probably just to uninstall it.

Ah sorry, there’s a typo. It should have been kde not dke. Copy-pasting an error message from the greeter isn’t an option sadly :upside_down_face:

That file is provided by kf5-plasma. At least on Fedora 37.

Alright, I think I’m satisfied at this point. As great as it would be to have every package version perfect, that just isn’t feasible – especially for EOL releases. Since the problem seems to be with mismatched library versions in rather old packages, there’s nothing that can be done here really, besides upgrading the affected systems.

I’m fairly certain the problem will simply go away after upgrading to Fedora 38.

To summarize:
In the Qt Widgets module from qtbase there’s a QToolButton::checkStateSet() method. It’s used and gets exported as an ELF symbol by several different libraries – in my case WidgetsAddons from KDE Frameworks. For some reason, in certain cases the symbol gains an @Qt_5 suffix, and the library exposing it becomes incompatible with software linking to it.

1 Like

Indeed, the issue isn’t present in Fedora 38 and dnf system-upgrade made it go away.

The upgrade succeeded and both SDDM and Plasma are working again. One thing I noticed though is that before doing dnf system-upgrade download --releasever=38, the dnf upgrade --refresh complained about dependency conflicts in some other Qt packages. Nothing to directly do with kf5-kwidgetsaddons, but it still might have influenced the version mismatch there.

dnf upgrade
Last metadata expiration check: 0:06:50 ago on Fri Feb 16 07:52:18 2024.
Dependencies resolved.

 Problem 1: cannot install the best update candidate for package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64
  - nothing provides qt5-qtbase(x86-64) = 5.15.9 needed by qt5-qtwebengine-freeworld-5.15.12-3.fc37.x86_64 from rpmfusion-free-updates
 Problem 2: problem with installed package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64
  - package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64 from @System requires qt5-qtbase(x86-64) = 5.15.6, but none of the providers can be installed
  - package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64 from rpmfusion-free requires qt5-qtbase(x86-64) = 5.15.6, but none of the providers can be installed
  - cannot install both qt5-qtbase-5.15.10-9.fc37.x86_64 from updates and qt5-qtbase-5.15.6-1.fc37.x86_64 from @System
  - cannot install both qt5-qtbase-5.15.10-9.fc37.x86_64 from updates and qt5-qtbase-5.15.6-1.fc37.x86_64 from fedora
  - cannot install the best update candidate for package qt5-qtbase-5.15.6-1.fc37.x86_64
  - nothing provides qt5-qtbase(x86-64) = 5.15.9 needed by qt5-qtwebengine-freeworld-5.15.12-3.fc37.x86_64 from rpmfusion-free-updates
================================================================================
 Package                   Arch   Version          Repository              Size
================================================================================
Skipping packages with conflicts:
(add '--best --allowerasing' to command line to force their upgrade):
 qt5-qtbase                x86_64 5.15.10-9.fc37   updates                3.5 M
Skipping packages with broken dependencies:
 qt5-qtwebengine-freeworld x86_64 5.15.12-3.fc37   rpmfusion-free-updates  45 M

Transaction Summary
================================================================================
Skip  2 Packages

Nothing to do.
Complete!
dnf upgrade --allowerasing
Last metadata expiration check: 0:07:12 ago on Fri Feb 16 07:52:18 2024.
Dependencies resolved.

 Problem: problem with installed package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64
  - cannot install the best update candidate for package qt5-qtwebengine-freeworld-5.15.10-2.fc37.x86_64
  - nothing provides qt5-qtbase(x86-64) = 5.15.9 needed by qt5-qtwebengine-freeworld-5.15.12-3.fc37.x86_64 from rpmfusion-free-updates
================================================================================
 Package                   Arch   Version          Repository              Size
================================================================================
Skipping packages with broken dependencies:
 qt5-qtwebengine-freeworld x86_64 5.15.12-3.fc37   rpmfusion-free-updates  45 M

Transaction Summary
================================================================================
Skip  1 Package

Nothing to do.
Complete!

So I still have two questions:

  • Why did dnf check and dnf repoquery --unsatisfied not say anything about that?
  • Why did dnf system-upgrade work? Does it not resolve dependency versions before installing them?

My guess is the installed packages all had their dependency constraints satisfied and it was only these potential updates that would not be valid. Is that correct?

It sounds like it was a packaging bug with regard to the specification of dependencies in Fedora Linux 37. dnf check is just going by what is recorded in the packages themselves. It won’t be able to discover missing/incorrect library APIs. That is supposed to be done manually by the packager when they create the RPM package.

rpm -q -R <package-name> will tell you a little about what requirements/dependencies the packager has specified for a package. You’ll often see specifications of >= or <= in the requirements list which shows that the packager is making some assumptions about how well the different components will integrate in future versions (the future is never certain).

dnf system-upgrade is checking the dependencies for the to-be-installed packages.

That sounds right to me.

Then it’s all clear and everything works now.

Thank you for your assistance. I really appreciate it!