I’m using Fedora Silverblue 35. I love the idea of Silverblue but I’m trying to understand how I am supposed to deal with a regression in Silverblue.
What happened: I upgraded my system one day and noticed a pretty big regression (different per-monitor scaling no longer works in GNOME, animations are choppy, etc). I ran rpm-ostree rollback and it solved the problem – awesome!
But what am I supposed to do now?
Is there a way to partially apply updates with rpm-ostree so I can at least upgrade packages unrelated to my issue?
Is there a way to identify which package update is causing the regression? The list is fairly big and growing every day.
Thanks, that’s what I’ve been doing, except daily ! I was hoping there was a way to help, maybe test a partial upgrade to at least figure out the package that’s causing this, then maybe I could file a bug.
Using this approach, I was able to pinpoint the regression to 35.20220118.0 (as in, 35.20220117.0 still works). Strangely enough, this version only upgrades the following packages:
OK, I feel pretty stupid for not noticing this before: what is actually broken is Wayland. After the upgrade above, my system runs on X11 instead of Wayland. It doesn’t support per-monitor scaling and it’s not smooth because… it’s not Wayland. Duh.
So either the initscripts change, or the selinux-policy change stops Wayland from working on my machine. I read through the changelogs but nothing stands out:
initscripts:
* Thu Jan 13 2022 Jan Macku <jamacku@redhat.com> - 10.13-1
- ifup-routes: Log when using `ip $type replace`
- ifup-routes: Use `ip route repace` to avoid race
- Translated using Weblate (German)
- Add LGTM badges to README
- ci: set default merge method to rebase
- ci: disable comments under opened PR in order to fix CI
- network scripts: do not use c-style for-loop
- network scripts: replace "<<<" with pipe
- rc.d/functions: do not use "+=" to concatenate string
- ci: Use default github-token (#395)
- ci(Mergify): configuration update (#394)
- ci: Output shellcheck results using PR comments (#393)
- ci: Update path to csdiff repository (#391)
- spec: Fix issue with $NEXT_VERSION (#390)
- Translated using Weblate (Indonesian)
- Translated using Weblate (Spanish)
- Translated using Weblate (Czech)
* Fri Sep 03 2021 Jan Macku <jamacku@redhat.com> - 10.12-1
- spec: Update relation between initscripts and initscripts-service (#386)
selinux-policy:
* Wed Jan 12 2022 Zdenek Pytela <zpytela@redhat.com> - 35.9-1
- Allow sshd read filesystem sysctl files
- Revert "Allow sshd read sysctl files"
- Allow tlp read its systemd unit
- Allow gssproxy access to various system files.
- Allow gssproxy read, write, and map ica tmpfs files
- Allow gssproxy read and write z90crypt device
- Allow sssd_kcm read and write z90crypt device
- Allow smbcontrol read the network state information
- Allow virt_domain map vhost devices
- Allow fcoemon request the kernel to load a module
- Allow sshd read sysctl files
- Ensure that `/run/systemd/*` are properly labeled
- Allow admin userdomains use socketpair()
- Change /run/user/[0-9]+ to /run/user/%{USERID} for proper labeling
- Allow lldpd connect to snmpd with a unix domain stream socket
- Dontaudit pkcsslotd sys_admin capability
(it seems like selinux-policy and selinux-policy-targeted depend on each other, so must be upgraded together)
Unfortunately, this fails with:
error: Checkout selinux-policy-targeted-35.9-1.fc35.noarch: Hardlinking a5/8b8b3f84fa2d588c41ae5fa6615dfe387b262737198f5b2a9c5f24b0b23045.file to commit_num: File exists
I don’t know nearly enough about rpm-ostree to figure that one out.
I’ve looked at the boot logs for the broken version, and it seems that GDM on Wayland is crashing at startup. It seems related to something called “User Runtime Directory”, which fails to mount/start:
Jan 29 23:10:18 fedora systemd[1]: Created slice User Slice of UID 42.
Jan 29 23:10:18 fedora systemd[1]: Starting Network Manager Script Dispatcher Service...
Jan 29 23:10:18 fedora systemd[1]: Starting User resource assignment daemon...
Jan 29 23:10:18 fedora systemd[1]: Starting User Runtime Directory /run/user/42...
Jan 29 23:10:18 fedora systemd[1]: Started Network Manager Script Dispatcher Service.
Jan 29 23:10:18 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/system>
Jan 29 23:10:18 fedora systemd-user-runtime-dir[1093]: Regex version mismatch, expected: 10.39 2021-10-29 actual: 10.37 2021-05-26
Jan 29 23:10:18 fedora audit[1093]: AVC avc: denied { create } for pid=1093 comm="systemd-user-ru" name="42" scontext=system_u:system_r:systemd_logind_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s>
Jan 29 23:10:18 fedora systemd-user-runtime-dir[1093]: Failed to mount per-user tmpfs directory /run/user/42: No such file or directory
Jan 29 23:10:18 fedora systemd[1]: user-runtime-dir@42.service: Main process exited, code=exited, status=1/FAILURE
Jan 29 23:10:18 fedora systemd[1]: user-runtime-dir@42.service: Failed with result 'exit-code'.
Jan 29 23:10:18 fedora systemd[1]: Failed to start User Runtime Directory /run/user/42.
Jan 29 23:10:18 fedora systemd[1]: Dependency failed for User Manager for UID 42.
Jan 29 23:10:18 fedora systemd[1]: user@42.service: Job user@42.service/start failed with result 'dependency'.
[...]
Jan 29 23:10:18 fedora gdm-launch-environment][1080]: pam_systemd(gdm-launch-environment:session): Failed to stat() runtime directory '/run/user/42': No such file or directory
Jan 29 23:10:18 fedora gdm-launch-environment][1080]: pam_systemd(gdm-launch-environment:session): Not setting $XDG_RUNTIME_DIR, as the directory is not in order.
[...]
Jan 29 23:10:19 fedora gnome-shell[1145]: WL: error: XDG_RUNTIME_DIR not set in the environment
Jan 29 23:10:19 fedora audit[1145]: ANOM_ABEND auid=4294967295 uid=42 gid=42 ses=4294967295 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 pid=1145 comm="gnome-shell" exe="/usr/bin/gnome-shell" sig=5 res=1
Jan 29 23:10:19 fedora org.gnome.Shell.desktop[1145]: == Stack trace for context 0x55a82c2301f0 ==
Jan 29 23:10:19 fedora gnome-shell[1145]: Failed to create socket
Jan 29 23:10:19 fedora systemd[1]: Created slice Slice /system/systemd-coredump.
Jan 29 23:10:19 fedora systemd[1]: Started Process Core Dump (PID 1216/UID 0).
Jan 29 23:10:19 fedora audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-coredump@0-1216-0 comm="systemd" exe="/usr/lib/systemd/system>
Jan 29 23:10:19 fedora systemd-coredump[1217]: [🡕] Process 1145 (gnome-shell) of user 42 dumped core.
Which seems suspicious given that one of the changes in selinux-policy is “Change /run/user/[0-9]+ to /run/user/%{USERID} for proper labeling”.
I’ll try to file a bug/contact the author of this commit.
Yeah, I added my own notes to the newer bug. If anyone’s facing the same issue, for now I’ve switched SELinux to permissive while I wait for Red Hat to fix the policy – I didn’t like staying so far behind on updates for the whole system.
I did install setroubleshoot and read up on how I could fix the policy myself, but there seems to be a lot to learn about SELinux, and for now I’ll just wait and see.
! I was hoping there was a way to help, maybe test a partial upgrade to at least figure out the package that’s causing this, then maybe I could file a bug.
– I didn’t like staying so far behind on updates for the whole system.