A consistent system freeze at every Sunday night

Ask Fedora
, , , , , ,
1

I have the weirdest issue where my desktop PC will just freeze consistently every Sunday night. Usually what I’m doing at this time is sitting away from the computer while watching something on it via MPV. Then the desktop freezes: sound and image are frozen and Bluetooth connections are lost. Fans are still spinning and the monitor does not goes to sleep, but I can’t even access a tty, so what I’m left to do is to force reboot by power-cycling the machine. Then I’m back as if nothing has happened. No heavy load process running on the background.

Weirdly, journalctl reports nothing, at least that I can find. Before the forced reboot, the last message on it is about dnf-makecache, which apparently succeeds, but this is probably irrelevant. Similarly, I could never find a relevant message on it the other times it happened.

Could be related that I recently changed to an AMD card (RX 7700xt). Before that, I had an Nvidia card and also every Sunday night suspend would fail (it’s cursed, I guess), but never this kind of system freeze that I remember of. I guess this started to happen after the 6.9 kernels, but I’m not sure now. The periodicity of it which makes me worried/curious. i usually do my system updates at the weekends too, but they always succeed. Never actually had an issue with an update. Other than that, I’m just using a standard installation of F40 Workstation, no heavy modifications.

So, how can I look deeper into this issue to figure out the cause? It doesn’t seem like a hardware issue to me, otherwise it would happen more often, right? I run videogames on this machine just fine, for example, which are far more demanding on the hardware than playing a MPV video and nothing of the sort happen during that :thinking:…

2

I believe fstrim runs weekly, maybe something related to that and SSD/NVME trim?

3

I believe this runs after boot, so it likely isn’t the cause of your issue.

Edit: it’s timer based:

OnBootSec=10min
OnUnitInactiveSec=1h
RandomizedDelaySec=60m
Unit=dnf-makecache.service
4

The first IBM PC we got at work would crash at the same time every afternoon. I watched the line voltage and it dropped from 120 to 90V. The outlet used for the PC was mistakenly connected to a circuit with a big ventilation fan on a timer.

4 Likes
5

Is there a way for me to look into its logs to see if it reports something?

6

So I have no clue what is wrong, but I’m going to confidently predict that your “system freezes on Sunday nights” issue surely must be related to the “suspend fails on Sunday nights” issue. That is just too weird to be coincidence.

1 Like
7

Perhaps you can check which weekly timers are scheduled on your system to run each Sunday:

$ systemctl list-timers
8

You should check user timers as well
systemctl --user list-timers

1 Like
9

If neither of the above brings you closer to the root cause, this might be a good time to share the logs around the time the freeze happens (using journalctl with the -S and -U options).

10

systemctl list-timers lists only raid-check.timer for Sundays, although not quite at the time the freeze happens, which is at night.

These are the journalctl logs of yesterday from the time MPV is launched until the time of the forced reboot:

Sep 15 21:35:27 pos systemd[2357]: Started app-flatpak-io.mpv.Mpv-70611.scope.
Sep 15 21:35:27 pos io.mpv.Mpv.desktop[70625]: [input] Command 'mute' not found.
Sep 15 21:35:27 pos io.mpv.Mpv.desktop[70625]: [input] Command was defined at /home/akira/.var/app/io.mpv.Mpv/config/mpv/input.conf:9.
Sep 15 21:35:27 pos io.mpv.Mpv.desktop[70625]: Cannot load libcuda.so.1
Sep 15 21:35:27 pos rtkit-daemon[1042]: Successfully made thread 70667 of process 70625 (/app/bin/mpv-bin) owned by '1000' RT at priority 20.
Sep 15 21:35:36 pos systemd[1]: fprintd.service: Deactivated successfully.
Sep 15 21:35:36 pos audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fprintd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 15 21:35:36 pos audit: BPF prog-id=156 op=UNLOAD
Sep 15 21:35:45 pos systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Sep 15 21:35:45 pos audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 15 21:35:45 pos audit: BPF prog-id=159 op=UNLOAD
Sep 15 21:35:45 pos audit: BPF prog-id=158 op=UNLOAD
Sep 15 21:35:45 pos audit: BPF prog-id=157 op=UNLOAD
Sep 15 21:45:37 pos kernel: usb 3-3.4: USB disconnect, device number 6
Sep 15 22:06:14 pos gnome-keyring-daemon[2397]: asked to register item /org/freedesktop/secrets/collection/login/4, but it's already registered
Sep 15 22:13:30 pos systemd[1]: Starting dnf-makecache.service - dnf makecache...
Sep 15 22:13:30 pos dnf[71490]: Metadata cache refreshed recently.
Sep 15 22:13:30 pos systemd[1]: dnf-makecache.service: Deactivated successfully.
Sep 15 22:13:30 pos systemd[1]: Finished dnf-makecache.service - dnf makecache.
Sep 15 22:13:30 pos audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 15 22:13:30 pos audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Sep 15 22:21:12 pos cupsd[1421]: REQUEST localhost - - "POST / HTTP/1.1" 200 184 Renew-Subscription successful-ok
-- Boot 26cd97745bda446caed74e446aaba6c7 --
11

Would you be able to post the outputs from one and two weeks ago (Sundays before forced reboots), to see if there is a pattern?

12

I couldn’t find signs of a forced reboot on the last two weeks, but I believe it happened on the August 25th. A similar thing happens: MPV starts, I stay away for around 50 minutes, then I have to force reboot. See the sudden boot at around 22:15. That would have been me forcing it.

(There is a bunch of Bluetooth junk on it, but it might simply be my mouse being weird since it goes to sleep all the time, I have no idea why, but it’s probably another unrelated issue).

Aug 25 20:32:17 pos systemd[15244]: Started app-gnome-nautilus-145600.scope - Application launched by gsd-media-keys.
Aug 25 20:32:17 pos nautilus[145600]: Connecting to org.freedesktop.Tracker3.Miner.Files
Aug 25 20:32:17 pos audit: BPF prog-id=201 op=LOAD
Aug 25 20:32:17 pos audit: BPF prog-id=202 op=LOAD
Aug 25 20:32:17 pos audit: BPF prog-id=203 op=LOAD
Aug 25 20:32:17 pos systemd[1]: Starting systemd-hostnamed.service - Hostname Service...
Aug 25 20:32:17 pos systemd[1]: Started systemd-hostnamed.service - Hostname Service.
Aug 25 20:32:17 pos audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 25 20:32:43 pos systemd[15244]: Started app-flatpak-io.mpv.Mpv-145682.scope.
Aug 25 20:32:43 pos io.mpv.Mpv.desktop[145699]: [input] Command 'mute' not found.
Aug 25 20:32:43 pos io.mpv.Mpv.desktop[145699]: [input] Command was defined at /home/akira/.var/app/io.mpv.Mpv/config/mpv/input.conf:9.
Aug 25 20:32:43 pos io.mpv.Mpv.desktop[145699]: Cannot load libcuda.so.1
Aug 25 20:32:43 pos rtkit-daemon[1040]: Successfully made thread 145749 of process 145699 (/app/bin/mpv-bin) owned by '1000' RT at priority 20.
Aug 25 20:32:47 pos systemd[1]: systemd-hostnamed.service: Deactivated successfully.
Aug 25 20:32:47 pos audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-hostnamed comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 25 20:32:47 pos audit: BPF prog-id=203 op=UNLOAD
Aug 25 20:32:47 pos audit: BPF prog-id=202 op=UNLOAD
Aug 25 20:32:47 pos audit: BPF prog-id=201 op=UNLOAD
Aug 25 20:32:47 pos rtkit-daemon[1040]: Successfully made thread 145802 of process 145699 (/app/bin/mpv-bin) owned by '1000' RT at priority 20.
Aug 25 20:36:51 pos kernel: input: Logitech M720 Triathlon Multi-Device Mouse as /devices/virtual/misc/uhid/0005:046D:B015.0011/input/input37
Aug 25 20:36:51 pos kernel: logitech-hidpp-device 0005:046D:B015.0011: input,hidraw3: BLUETOOTH HID v0.13 Keyboard [Logitech M720 Triathlon Multi-Device Mouse] on 8c:88:2b:13:1e:4f
Aug 25 20:36:51 pos kernel: logitech-hidpp-device 0005:046D:B015.0011: HID++ 4.5 device connected.
Aug 25 20:36:51 pos systemd-logind[1052]: Watching system buttons on /dev/input/event16 (Logitech M720 Triathlon Multi-Device Mouse)
Aug 25 20:45:10 pos gnome-keyring-daemon[15284]: asked to register item /org/freedesktop/secrets/collection/login/4, but it's already registered
Aug 25 20:46:11 pos kernel: usb 3-3.4: USB disconnect, device number 3
Aug 25 20:52:22 pos rtkit-daemon[1040]: Successfully made thread 15510 of process 15475 (/usr/bin/gnome-shell) owned by '1000' high priority at nice level 0.
Aug 25 20:52:22 pos rtkit-daemon[1040]: Successfully made thread 15510 of process 15475 (/usr/bin/gnome-shell) owned by '1000' RT at priority 20.
Aug 25 20:57:45 pos kernel: usb 3-3.4: new full-speed USB device number 4 using xhci_hcd
Aug 25 20:57:45 pos kernel: usb 3-3.4: New USB device found, idVendor=258a, idProduct=0059, bcdDevice=10.25
Aug 25 20:57:45 pos kernel: usb 3-3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Aug 25 20:57:45 pos kernel: usb 3-3.4: Product: RK Bluetooth Keyboar
Aug 25 20:57:45 pos kernel: usb 3-3.4: Manufacturer: SINO WEALTH
Aug 25 20:57:45 pos kernel: input: SINO WEALTH RK Bluetooth Keyboar as /devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4/3-3.4:1.0/0003:258A:0059.0012/input/input38
Aug 25 20:57:45 pos kernel: hid-generic 0003:258A:0059.0012: input,hidraw4: USB HID v1.11 Keyboard [SINO WEALTH RK Bluetooth Keyboar] on usb-0000:0b:00.3-3.4/input0
Aug 25 20:57:45 pos kernel: input: SINO WEALTH RK Bluetooth Keyboar System Control as /devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4/3-3.4:1.1/0003:258A:0059.0013/input/input39
Aug 25 20:57:45 pos rtkit-daemon[1040]: Successfully made thread 15510 of process 15475 (/usr/bin/gnome-shell) owned by '1000' high priority at nice level 0.
Aug 25 20:57:45 pos kernel: input: SINO WEALTH RK Bluetooth Keyboar Consumer Control as /devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4/3-3.4:1.1/0003:258A:0059.0013/input/input40
Aug 25 20:57:45 pos kernel: input: SINO WEALTH RK Bluetooth Keyboar Keyboard as /devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4/3-3.4:1.1/0003:258A:0059.0013/input/input41
Aug 25 20:57:45 pos kernel: hid-generic 0003:258A:0059.0013: input,hiddev97,hidraw5: USB HID v1.11 Keyboard [SINO WEALTH RK Bluetooth Keyboar] on usb-0000:0b:00.3-3.4/input1
Aug 25 20:57:45 pos mtp-probe[150231]: checking bus 3, device 4: "/sys/devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4"
Aug 25 20:57:45 pos mtp-probe[150231]: bus: 3, device: 4 was not an MTP device
Aug 25 20:57:45 pos systemd-logind[1052]: Watching system buttons on /dev/input/event20 (SINO WEALTH RK Bluetooth Keyboar Consumer Control)
Aug 25 20:57:45 pos systemd-logind[1052]: Watching system buttons on /dev/input/event21 (SINO WEALTH RK Bluetooth Keyboar Keyboard)
Aug 25 20:57:45 pos rtkit-daemon[1040]: Successfully made thread 15510 of process 15475 (/usr/bin/gnome-shell) owned by '1000' RT at priority 20.
Aug 25 20:57:45 pos systemd-logind[1052]: Watching system buttons on /dev/input/event19 (SINO WEALTH RK Bluetooth Keyboar System Control)
Aug 25 20:57:45 pos systemd-logind[1052]: Watching system buttons on /dev/input/event18 (SINO WEALTH RK Bluetooth Keyboar)
Aug 25 20:57:45 pos audit: BPF prog-id=204 op=LOAD
Aug 25 20:57:45 pos systemd[1]: Starting fprintd.service - Fingerprint Authentication Daemon...
Aug 25 20:57:45 pos mtp-probe[150278]: checking bus 3, device 4: "/sys/devices/pci0000:00/0000:00:08.1/0000:0b:00.3/usb3/3-3/3-3.4"
Aug 25 20:57:45 pos mtp-probe[150278]: bus: 3, device: 4 was not an MTP device
Aug 25 20:57:45 pos systemd[1]: Started fprintd.service - Fingerprint Authentication Daemon.
Aug 25 20:57:45 pos audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fprintd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 25 20:57:53 pos gdm-password][150263]: gkr-pam: unlocked login keyring
Aug 25 20:57:53 pos audit[150263]: USER_AUTH pid=150263 uid=0 auid=1000 ses=2 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:authentication grantors=pam_unix,pam_gnome_keyring acct="akira" exe="/usr/libexec/gdm-session-worker" hostname=pos addr=?>
Aug 25 20:57:53 pos audit[150263]: USER_ACCT pid=150263 uid=0 auid=1000 ses=2 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:accounting grantors=pam_unix acct="akira" exe="/usr/libexec/gdm-session-worker" hostname=pos addr=? terminal=/dev/tty1 re>
Aug 25 20:57:53 pos gdm-password][150263]: pam_env(gdm-password:setcred): Unable to read configuration file /etc//environment line 1: Missing delimiter
Aug 25 20:57:53 pos audit[150263]: CRED_REFR pid=150263 uid=0 auid=1000 ses=2 subj=system_u:system_r:xdm_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_unix,pam_gnome_keyring acct="akira" exe="/usr/libexec/gdm-session-worker" hostname=pos addr=? termin>
Aug 25 20:57:53 pos gnome-shell[15475]: meta_dbus_session_manager_uninhibit: assertion 'priv->inhibit_count > 0' failed
Aug 25 20:57:53 pos gnome-shell[15475]: meta_dbus_session_manager_uninhibit: assertion 'priv->inhibit_count > 0' failed
Aug 25 20:57:53 pos gnome-shell[15475]: meta_dbus_session_manager_uninhibit: assertion 'priv->inhibit_count > 0' failed
Aug 25 20:57:53 pos gnome-shell[15475]: loading user theme: /usr/share/themes/Yaru-blue-dark/gnome-shell/gnome-shell.css
Aug 25 20:57:53 pos NetworkManager[1505]: <info>  [1724630273.8312] agent-manager: agent[9e6da783af5678c3,:1.89/org.gnome.Shell.NetworkAgent/1000]: agent registered
Aug 25 20:58:15 pos systemd[1]: fprintd.service: Deactivated successfully.
Aug 25 20:58:15 pos audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=fprintd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Aug 25 20:58:15 pos audit: BPF prog-id=204 op=UNLOAD
Aug 25 21:01:36 pos kernel: input: Logitech M720 Triathlon Multi-Device Mouse as /devices/virtual/misc/uhid/0005:046D:B015.0014/input/input42
Aug 25 21:01:36 pos kernel: logitech-hidpp-device 0005:046D:B015.0014: input,hidraw3: BLUETOOTH HID v0.13 Keyboard [Logitech M720 Triathlon Multi-Device Mouse] on 8c:88:2b:13:1e:4f
Aug 25 21:01:36 pos kernel: logitech-hidpp-device 0005:046D:B015.0014: HID++ 4.5 device connected.
Aug 25 21:01:36 pos systemd-logind[1052]: Watching system buttons on /dev/input/event16 (Logitech M720 Triathlon Multi-Device Mouse)
Aug 25 21:12:17 pos kernel: usb 3-3.4: USB disconnect, device number 4
Aug 25 21:30:09 pos cupsd[1587]: REQUEST localhost - - "POST / HTTP/1.1" 200 184 Renew-Subscription client-error-not-found
Aug 25 21:45:05 pos systemd[1]: Starting dnf-makecache.service - dnf makecache...
Aug 25 21:45:06 pos dnf[158514]: Fedora 40 - x86_64                              6.5 kB/s | 2.8 kB     00:00
Aug 25 21:45:07 pos dnf[158514]: Fedora 40 openh264 (From Cisco) - x86_64        1.6 kB/s | 989  B     00:00
Aug 25 21:45:08 pos dnf[158514]: Fedora 40 - x86_64 - Updates                     78 kB/s |  62 kB     00:00
Aug 25 21:45:09 pos dnf[158514]: Fedora 40 - x86_64 - Updates                    1.8 MB/s | 1.9 MB     00:01
Aug 25 21:45:09 pos dnf[158514]: Errors during downloading metadata for repository 'updates':
Aug 25 21:45:09 pos dnf[158514]:   - Status code: 416 for http://mirror.i3d.net/pub/fedora/linux/updates/40/Everything/x86_64/repodata/60906d215a90cca03bb2c0aeb8fe42a9e738a636fdefc9c6c9c849c3bc29871d-primary.xml.zck (IP: 185.50.106.213)
Aug 25 21:45:09 pos dnf[158514]:   - Status code: 416 for http://mirror.i3d.net/pub/fedora/linux/updates/40/Everything/x86_64/repodata/fc54f5db20496a4815286c863ae6c41db70d49f5e0062b4d12c24c84b281aa96-updateinfo.xml.zck (IP: 185.50.106.213)
Aug 25 21:45:09 pos dnf[158514]: Error: Failed to download metadata for repo 'updates': Yum repo downloading error: Downloading error(s): repodata/60906d215a90cca03bb2c0aeb8fe42a9e738a636fdefc9c6c9c849c3bc29871d-primary.xml.zck - Download failed: Status c>
Aug 25 21:45:09 pos systemd[1]: dnf-makecache.service: Main process exited, code=exited, status=1/FAILURE
Aug 25 21:45:09 pos systemd[1]: dnf-makecache.service: Failed with result 'exit-code'.
Aug 25 21:45:09 pos systemd[1]: Failed to start dnf-makecache.service - dnf makecache.
Aug 25 21:45:09 pos audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=dnf-makecache comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Aug 25 21:45:10 pos gnome-keyring-daemon[15284]: asked to register item /org/freedesktop/secrets/collection/login/4, but it's already registered
**-- Boot a61f2105a46c48809a53c33bf77c04a7 --**
Aug 25 22:15:59 pos kernel: Linux version 6.10.6-200.fc40.x86_64 (mockbuild@f1069ead281040288cd8d3761ad1265a) (gcc (GCC) 14.2.1 20240801 (Red Hat 14.2.1-1), GNU ld version 2.41-37.fc40) #1 SMP PREEMPT_DYNAMIC Mon Aug 19 14:09:30 UTC 2024
13

So currently it appears that MPV may be a common factor? I see that you are running a flatpak version, can it be that it incorporates one of the unstable mesa or ffmpeg versions that have been causing problems with 6.10 kernels recently?

You could try running the RPM version instead and make sure that the rest of the system libraries are up to date.

14

It is somehow related to MPV, although I’m not sure how. I use MPV all the time, but I only have the issue every other Sunday specifically. I didn’t know there was an issue with the current mesa and kernels… I just got MPV from Flathub a while ago when installing this system. My system-wide version of ffmpeg is ffmpeg-full from the nonfree repositories. The only thing that might differ Sunday from every other day is that I usually update the system every Friday or Saturday, which usually updates the kernel as well. Could be some lingering Nvidia crap on my system, maybe? I will try the rpm version of MPV, but I guess I will only know if I’ll get a freeze or not on next Sunday :sweat_smile:

15

There have been issues with mesa and ffmpeg that especially hit AMD gpu’s under kernel 6.10.X, they cause the kind of issues that you experience.

Flatpaks can incorporate their own versions of libraries, so it’s possible that there are versions in the MPV flatpak that are problematic for your system and current kernel.

It’s possible that the timing of your crashes has just been coincidence. Or rather not coincidence because you were using MPV. But the fact that it crashed on Sunday 2 or 3 times may have been coincidence.

1 Like