F44: Kernel graphics driver crash results in system hang at boot

I ways using my desktop yesterday and all of a sudden the display went dark and the came back but the system was not responsive. Capslock was working.

I force shutdown the desktop and started it again, only to have system hang on the plymouth loading screen. (see First Reboot logs)

I had a live usb around of f41. It booted fine, no issues.

From there I used journalctl and pointed to the system’s journal.

It pointed to the nvidia driver crashing. (see first nvidia crash while system running logs)

I then rebooted and tried with the nouveau driver.

kernel command line (Remove): module.blacklist=nouveau,nova_core rd.module.blacklist=nouveau,nova_core quiet. Nouveau also crashed.

I booted into the live usb again and setup a chroot.

I removed nvidia drivers following the rpmfusion howto. dnf remove xorg-x11-nvidia-drv-\* to try and fallback to nouveau. Did not work nouveau crashes. (see Nouveau crash Log)

I eventually got into a graphical enviroment by using the fallback for nouveau nomodeset option.

I had not installed anything related to graphics that day. I think I had just install python3-devel and it crashed.

I currently have an almost unusable system.

Any help would be apprecitate.

Spec:

Motherboard: PRIME B450M-K II
CPU: AMD Ryzen 3 1300X Quad-Core Processor
System Memory: 3GB DDR4
GPU: NVidia TU117 [GeForce GTX 1650]

First Nvidia crash while system running ``` May 27 14:30:40 fedora kernel: NVRM: GPU at PCI:0000:09:00: GPU-990e7b3d-3c4b-0765-a7e9-2610a1b5fda9 May 27 14:30:40 fedora kernel: NVRM: Xid (PCI:0000:09:00): 79, pid=20736, name=kate, GPU has fallen off the bus. May 27 14:30:40 fedora kernel: NVRM: GPU 0000:09:00.0: GPU has fallen off the bus. May 27 14:30:40 fedora kernel: NVRM: GPU0 krcRcAndNotifyAllChannels_IMPL: RC all channels for critical error 79. May 27 14:30:40 fedora kernel: NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 78 sequence 21158! May 27 14:30:40 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273 May 27 14:30:40 fedora kernel: NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 78 sequence 21159! .... May 27 14:30:40 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273 May 27 14:30:40 fedora kernel: NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 78 sequence 21182! May 27 14:30:40 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273 May 27 14:30:40 fedora kernel: NVRM: GPU0 RmLogGpuCrash: RmLogGpuCrash: failed to save GPU crash data May 27 14:30:40 fedora kernel: NVRM: GPU0 _kgspLogRpcSanityCheckFailure: GPU0 sanity check failed 0xf waiting for RPC response from GSP. Expected function 76 (GSP_RM_CONTROL) sequence 21157 (0x20800a9a 0x8). May 27 14:30:40 fedora kernel: NVRM: GPU0 GSP RPC buffer contains function 78 (DUMP_PROTOBUF_COMPONENT) sequence 21182 and data 0x0000000000000000 0x0000000000000000. May 27 14:30:40 fedora kernel: NVRM: GPU0 RPC history (CPU -> GSP): May 27 14:30:40 fedora kernel: NVRM: entry function sequence data0 data1 ts_start ts_end duration actively_polling May 27 14:30:40 fedora kernel: NVRM: 0 76 GSP_RM_CONTROL 21157 0x0000000020800a9a 0x0000000000000008 0x000652cd74b5480c 0x0000000000000000 y May 27 14:30:40 fedora kernel: NVRM: -1 76 GSP_RM_CONTROL 21156 0x00000000a06c010a 0x0000000000000058 0x000652cd74b54705 0x000652cd74b547da 213us May 27 14:30:40 fedora kernel: NVRM: -2 103 GSP_RM_ALLOC 21155 0x000000000000a06c 0x0000000000000014 0x000652cd74b541ea 0x000652cd74b546fb 1297us May 27 14:30:40 fedora kernel: NVRM: -3 76 GSP_RM_CONTROL 21154 0x0000000020802a08 0x0000000000000004 0x000652cd74b540d4 0x000652cd74b541b2 222us May 27 14:30:40 fedora kernel: NVRM: -4 76 GSP_RM_CONTROL 21153 0x00000000a06c010a 0x0000000000000058 0x000652cd74b53ea9 0x000652cd74b540a4 507us May 27 14:30:40 fedora kernel: NVRM: -5 103 GSP_RM_ALLOC 21152 0x000000000000a06c 0x0000000000000014 0x000652cd74b53553 0x000652cd74b53e9c 2377us May 27 14:30:40 fedora kernel: NVRM: -6 76 GSP_RM_CONTROL 21151 0x0000000020802a08 0x0000000000000004 0x000652cd74b530db 0x000652cd74b5350a 1071us May 27 14:30:40 fedora kernel: NVRM: -7 10 FREE 21150 0x00000000beef0032 0x0000000000000000 0x000652cd74b525f8 0x000652cd74b527d4 476us May 27 14:30:40 fedora kernel: NVRM: GPU0 RPC event history (CPU <- GSP): May 27 14:30:40 fedora kernel: NVRM: entry function sequence data0 data1 ts_start ts_end duration during_incomplete_rpc May 27 14:30:40 fedora kernel: NVRM: 0 4134 GSP_LOAD_EXEC_GENERIC 0 0x0000000000000000 0x0000000000000000 0x000652cb9cd17c25 0x000652cb9cd17f62 829us May 27 14:30:40 fedora kernel: CPU: 3 UID: 1000 PID: 20736 Comm: kate Tainted: G OE 7.0.4-200.fc44.x86_64 #1 PREEMPT(lazy) May 27 14:30:40 fedora kernel: Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE May 27 14:30:40 fedora kernel: Hardware name: ASUS System Product Name/PRIME B450M-K II, BIOS 4604 03/19/2024 May 27 14:30:40 fedora kernel: Call Trace: May 27 14:30:40 fedora kernel: May 27 14:30:40 fedora kernel: dump_stack_lvl+0x5d/0x80 May 27 14:30:40 fedora kernel: _kgspRpcRecvPoll+0x714/0x1220 [nvidia] May 27 14:30:40 fedora kernel: ? os_get_system_time+0x3b/0x90 [nvidia] May 27 14:30:40 fedora kernel: ? _kgspRpcSendMessage+0xd3/0x1a0 [nvidia] May 27 14:30:40 fedora kernel: _issueRpcAndWait+0xd5/0x920 [nvidia] May 27 14:30:40 fedora kernel: ? osGetCurrentThread+0x26/0x60 [nvidia] May 27 14:30:40 fedora kernel: ? rmDeviceGpuLockIsOwner+0x29/0x90 [nvidia] May 27 14:30:40 fedora kernel: rpcRmApiControl_GSP+0x76f/0x940 [nvidia] May 27 14:30:40 fedora kernel: kperfBoostSet_IMPL+0x69/0x70 [nvidia] May 27 14:30:40 fedora kernel: resControl_IMPL+0x1b1/0x1c0 [nvidia] May 27 14:30:40 fedora kernel: gpuresControl_IMPL+0x4c/0xa0 [nvidia] May 27 14:30:40 fedora kernel: serverControl+0x47e/0x590 [nvidia] May 27 14:30:40 fedora kernel: _rmapiRmControl+0x4f2/0x820 [nvidia] May 27 14:30:40 fedora kernel: rmapiControlWithSecInfo+0x79/0x140 [nvidia] May 27 14:30:40 fedora kernel: rmapiControlWithSecInfoTls+0x8f/0xf0 [nvidia] May 27 14:30:40 fedora kernel: _nv04ControlWithSecInfo+0x8d/0xa0 [nvidia] May 27 14:30:40 fedora kernel: ? rmapiAllocWithSecInfoTls+0x7a/0x90 [nvidia] May 27 14:30:40 fedora kernel: ? __rmqueue_pcplist+0x56/0x2c0 May 27 14:30:40 fedora kernel: RmIoctl+0x934/0xda0 [nvidia] May 27 14:30:40 fedora kernel: ? rmqueue_pcplist+0x96/0x180 May 27 14:30:40 fedora kernel: ? os_acquire_spinlock+0x12/0x30 [nvidia] May 27 14:30:40 fedora kernel: ? portSyncSpinlockAcquire+0x18/0x30 [nvidia] May 27 14:30:40 fedora kernel: ? rm_ioctl+0x52/0x4f0 [nvidia] May 27 14:30:40 fedora kernel: rm_ioctl+0x66/0x4f0 [nvidia] May 27 14:30:40 fedora kernel: nvidia_ioctl.isra.0+0x51d/0xa30 [nvidia] May 27 14:30:40 fedora kernel: nvidia_unlocked_ioctl+0x1d/0x30 [nvidia] May 27 14:30:40 fedora kernel: __x64_sys_ioctl+0xb9/0x100 May 27 14:30:40 fedora kernel: do_syscall_64+0x109/0x6e0 May 27 14:30:40 fedora kernel: ? lruvec_stat_mod_folio+0x85/0xd0 May 27 14:30:40 fedora kernel: ? __folio_mod_stat+0x2d/0x90 May 27 14:30:40 fedora kernel: ? do_anonymous_page+0x374/0x5f0 May 27 14:30:40 fedora kernel: ? __handle_mm_fault+0x47c/0x6c0 May 27 14:30:40 fedora kernel: ? count_memcg_events+0xeb/0x220 May 27 14:30:40 fedora kernel: ? handle_mm_fault+0x248/0x330 May 27 14:30:40 fedora kernel: ? do_user_addr_fault+0x2cd/0x830 May 27 14:30:40 fedora kernel: ? irqentry_exit+0x7b/0x560 May 27 14:30:40 fedora kernel: ? exc_page_fault+0x90/0x1f0 May 27 14:30:40 fedora kernel: ? __irq_exit_rcu+0x4c/0xf0 May 27 14:30:40 fedora kernel: entry_SYSCALL_64_after_hwframe+0x76/0x7e May 27 14:30:40 fedora kernel: RIP: 0033:0x7fd8b2ef873d May 27 14:30:40 fedora kernel: Code: 04 25 28 00 00 00 48 89 45 c8 31 c0 48 8d 45 10 c7 45 b0 10 00 00 00 48 89 45 b8 48 8d 45 d0 48 89 45 c0 b8 10 00 00 00 0f 05 <89> c2 3d 00 f0 ff ff 77 1a 48 8b 45 c8 64 48 2b 04 25 28 00 00 00 May 27 14:30:40 fedora kernel: RSP: 002b:00007ffe97655590 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 May 27 14:30:40 fedora kernel: RAX: ffffffffffffffda RBX: 0000000000000020 RCX: 00007fd8b2ef873d May 27 14:30:40 fedora kernel: RDX: 00007ffe97655710 RSI: 00000000c020462a RDI: 0000000000000017 May 27 14:30:40 fedora kernel: RBP: 00007ffe976555e0 R08: 00007ffe97655710 R09: 00007ffe9765572c May 27 14:30:40 fedora kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000017 May 27 14:30:40 fedora kernel: R13: 00000000c020462a R14: 000000000000002a R15: 00007ffe976555f0 May 27 14:30:40 fedora kernel: May 27 14:30:40 fedora kernel: NVRM: Xid (PCI:0000:09:00): 154, GPU recovery action changed from 0x0 (None) to 0x1 (GPU Reset Required) May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Current device is not valid [NV_ERR_INVALID_DEVICE] (0x00000026) returned from rmDeviceGpuLocksAcquire(pGpu, GPUS_LOCK_FLAGS_NONE, RM_LOCK_MODULES_MEM_PMA) @ virtual_mem.c:115 May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 May 27 14:30:40 fedora kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_client.c:844 May 27 14:30:40 fedora kernel: NVRM: nvAssertFailedNoLog: Assertion failed: (status == NV_OK) || (status == NV_ERR_GPU_IN_FULLCHIP_RESET) @ rs_server.c:259 May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Current device is not valid [NV_ERR_INVALID_DEVICE] (0x00000026) returned from rmDeviceGpuLocksAcquire(pGpu, GPUS_LOCK_FLAGS_NONE, RM_LOCK_MODULES_FIFO) @ kernel_channel_group_api.c:116 May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Current device is not valid [NV_ERR_INVALID_DEVICE] (0x00000026) returned from rmDeviceGpuLocksAcquire(pGpu, GPUS_LOCK_FLAGS_NONE, RM_LOCK_MODULES_MEM_PMA) @ virtual_mem.c:115 May 27 14:30:40 fedora kernel: NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: Current device is not valid [NV_ERR_INVALID_DEVICE] (0x00000026) returned from rmDeviceGpuLocksAcquire(pGpu, GPUS_LOCK_FLAGS_NONE, RM_LOCK_MODULES_MEM_PMA) @ virtual_mem.c:115 May 27 14:30:58 fedora kernel: fbcon: Taking over console May 27 14:31:01 fedora kernel: [drm:nv_drm_atomic_commit [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000900] Flip event timeout on head 0 ```
First Reboot system
May 27 16:49:51 fedora kernel: nvidia: loading out-of-tree module taints kernel.
May 27 16:49:51 fedora kernel: nvidia: module verification failed: signature and/or required key missing - tainting kernel
May 27 16:49:51 fedora kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 235
May 27 16:49:51 fedora kernel:
May 27 16:49:51 fedora kernel: nvidia 0000:09:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none
May 27 16:49:51 fedora kernel: NVRM: loading NVIDIA UNIX Open Kernel Module for x86_64  595.71.05  Release Build  (dvs-builder@U22-I3-G08-03-1)  Fri Apr 24 06:42:30 UTC 2026
May 27 16:49:51 fedora kernel: nvidia-modeset: Loading NVIDIA UNIX Open Kernel Mode Setting Driver for x86_64  595.71.05  Release Build  (dvs-builder@U22-I3-G08-03-1)  Fri Apr 24 06:28:59 UTC 2026
May 27 16:49:51 fedora kernel: [drm] [nvidia-drm] [GPU ID 0x00000900] Loading driver
May 27 16:49:52 fedora kernel: NVRM: GPU at PCI:0000:09:00: GPU-990e7b3d-3c4b-0765-a7e9-2610a1b5fda9
May 27 16:49:52 fedora kernel: NVRM: Xid (PCI:0000:09:00): 79, GPU has fallen off the bus.
May 27 16:49:52 fedora kernel: NVRM: GPU 0000:09:00.0: GPU has fallen off the bus.
May 27 16:49:52 fedora kernel: NVRM: GPU0 krcRcAndNotifyAllChannels_IMPL: RC all channels for critical error 79.
May 27 16:49:52 fedora kernel: NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 78 sequence 0!
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273
....
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273
May 27 16:49:52 fedora kernel: NVRM: GPU0 _issueRpcAndWait: rpcSendMessage failed with status 0x0000000f for fn 78 sequence 22!
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from nvdEngineDumpCallbackHelper(pGpu, pPrbEnc, pNvDumpState, pEngineCallback) @ nv_debug_dump.c:273
May 27 16:49:52 fedora kernel: NVRM: GPU0 RmLogGpuCrash: RmLogGpuCrash: failed to save GPU crash data
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: expectedFunc == pHistoryEntry->function @ kernel_gsp.c:2447
May 27 16:49:52 fedora kernel: NVRM: GPU0 _kgspLogRpcSanityCheckFailure: GPU0 sanity check failed 0xf waiting for RPC response from GSP. Expected function 4097 (GSP_INIT_DONE) sequence 0 (0x0 0x0).
May 27 16:49:52 fedora kernel: NVRM: GPU0 GSP RPC buffer contains function 78 (DUMP_PROTOBUF_COMPONENT) sequence 22 and data 0x0000000000000000 0x0000000000000000.
May 27 16:49:52 fedora kernel: NVRM: GPU0 RPC history (CPU -> GSP):
May 27 16:49:52 fedora kernel: NVRM:     entry function                     sequence data0              data1              ts_start           ts_end             duration actively_polling
May 27 16:49:52 fedora kernel: NVRM:      0    73   SET_REGISTRY                   0 0x0000000000000000 0x0000000000000000 0x000652cdb95e2323 0x0000000000000000          y
May 27 16:49:52 fedora kernel: NVRM:     -1    72   GSP_SET_SYSTEM_INFO            0 0x0000000000000000 0x0000000000000000 0x000652cdb95e231f 0x0000000000000000
May 27 16:49:52 fedora kernel: NVRM: GPU0 RPC event history (CPU <- GSP):
May 27 16:49:52 fedora kernel: NVRM:     entry function                     sequence data0              data1              ts_start           ts_end             duration during_incomplete_rpc
May 27 16:49:52 fedora kernel: NVRM:      0    4134 GSP_LOAD_EXEC_GENERIC          0 0x0000000000000000 0x0000000000000000 0x000652cdb9666df9 0x000652cdb9667136    829us y
May 27 16:49:52 fedora kernel: CPU: 0 UID: 0 PID: 831 Comm: (udev-worker) Tainted: G           OE       7.0.4-200.fc44.x86_64 #1 PREEMPT(lazy)
May 27 16:49:52 fedora kernel: Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
May 27 16:49:52 fedora kernel: Hardware name: ASUS System Product Name/PRIME B450M-K II, BIOS 4604 03/19/2024
May 27 16:49:52 fedora kernel: Call Trace:
May 27 16:49:52 fedora kernel:  <TASK>
May 27 16:49:52 fedora kernel:  dump_stack_lvl+0x5d/0x80
May 27 16:49:52 fedora kernel:  _kgspRpcRecvPoll+0x714/0x1220 [nvidia]
May 27 16:49:52 fedora kernel:  kgspWaitForRmInitDone_IMPL+0x40/0x1a0 [nvidia]
May 27 16:49:52 fedora kernel:  kgspBootstrap_TU102+0x231/0x530 [nvidia]
May 27 16:49:52 fedora kernel:  kgspInitRm_IMPL+0x14db/0x24d0 [nvidia]
May 27 16:49:52 fedora kernel:  RmInitAdapter+0xf16/0x2100 [nvidia]
May 27 16:49:52 fedora kernel:  ? up+0x50/0xa0
May 27 16:49:52 fedora kernel:  ? down+0x1e/0x60
May 27 16:49:52 fedora kernel:  rm_init_adapter+0x135/0x140 [nvidia]
May 27 16:49:52 fedora kernel:  nv_start_device+0x273/0x840 [nvidia]
May 27 16:49:52 fedora kernel:  nv_open_device+0x96/0x260 [nvidia]
May 27 16:49:52 fedora kernel:  nvidia_dev_get+0x87/0xf0 [nvidia]
May 27 16:49:52 fedora kernel:  nvkms_open_gpu+0x59/0xa0 [nvidia_modeset]
May 27 16:49:52 fedora kernel:  AllocateDevice+0x45/0x410 [nvidia_modeset]
May 27 16:49:52 fedora kernel:  nv_drm_dev_load+0xb2/0x46c [nvidia_drm]
May 27 16:49:52 fedora kernel:  ? _tlsEntryRelease+0xe3/0x150 [nvidia]
May 27 16:49:52 fedora kernel:  ? rmapiFreeWithSecInfo+0x236/0x2b0 [nvidia]
May 27 16:49:52 fedora kernel:  ? __pfx_nv_drm_event_callback+0x10/0x10 [nvidia_drm]
May 27 16:49:52 fedora kernel:  nv_drm_register_drm_device.cold+0x1a/0x157 [nvidia_drm]
May 27 16:49:52 fedora kernel:  ? __pfx_nv_drm_register_drm_device+0x10/0x10 [nvidia_drm]
May 27 16:49:52 fedora kernel:  EnumerateGpus+0x180/0x190 [nvidia_modeset]
May 27 16:49:52 fedora kernel:  ? __pfx_nv_linux_drm_init+0x10/0x10 [nvidia_drm]
May 27 16:49:52 fedora kernel:  nv_drm_probe_devices+0x5c/0x70 [nvidia_drm]
May 27 16:49:52 fedora kernel:  do_one_initcall+0x8e/0x400
May 27 16:49:52 fedora kernel:  do_init_module+0x8c/0x290
May 27 16:49:52 fedora kernel:  init_module_from_file+0xc8/0x180
May 27 16:49:52 fedora kernel:  idempotent_init_module+0x23e/0x360
May 27 16:49:52 fedora kernel:  __x64_sys_finit_module+0x7f/0x110
May 27 16:49:52 fedora kernel:  do_syscall_64+0x109/0x6e0
May 27 16:49:52 fedora kernel:  ? filemap_read+0x42e/0x490
May 27 16:49:52 fedora kernel:  ? vfs_read+0x1d5/0x490
May 27 16:49:52 fedora kernel:  ? vfs_read+0x1d5/0x490
May 27 16:49:52 fedora kernel:  ? switch_fpu_return+0x5e/0x110
May 27 16:49:52 fedora kernel:  ? arch_exit_to_user_mode_prepare.isra.0+0x9d/0xc0
May 27 16:49:52 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 27 16:49:52 fedora kernel:  ? switch_fpu_return+0x5e/0x110
May 27 16:49:52 fedora kernel:  ? arch_exit_to_user_mode_prepare.isra.0+0x9d/0xc0
May 27 16:49:52 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 27 16:49:52 fedora kernel:  ? irqentry_exit+0x7b/0x560
May 27 16:49:52 fedora kernel:  ? exc_page_fault+0x90/0x1f0
May 27 16:49:52 fedora kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
May 27 16:49:52 fedora kernel: RIP: 0033:0x7fb760cfa7fd
May 27 16:49:52 fedora kernel: Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cb b5 0f 00 f7 d8 64 89 01 48
May 27 16:49:52 fedora kernel: RSP: 002b:00007ffd4a1a68a8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
May 27 16:49:52 fedora kernel: RAX: ffffffffffffffda RBX: 000055a96b28cd60 RCX: 00007fb760cfa7fd
May 27 16:49:52 fedora kernel: RDX: 0000000000000004 RSI: 00007fb76049e5c8 RDI: 0000000000000031
May 27 16:49:52 fedora kernel: RBP: 00007ffd4a1a6940 R08: 0000000000000000 R09: 0000000000000000
May 27 16:49:52 fedora kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 00007fb76049e5c8
May 27 16:49:52 fedora kernel: R13: 000055a96b285b60 R14: 0000000000020000 R15: 0000000000000000
May 27 16:49:52 fedora kernel:  </TASK>
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvCheckOkFailedNoLog: Check failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from rpcRecvPoll(pGpu, pRpc, NV_VGPU_MSG_EVENT_GSP_INIT_DONE, 0) @ kernel_gsp.c:6104
May 27 16:49:52 fedora kernel: NVRM: GPU0 nvAssertOkFailedNoLog: Assertion failed: GPU lost from the bus [NV_ERR_GPU_IS_LOST] (0x0000000F) returned from kgspWaitForRmInitDone(pGpu, pKernelGsp) @ kernel_gsp_tu102.c:569
May 27 16:49:52 fedora kernel: NVRM: GPU0 _kgspBootGspRm: unexpected WPR2 already up, cannot proceed with booting GSP
May 27 16:49:52 fedora kernel: NVRM: GPU0 _kgspBootGspRm: (the GPU is likely in a bad state and may need to be reset)
May 27 16:49:52 fedora kernel: NVRM: GPU0 RmInitAdapter: Cannot initialize GSP firmware RM
May 27 16:49:52 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x62:0x40:2168)
May 27 16:49:52 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:49:52 fedora kernel: [drm:nv_drm_dev_load [nvidia_drm]] *ERROR* [nvidia-drm] [GPU ID 0x00000900] Failed to allocate NvKmsKapiDevice
May 27 16:49:52 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:49:52 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:49:52 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:49:52 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
May 27 16:50:10 fedora kernel: NVRM: osInitNvMapping: *** Cannot attach gpu
May 27 16:50:10 fedora kernel: NVRM: RmInitAdapter: osInitNvMapping failed, bailing out of RmInitAdapter
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: RmInitAdapter failed! (0x22:0x56:894)
May 27 16:50:10 fedora kernel: NVRM: GPU 0000:09:00.0: rm_init_adapter failed, device minor number 0
Nouveau Crash
May 28 11:59:54 fedora kernel: nouveau 0000:09:00.0: NVIDIA TU117 (167000a1)
May 28 11:59:54 fedora kernel: nouveau 0000:09:00.0: gsp: RM version: 570.144
May 28 11:59:54 fedora kernel: nouveau 0000:09:00.0: bios: version 90.17.3d.00.73
May 28 11:59:54 fedora kernel: Console: switching to colour dummy device 80x25
May 28 11:59:54 fedora kernel: nouveau 0000:09:00.0: vgaarb: deactivate vga console
May 28 12:00:14 fedora kernel: ------------[ cut here ]------------
May 28 12:00:14 fedora kernel: WARNING: drivers/gpu/drm/nouveau/nvkm/subdev/gsp/rm/r535/rpc.c:164 at r535_gsp_msgq_wait+0xb4/0xd0 [nouveau], CPU#0: (udev-worker)/429
May 28 12:00:14 fedora kernel: Modules linked in: nouveau(+) drm_ttm_helper ttm video gpu_sched i2c_algo_bit drm_gpuvm nvme drm_exec mxm_wmi ghash_clmulni_intel drm_display_helper nvme_core sp5100_tco cec nvme_keyring nvme_auth hkdf wmi fuse i2c_dev
May 28 12:00:14 fedora kernel: CPU: 0 UID: 0 PID: 429 Comm: (udev-worker) Not tainted 7.0.10-200.fc44.x86_64 #1 PREEMPT(lazy)
May 28 12:00:14 fedora kernel: Hardware name: ASUS System Product Name/PRIME B450M-K II, BIOS 4604 03/19/2024
May 28 12:00:14 fedora kernel: RIP: 0010:r535_gsp_msgq_wait+0xb4/0xd0 [nouveau]
May 28 12:00:14 fedora kernel: Code: 8b 64 24 10 4c 8b 6c 24 18 48 8b 1c 24 48 83 c4 20 e9 60 7a 30 e5 0f 0b b8 ea ff ff ff 48 8b 1c 24 48 83 c4 20 e9 4c 7a 30 e5 <0f> 0b b8 92 ff ff ff 48 8b 6c 24 08 4c 8b 64 24 10 4c 8b 6c 24 18
May 28 12:00:14 fedora kernel: RSP: 0018:ffffd143c083f668 EFLAGS: 00010246
May 28 12:00:14 fedora kernel: RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000000002
May 28 12:00:14 fedora kernel: RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffd143c083f5e0
May 28 12:00:14 fedora kernel: RBP: 0000000000000003 R08: ffff8e4ad3025500 R09: 0000000000000000
May 28 12:00:14 fedora kernel: R10: 0000000000000001 R11: ffff8e4ad3025500 R12: ffff8e4ac1009000
May 28 12:00:14 fedora kernel: R13: ffffd143c083f6ac R14: ffff8e4ac1009000 R15: 0000000000000000
May 28 12:00:14 fedora kernel: FS:  00007f5a2e77ef00(0000) GS:ffff8e522709a000(0000) knlGS:0000000000000000
May 28 12:00:14 fedora kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 28 12:00:14 fedora kernel: CR2: 0000562c6fd85000 CR3: 000000010fc1b000 CR4: 00000000003506f0
May 28 12:00:14 fedora kernel: Call Trace:
May 28 12:00:14 fedora kernel:  <TASK>
May 28 12:00:14 fedora kernel:  r535_gsp_msg_recv+0x45/0x2d0 [nouveau]
May 28 12:00:14 fedora kernel:  ? nvkm_falcon_put+0x2a/0x70 [nouveau]
May 28 12:00:14 fedora kernel:  r535_gsp_rpc_poll+0x33/0x60 [nouveau]
May 28 12:00:14 fedora kernel:  r535_gsp_init+0x5c/0x250 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_subdev_init_+0x7e/0xe0 [nouveau]
May 28 12:00:14 fedora kernel:  ? nvkm_intr_rearm+0xd1/0x200 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_subdev_init+0x58/0x60 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_device_init+0x18f/0x230 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_udevice_init+0x57/0x70 [nouveau]
May 28 12:00:14 fedora kernel:  ? __pfx_nvkm_udevice_init+0x10/0x10 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_object_init+0x4d/0x140 [nouveau]
May 28 12:00:14 fedora kernel:  ? nvkm_udevice_new+0x8c/0xd0 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_ioctl_new+0x17a/0x250 [nouveau]
May 28 12:00:14 fedora kernel:  ? __pfx_nvkm_client_child_new+0x10/0x10 [nouveau]
May 28 12:00:14 fedora kernel:  ? __pfx_nvkm_udevice_new+0x10/0x10 [nouveau]
May 28 12:00:14 fedora kernel:  nvkm_ioctl+0xcc/0x1d0 [nouveau]
May 28 12:00:14 fedora kernel:  nvif_object_ctor+0x128/0x1e0 [nouveau]
May 28 12:00:14 fedora kernel:  nvif_device_ctor+0x2e/0x80 [nouveau]
May 28 12:00:14 fedora kernel:  nouveau_drm_device_new+0x1a0/0x370 [nouveau]
May 28 12:00:14 fedora kernel:  ? __pci_set_master+0x35/0xe0
May 28 12:00:14 fedora kernel:  nouveau_drm_probe+0xce/0x200 [nouveau]
May 28 12:00:14 fedora kernel:  local_pci_probe+0x4a/0x90
May 28 12:00:14 fedora kernel:  ? cpu_hotplug_disable+0x15/0x30
May 28 12:00:14 fedora kernel:  pci_call_probe+0x5a/0x1a0
May 28 12:00:14 fedora kernel:  ? pci_assign_irq+0x30/0x1a0
May 28 12:00:14 fedora kernel:  ? pci_match_device+0x152/0x1a0
May 28 12:00:14 fedora kernel:  pci_device_probe+0xa8/0x1b0
May 28 12:00:14 fedora kernel:  ? __pfx_pci_device_probe+0x10/0x10
May 28 12:00:14 fedora kernel:  call_driver_probe+0x23/0x100
May 28 12:00:14 fedora kernel:  ? driver_sysfs_add+0x55/0xc0
May 28 12:00:14 fedora kernel:  really_probe+0xd7/0x2d0
May 28 12:00:14 fedora kernel:  ? __pm_runtime_resume+0x5f/0x90
May 28 12:00:14 fedora kernel:  __driver_probe_device+0x8f/0x190
May 28 12:00:14 fedora kernel:  driver_probe_device+0x1f/0xa0
May 28 12:00:14 fedora kernel:  ? __pfx___driver_attach+0x10/0x10
May 28 12:00:14 fedora kernel:  __driver_attach+0xca/0x1f0
May 28 12:00:14 fedora kernel:  ? __pfx___driver_attach+0x10/0x10
May 28 12:00:14 fedora kernel:  bus_for_each_dev+0x97/0x100
May 28 12:00:14 fedora kernel:  bus_add_driver+0x15a/0x260
May 28 12:00:14 fedora kernel:  ? __pfx_nouveau_drm_init+0x10/0x10 [nouveau]
May 28 12:00:14 fedora kernel:  driver_register+0x75/0xe0
May 28 12:00:14 fedora kernel:  nouveau_drm_init+0x8a/0xff0 [nouveau]
May 28 12:00:14 fedora kernel:  do_one_initcall+0x8e/0x400
May 28 12:00:14 fedora kernel:  do_init_module+0x8c/0x290
May 28 12:00:14 fedora kernel:  init_module_from_file+0xc8/0x180
May 28 12:00:14 fedora kernel:  idempotent_init_module+0x23e/0x360
May 28 12:00:14 fedora kernel:  __x64_sys_finit_module+0x7f/0x110
May 28 12:00:14 fedora kernel:  do_syscall_64+0x109/0x6e0
May 28 12:00:14 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 28 12:00:14 fedora kernel:  ? switch_fpu_return+0x5e/0x110
May 28 12:00:14 fedora kernel:  ? __x64_sys_finit_module+0x7f/0x110
May 28 12:00:14 fedora kernel:  ? arch_exit_to_user_mode_prepare.isra.0+0x9d/0xc0
May 28 12:00:14 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 28 12:00:14 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 28 12:00:14 fedora kernel:  ? arch_exit_to_user_mode_prepare.isra.0+0x9d/0xc0
May 28 12:00:14 fedora kernel:  ? do_syscall_64+0x144/0x6e0
May 28 12:00:14 fedora kernel:  ? exc_page_fault+0x90/0x1f0
May 28 12:00:14 fedora kernel:  ? __irq_exit_rcu+0x4c/0xf0
May 28 12:00:14 fedora kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
May 28 12:00:14 fedora kernel: RIP: 0033:0x7f5a2ef556bd
May 28 12:00:14 fedora kernel: Code: ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 0b b7 0f 00 f7 d8 64 89 01 48
May 28 12:00:14 fedora kernel: RSP: 002b:00007fff5de10ee8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139
May 28 12:00:14 fedora kernel: RAX: ffffffffffffffda RBX: 0000557912c5c060 RCX: 00007f5a2ef556bd
May 28 12:00:14 fedora kernel: RDX: 0000000000000004 RSI: 00007f5a2e6455c8 RDI: 000000000000002f
May 28 12:00:14 fedora kernel: RBP: 00007fff5de10f80 R08: 0000000000000000 R09: 0000000000000000
May 28 12:00:14 fedora kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 00007f5a2e6455c8
May 28 12:00:14 fedora kernel: R13: 0000557912c5b120 R14: 0000000000020000 R15: 0000000000000000
May 28 12:00:14 fedora kernel:  </TASK>
May 28 12:00:14 fedora kernel: ---[ end trace 0000000000000000 ]---
May 28 12:00:14 fedora kernel: nouveau 0000:09:00.0: gsp: init failed, -110
May 28 12:00:14 fedora kernel: nouveau 0000:09:00.0: init failed with -110
May 28 12:00:14 fedora kernel: nouveau: drm:00000000:00000080: init failed with -110
May 28 12:00:14 fedora kernel: nouveau 0000:09:00.0: drm: Device allocation failed: -110
May 28 12:00:14 fedora kernel: nouveau 0000:09:00.0: probe with driver nouveau failed with error -110

This looks like a hardware problem. See https://docs.nvidia.com/deploy/xid-errors/index.html#xid-error-listing.

Just tried downgrading nvidia-gpu-firmware from nvidia-gpu-firmware-0:20260519-1.fc44.noarch to nvidia-gpu-firmware-0:20260309-1.fc44.noarch

I rebooted and can now get to the desktop. Still running the nouveau drivers.

I am now going to try bisecting where the it start break.

Also where can I report this, to fedora bugtracker, kernel or some other place.

Same issue also reported on nvidia-open-gpu-kernel-modules

From https://github.com/NVIDIA/open-gpu-kernel-modules/issues/1151 it looks like Nvidia is working on the problem. There isn’t much anyone else can do except revert to the working firmware version.

The firmware I downgraded to worked for a little bit but the screen went dark with graphical artefacts. I am not sure what happened as even the journal seemed to be corrupted after a force reboot.

Maybe it is the nouveau as I have only ever used the nvidia drivers as that is what I selected when I first installed fedora.

When the graphics failed which driver were you using? Nouveau also depends on Nvidia firmware, but only a subset, so it could be a clue to firmware bugs. Do you have a way to connect from another system with ssh (I often use termius on an Ipad)? If so, run journalctl —follow to try to capture details of the graphics crash.

the firmware for the tu117 are identical. And they are only used by the nouveau driver. This looks more like a HW issue.