AMD GPU not being used (Radeon RX 6700 XT - External / eGPU - Wayland)

fancyhatenjoyer · March 10, 2023, 8:45pm

Upon logging in Gnome with my eGPU connected, I experience a lag spike when the desktop is first being displayed, suggesting something related to graphics could be happening.

$ rocminfo

ROCk module is loaded
hsa api call failure at: /long_pathname_so_that_rpms_can_package_the_debug_info/src/rocminfo/rocminfo.cc:1148
Call returned HSA_STATUS_ERROR_OUT_OF_RESOURCES: The runtime failed to allocate the necessary resources. This error may also occur when the core runtime library needs to spawn threads or create internal OS-specific events.

$ clinfo

Number of platforms                               0

My installed packages hasn’t changed, but updates have taken place.

View package list

sudo dnf list installed '*amd-gpu*' '*amdgpu*' '*rocm*' '*roct*' '*hsa*' '*mesa*' '*vulkan*' 'kernel' | grep -v 'procmail|setproctitle'

Installed Packages
amd-gpu-firmware.noarch              20230210-147.fc37                @updates  
hsa-rocr.x86_64                      1.7.0.50400-72.el9               @rocm-copy
hsa-rocr-devel.x86_64                1.7.0.50400-72.el9               @rocm-copy
hsakmt.x86_64                        1.0.6-27.rocm5.4.1.fc37          @updates  
hsakmt-roct-devel.x86_64             20221020.0.2.50400-72.el9        @rocm-copy
kernel.x86_64                        6.1.11-200.fc37                  @updates  
kernel.x86_64                        6.1.13-200.fc37                  @updates  
kernel.x86_64                        6.1.14-200.fc37                  @updates  
mesa-dri-drivers.i686                22.3.6-1.fc37                    @updates  
mesa-dri-drivers.x86_64              22.3.6-1.fc37                    @updates  
mesa-filesystem.i686                 22.3.6-1.fc37                    @updates  
mesa-filesystem.x86_64               22.3.6-1.fc37                    @updates  
mesa-libEGL.i686                     22.3.6-1.fc37                    @updates  
mesa-libEGL.x86_64                   22.3.6-1.fc37                    @updates  
mesa-libGL.i686                      22.3.6-1.fc37                    @updates  
mesa-libGL.x86_64                    22.3.6-1.fc37                    @updates  
mesa-libGLU.x86_64                   9.0.1-7.fc37                     @fedora   
mesa-libOSMesa.i686                  22.3.6-1.fc37                    @updates  
mesa-libOSMesa.x86_64                22.3.6-1.fc37                    @updates  
mesa-libgbm.i686                     22.3.6-1.fc37                    @updates  
mesa-libgbm.x86_64                   22.3.6-1.fc37                    @updates  
mesa-libglapi.i686                   22.3.6-1.fc37                    @updates  
mesa-libglapi.x86_64                 22.3.6-1.fc37                    @updates  
mesa-libxatracker.x86_64             22.3.6-1.fc37                    @updates  
mesa-va-drivers.i686                 22.3.6-1.fc37                    @updates  
mesa-vulkan-drivers.i686             22.3.6-1.fc37                    @updates  
mesa-vulkan-drivers.x86_64           22.3.6-1.fc37                    @updates  
rocm-cmake.x86_64                    0.8.0.50400-72.el9               @rocm-copy
rocm-comgr.x86_64                    5.4.1-2.fc37                     @updates  
rocm-core.x86_64                     5.4.0.50400-72.el9               @rocm-copy
rocm-device-libs.x86_64              1.0.0.50400-72.el9               @rocm-copy
rocm-hip-libraries.x86_64            5.4.0.50400-72.el9               @rocm-copy
rocm-hip-runtime.x86_64              5.4.0.50400-72.el9               @rocm-copy
rocm-hip-runtime-devel.x86_64        5.4.0.50400-72.el9               @rocm-copy
rocm-hip-sdk.x86_64                  5.4.0.50400-72.el9               @rocm-copy
rocm-language-runtime.x86_64         5.4.0.50400-72.el9               @rocm-copy
rocm-llvm.x86_64                     15.0.0.22465.50400-72.el9        @rocm-copy
rocm-ocl-icd.x86_64                  2.0.0.50400-72.el9               @rocm-copy
rocm-opencl.x86_64                   2.0.0.50400-72.el9               @rocm-copy
rocm-opencl-runtime.x86_64           5.4.0.50400-72.el9               @rocm-copy
rocm-runtime.x86_64                  5.4.1-1.fc37                     @updates  
rocm-smi.noarch                      4.0.0-6.fc37                     @fedora   
rocm-smi-lib.x86_64                  5.0.0.50400-72.el9               @rocm-copy
rocminfo.x86_64                      1.0.0.50400-72.el9               @rocm-copy
rocthrust-devel.x86_64               2.10.9.50400-72.el9              @rocm-copy
vulkan-loader.i686                   1.3.216.0-3.fc37                 @fedora   
vulkan-loader.x86_64                 1.3.216.0-3.fc37                 @fedora   
vulkan-tools.x86_64                  1.3.216.0-2.fc37                 @fedora   
xorg-x11-drv-amdgpu.x86_64           23.0.0-1.fc37                    @updates

The rocminfo HSA_STATUS_ERROR_OUT_OF_RESOURCES error may be of interest, but I couldn’t find much about it.

fancyhatenjoyer · March 16, 2023, 3:30pm

A solution!

The reproducible solution for making my eGPU work is to plug an external monitor into the GPU.

Setup

Plug in eGPU and boot system.
Install hertg/egpu-switcher.
Execute sudo egpu-switcher enable and select the eGPU. The egpu-switcher daemon will now start on boot and know which device is the eGPU.
Execute sudo egpu-switcher switch --override.

How to use eGPU

Connect eGPU to laptop.
Connect GPU to a monitor and turn the monitor on, using the GPU input as its source.
Boot laptop

rocminfo, clinfo, radontop and lspci all show what I want to see and applications are using the GPU.

The external monitor can be turned off or unplugged at this stage, but if the laptop goes into suspend, when it wakes up the eGPU will no longer be working and the above commands will fail or show errors.

I plan to buy a HDMI emulator plug so I can do away with the monitor.

The only application which isn’t working with my eGPU at the moment is DaVinci Resolve. It detects the GPU and claims to use it via OpenCL, but the timeline won’t display/play any audio or video (even DNxHD). I’ll likely create a new thread dedicated to this.

Thank you all for your help over the months.

pdestefa · March 17, 2023, 6:26am

Wow, congratulations! Good sleuthing! You’re the expert here, now.

The egpu-switcher project talks about X.Org, but I’m not sure what that means. I mean, X11 and XWayland are both XOrg/FreeDesktop projects, so Xorg is either ambiguous or there is really configuration that is common to both setups, but AFAIK, that’s not the case. And, I’ve never seen anyone refer to both using the term XOrg. But, maybe; I’m not sure. Are you using Wayland or X11? I just wonder if this tool only works for X11, cause that’s how I read the project’s description.

fancyhatenjoyer · March 17, 2023, 9:29pm

Haha thank you.

You’re right, in my excitement I didn’t even realise. I’ve disabled and removed egpu-switcher from my system and I’m still able to use my eGPU when a monitor is connected to my GPU.

The new, more accurate instructions are as follows:

Setup
Non required in my environment (Gnome Wayland) other than the correct AMD/ROCm packages. If you have already followed the above instructions, run sudo egpu-switcher disable to undo the setup.

How to use the eGPU

Connect GPU to an external monitor via a display cable (HDMI, DisplayPort etc). Ensure the monitor is powered on and is communicating with the GPU by selecting the GPU display cable as its input source.
Connect eGPU to laptop via thunderbolt.
If already in a Gnome session, log out and log back in. If the eGPU disconnects, you will need to restart the laptop for it to work again.

pdestefa · March 18, 2023, 12:09am

Great news! Okay, thanks for explaining. I think I understand the situation better now.

Maybe update the post you marked as the solution so future readers go to the final word right away.

Cheers! Best of luck with your crunching!

computersavvy · March 18, 2023, 1:58am

In times past X was commonly referred to as X11. It has become more commonly referred to as xorg and on the desktop switcher on the fedora login it is shown as “Gnome on xorg”. Thus referring to it as X11 is somewhat archaic though still valid. Configs are still shown under /etc/X11/ as they have been for a long time.