Intel iGPU in Containers

Ask Fedora
1

Is passing through an Intel Arc GPU or iGPU to docker or podman containers expected to be supported in CoreOS? I was running some tests on a 12500T which has an iGPU that I wanted to use for a Jellyfin container.

For simplicity, I have temporarily disabled SELinux (setenforce 0). The iGPU shows up in the host:

lspci  
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC \[Natoma\] (rev 02)  
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA \[Natoma/Triton II\]  
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE \[Natoma/Triton II\]  
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB \[Natoma/Triton II\] (rev 01)  
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)  
00:02.0 VGA compatible controller: Device 1234:1111 (rev 02)  
00:03.0 Unassigned class \[ff80\]: XenSource, Inc. Xen Platform Device (rev 01)  
00:08.0 VGA compatible controller: Intel Corporation Alder Lake-S GT1 \[UHD Graphics 770\] (rev 0c)

I have layered the igt-gpu-tools package, however intel_gpu_top gives the following error:

intel_gpu_top
Failed to detect engines! (No such file or directory)
(Kernel 4.16 or newer is required for i915 PMU support.)

I suspect this suggest that the iGPU wouldn’t even work on the CoreOS host. And indeed, when I pass it through to Jellyfin, I do see the device inside the container, but once transcoding is enabled, the video just hangs.

CoreOS runs as an XCP-NG VM, but I’m able to use the iGPU in a Debian VM with the same passthrough, so I don’t think the hypervisor is the issue.

For reference, this is the docker compose file (I have also tested podman which has the same issue):

services:
  jellyfin:
    image: jellyfin/jellyfin
    container_name: jellyfin
    ports:
      - 8096:8096/tcp
      - 7359:7359/udp
    volumes:
      - /home/core/docker/jelly/config:/config:z
      - /home/core/docker/jelly/cache:/cache:z
      - /mnt/downloads:/media:z
    devices:
      # VAAPI Devices
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/dri/card0:/dev/dri/card0
      - /dev/dri/card1:/dev/dri/card1
    restart: 'unless-stopped'
2

It should work. Maybe we are missing some firmwares? Did you run that intel_gpu_top command as root? As you have a working VM and a non working one, can you compare the list of modules that are loaded?

3

Yes, I was running intel_gpu_top as root.

Thanks for the tip, there are indeed some missing modules.

Modules in the Debian VM:

xe                   3055616  0
drm_gpuvm              45056  1 xe
drm_exec               12288  2 drm_gpuvm,xe
gpu_sched              65536  1 xe
drm_suballoc_helper    12288  1 xe
nls_utf8               12288  2
cifs                 1507328  0
cifs_arc4              12288  1 cifs
nls_ucs2_utils          8192  1 cifs
cifs_md4               12288  1 cifs
dns_resolver           12288  1 cifs
netfs                 573440  1 cifs
xt_conntrack           12288  2
xt_MASQUERADE          16384  2
bridge                389120  0
xfrm_user              69632  1
xfrm_algo              16384  1 xfrm_user
xt_set                 20480  0
ip_set                 69632  1 xt_set
nft_chain_nat          12288  5
nf_nat                 65536  2 nft_chain_nat,xt_MASQUERADE
nf_conntrack          204800  3 xt_conntrack,nf_nat,xt_MASQUERADE
nf_defrag_ipv6         24576  1 nf_conntrack
nf_defrag_ipv4         12288  1 nf_conntrack
xt_addrtype            12288  4
nft_compat             20480  8
nf_tables             380928  80 nft_compat,nft_chain_nat
libcrc32c              12288  3 nf_conntrack,nf_nat,nf_tables
cfg80211             1392640  0
overlay               217088  0
rfkill                 40960  2 cfg80211
8021q                  53248  0
garp                   16384  1 8021q
stp                    12288  2 bridge,garp
mrp                    20480  1 8021q
llc                    16384  3 bridge,stp,garp
binfmt_misc            28672  1
nls_ascii              12288  1
nls_cp437              16384  1
vfat                   24576  1
fat                   102400  1 vfat
intel_rapl_msr         20480  0
intel_rapl_common      53248  1 intel_rapl_msr
intel_uncore_frequency_common    16384  0
intel_pmc_core        122880  0
intel_vsec             20480  1 intel_pmc_core
pmt_telemetry          16384  1 intel_pmc_core
pmt_class              16384  1 pmt_telemetry
crct10dif_pclmul       12288  1
ghash_clmulni_intel    16384  0
sha512_ssse3           53248  0
sha256_ssse3           32768  0
sha1_ssse3             32768  0
aesni_intel           122880  0
gf128mul               16384  1 aesni_intel
crypto_simd            16384  1 aesni_intel
cryptd                 28672  2 crypto_simd,ghash_clmulni_intel
i915                 4382720  0
ppdev                  24576  0
drm_buddy              24576  2 xe,i915
drm_display_helper    274432  2 xe,i915
cec                    69632  3 drm_display_helper,xe,i915
rc_core                73728  1 cec
i2c_algo_bit           16384  2 xe,i915
parport_pc             40960  0
video                  81920  2 xe,i915
parport                81920  2 parport_pc,ppdev
wmi                    28672  1 video
pcspkr                 12288  0
sg                     45056  0
joydev                 24576  0
evdev                  28672  3
button                 24576  0
configfs               69632  1
efi_pstore             12288  0
nfnetlink              20480  5 nft_compat,nf_tables,ip_set
vsock_loopback         12288  0
vmw_vsock_virtio_transport_common    61440  1 vsock_loopback
vmw_vsock_vmci_transport    45056  0
vsock                  61440  5 vmw_vsock_virtio_transport_common,vsock_loopback,vmw_vsock_vmci_transport
vmw_vmci              110592  1 vmw_vsock_vmci_transport
efivarfs               28672  1
ip_tables              28672  0
x_tables               53248  6 xt_conntrack,nft_compat,xt_addrtype,xt_set,ip_tables,xt_MASQUERADE
autofs4                57344  2
hid_generic            12288  0
usbhid                 77824  0
hid                   262144  2 usbhid,hid_generic
ext4                 1142784  1
crc16                  12288  1 ext4
mbcache                16384  1 ext4
jbd2                  200704  1 ext4
crc32c_generic         12288  0
sr_mod                 28672  0
cdrom                  81920  1 sr_mod
bochs                  16384  0
ata_generic            12288  0
drm_vram_helper        28672  1 bochs
ata_piix               45056  0
drm_ttm_helper         16384  4 bochs,drm_vram_helper,xe
libata                462848  2 ata_piix,ata_generic
ttm                   106496  4 drm_vram_helper,drm_ttm_helper,xe,i915
uhci_hcd               61440  0
drm_kms_helper        253952  6 bochs,drm_vram_helper,drm_display_helper,drm_ttm_helper,xe,i915
ehci_pci               16384  0
ehci_hcd              110592  1 ehci_pci
drm                   774144  14 gpu_sched,drm_kms_helper,drm_exec,bochs,drm_gpuvm,drm_vram_helper,drm_suballoc_helper,drm_display_helper,drm_buddy,drm_ttm_helper,xe,i915,ttm
scsi_mod              327680  3 libata,sg,sr_mod
usbcore               409600  4 ehci_pci,usbhid,ehci_hcd,uhci_hcd
psmouse               217088  0
xen_netfront           49152  1
crc32_pclmul           12288  0
xen_blkfront           53248  4
i2c_piix4              28672  0
floppy                 94208  0
crc32c_intel           16384  3
scsi_common            16384  4 scsi_mod,libata,sg,sr_mod
i2c_smbus              16384  1 i2c_piix4
usb_common             16384  3 usbcore,ehci_hcd,uhci_hcd
serio_raw              16384  0

Modules in the CoreOS VM:

Module                  Size  Used by
nft_nat                12288  1
nft_masq               12288  2
veth                   45056  0
bridge                475136  0
stp                    12288  1 bridge
llc                    16384  2 bridge,stp
xe                   4288512  0
drm_gpuvm              49152  1 xe
gpu_sched              69632  1 xe
drm_ttm_helper         16384  1 xe
drm_exec               12288  2 drm_gpuvm,xe
drm_suballoc_helper    20480  1 xe
nls_utf8               12288  4
cifs                 2056192  2
cifs_arc4              12288  1 cifs
nls_ucs2_utils          8192  1 cifs
cifs_md4               12288  1 cifs
dns_resolver           16384  1 cifs
rfkill                 45056  1
nft_fib_inet           12288  3
nft_fib_ipv4           12288  1 nft_fib_inet
nft_fib_ipv6           12288  1 nft_fib_inet
nft_fib                12288  3 nft_fib_ipv6,nft_fib_ipv4,nft_fib_inet
nft_reject_inet        12288  16
nf_reject_ipv4         12288  1 nft_reject_inet
nf_reject_ipv6         20480  1 nft_reject_inet
nft_reject             12288  1 nft_reject_inet
nft_ct                 28672  10
nft_chain_nat          12288  6
nf_nat                 65536  3 nft_nat,nft_masq,nft_chain_nat
nf_conntrack          212992  4 nf_nat,nft_ct,nft_nat,nft_masq
nf_defrag_ipv6         24576  1 nf_conntrack
nf_defrag_ipv4         12288  1 nf_conntrack
nf_tables             430080  517 nft_ct,nft_nat,nft_reject_inet,nft_fib_ipv6,nft_fib_ipv4,nft_masq,nft_chain_nat,nft_reject,nft_fib,nft_fib_inet
sunrpc                921600  1
i915                 5369856  0
intel_rapl_msr         20480  0
intel_rapl_common      61440  1 intel_rapl_msr
intel_uncore_frequency_common    16384  0
intel_pmc_core        147456  0
pmt_telemetry          16384  1 intel_pmc_core
drm_buddy              32768  2 xe,i915
pmt_discovery          20480  1 pmt_telemetry
ttm                   135168  3 drm_ttm_helper,xe,i915
pmt_class              20480  2 pmt_telemetry,pmt_discovery
intel_pmc_ssram_telemetry    16384  1 intel_pmc_core
intel_vsec             28672  3 intel_pmc_ssram_telemetry,pmt_telemetry,xe
ppdev                  24576  0
i2c_algo_bit           20480  2 xe,i915
drm_display_helper    331776  2 xe,i915
cec                   106496  3 drm_display_helper,xe,i915
video                  81920  2 xe,i915
i2c_piix4              40960  0
parport_pc             57344  0
wmi                    32768  1 video
parport                81920  2 parport_pc,ppdev
bochs                  24576  0
i2c_smbus              20480  1 i2c_piix4
joydev                 36864  0
tun                    77824  2
vsock_loopback         12288  0
vmw_vsock_virtio_transport_common    57344  1 vsock_loopback
vmw_vsock_vmci_transport    57344  0
vsock                  73728  5 vmw_vsock_virtio_transport_common,vsock_loopback,vmw_vsock_vmci_transport
vmw_vmci              118784  1 vmw_vsock_vmci_transport
overlay               253952  4
erofs                 184320  1
netfs                 614400  2 cifs,erofs
xfs                  3796992  2
xen_netfront           61440  2
xen_blkfront           57344  5
polyval_clmulni        12288  0
ghash_clmulni_intel    12288  0
serio_raw              20480  0
ata_generic            12288  0
nvme_tcp              102400  0
nvme_fabrics           49152  1 nvme_tcp
nvme_core             274432  2 nvme_tcp,nvme_fabrics
pata_acpi              12288  0
nvme_keyring           20480  3 nvme_tcp,nvme_core,nvme_fabrics
nvme_auth              32768  1 nvme_core
be2iscsi              233472  0
bnx2i                  94208  0
cnic                  102400  1 bnx2i
uio                    32768  1 cnic
cxgb4i                 73728  0
cxgb4                 782336  1 cxgb4i
tls                   167936  1 cxgb4
cxgb3i                 53248  0
cxgb3                 266240  1 cxgb3i
mdio                   12288  1 cxgb3
libcxgbi              110592  2 cxgb4i,cxgb3i
libcxgb                28672  2 libcxgbi,cxgb3i
qla4xxx               524288  0
iscsi_boot_sysfs       20480  2 be2iscsi,qla4xxx
iscsi_tcp              28672  0
libiscsi_tcp           40960  4 libcxgbi,iscsi_tcp,cxgb4i,cxgb3i
libiscsi               94208  8 be2iscsi,libiscsi_tcp,bnx2i,libcxgbi,iscsi_tcp,qla4xxx,cxgb4i,cxgb3i
scsi_transport_iscsi   192512  7 be2iscsi,libiscsi_tcp,bnx2i,libcxgbi,iscsi_tcp,qla4xxx,libiscsi
scsi_dh_rdac           16384  0
scsi_dh_emc            12288  0
scsi_dh_alua           28672  0
i2c_dev                28672  0
fuse                  274432  1
dm_multipath           53248  0
nfnetlink              20480  3 nf_tables

Having compared the two, there are 3 which are GPU related: drm, drm_vram_helper, drm_kms_helper.

I switched to a vanilla Fedora Server install to make things simpler, and the same modules are missing (and have the same issue as with CoreOS).

I tried to load the missing modules (modprobe drm, etc.), however they are not loaded. And I see the following error message in dmesg:

[    6.202136] systemd[1]: modprobe@drm.service - Load Kernel Module drm was skipped because of an unmet condition check (ConditionKernelModuleLoaded=!drm).

While in Debian:

[    1.386503] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[    1.405386] systemd[1]: modprobe@drm.service: Deactivated successfully.
[    1.416496] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.

I have found a similar thread in the XCP-NG forum, which suggested using the BIOS mode, or a dummy HDMI plug. I tried both, but neither worked. I will test Proxmox or ESXi during the weekend, to rule out if this is a hypervisor specific behaviour, but in the meantime let me know if there are other things I can test, e.g., how I can force the drm kernel module to be loaded.

4

I have installed a vanilla Fedora Server on baremetal, and actually I see that these modules are not loaded there either, while intel_gpu_top actually works. This may be an XCP-NG specific issue after all, I’ll report back once I try Proxmox.

1 Like
5

I have tested CoreOS as a VM under Proxmox, and can confirm that the iGPU is passed through without any issues, and I can use it with podman and docker containers. So it was an XCP-NG issue after all, but I wonder why it affected only Fedora and not Debian. If I have some time, I’ll do further testing and update this topic, but for the time being, I’ll stick to Proxmox.

1 Like