Low Nvidia performance with newer kernels

Ask Fedora
, , , , , ,
24

Interesting development:

I compared dmesg outputs from kernel 6.1 and 6.13. I’m seeing a limited PCIe bandwidth with 6.13 for some reason. Is this the cause instead of the pnp errors? (I see the BARs get assigned regardless of what pnp spits out)

6.1:

abijay@fedora:~$ sudo journalctl -b 0 -g "pci 0000:01"
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: [10de:24dd] type 00 class 0x030000
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x00ffffff]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x1ffffffff 64bit pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: reg 0x1c: [mem 0x00000000-0x01ffffff 64bit pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: reg 0x24: [io  0x0000-0x007f]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0007ffff pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 256)
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: PME# supported from D0 D3hot
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: 126.024 Gb/s available PCIe bandwidth, limited by 16.0 GT/s PCIe x8 link at 0000:00:01.0 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: Adding to iommu group 19
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: vgaarb: bridge control possible
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: [10de:228b] type 00 class 0x040300
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: reg 0x10: [mem 0x00000000-0x00003fff]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: Max Payload Size set to 256 (was 128, max 256)
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: Adding to iommu group 19
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: BAR 1: assigned [mem 0x6000000000-0x61ffffffff 64bit pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: BAR 3: assigned [mem 0x6200000000-0x6201ffffff 64bit pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x60ffffff]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: BAR 6: assigned [mem 0x61000000-0x6107ffff pref]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: BAR 0: assigned [mem 0x61080000-0x61083fff]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.0: BAR 5: assigned [io  0x5000-0x507f]
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: extending delay after power-on from D3hot to 20 msec
Mar 28 13:26:19 fedora kernel: pci 0000:01:00.1: D0 power state depends on 0000:01:00.0
Mar 28 15:50:29 fedora sudo[20814]:   abijay : TTY=pts/4 ; PWD=/home/abijay ; USER=root ; COMMAND=/usr/bin/journalctl -b - -g 'pci 0000:01'
Mar 28 15:50:37 fedora sudo[20848]:   abijay : TTY=pts/4 ; PWD=/home/abijay ; USER=root ; COMMAND=/usr/bin/journalctl -b 0 -g 'pci 0000:01'
Mar 28 15:52:14 fedora sudo[20886]:   abijay : TTY=pts/4 ; PWD=/home/abijay ; USER=root ; COMMAND=/usr/bin/journalctl -b 0 -g 'pci 0000:01'
Mar 28 15:52:24 fedora sudo[20917]:   abijay : TTY=pts/4 ; PWD=/home/abijay ; USER=root ; COMMAND=/usr/bin/journalctl -b -3 -g 'pci 0000:01'
Mar 28 15:54:00 fedora sudo[21388]:   abijay : TTY=pts/4 ; PWD=/home/abijay ; USER=root ; COMMAND=/usr/bin/journalctl -b 0 -g 'pci 0000:01'

6.13:

abijay@fedora:~$ sudo journalctl -b -3 -g "pci 0000:01"
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: [10de:24dd] type 00 class 0x030000 PCIe Legacy Endpoint
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 0 [mem 0x00000000-0x00ffffff]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 1 [mem 0x00000000-0x0fffffff 64bit pref]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 3 [mem 0x00000000-0x01ffffff 64bit pref]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 5 [io  0x0000-0x007f]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: ROM [mem 0x00000000-0x0007ffff pref]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: Max Payload Size set to 256 (was 128, max 256)
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: PME# supported from D0 D3hot
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x8 link at 0000:00:01.0 (capable of 252.048 Gb/s with 16.0 GT/s PCIe x16 link)
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: [10de:228b] type 00 class 0x040300 PCIe Endpoint
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: BAR 0 [mem 0x00000000-0x00003fff]
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: Max Payload Size set to 256 (was 128, max 256)
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: vgaarb: bridge control possible
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 1 [mem 0x6000000000-0x600fffffff 64bit pref]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 3 [mem 0x6010000000-0x6011ffffff 64bit pref]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 0 [mem 0x60000000-0x60ffffff]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: ROM [mem 0x61000000-0x6107ffff pref]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: BAR 0 [mem 0x61080000-0x61083fff]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: BAR 5 [io  0x5000-0x507f]: assigned
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: extending delay after power-on from D3hot to 20 msec
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: D0 power state depends on 0000:01:00.0
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.0: Adding to iommu group 16
Mar 28 12:15:04 fedora kernel: pci 0000:01:00.1: Adding to iommu group 16
25

I do have the same message with 6.13.8 kernel:

[ 0.327394] pci 0000:01:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x8 link at 0000:00:01.0 (capable of 126.024 Gb/s with 16.0 GT/s PCIe x8 link)

any idea what is it about?

26

in my case I think it’s about the Intel onboard graphics card which I disabled in BIOS anyways:

liviu@bobdenaut:~$ sudo lspci -vv -s 0000:00:01.0
0000:00:01.0 PCI bridge: Intel Corporation Raptor Lake PCI Express 5.0 Graphics Port (PEG010) (rev 01) (prog-if 00 [Normal decode])
	Subsystem: ASUSTeK Computer Inc. Device 14d3
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin A routed to IRQ 122
	IOMMU group: 2
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: 5000-5fff [size=4K] [16-bit]
	Memory behind bridge: 86000000-870fffff [size=17M] [32-bit]
	Prefetchable memory behind bridge: 6000000000-6201ffffff [size=8224M] [32-bit]
	Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16+ MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [40] Express (v2) Root Port (Slot+), IntMsgNum 0
		DevCap:	MaxPayload 256 bytes, PhantFunc 0
			ExtTag- RBE+ TEE-IO-
		DevCtl:	CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
			MaxPayload 256 bytes, MaxReadReq 128 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+ TransPend-
		LnkCap:	Port #2, Speed 32GT/s, Width x8, ASPM not supported
			ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt+ AutBWInt+
		LnkSta:	Speed 5GT/s, Width x8
			TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
		SltCap:	AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
			Slot #1, PowerLimit 75W; Interlock- NoCompl+
		SltCtl:	Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
			Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
		SltSta:	Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet+ Interlock-
			Changed: MRL- PresDet- LinkState-
		RootCap: CRSVisible-
		RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible-
		RootSta: PME ReqID 0000, PMEStatus- PMEPending-
		DevCap2: Completion Timeout: Range ABC, TimeoutDis+ NROPrPrP- LTR+
			 10BitTagComp+ 10BitTagReq+ OBFF Via WAKE#, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 2
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd+
			 AtomicOpsCap: Routing+ 32bit+ 64bit+ 128bitCAS+
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- ARIFwd-
			 AtomicOpsCtl: ReqEn+ EgressBlck-
			 IDOReq- IDOCompl- LTR+ EmergencyPowerReductionReq-
			 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
		LnkCap2: Supported Link Speeds: 2.5-32GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
		LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete+ EqualizationPhase1+
			 EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported
	Capabilities: [80] MSI: Enable+ Count=1/1 Maskable- 64bit+
		Address: 00000000fee00238  Data: 0000
	Capabilities: [98] Subsystem: ASUSTeK Computer Inc. Device 14d3
	Capabilities: [a0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP-
			ECRC- UnsupReq- ACSViol- UncorrIntErr- BlockedTLP- AtomicOpBlocked- TLPBlockedErr-
			PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt+ RxOF- MalfTLP-
			ECRC- UnsupReq- ACSViol- UncorrIntErr- BlockedTLP- AtomicOpBlocked- TLPBlockedErr-
			PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked-
		UESvrt:	DLP+ SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+
			ECRC- UnsupReq- ACSViol- UncorrIntErr- BlockedTLP- AtomicOpBlocked- TLPBlockedErr-
			PoisonTLPBlocked- DMWrReqBlocked- IDECheck- MisIDETLP- PCRC_CHECK- TLPXlatBlocked-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CorrIntErr- HeaderOF-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
		RootCmd: CERptEn+ NFERptEn+ FERptEn+
		RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
			 FirstFatal- NonFatalMsg- FatalMsg- IntMsgNum 0
		ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
	Capabilities: [220 v1] Access Control Services
		ACSCap:	SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
		ACSCtl:	SrcValid+ TransBlk- ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl- DirectTrans-
	Capabilities: [200 v1] L1 PM Substates
		L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
			  PortCommonModeRestoreTime=110us PortTPowerOnTime=500us
		L1SubCtl1: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+
			   T_CommonMode=255us LTR1.2_Threshold=783360ns
		L1SubCtl2: T_PwrOn=500us
	Capabilities: [150 v1] Precision Time Measurement
		PTMCap: Requester- Responder+ Root+
		PTMClockGranularity: 4ns
		PTMControl: Enabled+ RootSelected+
		PTMEffectiveGranularity: Unknown
	Capabilities: [a30 v1] Secondary PCI Express
		LnkCtl3: LnkEquIntrruptEn- PerformEqu-
		LaneErrStat: 0
	Capabilities: [a90 v1] Data Link Feature <?>
	Capabilities: [a9c v1] Physical Layer 16.0 GT/s <?>
	Capabilities: [edc v1] Lane Margining at the Receiver
		PortCap: Uses Driver-
		PortSta: MargReady- MargSoftReady-
	Capabilities: [adc v1] Physical Layer 32.0 GT/s <?>
	Kernel driver in use: pcieport
27

IIRC this is part of Nvidia’s power-saving features which downgrades the PCIe link to Gen1 x8 to save power and brings it back up to full speed when the GPU is active.

28

in my case it seems to be Intel Corporation Raptor Lake PCI Express.
See above

liviu@bobdenaut:~$ sudo lspci -vv -s 0000:00:01.0
0000:00:01.0 PCI bridge: Intel Corporation Raptor Lake PCI Express 5.0 Graphics Port (PEG010) (rev 01) (prog-if 00 [Normal decode])
29

So…possible kernel bug?

Not that I really know anything about how exactly the kernel works, but if this is happening on different hardware and exclusively with newer kernel versions, then that’s the only possibility left right?

Can someone familiar with the Linux kernel or with the Fedora kernel team please confirm?

30 
[    0.615983] pci 0000:04:00.0: 8.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x4 link at 0000:03:00.0 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)

[    0.618069] pci 0000:07:00.0: 32.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x16 link at 0000:00:03.1 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)

Remember that with 2 different GPUs, each is configured separately. My system shows both nvidia GPUs, one on an x4 link and one on an x16 link with different speeds.

Since you show the intel gpu on an x8 link why not look at the nvidia gpu as well in dmesg. It probably shows a different link with a different speed limit.

Also boot with the older kernel and verify that there is actually a recorded difference before reporting an issue. What is shown so far does not reveal an actual change.

31

6.1:

6.13:

Are these what you mean?

32

I have no explanation for that, but yes it is what I thought should be shown. Definitely a difference.

lspci | grep -i vga should show the addresses for both GPUs (if you have both) and you can then look in detail at the dmesg output for both. by using grep for the pci address for each card.

$ lspci | grep -i vga
00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-H GT2 [UHD Graphics 630]
01:00.0 VGA compatible controller: NVIDIA Corporation TU116M [GeForce GTX 1660 Ti Mobile] (rev a1)

sudo dmesg | grep 01:00.0 would show the output for my nvidia gpu.
sudo dmesg | grep 00:02.0 would show the output for my intel gpu.

33

Unfortunately, I’m not seeing the same bandwidth alert for my intel iGPU on 6.13:

abijay@fedora:~/Desktop$ cat journalctl_6_13.txt | grep 00:02.0
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: [8086:46a6] type 00 class 0x030000 PCIe Root Complex Integrated Endpoint
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: BAR 0 [mem 0x6203000000-0x6203ffffff 64bit]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: BAR 2 [mem 0x4000000000-0x400fffffff 64bit pref]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: BAR 4 [io  0x6000-0x603f]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: DMAR: Skip IOMMU disabling for graphics
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 0 [mem 0x00000000-0x00ffffff 64bit]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 0 [mem 0x00000000-0x06ffffff 64bit]: contains BAR 0 for 7 VFs
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 2 [mem 0x00000000-0x1fffffff 64bit pref]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 2 [mem 0x00000000-0xdfffffff 64bit pref]: contains BAR 2 for 7 VFs
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: vgaarb: setting as boot VGA device
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: vgaarb: bridge control possible
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
Mar 29 20:13:24 fedora kernel: pnp 00:03: disabling [mem 0xc0000000-0xcfffffff] because it overlaps 0000:00:02.0 BAR 9 [mem 0x00000000-0xdfffffff 64bit pref]
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 2 [mem 0x4020000000-0x40ffffffff 64bit pref]: assigned
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: VF BAR 0 [mem 0x4010000000-0x4016ffffff 64bit]: assigned
Mar 29 20:13:24 fedora kernel: pci 0000:00:02.0: Adding to iommu group 0
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] Found alderlake_p (device ID 46a6) display version 13.00 stepping D0
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] VT-d active for gfx access
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: vgaarb: deactivate vga console
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] Using Transparent Hugepages
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=io+mem
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] Finished loading DMC firmware i915/adlp_dmc.bin (v2.20)
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] [ENCODER:249:DDI B/PHY B] unusable PPS, disabling eDP
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: GuC firmware i915/adlp_guc_70.bin version 70.36.0
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: HuC firmware i915/tgl_huc.bin version 7.9.3
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: HuC: authenticated for all workloads
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: GUC: submission enabled
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: GUC: SLPC enabled
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] GT0: GUC: RC enabled
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] Protected Xe Path (PXP) protected content support initialized
Mar 29 20:13:27 fedora kernel: [drm] Initialized i915 1.6.0 for 0000:00:02.0 on minor 1
Mar 29 20:13:27 fedora kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
Mar 29 19:13:28 fedora kernel: mei_pxp 0000:00:16.0-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:00:02.0 (ops i915_pxp_tee_component_ops [i915])
Mar 29 19:13:28 fedora kernel: mei_hdcp 0000:00:16.0-b638ab7e-94e2-4ea2-a552-d1c54b627f04: bound 0000:00:02.0 (ops i915_hdcp_ops [i915])
Mar 29 19:13:28 fedora kernel: snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
34

There have been major updates to the kernel between version 6.1 and 6.13.
I would expect serious differences in operations.

35

True. I was hoping to find someone in the Kernel team for that exact reason.

Though from what I have seen, this issue began for me roughly around kernel 6.10 with Manjaro on my old Laptop. Though even 6.6 no longer works where it once did.

36

Just checked the lspci -vv outputs again. Specifically the PCIe bridges, and found that they are limited to 2.5GT/s on kernel 6.13 and are at 16GT/s on 6.1.

I’ve also tested Manjaro 23.0 and 24.0 live ISOs (kernel 6.5 and 6.9 respectively) and found the issues there as well. Links to lspci outputs below:

6.1: Pastebin
6.13: Pastebin

As an aside, is the problem described here related?

37

This is from my dmesg

[    0.463260] pci 0000:01:00.0: 16.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x8 link at 0000:00:01.1 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)

Ignore the low figures as they are bullshit.

Use nvidia-settings powermizer tab, it shows the pcie link rates, set the prefer mode to maximum.
Does that show the pcie link rates increase?

38

Also having two nvme devices can limit it, I removed one and the rate increased to x16
Some m2 slots can also limit the rate of other pcie slots, check the mobo manual for details.

[    0.464240] pci 0000:01:00.0: 32.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x16 link at 0000:00:01.1 (capable of 126.016 Gb/s with 8.0 GT/s PCIe x16 link)
39

you meant here to modify?

Screenshot From 2025-03-31 08-57-38
Screenshot From 2025-03-31 08-57-381234×985 155 KB

Because no matter what I manually enter down there it does not save.

40

adaptive mode

Screenshot from 2025-03-31 07-28-39
Screenshot from 2025-03-31 07-28-392162×1788 416 KB

performance mode

Screenshot from 2025-03-31 07-23-46
Screenshot from 2025-03-31 07-23-462162×1788 421 KB

Any app that uses the GPU should raise the power levels and increase link rates.

41

Interesting, mine looks totally different and missing that part

Screenshot From 2025-03-31 09-35-43
Screenshot From 2025-03-31 09-35-431639×1176 356 KB

42

what version is this?

mine:
Package "nvidia-settings-3:570.133.07-1.fc41.x86_64"

43

I have the same version.

leigh@leigh-pc:~$ rpm -q nvidia-settings
nvidia-settings-570.133.07-1.fc42.x86_64