Thunderbolt 3 nvme hotplug difficulties

I would like to use a LaCie Rugged SSD Pro and have run into a number of difficulties. I have had these difficulties for 3 to 5 fedora releases and am now asking for help.

The laptop has a 1920x1080 lcd. I am using a 1920x1200 monitor with it’s display port connected to a thunderbolt 3 port on the laptop. When I plug the LaCie nvme device into the only other thunderbolt 3 port on the laptop the display looses signal. Unplugging the LaCie and power cycling the monitor and re-enabling it in gnome and resetting the resolution in gnome allows me to get the external monitor back without rebooting.

Currently I have the LaCie labeled gpt with a single partition formatted xfs. When I plug in the LaCie it shows up in lsblk but the filesystem is not automounted. I can (u)mount the filesystem manually. I cannot see a way to power off or β€œsafely remove” this external storage.

I have even more problems when I use luks but I do not have the symptoms memorized and would like to resolve the more simple case first.

Rather than using the LaCie as a hotplug device, putting an entry in fstab and plugging it in before booting then unplugging it after poweroff works. Finding resolutions to hotplug problems would be beneficial though.

Let’s start here : I see you have a Dell Precision 7550

  1. Do you know if the hardware is configured to push the bandwidth needed to drive a Monitor from DP => Thunderbolt 3 + a LaCie Rugged SSD Pro at the same time? You can check with the Dell website for that info, I cannot speak from memory. You’ll be looking for Bandwidth per port. Depending how many ports you have, 1port could be full bandwidth + Charging, other could be limited to slower speeds etc.
  1. This is a symptom of the first question I asked, It can be a bandwidth issue, but could lead to a driver issue. Thunderbolt can be iffy. . .

Do you have LUKS set up on this device? Did you format the device and it set up for root user only? (happens if you partition with gparted under the root account ) This can cause the mount issue ex:

I find I have issues after formating my drives, and have to set permissions to allow user and groups read/write after doing so I can unmount from nautilus.

Well, let me know what you would like to achieve, --key-file entry, configure /etc/fstab/ & /etc/crypttab properly, Fedora Workstation mounts drives from /run/media/<User>/Your-drive and in nautilus you can simply use :eject_button: + notification of "

followed by β€œSafely Rremove device”

According to the Tech sheet you have a Mini DP 1.4 ? Why not use the DP => Mini DP 1.4 with an adapter ?

1 Like

@hamrheadcorvette, the loss of the display is likely due to bandwidth priority given to the LaCie as you suggested. Pretty cool to have fast hotplug storage!

LaCie Rugged SSD Pro
   β”œβ”€ type:          peripheral
   β”œβ”€ name:          Rugged SSD Pro
   β”œβ”€ vendor:        LaCie
   β”œβ”€ uuid:          00000000-0000-0000-ffff-ffffffffffff
   β”œβ”€ dbus path:     /org/freedesktop/bolt/devices/00000000_0000_0000_ffff_ffffffffffff
   β”œβ”€ generation:    Thunderbolt 3
   β”œβ”€ status:        authorized
   β”‚  β”œβ”€ domain:     c8010000-0072-7c0e-0330-02864a525102
   β”‚  β”œβ”€ parent:     c8010000-0072-7c0e-0330-02864a525102
   β”‚  β”œβ”€ syspath:    /sys/devices/pci0000:00/0000:00:1c.0/0000:02:00.0/0000:03:00.0/0000:04:00.0/domain0/0-0/0-3
   β”‚  β”œβ”€ rx speed:   40 Gb/s = 2 lanes * 20 Gb/s
   β”‚  β”œβ”€ tx speed:   40 Gb/s = 2 lanes * 20 Gb/s
   β”‚  └─ authflags:  none
   β”œβ”€ authorized:    Tue 16 Jan 2024 04:16:03 PM UTC
   β”œβ”€ connected:     Tue 16 Jan 2024 04:16:02 PM UTC
   └─ stored:        Tue 16 Jan 2024 04:13:34 PM UTC
      β”œβ”€ policy:     iommu
      └─ key:        no

The laptop has a miniDP and hdmi so I expect to be able to resolve display over thunderbolt 3 issues by purchasing an appropriate cable. Still, I would be happier if removal of the LaCie was followed by the monitor comming back automatically. Do you know if udev handles this? Or is this in the boltd domain?

I will get back to LUKS after the simpler non-LUKS case is working.

For the current xfs on gpt case

udisksctl mount -b /dev/nvme2n1p1

pops up an authentication dialog but that seems to be due to the LaCie not being handled as a hotplug external storage device by udisks. The /dev device nodes are owned by root:disk just like when plugging in a usb disk that does automount (it has the same xfs on gpt as the LaCie). So the automount should work and I would not expect the authentication dialog.

That thunderbolt 3 is still iffy is the reason I am coming here. This seems like a great issue to resolve on the road to better thunderbolt 3(4).

Some devices can bifurcate better than others, Intel has always had terrible bandwidth for devices. DMI is horrible. AMD just went with PCIe all the way. There could be a way to resolve this, but I cannot find a map of how Intel/Dell do Bifurcation on these models.

If Thunderbolt is straight from the CPU (it should be) Then there should be bandwidth to accommodate the Daisy Chaining of DP multiple monitors and have peripherals. They claim 4x4k Monitors on this model somehow.

You do not own the device, It’s owned by a user with higher privileges.
If you do not have any important data on the drive try this :

Set it to User ( I opened Nautilus as the root user and changed it. There are commands for this if you know them. ):

This has worked for me, in the past give it a try. I currently have this happen sometimes with USB Enclosures but not USB Drives. . .

1 Like

@hamrheadcorvette, receiving suggestions like this makes commiserating with the fedora community a wholesome activity I would recommend to anyone.

If I mount the LaCie as root

# mount /dev/nvme2n1p1 /mnt

and add a directory for a user

# install -d -o username -g usergroup -m 0750 /mnt/backups/home/username

then username can read/write to /mnt/backups/home/username

In the usb case the same works but hotplugging mounts it at /run/media/username/LABEL

Thunderbolt 3 hotplug storage automount working the same as usb hotplug would be a welcome improvement.

The thunderbolt 3 bandwidth issue being handled more gracefully would also be welcomed. Maybe through up a dialog allowing the user to direct bandwidth decisions.

Yep’ That is correct ! I have never asked why Fedora does this for OOTB Hotplugged drives. Every time I needed to work on it I was in a hurry, Vim :g was easier for me to fix the /etc/fstab & /etc/crypttab .

Poke around with the permissions, My USB/SS/USB-C drives work this way. We share a similar partiioning scheme too xfs

This. . . Has a more lengthy and complicated answer, I will throw a arrow in the dark here :

  1. In the BIOS Check if you have PCIe tunneling set to OFF/Disabled
    this can change the behavior of the devices potentially being bifurcated in some way DP > TB3 + TB3 NVMe . . .

Could be drivers, could be device could be. . something else ( even Cables . . knock off cables exits. )

Using the UUID for the device and putting that into fstab for mounting would ensure it always is mounted under /mnt even when hot plugged. To avoid hangs if the device is not connected at boot time add the nofail option into the fstab entry.

It seems udev checks the fstab entries before auto-mounting under /run/media

1 Like

@computersavvy, you point to udev. I think you are onto something.

Currently I would guess that udev makes the device nodes in /dev. I do not think there are any udev rules attempting to kick off any automout process.

lspci shows a bunch of Thunderbolt 3 pieces but not the LaCie

02:00.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] (rev 06)
03:00.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] (rev 06)
03:01.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] (rev 06)
03:02.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] (rev 06)
03:04.0 PCI bridge: Intel Corporation JHL7540 Thunderbolt 3 Bridge [Titan Ridge 4C 2018] (rev 06)
04:00.0 System peripheral: Intel Corporation JHL7540 Thunderbolt 3 NHI [Titan Ridge 4C 2018] (rev 06)

boltctl shows the LaCie
nvme shows that the LaCie has a Seagate Firecude inside.

Looking through /usr/lib/udev/rules.d there are a couple files dealing with thunderbolt net device stuff. Some dealing with nvme. But I am not able to follow how plugging in the LaCie goes through boltd and udev stuff to get the device nodes populated in /dev. Is the kernel doing it directly?

I think udev manages all auto-plugging needs. I know it does the mount of flash drives under /run/media, and that when I put a line in fstab to mount an exteral USB drive in a different location by UUID that also automatically happens when the device is attached. This is how I manage my backup drive in a USB case externally.

Whether that also carries over to TB connections I have no clue.

Recently there was an issue with an external drive drawing too much power on a USB port and it would cause an already mounted flash drive to drop off when the other device was attached. This may be similar on the TBolt port if there is not enough power available to handle the start-up power surge without dropping the voltage too low and causing other devices to lose their config.

1 Like

I am continuing my attempt to figure out what is going on by browsing files in /usr/lib/udev/rules.d and comparing what usb storage devices go through and nvme devices. Currently I have question about 80-udisks2.rules which seems promising. There are plenty of usb related rules with 60-persistent-storage.rules usb entries providing an oxymoron :slightly_smiling_face:

Progress has been made. At least now it can be hotplugged and udisks will mount it under /run/media/username/volumeid and udisksctl can umount it. The technique was to add a udev rule to set properties that inform udisks to automount without elevated permissions. The udev rules may not be as appropriate as they should be.

/etc/udev/rules.d/81-udisks-lacie-rugged-ssd-pro.rules contains

ACTION!="add", GOTO="fc_end"
SUBSYSTEM!="block", GOTO="fc_end"
KERNEL=="nvme*[0-9]n*[0-9]p*[0-9]", ENV{DEVTYPE}=="partition", ENV{ID_SERIAL}="Seagate_Firecuda_*" \

It looks like the chain of devices that get added when plugging the LaCie in is

partition β†’ disk β†’ nvme ns β†’ nvme β†’ pcie bus

This is not a complete solution. udevadm shows that the disk is not removable. The ATTR{removable} is not writable from a udev rule as far as my attempts have gone. Unmounting the filesystem and pulling the plug generates plenty of not very graceful log entries when udev is set to debug level logging.

Something that is working differently from usb storage occurs when powering up the laptop. If the hotplug storage device is plugged in while the laptop is powered down the usb device ends up getting mounted when the user logs in. The LaCie device does not. The device nodes are made though. Unplug/plugin the LaCie after login and it does automount.

Any suggestions on how to refine this solution?