I recently brought a NAS box back online after a few years of it being out of service. This box has a Supermicro AOC-SAS2LP-MV8 card, which uses the Marvell 9480 chip. This box was previously running Fedora 25, which seemed to support this card just fine. However, after upgrading to Fedora 32, none of the drives attached to this card is visible. As far as I know, this chip is still supported by the Linux kernel; but, I’m not really sure where to start with debugging this. Any suggestions?
anything like this bug? https://bugzilla.redhat.com/show_bug.cgi?id=531264 (not solves but closed)
1 Like
FWIW, it’s a fresh install.
That bug is ancient (2009). This was working in Fedora 25.
That chip should be covered by the mvsas kernel module. Can you check if it is loaded:
$ lsmod | grep mvsas
and if that does not report anything, load it:
$ sudo modprobe mvsas
That chip should be covered by the
mvsaskernel module. Can you check if it is loaded:$ lsmod | grep mvsas
That shows:
mvsas 69632 0
libsas 102400 1 mvsas
scsi_transport_sas 49152 2 mvsas,libsas
…so I gather the driver is loaded.
This box is also showing the following symptom, which may or may not be related (from Cockpit):
1:37 AM Failed to start udev Wait for Complete Device Initialization. systemd
1:37 AM systemd-udev-settle.service: Failed with result 'exit-code'. systemd
1:37 AM systemd-udev-settle.service: Main process exited, code=exited, status=1/FAILURE systemd
1:35 AM systemd-udev-settle.service is deprecated. udevadm
This is most likely related to what’s taking startup/shutdown so long. It seems like something takes about 2 minutes to time out before either of those procedures can complete.
OK, that’s interesting.
Yes, that seems likely: systemd calls udevadm settle, which times out. I don’t think that is usually much of a problem, but still, could point us in the right direction.
You can poke around the logs of the last boot (journalctl -b for the entire thing, or journalctl -b -u systemd-udev-settle -u systemd-udevd for just the udev-related stuff) to try and see if it is your card that is causing the holdup.
Also, you could override systemd-udev-settle to run udevadm settle with the --debug-flag to get more output.
1 Like
I did
$ sudo systemctl edit systemd-udev-settle
…and modified the override file to look like:
[Service]
ExecStart=udevadm --debug settle
This, unfortunately, didn’t seem to yield much more information (after rebooting):
$ journalctl -b -u systemd-udev-settle
-- Logs begin at Sat 2020-06-06 00:27:01 EDT, end at Thu 2020-06-18 16:05:30 EDT. --
Jun 18 15:33:40 hinge.endoframe.net udevadm[675]: systemd-udev-settle.service is deprecated.
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: systemd-udev-settle.service: Main process exited, code=exited, status=1/FAILURE
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: systemd-udev-settle.service: Failed with result 'exit-code'.
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: Failed to start udev Wait for Complete Device Initialization.
Adding messages from systemd-udev is a little more informative:
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[719]: could not read from '/sys/module/pcc_cpufreq/initstate': No such device
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[692]: Using default interface naming scheme 'v245'.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[692]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[692]: Could not set AlternativeName= or apply AlternativeNamesPolicy= on eno4: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[692]: eno4: Could not apply link config, ignoring: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[689]: Using default interface naming scheme 'v245'.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[689]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[689]: Could not set AlternativeName= or apply AlternativeNamesPolicy= on eno1: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[689]: eno1: Could not apply link config, ignoring: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[719]: Using default interface naming scheme 'v245'.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[719]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[719]: Could not set AlternativeName= or apply AlternativeNamesPolicy= on eno2: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[719]: eno2: Could not apply link config, ignoring: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[696]: Using default interface naming scheme 'v245'.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[696]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[696]: Could not set AlternativeName= or apply AlternativeNamesPolicy= on eno3: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[696]: eno3: Could not apply link config, ignoring: File exists
Jun 18 15:33:40 hinge.endoframe.net systemd-udevd[718]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
Jun 18 15:33:40 hinge.endoframe.net udevadm[675]: systemd-udev-settle.service is deprecated.
Jun 18 15:34:41 hinge.endoframe.net systemd-udevd[673]: 0000:00:16.0: Worker [722] processing SEQNUM=4087 is taking a long time
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: systemd-udev-settle.service: Main process exited, code=exited, status=1/FAILURE
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: systemd-udev-settle.service: Failed with result 'exit-code'.
Jun 18 15:35:39 hinge.endoframe.net systemd[1]: Failed to start udev Wait for Complete Device Initialization.
Jun 18 15:36:41 hinge.endoframe.net systemd-udevd[673]: 0000:00:16.0: Worker [722] processing SEQNUM=4087 killed
Jun 18 15:38:57 hinge.endoframe.net systemd-udevd[673]: Worker [722] terminated by signal 9 (KILL)
Jun 18 15:38:57 hinge.endoframe.net systemd-udevd[673]: 0000:00:16.0: Worker [722] failed
…but not much. Most of it looks extraneous; except for the bits about Worker [722]. But it’s not terribly informative about what this worker is doing.
Hm, I was hoping for the logs to be a bit more informative. Kernel 5.7 is being tested right now and should be available soon in Fedora 32, maybe that will take care of it - there a build here if you’d like to try it out. Otherwise, I’d file a bug. Sorry I couldn’t be of more help.
You were helpful; thank you.
I’d love to say I’m going to dig in and debug this; but the fact of the matter is that the equivalent LSI-based card (that works) is cheap enough that I’m just going to get one of those and follow the path of least resistance. While I’d like to see the Marvell chip support restored, I’m more interested in getting my NAS up and running.
1 Like