NFS (RPC) troubleshooting

Ask Fedora
42

I wonder if there is a required service that is not running?
This is the services with nfs in their name on my NFS server for you to compare against:

$ sudo systemctl | grep -i nfs
  proc-fs-nfsd.mount                                                                                                     loaded active mounted   NFSD configuration filesystem
  var-lib-nfs-rpc_pipefs.mount                                                                                           loaded active mounted   RPC Pipe File System
  nfs-idmapd.service                                                                                                     loaded active running   NFSv4 ID-name mapping service
  nfs-mountd.service                                                                                                     loaded active running   NFS Mount Daemon
  nfs-server.service                                                                                                     loaded active exited    NFS server and services
  nfsdcld.service                                                                                                        loaded active running   NFSv4 Client Tracking Daemon
  rpc-statd-notify.service                                                                                               loaded active exited    Notify NFS peers of a restart
  rpc-statd.service                                                                                                      loaded active running   NFS status monitor for NFSv2/3 locking.
  nfs-client.target                                                                                                      loaded active active    NFS client services
43

Can you try running a nmap scan of the port to make sure the service is open? Also I would advise focusing on the server for now.

What happens when you try to mount the NFS share on the same server that is hosting it? (using 127.0.0.1)

1 Like
44

Thanks. Mine looks exactly the same…

root@dell-kvmhost:~# sudo systemctl | grep -i nfs
  proc-fs-nfsd.mount                                                                       loaded active mounted   NFSD configuration filesystem
  var-lib-nfs-rpc_pipefs.mount                                                             loaded active mounted   RPC Pipe File System
  nfs-idmapd.service                                                                       loaded active running   NFSv4 ID-name mapping service
  nfs-mountd.service                                                                       loaded active running   NFS Mount Daemon
  nfs-server.service                                                                       loaded active exited    NFS server and services
  nfsdcld.service                                                                          loaded active running   NFSv4 Client Tracking Daemon
  rpc-statd-notify.service                                                                 loaded active exited    Notify NFS peers of a restart
  rpc-statd.service                                                                        loaded active running   NFS status monitor for NFSv2/3 locking.
  nfs-client.target                                                                        loaded active active    NFS client services
root@dell-kvmhost:~#
45

Here’s the output of nmap. Looks ok as far as I can tell?

root@dell-kvmhost:~# sudo nmap -p 2049,111 192.168.2.101
Starting Nmap 7.92 ( https://nmap.org ) at 2024-10-03 08:34 AEST
Nmap scan report for acer-bkp-server (192.168.2.101)
Host is up (0.0072s latency).

PORT     STATE SERVICE
111/tcp  open  rpcbind
2049/tcp open  nfs
MAC Address: F8:0F:41:CF:48:CA (Wistron Infocomm (Zhongshan))

Nmap done: 1 IP address (1 host up) scanned in 0.49 seconds

I get access denied even mounting from the server

root@acer-bkp-server:/mnt# mount 127.0.0.1:/data/nfs_share /mnt/nfs_share -o v3
mount.nfs: access denied by server while mounting 127.0.0.1:/data/nfs_share

I was told that setting up NFS is straight forward… might have to rethink that statement :slight_smile:

46

Let’s test this simple setup:

sudo mkdir -p /data/nfs_share /mnt/nfs_share
sudo tee /etc/exports << EOF > /dev/null
/data/nfs_share *(rw)
EOF
sudo systemctl restart nfs-server.service
sudo mount 127.0.0.1:/data/nfs_share /mnt/nfs_share
47

That worked

root@acer-bkp-server:~# df -h
Filesystem                                 Size  Used Avail Use% Mounted on
/dev/mapper/fedora_acer--bkp--server-root   35G   34G  2.0G  95% /
devtmpfs                                   4.0M     0  4.0M   0% /dev
tmpfs                                      1.9G     0  1.9G   0% /dev/shm
efivarfs                                   128K   58K   66K  48% /sys/firmware/efi/efivars
tmpfs                                      758M  1.4M  757M   1% /run
tmpfs                                      1.9G     0  1.9G   0% /tmp
/dev/sda2                                  2.0G  432M  1.6G  22% /boot
/dev/sda1                                  500M  7.5M  493M   2% /boot/efi
/dev/sda4                                  3.6T  491G  2.9T  15% /data
tmpfs                                      379M   24K  379M   1% /run/user/0
127.0.0.1:/data/nfs_share                  3.6T  491G  2.9T  15% /mnt/nfs_share

Interestingly when I did this earlier in response to @darin755 I got the access denied error with the same mount command

48

Perhaps there was a problem in the configuration.
Does it work from the client now?

49

No it still times out…

sudo mount 192.168.2.101:/data/nfs_share /mnt/nfs_share -o v3
50

I’d like to first thank you guys to support this puzzle of questions as this is clearly above me.

Bottomline: I can mount the exported directory locally from the server but not from the client

Here’s where I’m at:

  • Both nfs-server.service and rpcbind services are running (and enabled) on the server
  • All required services are running (as verified with another user); sudo systemctl | grep -i nfs
  • My directory is shared on the server; i.e. showmount -e <server IP>
  • I can ping in both direction via IPv4 and IPv6; latency of <1ms
  • Firewalls on both server and client are disabled (temporary)
  • SELinux is disabled on both server and client
  • I can mount the directory locally on the server (both loopback 127.0.0.1 and the IP address)
  • mount from the client times out
  • /etc/fstab not yet updated (as I can’t mount)
  • Server and client running Fedora server 40
  • showmount takes about 2 mins to show the exported directory; I have run this command from the client (Fedora 40 server) but also from another Linux client on the network (running on an RPi); both take the same amount of time
  • sudo tail -f /var/log/messages | grep -i nfs shows an entry Oct 4 17:01:32 acer-bkp-server rpc.mountd[17169]: authenticated mount request from 192.168.2.100:801 for /data/nfs_share (/data/nfs_share) when submitting a mount request from the client; this entry shows up immediately; mount times out
  • nmap scan shows ports 111 and 2049 open

I’d like to better understand what happens on the server when a mount or showmount command is issued from the network. What other tools can we employ here?

51

Have some great news.

The setup’s ok now. The issue is with my managed switch. It must block/process the requests from the client somehow. I found out through another post describing the same issue. It was resolved by temporarily connecting both server and client via a cheap LAN bridge. Did the same and it works. Could’ve probably figured this out with Wireshark but that’s another area I’m not really familiar with.

Have contacted the switch manufacturer re changes to settings and in the meantime put the firewall and SELinux back on.

Thanks again everyone.

3 Likes