So I had an issue at work, and found an amazing answer on StackOverflow. This post is just to copy that info so I can find it later, and help you find it easier.
Sometimes networks flake out. This can cause Linux to half lose its NFS mount. It’s still mounted, but you can’t read or write to it. And calls to unmount it just timeout and fail.
This seems to be caused by stateful firewalls. After the TCP connection has gone stale, the firewall wont forward the packets anymore and may just be dropping them. The client however won’t time out the connection and start over because it still thinks it has a valid connection. So when the client makes queries it just keeps trying over and over and never getting a reply.
I’ve seen people simply revert to rebooting a system to clear the NFS mount and cleanly mount it again. But there is a better way!
Just add the remote IP to the local system temporarily as a secondary interface. This lets the NFS client get a negative response back to its queries quickly and fail properly.
ifconfig eth0:fakenfs 192.0.2.55 netmask 255.255.255.255 # umount -f /path/to/nfs # ifdown eth0:fakenfs
192.0.2.55 is the IP of the NFS server you’ve lost connection to. Adding the IP locally lets the server get a reply from itself and safely unmount. You can then try mounting the NFS server again after removing the fake IP and if your network is happy again it should mount just fine.
Props to Daniel Papasian for the excellent answer.