NFS client Zombie Connection (620 Views)
Reply
Highlighted
Advisor
Posts: 15
Registered: ‎03-15-2010
Message 1 of 11 (620 Views)
Accepted Solution

NFS client Zombie Connection

[ Edited ]

Hi All,

 

I have a HPUX 11.11 workstation working as a nfs client. The nfs server is a Windows 2003 server.

 

After the nfs server is restarted, nfs client couldn't automout the Windows share.

 

I did:

 

/sbin/init.d/nfs.client stop

/sbin/init.d/nfs.server stop

/sbin/init.d/nfs.core stop

(nfs.client stop failed)

 

When I start nfs.server, it hangs at starting rpc.lockd.

 

"netstat -a"  shows:

 

tcp   0  0 hpux_nfs_client.865 windows_nfs_server.nfsd     ESTABLISHED

 

while on nfs server there is no such connection existed.

 

I couldn't do umount, shows "mount point is busy, umount return error 1"

 

Any advice would be appreciated.

 

Thanks,

Teresa

 

 

P.S. This thread has been moved from HP-UX > System Administration to HP-UX > networking - HP Forums Moderator

Advisor
Posts: 15
Registered: ‎03-15-2010
Message 2 of 11 (582 Views)

Re: NFS client Zombie Connection

I also tried to kill all the process that use on the mount point. However, that was unsuccessful. The command "fuser -ku mount_point_path" hung.

 

Thanks,

Teresa

HP Pro
Posts: 1,448
Registered: ‎10-10-2002
Message 3 of 11 (573 Views)

Re: NFS client Zombie Connection

Hi Teresa,

 

There are very few options at 11.11 when an NFS mount is blocked in the kernel and considered "busy".  At 11i v3 (11.31) we have the forcible unmount option where you might be able to avoid a reboot in the event of a hung mount point.  On an 11.11 system, if you cannot cleanly kill the biods and rpc daemons you're likely facing a reboot.

 

I wrote a white paper called "forcibly unmounting NFS filesystems" back in August 2003 that used to be available on the docs.hp.com site but it looks to have been removed for some reason.  It described ways to potentially resolve this issue on 11.11 systems.  I've attached a copy.

 

If the tips in the paper don't work, you're likely looking at a reboot.

 

Regards,

 

Dave

Advisor
Posts: 15
Registered: ‎03-15-2010
Message 4 of 11 (565 Views)

Re: NFS client Zombie Connection

Hi Dave,

 

I followed the instructions on your white paper.  It seems the process couldn't be terminated. "fuser -ku" and "kill -9" both wouldn't work.

 


# umount /mountpoint
nfs umount: nfs_unmount: /mountpoint: is busy
umount: return error 1.


# fuser -u nfs_server:/nfs_share
nfs_server:/nfs_share:    17659o(root)

 

# fuser -ku nfs_server:/nfs_share
nfs_server:/nfs_share:    17659o(root)

 

# fuser -u nfs_server:/nfs_share
nfs_server:/nfs_share:    17659o(root)

 

# ps -fp 17659
     UID   PID  PPID  C    STIME TTY       TIME COMMAND
    root 17659     1  0 03:30:37 ?        263:56 /someprogram 


# fuser -ku /some_program
/someprogram:    17659mt(root)

 

# fuser -u nfs_server:/nfs_share
nfs_server:/nfs_share:    17659o(root)

#

 

 

 

 

 

Thanks,

Teresa

HP Pro
Posts: 1,448
Registered: ‎10-10-2002
Message 5 of 11 (562 Views)

Re: NFS client Zombie Connection

Sounds like you're left with rebooting the client.

 

Dave

Acclaimed Contributor
Posts: 26,027
Registered: ‎03-06-2006
Message 6 of 11 (547 Views)

Re: NFS client hung

[ Edited ]

Note: This isn't a zombie since they can be cleaned up.  This is just a hung process.

Advisor
Posts: 15
Registered: ‎03-15-2010
Message 7 of 11 (501 Views)

Re: NFS client hung

 

Is there a way to clean up the hung process without rebooting?

 

Our nfs server is a Windows 2003 server using Services For Unix 3.5. We have lots of file I/O operations  between the two servers. Once in a while(probably every 3-4 months), the Windows server would complain about corrupted files and needs to run chkdsk to repair some security descriptors (the stage 3 of the chkdsk).  Last time when this happened I had to reboot HP UX server to clean up the process.

 

Thanks,

Teresa

Acclaimed Contributor
Posts: 26,027
Registered: ‎03-06-2006
Message 8 of 11 (496 Views)

Re: NFS client hung

>Is there a way to clean up the hung process without rebooting?

 

In general no, not if kill -9 fails.  Otherwise you must get that I/O to finish.  Possibly when the NFS server reboots?

Basically you need a more robust NFS server.

 
 
 
Advisor
Posts: 15
Registered: ‎03-15-2010
Message 9 of 11 (494 Views)

Re: NFS client hung

NFS server were rebooted several times and the hung process are still there. Do you know any better products for gateway serverices between Unix  and Windows? (other than Windows SFU3.5)

 

Thanks,
Teresa

Advisor
Posts: 15
Registered: ‎03-15-2010
Message 10 of 11 (482 Views)

Re: NFS client hung

I have two network interfaces on HPUX server. I dont't know if disabling the one (with hang process) and renable the interface would work or not.  (Got the idea from Dave's whitepaper...)

 

Tested on the other HPUX server, and it could umount the mount point after the network interface was disabled. (Don't know if it will work on the actual problem.)

 

What I will do:

 

ifconfig lanA down

umount /mountpoint

ifconfig lanA up

...

 

This way is less disruptive. I will report if this works or not.

 

Teresa

Advisor
Posts: 15
Registered: ‎03-15-2010
Message 11 of 11 (475 Views)

Re: NFS client hung

I did "ifconfig lanA down ", then tried to umount the stale mount point and it still couldn't umount.

 

Finally I gave up and did a clean reboot which solved the problem.

 

 

 

Dave and Dennis , thank you for your expertise and advice.  Every time I log on the forum, I knew I would learn something new.

 

 

Thanks,

Teresa

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.