HP ux root filesystem full. (3399 Views)
Reply
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 1 of 23 (3,399 Views)
Accepted Solution

HP ux root filesystem full.

Hi

In my server Hp9000-K series Hp ux 11 is installed .
The root filesystem is in a logical volume. Now the problem is that the root file system is showing 97 % .Its causing the warning message as "root full" & the application like "exceed" is exiting frequently.
Can someone tell me how to extend the root file system size in this condition without disturbing the existing OS & data on it.

Thanks in advance
Sanjay
Please use plain text.
Honored Contributor
Robert-Jan Goossens
Posts: 7,384
Registered: ‎04-04-2000
Message 2 of 23 (3,398 Views)

Re: HP ux root filesystem full.

Hi Sanjay,

The first thing you could check is if someone made an error using a tar command.

# ll /dev/rmt

check if there is a large file omn

check for other large files in /

# find / -xdev -size +1000 -exec ll {} \;

Best regards,
Robert-Jan
Please use plain text.
Honored Contributor
Joseph Loo
Posts: 1,981
Registered: ‎10-23-2000
Message 3 of 23 (3,398 Views)

Re: HP ux root filesystem full.

hi,

u may like to look for any core files or unwanted files in the "/" filesystem, cause there is no way of increasing the / filesystem without rebooting (that includes /stand and swap filesystem).

usually, we tend not to dump any unnecessary files in the root directory.

regards.
what you do not see does not mean you should not believe
Please use plain text.
HP Pro
melvyn burnard
Posts: 6,068
Registered: ‎04-06-1997
Message 4 of 23 (3,398 Views)

Re: HP ux root filesystem full.

well it might be quicker and easier to look for what may be filling up / file system.

For example, check in /dev and make sure no-one has accidentally created or put a large normal file there. A simple example is where someone does a tar or other such command and specifies an incorrect device, e.g. /dev/rmt/om instead of 0m.
The next place to look is if you do not have /var/adm/crash as a separate file system, do you have crash dumps in there? If yes, then unless they are relevant, remove them.

Another favourite is /var/adm/sw/save, where older versions of patches are saved to be used in the event of you wishing to revert to a previous patch.

Just a few thoughts.
My house is the bank's, my money the wife's, But my opinions belong to me, not HP!
Please use plain text.
Esteemed Contributor
baiju_3
Posts: 337
Registered: ‎01-14-2005
Message 5 of 23 (3,398 Views)

Re: HP ux root filesystem full.


Also ,

if /var is not a seperate file system check,

/var/adm/btmp
/var/adm/crash


Thanks,
BL.
Good things Just Got better (Plz,not stolen from advertisement -:) )
Please use plain text.
Frequent Advisor
Betty Fessler
Posts: 74
Registered: ‎06-06-2001
Message 6 of 23 (3,398 Views)

Re: HP ux root filesystem full.

Another option is to commit patches if the system is stable.
Please use plain text.
Honored Contributor
Hoang Chi Cong_1
Posts: 1,464
Registered: ‎07-29-2004
Message 7 of 23 (3,398 Views)

Re: HP ux root filesystem full.

And after you do everything to reduce the root filesystem but the trouble persit,
You have to extend root volume in single user mode. (Recommend make a full system backup before extend)
Regard,
HoangChiCong
Looking for a special chance.......
Please use plain text.
Respected Contributor
generic_1
Posts: 533
Registered: ‎10-09-2002
Message 8 of 23 (3,398 Views)

Re: HP ux root filesystem full.

You might want to keep a session open to that system as root until you get the problem resolved. That is a pretty full. If it fills up you are gonna have extra reboots :). Make tape recoverying with the interactive option is a easy way to rebuild the system and choose the new correct filesystem sizes/layouts too assuming your disks have enough space to have that flexability. Good Luck.
Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,199
Registered: ‎05-29-2000
Message 9 of 23 (3,398 Views)

Re: HP ux root filesystem full.

In your case, root filesystem does NOT have just OS files and directories. The reason is that an 11.00 version of HP-UX needs about 50 to 75 megs--no more. If your filesystem has a lot more in it, there have been one or more mistakes.

The first is the size of /dev. Do this:

du -ks /dev

It should report less than 200k, more like 80k. If it is MUCH larger then someone has misspelled /dev/null (ie, /dev/rmt/om, /dev/nul or /dev/null2 or soemthing similar) and it would have to be root or a root process (since /dev is 755 permissions). There are no ordinary files in /dev:

find /dev -type f -exec ll {} \;

If there are, move or remove them from /dev. Then there may be application of developer junk left in the / directory 'because it's easy to remember /' These directories do not belong in /...move them. And of course there may be some logfile (NEVER allow logfiles in /) that is growing. Find the culprit(s) by looking for big directories (NOT big files):

du -kx / | sort -rn | head -20

It should look something like this:

69232 /
41080 /etc
38448 /sbin
26680 /etc/vx
21656 /etc/vx/type
10160 /etc/opt
8616 /etc/vx/type/static
6344 /etc/vx/type/gen
5904 /sbin/fs
5400 /etc/opt/resmon

The numbers are in kbytes. These are all OS directories. I suspect that there are other directories at the top of your list that need to be moved. Or there are very large files in / which you can see with:

ll / | sort -rnk5 | head -20

NOTE: You root HOME directory is probably in / (very bad idea). Create a new location for root's HOME (perhaps /home/root) and move all the ordinary files to that location. Find ordinary files with:

ll / | grep ^-

NOTE: Some of those files may be part of some developer's package or a vendor's package. If they are and they are quite large, use symbolic links to move them to /opt.

Enlarging the / directory requires a re-installation (or Ignite/UX which is the same thing) and is normally not needed by managing the / space carefully.
Please use plain text.
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 10 of 23 (3,398 Views)

Re: HP ux root filesystem full.

Hi to all,

I will let you know the findings of the suggestions.

Thanks a lot
Sanjay
Please use plain text.
Honored Contributor
Nguyen Anh Tien
Posts: 923
Registered: ‎05-20-2004
Message 11 of 23 (3,400 Views)

Re: HP ux root filesystem full.

root (/) is unable to extend even you are in single mode.
The best way you can do is finding big file or core file inside this file system
1. find core file:
find / -name core -exec ll {} \;
2. find all big file or folder inside one folder
#du -sk *|sort -nr
Regard.
P/S: pls do not forgot to assign point to each solution.
tienna
HP is simple
Please use plain text.
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 12 of 23 (3,400 Views)

Re: HP ux root filesystem full.

Dear all,

1) There is no any unwanted file in the /dev directory.
2)checked /var no any big file , actually /var is on the separate vol.
3) The vol details are as ;
*Volgrop : vg00
*Physical volumes for this group : /dev/dsk/c0t6/d0
/dev/dsk/c0t5/d0
/dev/dsk/c0t4/d0

*Logical volumes on the group : /dev/vg00/lvol1 to /dev/vg00/lvol10

* lvol on /dev/dsk/c0t6d0
/ on /dev/vg00/lvol3
/stand on /dev/vg00/lvol1

* lvol on /dev/dsk/c0t4d0
/var on /dev/vg00/lvol6
/var/tmp on /dev/vg00/lvol9
.
.
.
.
and other data partitions on the remaining lvol.

Still the / size is showing 97% .

Is it possible using any Hp tool/software to online extension of the root partition without disturbing the exsting data on the volumes ?

Thanks
Sanjay


Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,199
Registered: ‎05-29-2000
Message 13 of 23 (3,400 Views)

Re: HP ux root filesystem full.

> Is it possible using any Hp tool/software to online extension of the root partition without disturbing the exsting data on the volumes ?

Not possible. The first 3 volumes on your boot disk must be contiguous (cannot be extended). If you don't find the files and directories that do not belong in /, you have no choice but to create an Ignite/UX backup for VG00, then restore it interactively and make / bigger.

However, before you go to all that trouble, can you post the results of this command:

du -kx / | sort -rn | head -20

If /sbin and /etc are not at the top of your list, use this command in the largest directories:

ll /largedir | sort -rnk5 | head -20

This will show the largest files in those rogue directories.
Please use plain text.
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 14 of 23 (3,400 Views)

Re: HP ux root filesystem full.

Hi
1)Please see the output of the
#du -kx / |sort -rn |head -20

it looks the /sbin / etc are on the top .

2) I have found two log files as /etc/rc.log
/etc/rc.log.old

I have moved these files to other location. with these it increases the no of files available in / but still the percenrage size remains 97% .
Is it Ok? or revet back these files ?

Thanking you
Sanjay

Please use plain text.
Honored Contributor
Kent Ostby
Posts: 1,836
Registered: ‎09-26-2000
Message 15 of 23 (3,400 Views)

Re: HP ux root filesystem full.

/sbin looks a bit bigger than one would expect.

Can you post the output from:

find /sbin -size +1000000c

This will be files greater than 1 megabyte.

"Well, actually, she is a rocket scientist" -- Steve Martin in "Roxanne"
Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,199
Registered: ‎05-29-2000
Message 16 of 23 (3,400 Views)

Re: HP ux root filesystem full.

Yes, /sbin is way too largte, by about 20megs. The problem is located in /sbin/init.d (it's the largest diectory in /sbin and should only be about 1000Kb (1 meg) NOT 28Megs!! DO this to find the bad file(s):

ll /sbin/init.d | sort -rnk5 | head -20

The very largest files in this directory should be less than 40Kb. Somone has dropped a giant file (or several) in this directory.

NOTE: rc.log and rc.log.old are always in the /etc directory. When you shutdown and reboot, these files will bve recreated. If these are very large (larger than 20-40Kb) then read them and look for problems. rc.log is the bootup log and rc.log.old is the shutdown log.
Please use plain text.
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 17 of 23 (3,400 Views)

Re: HP ux root filesystem full.

Hi ,
Please find the output of the commands,

Kindly suggest

Thanks
sanjay
Please use plain text.
Honored Contributor
Stephen Keane
Posts: 1,155
Registered: ‎11-24-2004
Message 18 of 23 (3,400 Views)

Re: HP ux root filesystem full.

Lose the nohup.out to start with!
Please use plain text.
Honored Contributor
RikTytgat
Posts: 432
Registered: ‎07-03-2000
Message 19 of 23 (3,400 Views)

Re: HP ux root filesystem full.

It's quite obvious: remove the file /sbin/init.d/nohup.out

It hasn't been changed recently, so the nohupped process has probably exited.

Rgds,
Rik
Please use plain text.
Valued Contributor
Anthony Lennan
Posts: 88
Registered: ‎07-10-2003
Message 20 of 23 (3,400 Views)

Re: HP ux root filesystem full.

You should remove nohup.out. That is just a log file that has been created when some one has used the nohup command.

Moving it or removing it should resolve help resolve your disk space issues.

Rgds,
Anthony
Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,199
Registered: ‎05-29-2000
Message 21 of 23 (3,398 Views)

Re: HP ux root filesystem full.

As suggested, the problem is with the nohup.out command. Someone ran a program using the nohup command, probably in the background:

nohup some_program &

and this program produced an immense amount of data. Since this file is in the /sbin/init.d directory, it was very likely cause by a locally developed startup script that was improperly designed.

The /sbin/init.d directory is for startup scripts that start processes and the process runs by itself, in the background. While the above commands (nohup and &) will do this, processes that are to run forever at startup must produce *NO* output to stdout and stderr. The reason is that there is no terminal associated with a process that is to run by itself. Instead, any output should be directed to a logfile, ideally loacted in /var/adm with the rest of the logfiles.

Take a look at the contents of nohup.out and you'll likely recognize the program or script that produced this giant file. Then move it or remove it and you'll have avoided reloading you operating system because of a full / disk.
Please use plain text.
Occasional Advisor
sanjay_20
Posts: 8
Registered: ‎02-15-2004
Message 22 of 23 (3,398 Views)

Re: HP ux root filesystem full.

Hi ,

The File " nohup.out" is moved to other filesystem & now the root size is around 70%.
This avoided to resize the root filesystem.

Thanks to all of you for your valuable suggestions.Specially my very thanks to Bill Hassel for his valuable suggestions & detailed informatiion provided for my queries.

with thanks again !!
Sanjay
Please use plain text.
Honored Contributor
Michael Steele_2
Posts: 4,934
Registered: ‎08-14-2002
Message 23 of 23 (3,197 Views)

Re: HP ux root filesystem full.

this is nonesense - /var is not apart of / where the problem is

Support Fatherhood - Stop Family Law
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation