Re: Ignite backup find_files command fails status 141 (2128 Views)
Reply
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 1 of 12 (2,128 Views)
Accepted Solution

Ignite backup find_files command fails status 141

Greetings,
We run and Ignite backup of our 3 unix servers each night. Last night we recieved these error(s) on all three servers:
WARNING: The pax command returned a non-zero exit status (exit status 1).
ERROR: The find_files command failed (exit status 141).

From what I understand a single occurance of this may indicate a bad tape or a drive problem. Three in one night makes me think otherwise. Here's the configuration:
HP-UX 11.23 all three servers.
DDS3 tape drives (one for each server)
Host machine is RX6600 with HP-UX Host.
Each is a VM guest on the RX6600 (again all three running UX 11.23)
Ignite Version C.7.4.155 (x3)

I've run tar backups to each of the drives this morning with no problem. I ran Ignite backups to all three servers with no problem every other morning this week.

No errors that I can see in syslog.log on any of the servers (including the VM host). What should I be looking at next, and after that?
Please use plain text.
Respected Contributor
YAQUB_1
Posts: 614
Registered: ‎04-24-2006
Message 2 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

Hi Frank,

Pls go through the below thread may be it will help 4 U.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1215712559643+28353475&...

BR/
Yaqub
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 3 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

The note you sent was imformative but focused on exit status 1. Our Ignite backup didn't finish with warnings but with failure that I assume is due to the "find_files command failed (exit status 141)". Attached is a standard out/error file from one of the servers.
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,958
Registered: ‎03-06-2006
Message 4 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

>Attached is a standard out/error file from one of the servers.

That has:
* Archiving contents of lex06 via tar image to local device /dev/rmt/0mn.
pax: write : No such file or directory
pax: A write error has occurred.
WARNING: The pax command returned a non-zero exit status (exit status 1).

It appears there is a file "write" that isn't there. Like:
pax -w -f foo.tar write

Do you have such a file named "write"? Or is ignite messing up and putting a bogus "write" on the pax command line?
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 5 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

There is a file called "write" on each guest and the host VM. Listing for that file follows:
[/] lex08-ROOT->ll /usr/bin/write
-r-xr-xr-x 1 bin bin 23852 Mar 26 2004 /usr/bin/write

[/roots] uxvmadm1-ROOT->file /usr/bin/write
/usr/bin/write: ELF-32 executable object file - IA64

I would suppose it belongs there and is used by Ignite and other processes. It has been part of the OS for all earlier versions of UX I've used.
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 6 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

I see. There's an implication that the PAX command or Ignite are trying to execute the write command but are possibly trying to execute with a corrupted PATH variable, or have the explicit /dir/path/write coded in the command, but for the wrong dir path. The script that our root cron runs to execute make_tape_recovery doesn't do anything to set or modify the PATH. I will modify that script with PATH=/usr/bin:$PATH and see if the performance improves.
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 7 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

Modifing Path didn't provide any change in the behavior. Still bombing at the end with:
pax: write : No such file or directory
pax: A write error has occurred.
WARNING: The pax command returned a non-zero exit status (exit status 1).
ERROR: The find_files command failed (exit status 141).

Also did a for file named ":" and came up empty. If you know of someone who may have an idea please send them here. Thanks
Please use plain text.
Honored Contributor
Hasan  Atasoy
Posts: 768
Registered: ‎06-05-2007
Message 8 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

hi frank ;

are you using dds2 cartridges ?

please test with a larger capacity cartridges.
it seems that cartridges full at some point and pax exit with an exit code.

Hasan
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 9 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

They ARE dds2 tapes. Is theimplication that the process has reached the end of the media and just errors out? Once upon a time if we reached the end of a tape during a make_tape_recovery it prompted that we needed to insert a new tape, didn't it? I could be thinking of sysback or mksysb from AIX but should be similar regardless of OS. We'll I've replaced with DDS3 tapes now and I'll see later today if that solves the problem. That is the best notion I never thought of so far.
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,958
Registered: ‎03-06-2006
Message 10 of 12 (2,128 Views)

Re: Ignite backup find_files command fails status 141

>Also did a for file named ":" and came up empty.

The name of the file is "write". If you do the pax command I showed, you get the error message as you got:
pax -w -f foo.tar write

As to how ignite gets that, I don't know. I suppose there could be unprintable chars? So use "find / -name "*write*".

>I would suppose it belongs there

Yes, that's write(1), not the one with your problem.

>PAX command or Ignite are trying to execute the write command

No, the fact that the word "write" was placed on the pax command line.
Can you look in these for "write":
/var/opt/ignite/recovery/2008-07-10,01:30/flist
/var/opt/ignite/recovery/2008-07-10,01:30/recovery.log
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 11 of 12 (2,123 Views)

Re: Ignite backup find_files command fails status 141

Mr. Handly I thank you much for your suggestions and I owe you some points for trying. You were at the same disadvantage I was on this in that you couldn't actually SEE the DDS tapes being used for the backup. The machine room/data center is across town for me and in cyberspace to you which put us both in the wrong perspective. The problem is indeed media after all. I was trying to get the Ignite backups for our new "itanium-integrity" servers (I left that out didn't I, silly me) on the old tapes we used to run our "PA Risc" Ignite backups on. That rotation of tapes included both DDS2 and DDS3 tapes. vg00 on the new servers have enough extra data and larger files stored on them that, even with compression, do not fit on a DDS2 tape. After this realization I switched tapes out yesterday (made them all DDS3) and I now have 3 successful Ignite tape backups of the 3 servers. Special thanks to a back level support team at HP for getting me to check on that.
Please use plain text.
Advisor
Frank Copeland
Posts: 24
Registered: ‎11-21-2004
Message 12 of 12 (2,123 Views)

Re: Ignite backup find_files command fails status 141

Also thanks Hasan for your note. You nailed it as well as back level did. The tapes our operator placed in each of the drives on Wednesday night were all DDS2 (small capacity) tapes. If I worked in the same building as the server I would have easily walked into the data center and pulled out the tapes to examine. Then seeing all 3 were OLD tapes I probably would have just changed them out for the sake of change. But I may not have thought about the extra capacity being stored from IA servers and just thought it was a media problem. Problem would have come back to bite again. No more. I will be flushing all DDS2 tapes from our media pool now.
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation