Re: Incorrect file size reported (138 Views)
Reply
Frequent Advisor
user001
Posts: 84
Registered: ‎11-30-2010
Message 1 of 12 (138 Views)
Accepted Solution

Incorrect file size reported


Hello,
I have the following disk layout
vg00
lvol1
/mnt1
vg01
lvol2
/mnt2

I'm trying to copy all the data from /mnt1 to /mnt2, i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2. All files copied from /mnt1 to /mnt2 seem to have this problem.

11.31
vxfs v6
20GB lv's

Any ideas. This is driving me insane, /mnt1 is about 10GB but because its making them large on /mnt2 i'm unable to copy the entire contents.

Any help would be great.
Thank you.
Please use plain text.
Honored Contributor
Viktor Balogh
Posts: 1,007
Registered: ‎03-15-2009
Message 2 of 12 (138 Views)

Re: Incorrect file size reported

[ Edited ]

how did you copy it? I think these are sparse files, that's why you end up with much bigger files on /mnt2.

use dd or pax to move them, see also this thread:

http://h30499.www3.hp.com/t5/System-Administration/How-can-I-copy-a-sparse-file/m-p/4859869#M396243

****
Unix operates with beer.
Please use plain text.
Honored Contributor
Viktor Balogh
Posts: 1,007
Registered: ‎03-15-2009
Message 3 of 12 (138 Views)

Re: Incorrect file size reported

[ Edited ]

here is a way of identifying a sparse file:

http://h30499.www3.hp.com/t5/System-Administration/Find-sparse-file/m-p/3875418#M279127

****
Unix operates with beer.
Please use plain text.
Frequent Advisor
user001
Posts: 84
Registered: ‎11-30-2010
Message 4 of 12 (138 Views)

Re: Incorrect file size reported


i tried dd but it never ended :(

dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

this seem to go forever, the new partition is slightly larger but that shouldn't be a problem. Is there a way to monitor dd?

i have tried cpio, mv, cp, fbackup, snapshot + those commands (in case).

i wasn't aware of pax, i noticed someone else commented that it doesn't support files larger than 2GB?

Thanks for the help.
Please use plain text.
Honored Contributor
Viktor Balogh
Posts: 1,007
Registered: ‎03-15-2009
Message 5 of 12 (138 Views)

Re: Incorrect file size reported

> i tried dd but it never ended :(

or you just couldn't wait till the end of it. :)
you can also copy it on a file level like this:

# dd if=/mnt1/file1 of=/mnt2/file1 bs=1024

AFAIK dd can handle files >2gig
****
Unix operates with beer.
Please use plain text.
Frequent Advisor
user001
Posts: 84
Registered: ‎11-30-2010
Message 6 of 12 (138 Views)

Re: Incorrect file size reported


I left it run over night, i thought this would have been enough for dd.

I'll give pax a try first, this sounds like it could do the job. failing that i'll try dd again.

i'll update the thread / points tomorrow when i get to try it out.

thanks for the suggestions.
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,762
Registered: ‎03-06-2006
Message 7 of 12 (138 Views)

Re: Incorrect file size reported

>I've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Hmm. I wouldn't expect tar to be smart enough to create sparse files?

>I noticed someone else commented that it doesn't support files larger than 2GB?

No, pax(1) can support > 8 Gb. After all, after fbackup(1m) is obsoleted, pax is its replacement.
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,762
Registered: ‎03-06-2006
Message 8 of 12 (138 Views)

Re: Incorrect file size reported

>I tried dd(1) but it never ended :(
dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

Your blocksize is too small. Try bs=1024k
Please use plain text.
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 9 of 12 (138 Views)

Re: Incorrect file size reported

Hmmm.
From my testing, it seems that neither 'tar' , 'fbackup', nor pax support sparse files.
I'm surprised a little.

Just for grins, why don't you untar (watch out for absolute pathnames -- be sure to use the -C option) the tar archive into /mnt2 and see what the stuff *looks* like and actually *is*.


bv
"The lyf so short, the craft so long to lerne." - Chaucer
Please use plain text.
Honored Contributor
Hein van den Heuvel
Posts: 6,580
Registered: ‎05-19-2003
Message 10 of 12 (138 Views)

Re: Incorrect file size reported

Can we take a few steps back and get a clear problem statement with details?


>> I'm trying to copy all the data from /mnt1 to /mnt2,

Which tool(s) used for the copy? cp? details?

>> i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Best I know there is no tool called 'it' on HPUX, although that would be neat to maximize confusion!

So what tool did you use to report the size? ls? du? df? options used for those?
Did you try alternatives? What did they all report?

hth,
Hein.



All files copied from /mnt1 to /mnt2 seem to have this problem.
Please use plain text.
Frequent Advisor
user001
Posts: 84
Registered: ‎11-30-2010
Message 11 of 12 (132 Views)

Re: Incorrect file size reported


Viktor Balogh is the champion, 100% sparse files they were and pax worked a treat.

I only half expected it to work but when it did wow was i happy.

thanks for everyones help much appreciated.

cheers.

Please use plain text.
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 12 of 12 (132 Views)

Re: Incorrect file size reported

Wow.
That's interesting.
When I tested it, as I said, I could not get 'pax', 'tar', or 'fbackup' to *CREATE* an archive of sparse files that was small.
A 'du' of the archive was always as large as the sparse files showed in 'll', not 'du'.

I did the testing on both 11.23 and 11.31.

HOWEVER, when I *extracted* from the 'pax' archive, it DID restore a sparse file !!! ('tar' did not), even though the archive itself was large !?!?


So, I'm wondering how your archive got created as such a small size in the first place.

bv


((((((((((((((((((((((((((((((

Here is my testing:

create a sparse file:

## dd if=/dev/zero of=/tmp/sparsefile seek=1024000 count=1 bs=1024


Check sizes:

## ll /tmp/sparsefile ; du -k /tmp/sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 08:41 /tmp/sparsefile
8 /tmp/sparsefilep/sparsefile

So, 'll' reports 1G, while the actual 'du' size is 8K


Create archives with 'pax', 'tar', 'fbackup' :

## (cd /tmp ; pax -w -f pax1 sparsefile)
## (cd /tmp ; tar cf tar1 sparsefile)
## (cd /tmp ; fbackup -f fb1 -i sparsefile)


Check sizes:

## ( for F in /tmp/pax1 /tmp/tar1 /tmp/pax1
do ll $F; du -k $F
done
)
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:11 /tmp/tar1
1024016 /tmp/tar1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1

All archives show 1G with BOTH 'll' and 'du'.

Now, extract:

## mkdir /tmp/td
## (cd /tmp/td ; pax -rv < /tmp/pax1 ; ll sp* ; du -sk sp* ; rm sp*)
USTAR format archive
sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 09:09 sparsefile
8 sparsefile

So, the recovered file *is* sparse, even though the archive was 1G !?!?!?!

)))))))))))))))))))))))))))))))))))))))))
"The lyf so short, the craft so long to lerne." - Chaucer
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation