Re: Using "touch" command to create large files?? Is it possible?? (721 Views)
Reply
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 1 of 20 (721 Views)
Accepted Solution

Using "touch" command to create large files?? Is it possible??

Is it possible to use the touch comand to create large files? Like 50 GIG size files?

Thanks,
Angie
Outstanding Contributor
Pete Randall
Posts: 16,205
Registered: ‎11-03-1996
Message 2 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

No, touch, by definition, creates a 0 byte file. Try prealloc.


Pete

Pete
Honored Contributor
Victor BERRIDGE
Posts: 2,576
Registered: ‎07-10-2000
Message 3 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Hi Angie,
No but you could use prealloc e.g
prealloc ./test 100000000 >> /tmp/testperf2

All the best
Victor
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 4 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

I have never heard of that command "prealloc"... does this below create a 50 GIG file? Looks like a 100 meg file?

prealloc ./test 100000000 >> /tmp/testperf2

Thanks!
Angie
Honored Contributor
Fred Ruffet
Posts: 1,366
Registered: ‎08-17-2003
Message 5 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

... or dd.

Regards,

Fred
--

"Reality is just a point of view." (P. K. D.)
Acclaimed Contributor
A. Clay Stephenson
Posts: 17,825
Registered: ‎07-16-1998
Message 6 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

No but you could use dd to create a large (though sparse) file very quickly.

dd if=/dev/zero bs=1k seek=52428799 count=1 of=mybigfile


If you do an ls -l mybigfile, it will display 50GB but a du -k will display only only 1 block because a sparse file has "holes". When a sparse file is read, the "holes" are filled in with NUL's so a copy of mybigfile will actually use 50GB of space.
If it ain't broke, I can fix that.
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 7 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

So in order to get the actual 50 GIG of space to be used up, I would have to then copy that file after creating it with dd? Let me explain what I am doing. I am purposely filling up a disk with some files to control the amount of disk space for some users. Yes I could use a quota but this is easier.

Angie
Outstanding Contributor
Pete Randall
Posts: 16,205
Registered: ‎11-03-1996
Message 8 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

$ prealloc filler 100
$ ll filler
-rw-rw-r-- 1 plr support 100 May 18 12:20 filler
$


Pete

Pete
Honored Contributor
john korterman
Posts: 1,117
Registered: ‎11-15-2000
Message 9 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Hi,
if you have all that space available, you could try something like this in the directory you want the file "big" to be put:
# prealloc big $((50*1024*1024*1024*1024))

regards,
John K.
it would be nice if you always got a second chance
Acclaimed Contributor
A. Clay Stephenson
Posts: 17,825
Registered: ‎07-16-1998
Message 10 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

No, a modified dd command will do this nicely although it will take some time. I thought you wanted something quick like touch.

dd if=/dev/zero bs=256k count=204800 of=mybigfile


and unlike prealloc, dd is a standard UNIX/Linix command
If it ain't broke, I can fix that.
Honored Contributor
Victor BERRIDGE
Posts: 2,576
Registered: ‎07-10-2000
Message 11 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Hi Angie,
I did put e.g. Yes it creates (or reseves) a 100Mb space Up to you to adapt to your needs
note that
prealloc ./test 100000000 creates test of about 100MB
>> /tmp/testperf2 creates a file of 100MB in /tmp
I use this to see how my disks work when I suspect an issue...
You would need then >50000000000

All the best
Victor
Outstanding Contributor
Pete Randall
Posts: 16,205
Registered: ‎11-03-1996
Message 12 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Here's another demo of doing this with prealloc:

$ bdf
File System......Mbytes....Used.. Avail %Used Mounted on
/dev/vg00/lvol3.... 200.... 169......31.. 85% /
/dev/vg00/lvol1.... 288......56.... 203.. 22% /stand
/dev/vg00/lvol8....3000.... 431....2550.. 14% /var
/dev/vg00/lvol7....3000....1067....1918.. 36% /usr
/dev/vg00/lvol6.... 200......40.... 159.. 20% /tmp
/dev/vg00/lvol5....2000....1625.... 372.. 81% /opt
/dev/vg00/lvol4....1600.... 582....1012.. 36% /home
$ prealloc filler 1000000000
$ bdf
File System......Mbytes....Used.. Avail %Used Mounted on
/dev/vg00/lvol3.... 200.... 169......31.. 85% /
/dev/vg00/lvol1.... 288......56.... 203.. 22% /stand
/dev/vg00/lvol8....3000.... 431....2550.. 14% /var
/dev/vg00/lvol7....3000....1067....1918.. 36% /usr
/dev/vg00/lvol6.... 200......40.... 159.. 20% /tmp
/dev/vg00/lvol5....2000....1625.... 372.. 81% /opt
/dev/vg00/lvol4....1600....1535......65.. 96% /home


Pete

Pete
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 13 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

WOW look at all the feedback I got an so quick! Thank you all of you.

I ended up using dd if=/dev/zero bs=1k seek=52428799 count=1 of=mybigfile and then I copied the file it created to another file, and it worked good. Didn't take too long either. I wanted to use this command too because I may use this on a Linux system here too.

Thanks for all the great feedback again and will post some points to all you brainiacs!

Have a nice day!
Angie
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 14 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Ok I really appreciate the efforts earlier but have ran into a snag. Though the files are getting created, 50 GIGs a piece , when I do a "df" command it says that still only 1% of the filesystem is in use. But of course when I do an "ll" it shows all the 50 gig files. Now someone mentioned if I copy this "dd" created file to another file it would fix this but this doesn't seem to be the case.

I did the following:

dd if=/dev/zero bs=2k seek=26214399 count=1 of=file1

See below:

-rw-r--r-- 1 root root 53687091200 2005-05-18 11:52 file1
-rw-r--r-- 1 root root 53687091200 2005-05-18 11:58 file2
-rw-r--r-- 1 root root 53687091200 2005-05-18 12:00 file3
-rw-r--r-- 1 root root 53687091200 2005-05-18 12:02 file4

But again with a "df" command still shows only 1% is in use when it should be like 50% in use with 20 of these 50-gig files.

Thanks for responding.
Angie
Acclaimed Contributor
A. Clay Stephenson
Posts: 17,825
Registered: ‎07-16-1998
Message 15 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

What does a du -k report in this directory? The cp command should have worked because on sparse files the read() system call silently pads the unallocated blocks with ASCII NUL's completely invisible to the application.

I just tried a 20MB sparse file and copied it and ls -l and du -k reported just as expected as did df -k.
If it ain't broke, I can fix that.
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 16 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

The following is shown and still with df it shows just 1% use.

host:/test/temp # du -k
1024005

Angie
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 17 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Ok I missed an earlier post... I am trying this...as it seems to be working... a bit modified as was mentioned. I don't understand though why changing the block size to 256k?

dd if=/dev/zero bs=256k count=204800 of=mybigfile

Angie
Acclaimed Contributor
A. Clay Stephenson
Posts: 17,825
Registered: ‎07-16-1998
Message 18 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Because it would take forever and a day to write 50GB in 1k blocks. You should set the blocksize to at least 64k for that operation and not much is to be gained above 256k; 256k is a reasonable chunk. Reads above 1MB are silently broken into (at most) 1MB chunks.

I still don't know why cp did not completely copy the sparse file. I would try "cat < file1 > file2".
If it ain't broke, I can fix that.
Regular Advisor
Angie_1
Posts: 184
Registered: ‎04-27-1998
Message 19 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

Ok thanks for explaining... I will use the 256k then as that makes more sense.

Thank you.. this is going to work out great!

Angie
Trusted Contributor
Amit Agarwal_1
Posts: 153
Registered: ‎06-02-2004
Message 20 of 20 (721 Views)

Re: Using "touch" command to create large files?? Is it possible??

somtimes I use tail command to create huge files.

$ echo "some dummy stuff" >> filename
$ tail -1000 -f filename >> filename

After few minutes of runnning it would grow to large size. You can press Ctrl-C to interrupt and check the size, if it is not yet 5GB, you can continue with the tail command.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.