Re: Max arguments for rm command (1577 Views)
Reply
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 1 of 11 (1,583 Views)

Max arguments for rm command

Dear Admins,

How do i determine the maximum arguments allowed in rm command? i.e number of files allowed in rm command to delete at a time.

 

SERVER1:/home/sit>ls -lrt|wc -l
34813
SERVER:/home/sit>rm *
sh: /usr/bin/rm: The parameter list is too long.

Regards,

V2P

Honored Contributor
rariasn
Posts: 1,135
Registered: ‎08-09-1999
Message 2 of 11 (1,577 Views)

Re: Max arguments for rm command

Hi:

 


[E2BIG] The number of bytes in the new program'sargument list plus environment is greater
than the system-imposed limit. This limit isat least 5120 bytes on HP-UX systems.

 

man 2 exec

 

rgs,

Acclaimed Contributor
Dennis Handly
Posts: 25,284
Registered: ‎03-06-2006
Message 3 of 11 (1,575 Views)

Re: Max arguments for rm command

[ Edited ]

>How do I determine the maximum arguments allowed in rm command?

 

You don't.  You have your plan A which is just "rm *".

Note: A closer estimate would be: { env; ls; } | wc -c

 

If you have a total length of all of the files about 2 Mb, then you switch to plan B:

find . -type -f -exec rm -f {} +

 

(Or you always use plan B.)

 

>This limit is at least 5120 bytes on HP-UX systems.


It is about 2 Mb and "-exec ... +" knows what it is, compared to dumb xargs(1).

V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 4 of 11 (1,567 Views)

Re: Max arguments for rm command

Hello Denis,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 5 of 11 (1,567 Views)

Re: Max arguments for rm command

Hello rariasn,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?
Acclaimed Contributor
Dennis Handly
Posts: 25,284
Registered: ‎03-06-2006
Message 6 of 11 (1,563 Views)

Re: Max arguments for rm command

[ Edited ]

> ls | wc -c 2022086
>Bytes would be 2022086
>Here am able to delete the 31707 files with rm *. Is this the same calc you meant?

 

I'm saying that 2 Mb may be too large to pass as arguments.

V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 7 of 11 (1,559 Views)

Re: Max arguments for rm command

But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

Honored Contributor
rariasn
Posts: 1,135
Registered: ‎08-09-1999
Message 8 of 11 (1,553 Views)

Re: Max arguments for rm command

Acclaimed Contributor
Dennis Handly
Posts: 25,284
Registered: ‎03-06-2006
Message 9 of 11 (1,548 Views)

Re: Max arguments for rm command

>But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

 

Ok, I found the limit is more like 2 Mb:

http://h30499.www3.hp.com/t5/System-Administration/usr-bin-cp-the-parameter-list-is-too-long/td-p/44...

$ getconf ARG_MAX
2048000

 

This is a fixed value.

Honored Contributor
Bill Hassell
Posts: 14,220
Registered: ‎05-29-2000
Message 10 of 11 (1,538 Views)

Re: Max arguments for rm command

[ Edited ]

The correct answers are:

 

  • There is no limit to the number of files that can be deleted, but
  • there is a limit to the number of characters on the command line

So for the specific example: rm *

The star * is a special character for the shell and for default environments, it will effectively rewrite the command line with rm plus the names of all the files in the current directory. If every file is exactly two letters (as in aa, ab, ac, ad...Aa, Ab, Ac...11, 12, 13,...) then about 666,000 files can be removed with the rm command (2 characters plus 1 space).

 

So it has nothing to do with the number of files, it has to do with the length of the resultant command line. It is not a good idea to think of managing massive numbers of files with simple file matching characters like "*".  The first consideration is what can you do to stop the madness of having thousands of files in one directory. HP-UX has no problems with thousands, even millions of files in one directory. But the fact that it can be done does not make it a good idea. Most of the massive directory problems I have seen have been due to bad designs trying to create a database with files rather than using a real database.

 

Now if you are forced to deal with a bad design that will not be fixed, the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O). Assuming your current directory has 1 million files, the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen, onto the floor and down the stairs...

 

Instead, you limit the listing to what you are really looking for. If you are looking for files that start with the characters abc and end with 123, then limit the match: ls abc*123 . If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

 

If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

Acclaimed Contributor
Dennis Handly
Posts: 25,284
Registered: ‎03-06-2006
Message 11 of 11 (1,532 Views)

Re: Max arguments for rm command

>the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O).

 

What's why plan B should be used.

 

>the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen,

 

You can always redirect the output.

>If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

 

Right.  And copy the permissions/owners if needed to recreate the directory.

>If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

 

Plan B doesn't have any issues with this, unless you don't want to remove subdirectories.  And then with the appropriate -prune, that will work too.

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.