Re: Max arguments for rm command (1152 Views)
Reply
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 1 of 11 (1,182 Views)

Max arguments for rm command

Dear Admins,

How do i determine the maximum arguments allowed in rm command? i.e number of files allowed in rm command to delete at a time.

 

SERVER1:/home/sit>ls -lrt|wc -l
34813
SERVER:/home/sit>rm *
sh: /usr/bin/rm: The parameter list is too long.

Regards,

V2P

Please use plain text.
Honored Contributor
rariasn
Posts: 1,135
Registered: ‎08-09-1999
Message 2 of 11 (1,176 Views)

Re: Max arguments for rm command

Hi:

 


[E2BIG] The number of bytes in the new program'sargument list plus environment is greater
than the system-imposed limit. This limit isat least 5120 bytes on HP-UX systems.

 

man 2 exec

 

rgs,

Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,853
Registered: ‎03-06-2006
Message 3 of 11 (1,174 Views)

Re: Max arguments for rm command

[ Edited ]

>How do I determine the maximum arguments allowed in rm command?

 

You don't.  You have your plan A which is just "rm *".

Note: A closer estimate would be: { env; ls; } | wc -c

 

If you have a total length of all of the files about 2 Mb, then you switch to plan B:

find . -type -f -exec rm -f {} +

 

(Or you always use plan B.)

 

>This limit is at least 5120 bytes on HP-UX systems.


It is about 2 Mb and "-exec ... +" knows what it is, compared to dumb xargs(1).

Please use plain text.
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 4 of 11 (1,166 Views)

Re: Max arguments for rm command

Hello Denis,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?
Please use plain text.
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 5 of 11 (1,166 Views)

Re: Max arguments for rm command

Hello rariasn,
SERVER1:/home/sit>ls |wc -c
2022086
Bytes would be 2022086
SERVER1:/home/sit>ls |wc -l
31707
Here am able to delete the 31707 files with rm*. Is this the same calc u meant?
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,853
Registered: ‎03-06-2006
Message 6 of 11 (1,162 Views)

Re: Max arguments for rm command

[ Edited ]

> ls | wc -c 2022086
>Bytes would be 2022086
>Here am able to delete the 31707 files with rm *. Is this the same calc you meant?

 

I'm saying that 2 Mb may be too large to pass as arguments.

Please use plain text.
V2P
Occasional Visitor
V2P
Posts: 4
Registered: ‎12-27-2011
Message 7 of 11 (1,158 Views)

Re: Max arguments for rm command

But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

Please use plain text.
Honored Contributor
rariasn
Posts: 1,135
Registered: ‎08-09-1999
Message 8 of 11 (1,152 Views)

Re: Max arguments for rm command

Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,853
Registered: ‎03-06-2006
Message 9 of 11 (1,147 Views)

Re: Max arguments for rm command

>But here am able to delete the files with rm * command. Then what parameter manages the rm max args?

 

Ok, I found the limit is more like 2 Mb:

http://h30499.www3.hp.com/t5/System-Administration/usr-bin-cp-the-parameter-list-is-too-long/td-p/44...

$ getconf ARG_MAX
2048000

 

This is a fixed value.

Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,199
Registered: ‎05-29-2000
Message 10 of 11 (1,137 Views)

Re: Max arguments for rm command

[ Edited ]

The correct answers are:

 

  • There is no limit to the number of files that can be deleted, but
  • there is a limit to the number of characters on the command line

So for the specific example: rm *

The star * is a special character for the shell and for default environments, it will effectively rewrite the command line with rm plus the names of all the files in the current directory. If every file is exactly two letters (as in aa, ab, ac, ad...Aa, Ab, Ac...11, 12, 13,...) then about 666,000 files can be removed with the rm command (2 characters plus 1 space).

 

So it has nothing to do with the number of files, it has to do with the length of the resultant command line. It is not a good idea to think of managing massive numbers of files with simple file matching characters like "*".  The first consideration is what can you do to stop the madness of having thousands of files in one directory. HP-UX has no problems with thousands, even millions of files in one directory. But the fact that it can be done does not make it a good idea. Most of the massive directory problems I have seen have been due to bad designs trying to create a database with files rather than using a real database.

 

Now if you are forced to deal with a bad design that will not be fixed, the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O). Assuming your current directory has 1 million files, the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen, onto the floor and down the stairs...

 

Instead, you limit the listing to what you are really looking for. If you are looking for files that start with the characters abc and end with 123, then limit the match: ls abc*123 . If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

 

If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,853
Registered: ‎03-06-2006
Message 11 of 11 (1,131 Views)

Re: Max arguments for rm command

>the next best is to use command lines that will work and avoid those that fail or take enormous amounts of time (and disk I/O).

 

What's why plan B should be used.

 

>the first commands to avoid are simple ls and ll commands. The result will scroll off the top of your screen,

 

You can always redirect the output.

>If you need to remove all the files in a directory (with no subdirectories), then use rm -r mydirname and *not* rm mydirname/*

 

Right.  And copy the permissions/owners if needed to recreate the directory.

>If the directory has subdirectories, you'll have to be more creative by breaking the filenames into manageable groups.

 

Plan B doesn't have any issues with this, unless you don't want to remove subdirectories.  And then with the appropriate -prune, that will work too.

Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation