Re: Powerfull Find (611 Views)
Reply
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 1 of 33 (611 Views)
Accepted Solution

Powerfull Find

Dears,

Good day,

I have an issue with find command I would like to find files older than 2 days and move them to specific location. The issue is I want to build the exact same tree in the destination directory.

For example, I want to search /var/LOGS and move anything older than two days to /backup/var/LOGS. If there was a sub directories in the source I want to create it in the destination.

Regards,
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Honored Contributor
Jeeshan
Posts: 2,084
Registered: ‎11-05-2007
Message 2 of 33 (611 Views)

Re: Powerfull Find

Salam Ahmed

try this command

#find /var/LOGS -type f -mtime +7 -exec mv -r {} /backup/var/LOGS \;
a warrior never quits
Honored Contributor
Jeeshan
Posts: 2,084
Registered: ‎11-05-2007
Message 3 of 33 (611 Views)

Re: Powerfull Find

mistake

#find /var/LOGS -type f -mtime +2 -exec mv -r {} /backup/var/LOGS \;
a warrior never quits
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 4 of 33 (611 Views)

Re: Powerfull Find

Hi:

# cd /var
# find ./LOGS -depth -mtime +2 -print | cpio -pudlvm /backup/var

...will *copy* and replicate the directory/file structure in the destination directory. You will then need to remove what you don't want from the source tree in a separate pass.

Regards!

...JRF...
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 5 of 33 (611 Views)

Re: Powerfull Find

This solution does it in one pass but you'll need to write a customized script:

Starting from Ahsan's basic find:
find /var/LOGS -type f -mtime +2 -exec fancy_script {} +

Then create fancy_script:
#!/usr/bin/ksh
# Move files to /backup, keeping directory paths
for file in $*; do
mkdir -p /backup/$(dirname $file)
mv $file /backup/$file
done
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 6 of 33 (611 Views)

Re: Powerfull Find

Thanks for reply,

Dear ahsan, mv doesn't have -r option it's only in cp.

Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command. the script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command. This solution is very good but I prefer mv.

FANCY SCRIPT :

cp -r -p $1 $2
rm $1


FIND COMMAND :

find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

Any suggestion.

Regards,
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Respected Contributor
OFC_EDM
Posts: 740
Registered: ‎07-13-2004
Message 7 of 33 (611 Views)

Re: Powerfull Find

Thought I'd throw my hat in the ring with another idea.

Step 1. Copy the entire directory structure to the new location. Including the files you don't want.
"cp -r" or "cp -p -r" to preserve permisssions

Step 2. Nuke the files from the new structure which you don't want.

I'd test this out first by replacing rm with ls -l just to make certain....I don't have a system to test this on today and this is from memory.

Example:
Find files in current directory only older than June 3rd 2008

1. Create reference file
touch -t 200806030000 /tmp/june3.ref
Format of the numeric is YYYYMMDDHHMM
- use some logic in the script to calculate proper value to represent 2 days agos. I like this because you can control it down to the minute.

2 Run the command
find ./loc/A/* ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

If you like it then stick this in a script to automate.

I'll try to find my script which has all the little things like calculating the date etc for the touch command. But it'll be a few days.

Cheers
The Devil is in the detail.
Respected Contributor
OFC_EDM
Posts: 740
Registered: ‎07-13-2004
Message 8 of 33 (611 Views)

Re: Powerfull Find

I'll add one more option.

The previous example I put in goes into sub-directories.

However if at some point you only want to process the current directory stick in the -prune argument

find ./loc/A/* -prune ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

Regards
The Devil is in the detail.
Exalted Contributor
Steven E. Protter
Posts: 33,806
Registered: ‎08-15-2002
Message 9 of 33 (611 Views)

Re: Powerfull Find

Shalom,

A word of caution.

All it takes is about 4 finds starting out at root to totally bring even a powerful system to its knees.

Use carefully, preferably one at a time.

Very interesting thread.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 10 of 33 (611 Views)

Re: Powerfull Find

>I have this idea already, creating a script and calling it from find command. The script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command.

My script already does mv(1). What's wrong with it? It will only create the needed parts of the tree.

>find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

If you are changing the path in your target, then you'll need to fiddle with my script a little:
find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT /backup/var/LOGS-DATE {} +

FANCY_SCRIPT:
#!/usr/bin/ksh
TARGET=$1
shift
for file in $*; do
mkdir -p $TARGET/$(dirname $file)
mv $file $TARGET/$file
done
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 11 of 33 (611 Views)

Re: Powerfull Find

Hi (again):

> Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command.

Well, that's exactly what Dennis suggested originally to you! I presented an alternative, less direct answer to your original query (using 'find' and a pipe to 'cpio'). I think you have everything you need to make your decision.

Regards!

...JRF...
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 12 of 33 (611 Views)

Re: Powerfull Find

Hi all, and thanks for your all ideas,

Dear Kevin, I would like to ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec ?

I think I can solve this by moving the all directory to temporary directory and then find all the files newer than certain date as Kevin suggest and the returen those back with there directories to source directory. doing this I can save a lot of I/O because I'm dealing with more than 2,500,000 files.

-----------------

Again Kevin, thank for -prune option, I will write tomorowo script that have to go to every directory and count the number of files. So, I think this can help me with find.

-----------------

Dear JRF, I will read more about cpio, and feedback soon.

Thanks all.
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 13 of 33 (611 Views)

Re: Powerfull Find

>ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec?

The opposite of newer is "NOT newer". Which is older or the same time.

>I think I can solve this by moving the all directory to temporary directory

What's wrong with my "mv" solution? (You can still use -newer if you want.)
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 14 of 33 (611 Views)

Re: Powerfull Find

Hi folks,

I would like to ask about -mtime option.

In case you need to take all the files that were modified more than to days you need to use "-mtime +2". But this actually not finding all the files. Is this happened with you guys?

I have to put another find with -mtime 2 option to achieve my goal.

Any ideas ?
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 15 of 33 (611 Views)

Re: Powerfull Find

>I have to put another find with -mtime 2 option to achieve my goal.

What's your goal? -mtime +2 should be all files modified more than 2*24 hours ago.
Have you read find(1)?
http://docs.hp.com/en/B2355-60130/find.1.html
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 16 of 33 (611 Views)

Re: Powerfull Find

Hi Dennis,

I want to know the difference between -mtime 1 & -mtime +1. In Sun +1 mean find all files that modified time is more than 24 hour.

Look at this:

touch -t 0810250000 file
touch -t 0810260000 file1
touch -t 0810270000 file2
touch -mt 0810270000 file3
touch -mt 0810260000 file4

#ls :

Oct 25 00:00 file
Oct 26 00:00 file1
Oct 26 00:00 file4
Oct 27 00:00 file2
Oct 27 00:00 file3

# find . -mtime +1
./file
# find . -mtime 1
./file1
./file4

----

I think "find . -mtime +1" suppose to display "find . -mtime 1" result.

Thanks
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Honored Contributor
Aneesh Mohan
Posts: 804
Registered: ‎07-02-2007
Message 17 of 33 (611 Views)

Re: Powerfull Find

Hi Ahmed,


+n means more than n, -n means less than n, and n means exactly n.

here it is n * 24 hours.


Aneesh
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 18 of 33 (611 Views)

Re: Powerfull Find

Hi,

So why the first command didn't find ./file1 , ./file4 files ?

Regards,
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Honored Contributor
Aneesh Mohan
Posts: 804
Registered: ‎07-02-2007
Message 19 of 33 (611 Views)

Re: Powerfull Find

Hi Ahmad,

Try #find . -mtime +0 you will get it .

fyi:-
True if the file modification time subtracted
from the initialization time is n-1 to n
multiples of 24 h. The initialization time
shall be a time between the invocation of the
find utility and the first access by that
invocation of the find utility to any file
specified in its path operands.


Aneesh

Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 20 of 33 (611 Views)

Re: Powerfull Find

>So why the first command didn't find ./file1, ./file4 files?

How can we tell if you don't tell us what time it is?
From find(1) there is a 24 hour fuzziness in -mtime n, perhaps there is for +n?
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 21 of 33 (611 Views)

Re: Powerfull Find

find(1) works two ways depending if UNIX95 is exported. The find(1) describes UNIX95, which isn't the default.
time is now 27:1800
25:0000 2*24:1800 26:0000 1*24:1800 27:0000 0*24:1800
file X file1 X file2 NOW
Default: takes time / day and truncates
2 X 1 X 0
+0 X +0 X X
+1 X X X -1

UNIX95: takes time in seconds and compares N thru N-1 as =, - as < N-1, + as > N
file X file1 X file2 NOW
3 X 2 X 1
+0 X +0 X +0
+1 X +1 X X
+2 X X X -2
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 22 of 33 (611 Views)

Re: Powerfull Find

If you are happy with the answers you have gotten, please read the following about assigning points:
http://forums.itrc.hp.com/service/forums/helptips.do?#33
Super Advisor
AZayed
Posts: 391
Registered: ‎04-12-2008
Message 23 of 33 (611 Views)

Re: Powerfull Find

Hi Dennis,

I didn't understand your last example, can you explain it a little.

Thanks
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Acclaimed Contributor
Dennis Handly
Posts: 25,290
Registered: ‎03-06-2006
Message 24 of 33 (611 Views)

Re: Powerfull Find

>can you explain it a little.

Which parts? How find(1) works with and without UNIX95?

Default:
25:0000 2*24:1800 26:0000 1*24:1800 27:0000 0*24:1800

This is a timeline, Oct 25, 0000; a time 2 days ago at 1800; then Oct 26, ... Oct 27, then "now at Oct 27, 1800.
Default: takes time / day and truncates

With UNIX95, it does this fuzzy compare that is documented in find(1).
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 25 of 33 (611 Views)

Re: Powerfull Find

Hi (again):

> Dennis: With UNIX95, it does this fuzzy compare that is documented in find(1).

And just where in the HP-UX manpages did you find [ no pun intended ] that? I can't seem to locate a reference to either XPG4 or UNIX95 that points at this logic. The GNU Linux manpages are a bit more explicit in stating that "... When find figures out how many 24-hour periods ago the file was last accessed, any fractional part is ignored..."

Regards!

...JRF...

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.