Re: Deleting files oler than 20 minutes (2318 Views)
Reply
Frequent Advisor
Salman Ahmed Qureshi
Posts: 68
Registered: ‎01-28-2009
Message 1 of 16 (2,318 Views)
Accepted Solution

Deleting files oler than 20 minutes

Hi,
On my database server, archives are created in a file system very frequently and i move those archives to another file system so that this file system doesn't get full. My current job moves all archives to the other file system but some times it corrupts my archives because it also moves the archive currently being written and has not finished yet, so the solution in my mind is to move only archive files older than, let say 15 minutes so i will be sure that only finished archives are being moved to the other file system. Can anyone tell me how to edit my current script to accomplish this. Please see my script.
##########################################
date
for fstr in `find /odsdata1/archive/ods_* -type f -print`
do
echo "Moving $fstr"
mv $fstr /odsarchive2/backup/ods/archive/
done
date
echo "End of transfer"
######################################
Thanks

Salman
Please use plain text.
Honored Contributor
Jeeshan
Posts: 2,084
Registered: ‎11-05-2007
Message 2 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

find files older than 1 hr

# perl -MFile::Find -le 'find(sub{print $File::Find::name if -f $_ && -M _ <= 1/24},@ARGV)' /path
a warrior never quits
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,953
Registered: ‎03-06-2006
Message 3 of 16 (2,318 Views)

Re: Deleting files older than 20 minutes

[ Edited ]

You'll need to create a reference file with touch for 20 minutes ago. (Or use gnu find, -mmin +20.)

The problem with 20 minutes ago is that you need to do date arithmetic on touch. If now is 16:00, then the reference file would be:
touch 02091540 ref_file

You can also solve the reference file issue by creating it manually the first time then just use your sleep or cron to remember 20 minutes ago:
touch ref_file_new # for next time
for fstr in $(find /odsdata1/archive/ods_* -type f ! -newer ref_file); do
done
mv ref_file_new ref_file
sleep $(( 20 * 60 ))

I suppose you could put the touch at the bottom if the find and mv doesn't take too long.

Please use plain text.
Frequent Advisor
Salman Ahmed Qureshi
Posts: 68
Registered: ‎01-28-2009
Message 4 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi Dennis,
Thanks for your reply. I would be rather interested in, "find -mmin" command. Can you please edit my script to add this portion. If you think that i can't achieve 20 mins, you can make it 1 hour.
Thanks in advance

Salman
Please use plain text.
Exalted Contributor
Steven E. Protter
Posts: 33,806
Registered: ‎08-15-2002
Message 5 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Shalom Salman,

Example script:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=131801

How to overcome issues that may come up in such a script:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1227620

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,953
Registered: ‎03-06-2006
Message 6 of 16 (2,318 Views)

Re: Deleting files older than 20 minutes

[ Edited ]

>"find -mmin" command. Can you please edit my script to add this portion?

Unless you install GNU find, you can't. HP-UX's find only has the standard day granularity with -mtime. If you don't want to use GNU, you will have to use a reference file or use ahsan's perl solution.

>you can make it 1 hour.

That is still 1/24 too small. :-)

Please use plain text.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 7 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi Salman:

To find your candidates, use Perl like this (sorry Ahsan, you can't just copy and paste previous solutions):

# perl -MFile::Find -le '@ARGV=(".") unless @ARGV;find(sub{print $File::Find::name if -f $_ && -M _ >= 1/96},@ARGV)' /path

You can pass multiple /path arguments if you like. RUnning without any argument ('/path') infers the current directory.

The "1/96" is "1/96" of one day or one 15-miunute period of age.

Regards!

...JRF...

Please use plain text.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 8 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi (atain) Salman:

Actually, it is just as easy to do everything in one Perl script:

# cat ./mymover
#!/usr/bin/perl
use strict;
use warnings;
use File::Find;
use File::Copy;
my $dstdir='/odsarchive2/backup/ods/archive';
@ARGV = (".") unless @ARGV;
print STDERR "Begin @ ", scalar localtime, "\n";
find(
sub {
if ( -f $_ && -M _ >= 1 / 96 ) {
print STDERR "Moving '$_'\n";
move( $File::Find::name, $dstdir ) or die "$!\n";
}
},
@ARGV
);
print STDERR "Ended @ ", scalar localtime, "\n";
1;

...Again, pass the source directory name or "." or nothing at all to denote the current working directory for the source path.

Regards!

...JRF...
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,953
Registered: ‎03-06-2006
Message 9 of 16 (2,318 Views)

Re: Deleting files older than 20 minutes

[ Edited ]

>JRF: The "1/96" is one 15-minute period

You might want to use an expression so you don't have to explain it:
15 / (24 * 60)

Please use plain text.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 10 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi (again):

> Dennis: You might want to use an expression so you don't have to explain it:
15 / (24 * 60)

Yes, I agree, in this case that would be good. So change:

if ( -f $_ && -M _ >= 1 / 96 ) {

...to:

if ( -f $_ && -M _ >= ( 15 / ( 24 * 60 ) ) ) {

...which makes the 15-minute interval much clearer and allows leveraging further:

...
my $age=15
@ARGV = (".") unless @ARGV;
...
if ( -f $_ && -M _ >= ( $age / ( 24 * 60 ) ) ) {
...

Regards!

...JRF...
Please use plain text.
Esteemed Contributor
Arturo Galbiati
Posts: 830
Registered: ‎02-10-2003
Message 11 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi Salman,
all the above solution are fine for your problem, but you have to be sure tht the log file you are trying to move is not really in use by the database.
I suggest you to include in your script some Oracle command to shiwtch the log file in use (in sorry but now I don't rememeber those). IN this way you are sure that file older than 'x' min is really not in use.
Just my .02$.

HTH,
Art
Please use plain text.
Honored Contributor
likid0
Posts: 979
Registered: ‎03-01-2005
Message 12 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hy,

just wanted to leave you the zsh line for what you need:

ls -ltr **/*(.mm+15)


Windows?, no thanks
Please use plain text.
Honored Contributor
likid0
Posts: 979
Registered: ‎03-01-2005
Message 13 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Ops sorry you actually wanted to delete them:

If files, dirs and subdirs then:
rm -Rf **/*(.mm+15)
If only files:
rm -f *(.mm+15)
Windows?, no thanks
Please use plain text.
Honored Contributor
OldSchool
Posts: 3,372
Registered: ‎09-09-2004
Message 14 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

I saw "move", but the above deletes.....
Please use plain text.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 15 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

Hi:

> OldShool: I saw "move", but the above deletes.....

Well, not mine, the code even says "move" :-))

Regards!

...JRF...
Please use plain text.
Honored Contributor
OldSchool
Posts: 3,372
Registered: ‎09-09-2004
Message 16 of 16 (2,318 Views)

Re: Deleting files oler than 20 minutes

seems to be that fuser might suffice to check if one of the files you want to move is open, rather than assuming that at some fixed point in time that its done.

or, as one of the other posters noted, force oracle to switch archives. in such a case, build the list first, switch the file, then process everything in the list
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation