Re: script to move files (230 Views)
Reply
Super Advisor
lawrenzo_1
Posts: 560
Registered: ‎06-06-2003
Message 1 of 14 (230 Views)
Accepted Solution

script to move files

i am trying to write a function that checks the date of files in a directory and if the files are not equal to today then create the dir:

set -x
for i in `ls -l |awk '{print $6,$7}' |uniq`
do
if [ $i != $TODAY ] ; then
DIR=$($i |sed s/' '//)
mkdir $DIR
fi
done
+ ls -l
+ uniq
+ awk {print $6,$7}
+ [ 06 != 20 Jul ]
+ + sed s/ //
+ 06


how can i parse "06 Jun" into [ $i != $TODAY ]as one string?

Manyb Thanks
hello
Honored Contributor
Jeff_Traigle
Posts: 1,354
Registered: ‎03-04-2004
Message 2 of 14 (230 Views)

Re: script to move files

I don't see where you set TODAY. Setting it as such, should do the trick for you:

TODAY=$(date +"%d %b")
--
Jeff Traigle
Honored Contributor
Enrico P.
Posts: 368
Registered: ‎06-10-2003
Message 3 of 14 (230 Views)

Re: script to move files

Hi,
try with this:

ls -l |awk '{print $6,$7}' |uniq|while read i
do
echo $i
done

Enrico
Honored Contributor
Enrico P.
Posts: 368
Registered: ‎06-10-2003
Message 4 of 14 (230 Views)

Re: script to move files

If you want exclude the directory from the ls command:

ls -l |grep -v ^d|awk '{print $6,$7}' |uniq|while read i
do
if [ $i != $TODAY ] ; then
DIR=$($i |sed s/' '//)
mkdir $DIR
fi
done

Enrico
Honored Contributor
Enrico P.
Posts: 368
Registered: ‎06-10-2003
Message 5 of 14 (230 Views)

Re: script to move files

And you need to quote the variables in the:

if [ "$i" != "$TODAY" ]

Sorry
Enrico
Honored Contributor
Victor Fridyev
Posts: 937
Registered: ‎07-01-1998
Message 6 of 14 (230 Views)

Re: script to move files

Hi,

If I understand you correctly, you need something like that:

#!/bin/sh
TODAY=$(date +%d%b)
/bin/ls -lt |grep -v ^d |while read FN; do
DATE=$(echo $FN|awk '{printf("%02d%s\n",$7,$6)}')
if [ "$DATE" != $TODAY ]; then
if [ ! -d $DATE ] ; then
mkdir $DATE
fi
fi
done
This works if all files in the current directory have been created on this year, otherwise you have to lok at $8, which is either year or HH:MM, but these are details.

HTH
Entities are not to be multiplied beyond necessity - RTFM
Honored Contributor
Sandman!
Posts: 2,220
Registered: ‎01-13-2005
Message 7 of 14 (230 Views)

Re: script to move files

Hi Lawrenzo,

Change the awk statement on the for-loop line so that it outputs a string w/o any embedded spaces. This also helps in compacting code since you don't need the sed statement anymore, which basically removes embedded spaces in $i, and make sure $TODAY contains the same date format as $i. Here's your code with the suggested modifications.

change -> awk '{print $6,$7}'
to -> awk '{print $6$7}'

remove -> DIR=$($i |sed s/' '//)

set -x
TODAY=$(date +"%b%d")
for i in `ls -l |awk '{print $6$7}' |uniq`
do
if [ $i != $TODAY ]; then
mkdir $i
fi
done
Super Advisor
lawrenzo_1
Posts: 560
Registered: ‎06-06-2003
Message 8 of 14 (230 Views)

Re: script to move files

ok thats great to start my script off the main challenge I have here:

I want to list all files in the directory - if the directory doesn't exist create it, then move the relevant date files to the directory

if the directory does exist just move the files there.

TY
hello
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 9 of 14 (230 Views)

Re: script to move files

Hi Lawrenzo:

I'd archive this way:

# touch -amt 07170000 /tmp/myref #...today
# mkdir /tmp/myarchive
# cd /tmp/myarchive
# find /path ! -newer /tmp/me|cpio -padmxv .

...This will find all files (and directories) that have been modified *before* today and copy then to your archive directory. A listing of the files copied is automatically generated. Modification and access times of the files are preserved.

Regards!

...JRF...
Super Advisor
lawrenzo_1
Posts: 560
Registered: ‎06-06-2003
Message 10 of 14 (230 Views)

Re: script to move files

ok getting there now here is the function within the script

#!/bin/ksh

DIR=/usr/dump/JDE/data/out/ARCHIVE
LOGFILE=/sysadmin/logs/mvdata_JDE.log
REMOVE=/usr/dump/JDE/toberemoved
TODAY=$(date +"%d %b")



for a in `find $DIR -ctime 1 -type f`
do

month=`ls -l $a | awk -F' ' '{print $6$7}'`
file=`ls -l $a | awk -F' ' '{print $9}'`


if [ ! -d $month ] ; then

mkdir /usr/dump/JDE/data/out/ARCHIVE/$month
mv $file /usr/dump/JDE/data/out/ARCHIVE/$month

else
mv $file /usr/dump/JDE/data/out/ARCHIVE/$month
fi
done


this will create the directory and move the file.

The issue I have now is when the script runs the second day it still attempts to move all files that where moved the previous day.

how can i get find just to search the directory /usr/dump/JDE/data/out/ARCHIVE and not /usr/dump/JDE/data/out/ARCHIVE/day.mnth

thanks
hello
Honored Contributor
Peter Nikitka
Posts: 1,575
Registered: ‎02-10-2003
Message 11 of 14 (230 Views)

Re: script to move files

Hi,

the easiest thing would be to keep the monthly archive outside the find tree:

...
M_ARCHIVE=${DIR}-bymonth

[ -d $M_ARCHIVE ] || mkdir $M_ARCHIVE

for a in `find $DIR -ctime 1 -type f`
do
...
[ -d $M_ARCHIVE/$month ] || mkdir $M_ARCHIVE/$month
mv $file $M_ARCHIVE/$month
...
done

mfG Peter
The Universe is a pretty big place, it's bigger than anything anyone has ever dreamed of before. So if it's just us, seems like an awful waste of space, right? Jodie Foster in "Contact"
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 12 of 14 (230 Views)

Re: script to move files

Hi (again) Lawrenzo:

You are using the 'ctime' or last change time for a file's inode. The 'ctime' changes whenever ownership, permissions are changed or whenever a file is renamed.

Using 'ctime' tracks metadata changes. If you truly want to find files based on *modification* of their data, you need to use 'mtime'.

The manpages for 'stat(2)' will clarify things for you.

BTW, I still suggest the 'cpio' method I posted above for your archiving needs.

Regards!

...JRF...
Honored Contributor
Sandman!
Posts: 2,220
Registered: ‎01-13-2005
Message 13 of 14 (230 Views)

Re: script to move files

Modify the find command to exclude the other dir as follows:

find $DIR ! -path "/usr/dump/JDE/data/out/ARCHIVE/day.mnth/*" -ctime 1 -type f

~hope it helps
Super Advisor
lawrenzo_1
Posts: 560
Registered: ‎06-06-2003
Message 14 of 14 (230 Views)

Re: script to move files

Thank you everyone - got there in the end!

James thanks for the suggestion however the business have request the archive directories this was.

I like your idea however it wouldn't work for what is required as there are over 3000 files daily that sometimes require a search etc etc.


Chris.
hello
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.