Re: Need Script to monitor file transfer has occurred (338 Views)
Reply
Frequent Advisor
Clara Rowe
Posts: 66
Registered: ‎01-29-1998
Message 1 of 6 (338 Views)
Accepted Solution

Need Script to monitor file transfer has occurred

Hi Experts! I need your help again. I am in need of a script or software to perform the following function:

Monitor a specific directory.
If a file is there for longer than one minute, notify people.

There is a process that is dropping orders on to this directory. The files are moved when they have processed. If any one of the files stays in the directory for more than a minute, there is a big problem.

Some how this has become my problem, and they want me to write a script to monitor this and several other directories. It seems to me that this is going to get pretty ridiculous pretty quickly. I would love software that could do the monitoring, but would settle for help on the scripting.

As always, any help you can provide will be greatly appreciated, and rewarded with points.

Clara
Take time to smell the roses.
Honored Contributor
Muthukumar_5
Posts: 4,030
Registered: ‎06-09-2004
Message 2 of 6 (338 Views)

Re: Need Script to monitor file transfer has occurred

I hope we can do it with shell scripting.

#!/usr/bin/ksh
set -x

# Directory Path
DIR=/home/test/

for file in `ls $DIR`
do

min=`ll $file | awk '{ print $8 }' | cut -d : -f2`
org=`date +'%M'`

if [[ $min -gt `date +'%M'` ]]
then

# Make a alert or mail with $file info
# mailx

fi
done

Make changes more appropriate to your requirement.

HTH.
+muthu+
Easy to suggest when don't know about the problem!
Frequent Advisor
Clara Rowe
Posts: 66
Registered: ‎01-29-1998
Message 3 of 6 (338 Views)

Re: Need Script to monitor file transfer has occurred

Test directory:
root in /home/rowec/tevo
sfitevo1 850 #ll
total 41
drwxr-xr-x 2 root system 512 Aug 19 13:35 .
drwxrwxrwx 3 rowec system 512 Aug 19 13:24 ..
-rw-r--r-- 1 root system 1961 Aug 19 13:36 20040819133526.udf204.gwn
-rw-r--r-- 1 root system 1962 Aug 19 13:36 20040819133530.udf204.gwn
-rw-r--r-- 1 root system 1843 Aug 19 13:36 20040819133533.udf204.gwn
-rw-r--r-- 1 root system 1961 Aug 19 13:36 20040819133536.udf204.gwn
-rw-r--r-- 1 root system 1790 Aug 19 13:36 20040819133539.udf204.gwn
-rw-r--r-- 1 root system 2262 Aug 19 13:36 20040819133542.udf204.gwn
-rw-r--r-- 1 root system 2713 Aug 19 13:36 20040819133545.udf204.gwn
-rw-r--r-- 1 root system 1984 Aug 19 13:36 20040819133550.udf204.gwn
-rw-r--r-- 1 root system 1866 Aug 19 13:36 20040819133604.udf204.gwn


Result of running script:
sfitevo1 853 #sh -vx tevo2
#!/usr/bin/ksh
set -x
+ set -x

# Directory Path
DIR=/home/rowec/tevo
+ DIR=/home/rowec/tevo

for file in `ls $DIR`
do

min=`ll $file | awk '{ print $8 }' | cut -d : -f2`
org=`date +'%M'`

if [[ $min -gt `date +'%M'` ]]
then

# Make a alert or mail with $file info
# mailx
echo "files not moving"
fi
done
+ ls /home/rowec/tevo
+ + ll 20040819133526.udf204.gwn
+ awk { print $8 }
ls: 0653-341 The file 20040819133526.udf204.gwn does not exist.
+ cut -d : -f2
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133530.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
ls: 0653-341 The file 20040819133530.udf204.gwn does not exist.
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133533.udf204.gwn
+ awk { print $8 }
ls: 0653-341 The file 20040819133533.udf204.gwn does not exist.
+ cut -d : -f2
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133536.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
ls: 0653-341 The file 20040819133536.udf204.gwn does not exist.
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133539.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
ls: 0653-341 The file 20040819133539.udf204.gwn does not exist.
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133542.udf204.gwn
+ awk { print $8 }
ls: 0653-341 The file 20040819133542.udf204.gwn does not exist.
+ cut -d : -f2
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133545.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
ls: 0653-341 The file 20040819133545.udf204.gwn does not exist.
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133550.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
ls: 0653-341 The file 20040819133550.udf204.gwn does not exist.
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]
+ + ll 20040819133604.udf204.gwn
+ awk { print $8 }
ls: 0653-341 The file 20040819133604.udf204.gwn does not exist.
+ cut -d : -f2
min=
+ + date +%M
org=36
+ date +%M
+ [[ -gt 36 ]]

root in /home/rowec
sfitevo1 854 #


Something is not quite right.




Take time to smell the roses.
Frequent Advisor
Clara Rowe
Posts: 66
Registered: ‎01-29-1998
Message 4 of 6 (338 Views)

Re: Need Script to monitor file transfer has occurred

I think I got it fixed!

New script:
#!/usr/bin/ksh
set -x

# Directory Path
DIR=/home/rowec/tevo
cd $DIR
for file in *
do

min=`ll $file | awk '{ print $8 }' | cut -d : -f2`
org=`date +'%M'`

if [[ $min -gt `date +'%M'` ]]
then

# Make a alert or mail with $file info
# mailx
echo "files not moving"
fi
done

New result:

#!/usr/bin/ksh
set -x
+ set -x

# Directory Path
DIR=/home/rowec/tevo
+ DIR=/home/rowec/tevo
cd $DIR
+ cd /home/rowec/tevo
for file in *
do
echo $file
min=`ll $file | awk '{ print $8 }' | cut -d : -f2`
org=`date +'%M'`
echo $min
if [[ $min -gt `date +'%M'` ]]
then

# Make a alert or mail with $file info
# mailx
echo "files not moving"
fi
done
+ echo 20040819133526.udf204.gwn
20040819133526.udf204.gwn
+ + ll 20040819133526.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133530.udf204.gwn
20040819133530.udf204.gwn
+ + ll 20040819133530.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133533.udf204.gwn
20040819133533.udf204.gwn
+ + ll 20040819133533.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133536.udf204.gwn
20040819133536.udf204.gwn
+ + ll 20040819133536.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133539.udf204.gwn
20040819133539.udf204.gwn
+ + ll 20040819133539.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133542.udf204.gwn
20040819133542.udf204.gwn
+ + ll 20040819133542.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133545.udf204.gwn
20040819133545.udf204.gwn
+ + ll 20040819133545.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133550.udf204.gwn
20040819133550.udf204.gwn
+ + ll 20040819133550.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]
+ echo 20040819133604.udf204.gwn
20040819133604.udf204.gwn
+ + ll 20040819133604.udf204.gwn
+ awk { print $8 }
+ cut -d : -f2
min=36
+ + date +%M
org=45
+ echo 36
36
+ date +%M
+ [[ 36 -gt 45 ]]


Can't thank you enough!
Points will have to do.

Thanx again
Clara
Take time to smell the roses.
Frequent Advisor
Clara Rowe
Posts: 66
Registered: ‎01-29-1998
Message 5 of 6 (338 Views)

Re: Need Script to monitor file transfer has occurred

I ran into a few snags with the script you suggested. Midnight was a huge issue. This is what I ended up with:

sfitevo1 1150 #cat tevotest
#################################################################################
# Script to see if files are stalled.
#################################################################################
# Variables
###########
lagtime="0"
earl="9999"
dropdir=/home/rowec/tevo
tevotmp=/u15/tmp
curtime=`date +%H%M`
curhour=`date +%H`
curmin=`date +%M`
#################################################################################
# Go to the Directory to be checked
#################################################################################
cd $dropdir
#################################################################################
# Create a listing of the files
#################################################################################
ls -l > $tevotmp/tevo
#################################################################################
# Pull out the time and remove duplicates and :
#################################################################################
cut -c 52-56 $tevotmp/tevo > $tevotmp/tevotime
sort -u $tevotmp/tevotime > $tevotmp/tevocheck
sed -e 's/://g' $tevotmp/tevocheck > $tevotmp/tevock
#################################################################################
# Find the earliest time
#################################################################################
for a in `cat $tevotmp/tevock`
do
[ $a -lt $earl ] && {
earl=$a
}
done
#################################################################################
# Check to see if it is midnight and change current hour to 24 if it is.
#################################################################################
[ $earl -ge "2300" -a $curhour -eq "00" ] && {
curhour=24
curtime=$curhour$curmin
}
#################################################################################
# Subtract the earliest time from the current time to get lag time.
#################################################################################
lagtime=`expr $curtime - $earl`
if [ $lagtime -ge "3" ]
then {
##mailx here
echo "count equals $lagtime"
}
fi
exit 0
Take time to smell the roses.
Honored Contributor
John Poff
Posts: 2,448
Registered: ‎05-22-2001
Message 6 of 6 (338 Views)

Re: Need Script to monitor file transfer has occurred

Hi,

That sure seems like a lot of work, especially for somebody as lazy as I am. :)

I thought about it and I came up with a little shell script that seems to work. I just use the 'touch' command to touch a file, sleep for 60 seconds, and then use the 'find' command to find any files older (actually not newer) than the file I touched. If I do, I e-mail a list of the files I found. The "reference" file can live anywhere, if you don't want it in your current directory.



#!/bin/sh

# oldfiles.sh

while true
do
touch reference.file
sleep 60
find . ! -newer reference.file -name "*.udf204.gwn" >/var/tmp/oldfiles
if [[ -s /var/tmp/oldfiles ]];
then
mailx -s "Files older than 1 minute found!" jpoff fi
done


JP
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.