Re: some help needed for scripting (799 Views)
Reply
Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 1 of 16 (831 Views)
Accepted Solution

some help needed for scripting

Hi ,

 

pfa my file which contains many jobs.

my reqmnt is to print all o/p from SY3100 . i.e start and end timings .

it also must print only single entries and no duplicate entries.

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 2 of 16 (828 Views)

Re: some help needed for scripting

[ Edited ]

It appears you can just use grep:  grep SY3100 file

 

I only see the Start and End lines, no duplicates to handle.

 

Honored Contributor
rariasn
Posts: 1,135
Registered: ‎08-09-1999
Message 3 of 16 (822 Views)

Re: some help needed for scripting

Hi:

 

$ grep SY3100 smartlog.20120117 | uniq -n

 

rgs

Regular Visitor
Riccardo_Marini
Posts: 5
Registered: ‎01-16-2012
Message 4 of 16 (814 Views)

Re: some help needed for scripting

try this :

grep SY3100 FILENAME |awk '/Start|End/ { {printf "%s : %s %s %s %s %s %s ",$2,$6,$7,$8,$9,$10,$11  } ;  if ( $2 == "End")  {printf "\n"  } }' |sort -u

 

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 5 of 16 (799 Views)

Re: some help needed for scripting

Hi guys,

Really sorry for not making my reqmnt clean.

i want to grep start and end timings from this file along with jobs name for ex sy3100

it shold give me o/p like, it sould subtract timings

job name   time taken

sy3100         20 min

 

Also if i have multistreams A,B,C,D it should display the max time taken for that job.

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 6 of 16 (781 Views)

Re: some help needed for scripting

>I want to grep start and end timings from this file along with jobs name for ex sy3100

>sy3100         20 min

 

Do you want it to do ALL jobs in the file or just the one selected?

 

>if I have multistreams A,B,C,D it should display the max time taken for that job.

 

What do you mean by "multistreams"?

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 7 of 16 (779 Views)

Re: some help needed for scripting

Hi Dennis ,

PFA file from which i want all jobs, after this specific  job SY3100 with total time taken

i.e end time - start time .

 

the o/p should be like ;

 

SY3100   0 sec/min

 

 

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 8 of 16 (750 Views)

Re: some help needed for scripting

Hi Dennis ,

 

I hope my reqmnt is clear now.

i just need a hint to subtract time.

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 9 of 16 (736 Views)

Re: some help needed for scripting

>I hope my requirement is clear now.

 

I still don't know what multistreams are?

 

>I just need a hint to subtract time.

 

# subtract times
# Tue Jan 17 21:56:18 IST 2012 End: Tue Jan 17 21:56:18 IST 2012
grep sy3100 smartlog.20120117.txt |
  awk '
# day of month, HH:MM:SS (24 hour clock)
function convert_to_secs(day, hhmmss) {
#   print day, hhmmss
   split(hhmmss, t_hhmmss, ":")
   return (((day - 1) * 24 + t_hhmmss[1]) * 60 + t_hhmmss[2]) * 60 +  t_hhmmss[3]
}
BEGIN {
   print "job name   time"
}
$2 == "Start" {
   yyyymmmddzz[$4] = $7 $11 $10  # check for unhandled date straddling
   start_time[$4] = convert_to_secs($8, $9)
   next
}

$2 == "End" {
   if (yyyymmmddzz[$4] == "") {
      print "Cannot find Start time for", $4
      next
   }
   stop_yyyymmmddzz = $7 $11 $10  # check for unhandled date straddling
   if (stop_yyyymmmddzz != yyyymmmddzz[$4]) {
      print "Start/stop for", $4", straddles complex date boundary",  stop_yyyymmmddzz
   }
   stop_time = convert_to_secs($8, $9)
#   printf "%s: %d\n", $4, start_time[$4]
#   printf "%s: %d\n", $4, stop_time
   printf "%-12s %.1f min\n", $4, (stop_time - start_time[$4]) / 60
   next
}'

 

This doesn't handle the case where the times span month boundaries.  It does detect it.

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 10 of 16 (729 Views)

Re: some help needed for scripting

Hi Dennis ,

 

After grepping the required pattern from logfile  m  getting ;

 

GB_PROD /home//sysout > grep "IN0800" smartlog.??????.txt
*** Start of  IN0800A  at  Mon Nov 28 15:21:22 IST 2011
*** End   of  IN0800A  at  Mon Nov 28 15:21:24 IST 2011  - RC =  0
*** Start of  IN0800B  at  Mon Nov 28 15:21:27 IST 2011
*** End   of  IN0800B  at  Mon Nov 28 15:21:29 IST 2011  - RC =  0
*** Start of  IN0800C  at  Mon Nov 28 15:21:32 IST 2011
*** End   of  IN0800C  at  Mon Nov 28 15:21:33 IST 2011  - RC =  0
*** Start of  IN0800D  at  Mon Nov 28 15:21:37 IST 2011
*** End   of  IN0800D  at  Mon Nov 28 15:21:38 IST 2011  - RC =  0
*** Start of  IN0800E  at  Mon Nov 28 15:21:42 IST 2011
*** End   of  IN0800E  at  Mon Nov 28 15:21:44 IST 2011  - RC =  0
*** Start of  IN0800F  at  Mon Nov 28 15:21:47 IST 2011
*** End   of  IN0800F  at  Mon Nov 28 15:21:48 IST 2011  - RC =  0
*** Start of  IN0800G  at  Mon Nov 28 15:21:52 IST 2011
*** End   of  IN0800G  at  Mon Nov 28 15:21:53 IST 2011  - RC =  0
*** Start of  IN0800H  at  Mon Nov 28 15:21:57 IST 2011
*** End   of  IN0800H  at  Mon Nov 28 15:21:58 IST 2011  - RC =  0

==========================================================

I tried running that time function ;

 

i got o/p as ;

 

job name   time
IN0800A      0.0 min
IN0800B      0.0 min
IN0800C      0.0 min
IN0800D      0.0 min
IN0800E      0.0 min
IN0800F      0.0 min
IN0800G      0.0 min
IN0800H      0.0 min

I understand some column paramters needs to be edited ,new to scripting .

Any more help  would be great.

Thanks .

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 11 of 16 (724 Views)

Re: some help needed for scripting

>I got output as: IN0800A      0.0 min

 

If you want the time in seconds:

printf "%-12s %.1f sec\n", $4, (stop_time - start_time[$4])

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 12 of 16 (722 Views)

Re: some help needed for scripting

Hi Dennis ,

 

Thanks its working.

Now a final question ,

 

Suppose i have a file with below lines in it ;

 

*** Start of chkrerun at  Thu Jan 26 22:06:25 IST 2012

*** End   of chkrerun at  Thu Jan 26 22:06:25 IST 2012  - RC =  0
*** Start of  SY0007  at  Thu Jan 26 22:06:31 IST 2012
*** End   of  SY0007  at  Thu Jan 26 22:06:32 IST 2012  - RC =  0
*** Start of  SY3100  at  Thu Jan 26 22:06:44 IST 2012
*** End   of  SY3100  at  Thu Jan 26 22:06:44 IST 2012  - RC =  0
*** Start of  SP0100  at  Thu Jan 26 22:06:45 IST 2012
*** End   of  SP0100  at  Thu Jan 26 22:07:20 IST 2012  - RC =  0
*** Start of spms0122 at  Thu Jan 26 22:07:21 IST 2012

*** End   of spms0122 at  Thu Jan 26 22:07:52 IST 2012  - RC =  0
*** Start of  SY9507  at  Thu Jan 26 22:07:53 IST 2012
*** End   of  SY9507  at  Thu Jan 26 22:07:57 IST 2012  - RC =  0
*** Start of stopgateway at  Thu Jan 26 22:08:02 IST 2012

*** End   of stopgateway at  Thu Jan 26 22:08:02 IST 2012  - RC =  0
*** Start of shutbtm at  Thu Jan 26 22:08:03 IST 2012

*** End   of shutbtm at  Thu Jan 26 22:08:03 IST 2012  - RC =  0
*** Start of  SY9600  at  Thu Jan 26 22:08:04 IST 2012
*** End   of  SY9600  at  Thu Jan 26 22:08:04 IST 2012  - RC =  0
*** Start of  UTDEBUG1  at  Thu Jan 26 22:08:05 IST 2012
*** End   of  UTDEBUG1  at  Thu Jan 26 22:08:05 IST 2012  - RC =  0
*** Start of  SY0007  at  Thu Jan 26 22:08:13 IST 2012
*** End   of  SY0007  at  Thu Jan 26 22:08:14 IST 2012  - RC =  0
*** Start of  SY3300  at  Thu Jan 26 22:08:25 IST 2012
*** End   of  SY3300  at  Thu Jan 26 22:08:25 IST 2012  - RC =  0
*** Start of  SY0199  at  Thu Jan 26 22:08:40 IST 2012
*** End   of  SY0199  at  Thu Jan 26 22:08:40 IST 2012  - RC =  0
*** Start of  CI0251  at  Thu Jan 26 22:08:41 IST 2012
*** End   of  CI0251  at  Thu Jan 26 22:25:43 IST 2012  - RC =  0
*** Start of  IN0869  at  Thu Jan 26 22:25:44 IST 2012
*** End   of  IN0869  at  Thu Jan 26 22:25:44 IST 2012  - RC =  0
*** Start of  IN0800A  at  Thu Jan 26 22:25:45 IST 2012
*** Start of  IN0800B  at  Thu Jan 26 22:25:50 IST 2012
*** Start of  IN0800C  at  Thu Jan 26 22:25:55 IST 2012
*** Start of  IN0800D  at  Thu Jan 26 22:26:00 IST 2012
*** Start of  IN0800E  at  Thu Jan 26 22:26:06 IST 2012
*** Start of  IN0800F  at  Thu Jan 26 22:26:11 IST 2012
*** Start of  IN0800G  at  Thu Jan 26 22:26:16 IST 2012
*** Start of  IN0800H  at  Thu Jan 26 22:26:21 IST 2012
*** End   of  IN0800B  at  Thu Jan 26 23:00:02 IST 2012  - RC =  0
*** End   of  IN0800D  at  Thu Jan 26 23:01:41 IST 2012  - RC =  0
*** End   of  IN0800F  at  Thu Jan 26 23:02:22 IST 2012  - RC =  0
*** End   of  IN0800C  at  Thu Jan 26 23:02:51 IST 2012  - RC =  0
*** End   of  IN0800A  at  Thu Jan 26 23:05:47 IST 2012  - RC =  0
*** End   of  IN0800G  at  Thu Jan 26 23:10:13 IST 2012  - RC =  0
*** End   of  IN0800E  at  Thu Jan 26 23:21:37 IST 2012  - RC =  0
*** End   of  IN0800H  at  Thu Jan 26 23:56:23 IST 2012  - RC =  0
*** Start of  IN0806A  at  Thu Jan 26 23:56:25 IST 2012
*** End   of  IN0806A  at  Thu Jan 26 23:56:25 IST 2012  - RC =  0
*** Start of  IN0806B  at  Thu Jan 26 23:56:27 IST 2012
*** End   of  IN0806B  at  Thu Jan 26 23:56:27 IST 2012  - RC =  0
*** Start of  IN0806C  at  Thu Jan 26 23:56:29 IST 2012
*** End   of  IN0806C  at  Thu Jan 26 23:56:29 IST 2012  - RC =  0

 

From the above i want to grepall o/p  from job SY3100 and continue till end of file;

 

i.e  the o/p should be like ;

       job name  time

       SY3100   0

       SP0100   yy 

       spms0122 zz

and so on till end of line....

..................

....................

   IN0806C   tt

 

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 13 of 16 (708 Views)

Re: some help needed for scripting

>From the above I want to grepall output from job SY3100 and continue till end of file:

 

You could do something like this.  Where the awk variable job_start is the first that prints:

awk -v job_start="SY3100" '
# day of month, HH:MM:SS (24 hour clock)
function convert_to_secs(day, hhmmss) {
#   print day, hhmmss
   split(hhmmss, t_hhmmss, ":")
   return (((day - 1) * 24 + t_hhmmss[1]) * 60 + t_hhmmss[2]) * 60 +  t_hhmmss[3]
}
BEGIN {
   print "job name   time"
}
$2 == "Start" {
   if ($4 != job_start && job_start != "") next  # skipping
   job_start = ""  # now handle rest of the jobs
   yyyymmmddzz[$4] = $7 $11 $10  # check for unhandled date straddling
   start_time[$4] = convert_to_secs($8, $9)
   next
}
$2 == "End" {
   if ($4 != job_start && job_start != "") next  # skipping
   if (yyyymmmddzz[$4] == "") {
      print "Cannot find Start time for", $4
      next
   }
   stop_yyyymmmddzz = $7 $11 $10  # check for unhandled date straddling
   if (stop_yyyymmmddzz != yyyymmmddzz[$4]) {
      print "Start/stop for", $4", straddles complex date boundary",
            stop_yyyymmmddzz
   }
   stop_time = convert_to_secs($8, $9)
#   printf "%s: %d\n", $4, start_time[$4]
#   printf "%s: %d\n", $4, stop_time
   printf "%-12s %.1f sec\n", $4, (stop_time - start_time[$4])
   next
}' smartlog.20120117.txt

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 14 of 16 (696 Views)

Re: some help needed for scripting

Hi Dennis ,

I got the o/p what i was looking for .

 

Can we sort this o/p as per time taken ?

 

Thanks very much for ur valuable time and efforts  for this solution.

Acclaimed Contributor
Dennis Handly
Posts: 25,296
Registered: ‎03-06-2006
Message 15 of 16 (685 Views)

Re: some help needed for scripting

[ Edited ]

>Can we sort this o/p as per time taken?

 

Yes, just pass the output to sort.  Unfortunately the title probably won't sort well and you may have to remove it from awk and add it back after the sort.  You may be in luck since the title sorts first:

awk ....

}' smartlog.20120117.txt | sort -n -k2,2  # sort on key 2, using numeric compares

 

Super Advisor
zxcv
Posts: 226
Registered: ‎04-09-2010
Message 16 of 16 (669 Views)

Re: some help needed for scripting

Hi Dennis ,

 

Thank you very much for the solution.

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.