Re: Script needed... (189 Views)
Reply
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 1 of 11 (189 Views)
Accepted Solution

Script needed...

Hi!

I have a file which has a list of host, I want to ssh to each of the first box for an app and grep through the logs for version of the application and look for a certain string "Connection is open" in the log file. The file I am refering to looks like this -


prod1-doms-2001.corp.zap.com:App1-America:PROD:8000
prod2-doms-2001.corp.zap.com:App1-Asia:PROD:8020
prod3-doms-2002.corp.zap.com:App1-Europe:PROD:8040
prod1-doms-2001.corp.zap.com:App2-America:PROD:9000
prod2-doms-2001.corp.zap.com:App2-Asia:PROD:9020
prod3-coms-2002.corp.zap.com:App2-Europe:PROD:9040

where columns are separated by colons -

First column - hostname
Second column - app name
Third column - env name
Forth column - port #

I want to ssh to first host of each app then grep for a string called "connection is open" and also parse out (20100613) from a string of line like the following from the log file -

.... -JbossRootDirectory=/remote/QB/packages/App1-20100712-133944-249b7220ac454586864a7...


Currently I am doing this manually but since there are some 30 hosts, I want to have this executed from a central admin box.

The log file name changes with appname and hostname /port and is of the pattern - /App1-region_prod1-doms-2001.corp_8000

Please share if someone has developed a script which can do this. I want to do this after restart of each app.

Thanks,
Allan
Acclaimed Contributor
Dennis Handly
Posts: 25,291
Registered: ‎03-06-2006
Message 2 of 11 (189 Views)

Re: Script needed...

>parse out (20100613) from a string of line like the following from the log file

What do you mean by "parse out"? Exclude, select?

>The log file name changes with appname and hostname /port and is of the pattern - /App1-region_prod1-doms-2001.corp_8000

Is the pattern:
/${FIELD2}_${FIELD1e}_${FIELD4}
Where $FIELD1e is FIELD1 but without ".zap.com"?
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 3 of 11 (189 Views)

Re: Script needed...

Hi Dennis,

"parse out"? Exclude, select? - I mean select.

Is the pattern:
/${FIELD2}_${FIELD1e}_${FIELD4}
Where $FIELD1e is FIELD1 but without ".zap.com"?

Appname-region__

Appname could be App1,App2 etc.
region could be America,Asia or Europe.

Thanks,
Allan



Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 4 of 11 (189 Views)

Re: Script needed...

Hi Allan:

> I want to ssh to first host of each app then grep for a string called "connection is open" and also parse out (20100613) from a string of line like the following from the log file -

This isn't a very clear description of what you want. Are you saying that if you find the string "connection is open" in the log, then and only then do you want to report lines that contain the string "20100613"?

A simple shell script could read you list of hosts and derive the logfile names associated with each host. This could look like:

# cat ./mklognames
#!/usr/bin/sh
typeset INFILE=$1
OLDIFS=${IFS}
IFS=":"
while read HOST APP ENVRN PORT X
do
echo ${APP}_${HOST%%.corp.zap.com}_${PORT}
done < ${INFILE}
IFS=${OLDIFS}
exit 0

...run as:

# ./mklognames ./mklognames.data

...which using your data as input gives:

App1-America_prod1-doms-2001_8000
App1-Asia_prod2-doms-2001_8020
App1-Europe_prod3-doms-2002_8040
App2-America_prod1-doms-2001_9000
App2-Asia_prod2-doms-2001_9020
App2-Europe_prod3-coms-2002_9040

Regards!

...JRF...
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 5 of 11 (189 Views)

Re: Script needed...

Hi JRF!

20100613 is just an example, it could 20100702.

> Are you saying that if you find the string "connection is open" in the log, then and only then do you want to report lines that contain the string "20100613"?

No, we can look for version first before looking for that string - "connection is open".

Thanks,
Allan.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 6 of 11 (189 Views)

Re: Script needed...

Hi Allan:

OK, as a simplistic shell script you might do:

# cat ./mklognames
#!/usr/bin/sh
typeset TOFIND=$1
typeset INFILE=$2
OLDIFS=${IFS}
IFS=":"
while read HOST APP ENVRN PORT X
do
ssh -n ${HOST} "grep ${TOFIND} ${APP}_${HOST%%.corp.zap.com}_${PORT}"
done < ${INFILE}
IFS=${OLDIFS}
exit 0

...run this like:

# ./mklognames 20100702 ./mklognames.data

...where the first argument is the string to match and the second argument is the file of hostnames.

Regards!

...JRF...
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 7 of 11 (189 Views)

Re: Script needed...

Thanks for your script, but as I run it there is one catch here,

Since the list of apps/hosts are too many, I would also like to give appname as an argument so that only the first or all the boxes that belong to that app are searched and not all.

Thanks,
Allan

Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 8 of 11 (189 Views)

Re: Script needed...

Hi (again) Allan:

> Since the list of apps/hosts are too many, I would also like to give appname as an argument so that only the first or all the boxes that belong to that app are searched and not all.

Then, perhaps:

# cat ./mklognames
#!/usr/bin/sh
typeset TOFIND=$1
typeset WHATAP=$2
typeset INFILE=$3
OLDIFS=${IFS}
IFS=":"
while read HOST APP ENVRN PORT X
do
[ $(echo ${APP}|cut -f1 -d"-") != ${WHATAP} ] && continue
ssh -n ${HOST} "grep ${TOFIND} ${APP}_${HOST%%.corp.zap.com}_${PORT}"
done < ${INFILE}
IFS=${OLDIFS}
exit 0


...Notice that there are now three arguments. Run as:

# ./mklognames 20100702 App1 ./mklognames.data

If I understand correctly, the "application" is "App" followed by a hyphen and a "region", hence we snip away the "-" and region suffix.

Regards!

...JRF...
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 9 of 11 (189 Views)

Re: Script needed...

Hey JRF,

Currently I am not able to search for strings through this script, I tried grep -F but with --color but that did not work and was looking for a single word instead of string.

Can you help?

Thanks,
Allan.
Acclaimed Contributor
James R. Ferguson
Posts: 21,184
Registered: ‎07-06-2000
Message 10 of 11 (189 Views)

Re: Script needed...

Hi Allan:

> looking for a single word instead of string.

Try using 'grep -w'. This matchs whole "words". See the 'grep' manpages for a full explanation.

Regards!

...JRF...
Super Advisor
Allanm
Posts: 287
Registered: ‎08-13-2007
Message 11 of 11 (189 Views)

Re: Script needed...

Thanks JRF!
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.