Re: Pattern matching - Need some help (198 Views)
Reply
Frequent Advisor
tamilrain9
Posts: 38
Registered: ‎12-25-2009
Message 1 of 8 (272 Views)

Pattern matching - Need some help

[ Edited ]

Hi guys

 

i have below three lines of logs

 

error occured at 8:44AM processes id 0x234
error occured at 8:45AM processes id 0x244
error occured at 8:46AM processes id 0x294

 

i want to extract only the alphabets from each line and store it in a variable..

 

Ex Pattern: error <*.var1>

 

The var1 should contain only the alphabets and no numbers/special characters.

 

erroroccuredatAMprocessesidx

erroroccuredatAMprocessesidx

erroroccuredatAMprocessesidx

 

Is that possible? please advise the exact pattern that can be used ,

Valued Contributor
Raymond Meijer
Posts: 98
Registered: ‎02-20-2008
Message 2 of 8 (254 Views)

Re: Pattern matching - Need some help

You can do this by writing a script that parses the actual logfile and sends its output to an output file.

Then you can instruct your logfile monitor to run the script first and then read the output file instead of the original logfile.

 

The options are called Execute and Read file and can be found under Monitoring Options

Frequent Advisor
tamilrain9
Posts: 38
Registered: ‎12-25-2009
Message 3 of 8 (250 Views)

Re: Pattern matching - Need some help

hi raymond,

 

Actually, i am trying to use this logic for message key correlation.

 

my message text contains timestamp and when there are many messages, trying to extract only the alphabets and store it in a var1

 

then this value will be used in message key and message key relation.

 

so the duplicate messages will be supressed.

 

Do you know any other way to supress the duplicate events when he message text has time stamp in it?

 

Ex logs:

error occured at 8:44AM processes id 0x234
error occured at 8:45AM processes id 0x244
error occured at 8:46AM processes id 0x294

 

Valued Contributor
Raymond Meijer
Posts: 98
Registered: ‎02-20-2008
Message 4 of 8 (243 Views)

Re: Pattern matching - Need some help

Do you mean something like this:

 

Match: ^error occured at <#>:<#>AM processes id <#>x<#>$

Output: erroroccuredatAMprocessesidx

 

Or more flexible:

 

Match: ^<@.t1><_><@.t2><_><@.t3><_><#>:<#>AM<_><@><_><@><_><#>x<#>$

Output: erroroccuredatAMprocessesidx

 

Or more flexible with AM/PM:

 

Match: ^<@.t1><_><@.t2><_><@.t3><_><#>:<#><@.ampm><_><@><_><@><_><#>x<#>$

Output: erroroccuredat<ampm>processesidx

 

?

 

OM's regular expressions are quite powerful.

Frequent Advisor
tamilrain9
Posts: 38
Registered: ‎12-25-2009
Message 5 of 8 (228 Views)

Re: Pattern matching - Need some help

hi raymond,

 

The challenge here is,

 

There is no specific format, i do not know the incoming log lines.

 

since i am trying to apply this correlation logic for all logfile template, there are lot of logfiles containing lines in different format.

 

so ideally the input line may be anything 

EX

Source line1: aaaaaaaaaaaaaaaaaaabbbbbbbbbbbb  sdssss 12222

or

Source line2: 172.16.215.12011153 is being created with scope DISTRIBUTED_NO_ACK 34534

or

any log lines.

 

however, we should strip all the numbers and special characters from them and get only the alphabets

these alphabets will be stored in a variable.

 

value from line 1 will be var1=aaaaaaaaaaaaaaaaaaabbbbbbbbbbbbsdssss

value from line 2 will be var2=isbeingcreatedwithscopeDISTRIBUTEDNOACK

 

were you able to understand?

 

Honored Contributor
Ram_21
Posts: 3,655
Registered: ‎04-30-2003
Message 6 of 8 (223 Views)

Re: Pattern matching - Need some help

Hello

 

I am not sure you can do this with pattern matching for your incoming text does not really have a pattern and the end result you are looking for is also not a pattern but just 'keeping' the alphabets in the incoming message text no matter what the actual text is.

 

In your case the solution may be what Raymond had suggested eariler in this post of creating a script that will read the log file and strip off the numeric characters in the log entry and put that parsed information to another log file.

 

Hope you get a better answer.

Regards

Ram

Frequent Advisor
tamilrain9
Posts: 38
Registered: ‎12-25-2009
Message 7 of 8 (218 Views)

Re: Pattern matching - Need some help

Thanks Ram,

 

But my intension of doing the whole thing is to have a effective correlation using message key and message key relation.

 

Let me give some scenarios, please have a look if i am on a right direction.

 


-----------------------------------------------------------------------------------------------------------------------
scenario 1: Trigger a alert if the line contains warning
-----------------------------------------------------------------------------------------------------------------------

incoming log message :
warning-client disconnected at 4:35 AM with error code 00000x45

Matching text:
warning-<*[a-z].var1>

 


Display Message text:
<$MSG_TEXT>

Correlation:
Message Key and Message key relation based on var1

 

-----------------------------------------------------------------------------------------------------------------------
scenario 2:Trigger a alert if the line contains critical
-----------------------------------------------------------------------------------------------------------------------

incoming log message :
critical-client disconnected at 4:35 AM with error code 00000x45

Matching text:
critical-<*[a-z].var1>

 

Display Message text:
<$MSG_TEXT>

Correlation:
Message Key and Message key relation based on var1


-----------------------------------------------------------------------------------------------------------------------
scenario 3:Trigger a alert if the line contains disconnected
-----------------------------------------------------------------------------------------------------------------------

incoming log message :
warning-client disconnected at 4:35 AM with error code 00000x45

Matching text:
<*[a-z].var1>disconnected<*[a-z].var2>

 


Display Message text:
<$MSG_TEXT>

Correlation:
Message Key and Message key relation based on var1 and var2

 

This match pattern "<*[a-z].var1>" is just a assumption.

instead the actual pattern which can match any alphabets and capture only alphabets whill help.

 

Occasional Contributor
Steph0805
Posts: 5
Registered: ‎07-18-2013
Message 8 of 8 (198 Views)

Re: Pattern matching - Need some help

Hello,

 

You can do that using the script like Raymond says you put only the alphabetical on a log file, then you make a policy that parse this log file and make a patterne matching like <*.var1> like this you'll had only the alphabetical in your variable and use it for your correlation

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.