JVM options (988 Views)
Reply
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 1 of 12 (988 Views)
Accepted Solution

JVM options

Experts,

Recently our application SM7.10.0123 died of Java heap errors. This started happening once in a week, with sm.log getting dumped with java heap errors, thereby killing system processes. Did some digging on this, came to knw it as an issue in sm7.10 which has been fixed in SM7.11 (release notes). Now I was trying with JVM options to control
this occurrances.

I tried putting the parameters directly in sm.ini like below

JVMOption0:-Xms384m
JVMOption1:-Xmx512m
JVMOption2:-Xrs
JVMOption3:-XX:+HeapDumpOnOutOfMemoryError

and found the below error

4804( 2360) 10/23/2009 04:50:14 RTE E **** CreateJavaVM() failed! rc = -4
4804( 2360) 10/23/2009 04:50:14 RTE D Xmx512M
4804( 2360) 10/23/2009 04:50:14 RTE D -Xmx512m
4804( 2360) 10/23/2009 04:50:14 RTE D -Xrs
4804( 2360) 10/23/2009 04:50:14 RTE D -XX:+HeapDumpOnOutOfMemoryError


I tried putting the parameters in the sm.cfg like below

sm -JVMOption0:Xms384m -JVMOption1:Xmx512m -JVMOption2:Xrs -JVMOption3:XX:+HeapDumpOnOutOfMemoryError system.start

and checked the sm.log after application restart

1272( -1) 10/26/2009 06:26:12 RTE I smservice: starting process sm -JVMOption0:Xms384m -JVMOption1:Xmx512m -JVMOption2:Xrs -JVMOption3:XX:+HeapDumpOnOutOfMemoryError system.start
5176( 2740) 10/26/2009 06:26:12 JGroups version: 2.5.0
4472( 3308) 10/26/2009 06:26:12 RTE I Using "utalloc" memory manager
4472( 3308) 10/26/2009 06:26:12 RTE I Command line: sm -JVMOption0:Xms384m -JVMOption1:Xmx512m -JVMOption2:Xrs -JVMOption3:XX:+HeapDumpOnOutOfMemoryError system.start
4472( 3308) 10/26/2009 06:26:13 RTE I Process sm 7.10.023 (023) System: 13080 (0x784dfb00) on PC running Windows Server 2003 (5.2 Build 3790) from SDEDAPS058VS (10.170.102.50)
1272( -1) 10/26/2009 06:26:13 RTE I smservice: initialization complete
5176( 2740) 10/26/2009 06:26:15 JRTE I Channel connected: SDEDAPS058VS:13080; address: 10.170.102.50:1070
4472( 3308) 10/26/2009 06:26:15 RTE I Attaching to resources with key 0x784dfb00
4472( 3308) 10/26/2009 06:26:15 RTE I JVM successfully created



Could any1 tell me, now how should I start monitoring? WIll i get a seperate log file for heapDump??
Please guide.

Thanks
DEV---
I am Listening..
Esteemed Contributor
Leon_Van_Dongen
Posts: 278
Registered: ‎09-30-2008
Message 2 of 12 (988 Views)

Re: JVM options

We recently had the same issue with SM7.10. Servlets would crash every few days and we were restarting every few days as well. We upgraded the binaries only to 7.11 and it's been running stable for a week. It's a simple upgrade and very easy to rollback. I'd recommend upgrading to it to resolve this.

We tried everything to get 7.10 stable but nothing really helped.
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 3 of 12 (988 Views)

Re: JVM options

Leon,

I know that this has been fixed in SM7.11. I came to know about this after going thru the SM7.11 release notes.
Thanks for the input.

But we cannot upgrade the prod directly to sm7.11. we will be doing it in another couple of months,till that I need to make sure that Sm7.10 run kind of stable.

Have you come across these JVM options in your sm.ini and sm.cfg . How do I get to see the
HeapDumpOnOutOfMemoryError log file. Does it come in the logs folder ???

Please guide.

Thanks
DEV---
I am Listening..
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 4 of 12 (988 Views)

Re: JVM options

Experts,

I need help on this. I have applied the JVM options like below in our environment. But still I am finding JAVA heap errors which clearly tells me that, the min and max heap size which I have set is not in effect.

JRTE I Running Low On JavaHeapMemory PercentUsed: 80.0 Max:61997056 Used: 49598840

Below is how I have set the parameters in my sm.cfg

sm -JVMOption0:Xms384m -JVMOption1:Xmx512m -JVMOption2:Xrs -JVMOption3:XX:+HeapDumpOnOutOfMemoryError system.start


Today I was going thru one of the KB article
'KM725908' . The way in which it is mentioned in the article is how I have set in the sm.ini . But when I restart the sm service, i could see the errors like

create JVM failed.
Shutting down sm application --(similar msgs)

Could you please guide me in how should i put this parameter correctly? For the JVMOption3:XX:+HeapDumpOnOutOfMemoryError

I could see the log file created of this in the RUN directory.

Thanks
DEV---
I am Listening..
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 5 of 12 (988 Views)

Re: JVM options

Is there any way to check the
TotalMemory ()
UsedMemory () for the smservice?

can we write a java program to find this ?

Thanks
DEV ---
I am Listening..
Honored Contributor
James Herold
Posts: 861
Registered: ‎07-20-2006
Message 6 of 12 (988 Views)

Re: JVM options

Should those commands be put in the JVM configuration and not SM?

Just asking...
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 7 of 12 (988 Views)

Re: JVM options

James,

Thanks for the reply.

which commands you are speaking abot? the JVM options?

In the KB article of SM7, i have read that the JVM options shud be put in sm.ini

JVMOption0:-Xms256m
JVMOption1:-Xmx256m
JVMOption2:-Xrs
JVMOption3:-XX:+HeapDumpOnOutOfMemoryError

If I put the above parameters in the sm.ini not in sm.cfg, the JVM gets created successfully. But as I understand tat, if I use 256M as min and max, the garbage collection wont happen and unused memory wont get freed up.

If I amend the above same parameters, with
Xms384m and Xmx512m in sm.ini, after the recycle, i get the below error in sm.log



5248( 5252) 11/19/2009 21:18:28 RTE E **** CreateJavaVM() failed! rc = -4
5248( 5252) 11/19/2009 21:18:28 RTE D -Xms384m
5248( 5252) 11/19/2009 21:18:28 RTE D -Xmx512m
5248( 5252) 11/19/2009 21:18:28 RTE D -Xrs
5248( 5252) 11/19/2009 21:18:28 RTE D -XX:+HeapDumpOnOutOfMemoryError
5248( 5252) 11/19/2009 21:18:28 RTE D -Djava.class.path=.\lib\common-7.10.jar;.\lib\commons-codec-1.3.jar;.\lib\commons-logging-1.1.jar;.\lib\embedded-7.10.jar;.\lib\FastInfoset-1.1.1.jar;.\lib\htmlparser-1.6.jar;.\lib\jgroups-all-2.5.0.jar;.\lib\k2.jar;.\lib\kmplugin-7.10.jar;.\lib\log4j-1.2.15.jar;.\lib\mail-1.4.jar;.\lib\saaj-api-2.1.jar;.\lib\saaj-impl-2.1.jar;.\lib\scautoserver-7.10.jar;.\lib\utility-7.10.jar;C:\Program Files\Common Files\Hewlett-Packard\HPOvLIC\lib\OvLic.jar
5248( 5252) 11/19/2009 21:18:28 RTE D -Djava.library.path=D:\Program Files\HP\Service Manager 7.10\Server\RUN
5248( 5252) 11/19/2009 21:18:28 RTE D -Djava.endorsed.dirs=.\lib\endorsed
5248( 5252) 11/19/2009 21:18:28 RTE D -Djavax.net.ssl.trustStore=D:\Program Files\HP\Service Manager 7.10\Server\RUN\jre\lib\security\cacerts
5248( 5252) 11/19/2009 21:18:28 RTE D -Djavax.net.ssl.trustStorePassword=changeit
5248( 5252) 11/19/2009 21:18:28 RTE D -Djavax.net.ssl.keyStore=D:\Program Files\HP\Service Manager 7.10\Server\RUN\smsrv.keystore
5248( 5252) 11/19/2009 21:18:28 RTE D -Djavax.net.ssl.keyStorePassword=changeit
5248( 5252) 11/19/2009 21:18:28 RTE D -Djava.net.preferIPv4Stack=true
5248( 5252) 11/19/2009 21:18:28 RTE E Error! CreateJavaVM() failed! rc = -4
5248( 5252) 11/19/2009 21:18:28 RTE E HP Service Manager is unable to start. Failed to initialize or attach to shared memory environment
5248( 5252) 11/19/2009 21:18:28 RTE E Could not create JVM.


Y is that so?? Please guide.

Thanks
DEV---
I am Listening..
Honored Contributor
James Herold
Posts: 861
Registered: ‎07-20-2006
Message 8 of 12 (988 Views)

Re: JVM options

You mention:

"In the KB article of SM7, i have read that the JVM options shud be put in sm.ini "

Can you send me that article?

Thanks.

jherold@goodyear.com
Valued Contributor
Igor 'bis' Barinov
Posts: 69
Registered: ‎02-16-2008
Message 9 of 12 (988 Views)

Re: JVM options

dev0085,
you should play with parameters starting additional port in command line (easy to control-c it) and see with how much memory your process actually will start (hint: try -Xmx350m for start)
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 10 of 12 (988 Views)

Re: JVM options

Hi,

Below is the link to KB article. Let me know if you are able to see this , james.

http://support.openview.hp.com/selfsolve/document/KM725908?searchIdentifier=5c781df%3a1251319cae1%3a...


Barinov,

Did you mean to try starting an sm.exe process from the command line by giving JVM options as arguments???
Is there any way to record that into a log rather than checking the task manager for seeing the cpu usage and memory ???

Does verbose cmd works?? Please clarify .

Thanks
DEV ---

I am Listening..
Valued Contributor
Igor 'bis' Barinov
Posts: 69
Registered: ‎02-16-2008
Message 11 of 12 (981 Views)

Re: JVM options

Look at pic.
First time i have to start sm with incompatible by host amount of memory for Xmx setting (512) and got relevant error (Could not reserve). In case i will start sm with the same settings as a service i will not get same error - only rc=-4 one which says nothing.

Second time 100M lower - sm started in cmd.
So i can play more, or put parameter in config file.
Bingo.

HTH,
Igor
Respected Contributor
brav0
Posts: 931
Registered: ‎12-07-2008
Message 12 of 12 (981 Views)

Re: JVM options

Hi Igor,

In 7.10 I tried with the JVM options , it helped to bring down the number of times we restarted the servlet containers due to heap memory issues.

But everytime i tried with Xmx:512MB it failed.

Later as per the docs, i put Xms:254 and Xmx:254, it worked for me. why i didnt try this option is becoz in the KB article, HP has mentioned that for the garbage collection to work properly both xms and xmx shud be different.

Right now, after the upgrade to sm7.11, we are again facing with memory leakage issues which Hp is figuring out in the next release.

Now in my environment, instead of trying to increase the entire application heap, i have tried to increase the heap of each of the servlets . I have done this in the dev environment.

So are you aware of any work arounds for memory issues in sm7.11

Thanks
DEV---
I am Listening..
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.