Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS (1514 Views)
Reply
Occasional Advisor
Blichew
Posts: 14
Registered: ‎08-06-2012
Message 1 of 13 (1,670 Views)

BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Hello,

 

the topic says it all:

 

I have one GTW and one DPS both installed on RHEL 5.9.

 

DPS is starting fine while GTW is stuck on mercuryAS with LAUNCHED status. Logs say that port checker found that 1098 is used. Well, it is used, but by mercuryAS. Seems like component order is messed out.

 

It'll be a problem with attaching log files as botyh servers run outside my LAN and I have to connect via multiple VPNs so it's not possible to take out any file from there...

 

 

Regards,

Blichew

HP Expert
Dmitry Shevchenko
Posts: 4,881
Registered: ‎01-30-2008
Message 2 of 13 (1,637 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Try to simply reboot the GW machine (not just restart BSM service). Usually such errors are gone after reboot.

Advisor
bednarcm
Posts: 20
Registered: ‎09-02-2011
Message 3 of 13 (1,613 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Simple restart not help. On second BSM Gateway server (it will be gtw cluster) behaviour is exactly the same. Port 1098 in use by MercuryAS. Any other ideas?

Serwer was installed with all required patches. First hpbsmd run was after all installations.

HP Expert
PatWest
Posts: 836
Registered: ‎03-06-2008
Message 4 of 13 (1,609 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

try running, while the GW is stopped completely:

 

..\HPBSM\tools\CheckPort\checkfreeport 1098

 

to see if there is any other process taking port 1098 before the GW is started.

 

PatW

 

 

Advisor
bednarcm
Posts: 20
Registered: ‎09-02-2011
Message 5 of 13 (1,601 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Before GTW stop:

[]# ./check_free_port.sh 1098
New check free ports
Try #1: Port '1098' in use

Exit code: 1

 After GTW stop:

[]# ./check_free_port.sh 1098
New check free ports
Port '1098' is free.

Exit code: 0

 After hpbsmd restart port is still "in use":

[]# netstat -anp | grep 1098
tcp 0 0 0.0.0.0:1098 0.0.0.0:* LISTEN 26684/MercuryAS

 ... and Gateway is not working, mercuryAS has LAUNCHED status.

HP Expert
PatWest
Posts: 836
Registered: ‎03-06-2008
Message 6 of 13 (1,597 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

please upload JBOSS_BOOT.LOg and TOPAZ_ALL.EJB.LOG from the HPBSM\log direcotory.
Advisor
bednarcm
Posts: 20
Registered: ‎09-02-2011
Message 7 of 13 (1,591 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Logs are attached.

Thanks.

HP Expert
PatWest
Posts: 836
Registered: ‎03-06-2008
Message 8 of 13 (1,576 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

it seems the message broker can't connect to port 2507, this needs to be fixed first:

 


2013-09-23 11:22:29,542 [ReconnectThread-Monitor-[CLUSTER]] (RetryWork.java:70) WARN  - Work Thread failed retry in 5000ms. [name=ReconnectThread-Monitor-[CLUSTER];attempt=3;reason=java.net.UnknownHostException: sap-lnx-068 (broker url: tcp://sap-lnx-068:2507)]
2013-09-23 11:22:30,052 [UCMDB Notification Services Reregister Thread] (BsmSDKLoggerImpl.java:96) ERROR - customer id: 1 , console ==> Failed to register listener due to following failure :java.net.ConnectException: Connection refused
com.hp.ucmdb.api.CommunicationException: java.net.ConnectException: Connection refused

.............

2013-09-23 11:41:23,268 [ClientListener sap-lnx-069_mercury_as_tracker:LOCAL$CONNECTION$  (sap-lnx-069:2507)] (ConnectionTracker.java:99) WARN  - monitor is trying to reconnect. [type=LOCAL;connectId=sap-lnx-069_mercury_as_tracker:LOCAL;status=RECONNECTING]
20

 

I see many more errors in the TOPAZ_ALL file. Is this a new installation, is it a virtual system?

 

BR,

PatW

Advisor
bednarcm
Posts: 20
Registered: ‎09-02-2011
Message 9 of 13 (1,568 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

sap-lnx-068 is the second gateway server (described in first post). Localhost, second gateway and dps is listening on port 2507. After restart I don't see new errors with port 2507.

This is fresh installation on fresh OS. Environment is virtual on  Hyper-V.

 

After BSM 9.20 installation there was run ovii-postinstall.sh. After this there were installed updates 816, 820 and 831. The next step was config-server-wizard.sh and connection to DB.

 

In LAB environment we have once exactly the same problem, when updates were installed after run bsm 9.20. The above described installation steps helped. But not this time.

HP Expert
PatWest
Posts: 836
Registered: ‎03-06-2008
Message 10 of 13 (1,546 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

hi,

could you tell a bit more about your deployment.

Waht database are you using and which (exact) version?

What is the BSM deployment size (small, medium, large) ?

What are CPU/memory sizes on the Linux machines?

Is there a firewall in between GW & DPS server?

Please upload new logs.

Best regards,
Patrick



Advisor
bednarcm
Posts: 20
Registered: ‎09-02-2011
Message 11 of 13 (1,539 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

DB: MS SQL Server 2008 R2 Ent. SP1
BSM deployment size after install: small
CPU GTW: 6x; RAM: 8GB
CPU DPS: 8x; RAM: 24GB
There is no firewall between GTW and DPS. This environment is prepared for production system, but BSM is not yet scaled.

 

GTW can't connect to port 2507 DPS if DPS is disabled - these errors we can ignore.

BSM was working properly after installation 9.20 without patches and updates to 9.22.

 

New logs are attached. Last restart of this GTW was yesterday, about 15:38.

 

Best regards.

HP Expert
PatWest
Posts: 836
Registered: ‎03-06-2008
Message 12 of 13 (1,524 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

hi,

 

There are two things you need to check , in case they are not causing the issue then it's time to open a support case.

 

1. In time in sync between all servers, that includes the DB server too?

 

2. isn't there any other applications using a port (1098 seems to be taken) perhaps a (separate) OM agent installation ?

 

BR,

PatW

Occasional Advisor
Blichew
Posts: 14
Registered: ‎08-06-2012
Message 13 of 13 (1,514 Views)

Re: BSM 9.22 one DPS/one GTW deployment - GTW doesn't start with LAUNCHED MercuryAS

Hello,
the time thing - we have to check (probably tomorrow).
As for port 1098 being taken - that is not the issue. There's nothing else installed there. The port is being used by mercuryAS itself (checked with netstat). It's like mercuryAS is starting and then the check is initiated.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.