Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout (586 Views)
Reply
Frequent Advisor
Stefano Ferri
Posts: 45
Registered: ‎05-22-2006
Message 1 of 8 (938 Views)

uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

[ Edited ]

Hi all,

we have problem with our ucmdb server, the issue is the following:

While we try to discovery a virtual center by vim we have an error: "remote agent timeout" while the pattern is discovering the datastores. We have tried to increase the wmi and vmware protocol timeout in the credential form and we tried to increase this value on the variable remoteProcessTimeout (set to 14400000) within the file globalSettings.xml and the variable appilog.agent.local.max.worker.runtime within the file discoveryProbe.properties, but this don't has effect to agent remote timeout that seems to be fixed to value 900000 (15minutes).

We have a virtual center that manage over 150 virtual machines and we have to increase the timeout to value around 3+ hours.

Can anyone help me to find how can I increase this time out value?
The error the communication log show is :

log start="18:16:48" severity="debug"> ...... 381 datastores</log>
    <log start="18:28:5" severity="debug">AxisFault
 faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
 faultSubcode:
 faultString: java.net.SocketTimeoutException: Read timed out
 faultActor:
 faultNode:
 faultDetail:
    {http://xml.apache.org/axis/}stackTrace:java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:312)
    at sun.security.ssl.InputRecord.read(InputRecord.java:350)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:893)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:850)
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
    at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
    at org.apache.axis.transport.http.HTTPSender.readHeadersFromSocket(HTTPSender.java:583)
    at org.apache.axis.transport.http.HTTPSender.invoke(HTTPSender.java:143)
    at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
    at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
    at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
    at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
    at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
    at org.apache.axis.client.Call.invoke(Call.java:2767)
    at org.apache.axis.client.Call.invoke(Call.java:2443)
    at org.apache.axis.client.Call.invoke(Call.java:2366)
    at org.apache.axis.client.Call.invoke(Call.java:1812)
    at com.vmware.vim25.VimBindingStub.retrieveDiskPartitionInfo(VimBindingStub.java:64025)
    at sun.reflect.GeneratedMethodAccessor958.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java)
    at org.python.core.PyMethod.__call__(PyMethod.java)
    at org.python.core.PyObject.__call__(PyObject.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx24.retrieveDiskPartitionInfo$113(_vmware_vim_base:830)
    at org.python.pycode._pyx24.call_function(_vmware_vim_base)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx24._discoverExtents$116(_vmware_vim_base:876)
    at org.python.pycode._pyx24.call_function(_vmware_vim_base)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx24._discoverDatacenter$119(_vmware_vim_base:953)
    at org.python.pycode._pyx24.call_function(_vmware_vim_base)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx24._discoverDatacenters$118(_vmware_vim_base:914)
    at org.python.pycode._pyx24.call_function(_vmware_vim_base)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx24.discoverVirtualCenter$125(_vmware_vim_base:1037)
    at org.python.pycode._pyx24.call_function(_vmware_vim_base)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyInstance.invoke(PyInstance.java)
    at org.python.pycode._pyx30.DiscoveryMain$1(vmware_virtualcenter_topology_by_vim:52)
    at org.python.pycode._pyx30.call_function(vmware_virtualcenter_topology_by_vim)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyFunction.__call__(PyFunction.java)
    at org.python.core.PyObject.invoke(PyObject.java)
    at org.python.pycode._pyx31.f$0(&amp;lt;string&amp;gt;:2)
    at org.python.pycode._pyx31.call_function(&amp;lt;string&amp;gt;)
    at org.python.core.PyTableCode.call(PyTableCode.java)
    at org.python.core.PyCode.call(PyCode.java)
    at org.python.core.Py.runCode(Py.java)
    at org.python.core.Py.exec(Py.java)
    at org.python.util.PythonInterpreter.exec(PythonInterpreter.java)
    at com.hp.ucmdb.discovery.library.execution.impl.ExecutionEngineImpl.executeScript(ExecutionEngineImpl.java:214)
    at com.hp.ucmdb.discovery.library.execution.impl.ExecutionEngineImpl.executeScript(ExecutionEngineImpl.java:189)
    at com.hp.ucmdb.discovery.library.execution.impl.ExecutionEngineImpl.executeScript(ExecutionEngineImpl.java:185)
    at com.hp.ucmdb.discovery.library.execution.impl.ExecutionEngineImpl.execute(ExecutionEngineImpl.java:102)
    at com.hp.ucmdb.discovery.probe.services.dynamic.core.TablesProcessorUtil.processDestinationTables(TablesProcessorUtil.java:136)
    at com.hp.ucmdb.discovery.probe.services.dynamic.core.DynamicService.discover(DynamicService.java:71)
    at com.hp.ucmdb.discovery.probe.agents.probemgr.taskexecuter.JobExecuter.launchTask(JobExecuter.java:1238)
    at com.hp.ucmdb.discovery.probe.agents.probemgr.taskexecuter.JobExecuter$JobExecuterWorker.launch(JobExecuter.java:993)
    at com.hp.ucmdb.discovery.probe.agents.probemgr.taskexecuter.JobExecuter$JobExecuterWorker.executeTask(JobExecuter.java:930)
    at com.hp.ucmdb.discovery.probe.agents.probemgr.taskexecuter.JobExecuter$JobExecuterWorker.run(JobExecuter.java:836)

    {http://xml.apache.org/axis/}hostname:******
</log>
    <log start="18:28:5" severity="debug">VMware VIM: Connection timed out</log>
    <log start="18:28:5" severity="debug">Reporting error code 401 to framework.</log>
    <log start="18:28:5" severity="debug">Error message is: VMware VIM: Connection timed out</log>
    <DISCONNECT start="18:28:6" duration="241" CMD="client_disconnect" RESULT="" IS_NULL="Y" type="vmware" credentialsId="51_1_CMS" />
</execution>

Thanks for any help,

Stefano FERRI

HP Expert
Dima Gomel
Posts: 919
Registered: ‎03-09-2010
Message 2 of 8 (924 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

Dear Stefano,

To get better understanding please provide full  communication log.

 

 

Regards
-Dima, HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at bottom to say 'Thanks'
I'll be happy to be in touch at LinkedIn .
Frequent Advisor
Stefano Ferri
Posts: 45
Registered: ‎05-22-2006
Message 3 of 8 (903 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

HI Dima,

 

thank you for your support, the log is in attachment.

 

Stefano

HP Expert
Dima Gomel
Posts: 919
Registered: ‎03-09-2010
Message 4 of 8 (898 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

Dear Stefano,

 

I'd ask you to open support case with the problem.

Please decribe the problem, add full communication log  with results and information from JMX console -> Server Services ->viewSystemInformation.

Please state on the case that I asked about elevation of the case.

Regards
-Dima, HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at bottom to say 'Thanks'
I'll be happy to be in touch at LinkedIn .
Occasional Advisor
John Bethanis
Posts: 7
Registered: ‎09-16-2009
Message 5 of 8 (594 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

Is there an answer to this? I am having the same issue, since we have upgraded VCentre 5.1.

 

Cheers

John

HP Expert
Dima Gomel
Posts: 919
Registered: ‎03-09-2010
Message 6 of 8 (586 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

Dear  John,

Please note that VMware 5.1 discovery isn't officially supported by CP11.  But R&D do all possible, and it's known to be working.

Please install latest Update for CP11. It contains many defect fixed toward VMware 5.1 discovery and will be beneficial in your situation. It's availale on HP LN. Let me know if you have problems to find it.

Regards
-Dima, HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at bottom to say 'Thanks'
I'll be happy to be in touch at LinkedIn .
Occasional Advisor
John Bethanis
Posts: 7
Registered: ‎09-16-2009
Message 7 of 8 (578 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

I had discorvery working with 5.0 and with 5.1. A few weeks later on 5.1, im getting this timeout errors.

 

 Nothing had changed in the back end. ( That I know of anyway ).

 

Thanks for your advise!

Cheers

John

HP Expert
Dima Gomel
Posts: 919
Registered: ‎03-09-2010
Message 8 of 8 (567 Views)

Re: uCMDB 9.0.5 CP11 Up 5 : VMWare VCenter Timeout

[ Edited ]

Dear John,

The change could not be only on UCMDB side, but on vCenter as well. For instance, Bunch on new ESX server were added, hundreeds of new VMs raised or security patch  was installed. The CP Updates mostly contains bug fixes comming from current UCMDB customers.

I highly recommend installation of CP Update. Please open support case if you need official statement.

 

Regarding the timeout.

CP 11 has mplementation of chunking mechanism that allow us to select data by lower partitions. It also can help us to avoid the "time out error". Could you please try following:
1.Please do backup  of open _vmware_vim41.py and open it for editing:
2. Find following section:
_DEFAULT_PAGE_SIZE_DATACENTER = 50
_DEFAULT_PAGE_SIZE_COMPUTE_RESOURCE = 50
_DEFAULT_PAGE_SIZE_HOST = 25
_DEFAULT_PAGE_SIZE_VM = 25
_DEFAULT_PAGE_SIZE_POOL = 50
_DEFAULT_PAGE_SIZE_NETWORK = 50
_DEFAULT_PAGE_SIZE_DATASTORE = 50
_DEFAULT_PAGE_SIZE_DVS = 50
_DEFAULT_PAGE_SIZE_DVPG = 50
 
3. Decrease parameters with such values:
 
_DEFAULT_PAGE_SIZE_DATACENTER = 10
_DEFAULT_PAGE_SIZE_COMPUTE_RESOURCE = 10
_DEFAULT_PAGE_SIZE_HOST = 1
_DEFAULT_PAGE_SIZE_VM = 10
_DEFAULT_PAGE_SIZE_POOL = 10
_DEFAULT_PAGE_SIZE_NETWORK = 10
_DEFAULT_PAGE_SIZE_DATASTORE = 10
_DEFAULT_PAGE_SIZE_DVS = 10
_DEFAULT_PAGE_SIZE_DVPG = 10
 
4. Save changes and rediscover destination.
5. This most probably will resolve the problem or improve the situation( please notice that job will work longer then now).
 
Regards
-Dima, HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Click the Like button at bottom to say 'Thanks'
I'll be happy to be in touch at LinkedIn .
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.