Re: Traceroute failed - nnmi 9.2 (526 Views)
Reply
Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 1 of 11 (546 Views)
Accepted Solution

Traceroute failed - nnmi 9.2

Hello !

 

Environment;
NNM 9.20,9.20.001,9.22.002,9.23.003,9.23.004 on RHEL 5.10 (v2.6.18-371.4.1.el5)

 

Getting an error message when i try to run a path view in NNMi;
Traceroute failed, node is unreachable; unable to find access router for node Abb-bjorndal1ar1; Traceroute failed, node is unreachable; unable to find access router for node Abb-aseral1ar1

 

I think I get this error message because in our datacenter environment we are not allowed to run traceroute <host>.
We have to run traceroute -I <host> so that it uses ICMP packets instead of UDP(UDP is blocked)

 

So to make nnmi run traceroute with this option I have created an alias for the user nmsproc in bashrc so when this user tries to run the command traceroute it will automatically apply -I

 

Still I cant get the traceroute to work.

 

I found this;
Trace Route (from server)
/nnm/launch?cmd=runTool&tool=traceroute&nodename=${getAttrOrName(hostname)}

 

Where can i find this tool, i can i modify it ?

 

I was wondering if i have missed something or can i solve this problem another way?

Thanks in advance!

Honored Contributor
dieter boschung
Posts: 236
Registered: ‎12-18-2008
Message 2 of 11 (526 Views)

Re: Traceroute failed - nnmi 9.2

Hi Kristoffers

 

The traceroute command is part of a set of pre-defined command, you can get a list by pointing your browser to:

 

http://server.name/nnm/cmd

 

As you will see, the traceroute command will not accept any option on that level. Another problem is that the Linux traceroute command with the option "-I" can only be run by the user root. ovjboss runs as nmsproc and has no rights to run that command option.

 

However, what you could do is to add the -I option to the traceroute command by.

 

- mv /bin/traceroute /bin/traceroute.UDP

- create a script /bin/traceroute with this contents:

 

#!/bin/sh
/bin/traceroute.UDP -I $*

- make the script executable: chmod 755 /bin/traceroute

 

- allow the user nmsproc run the sudo command to execute /bin/traceroute.UDP

 

Please consult the sudo and sudoers man pages if needed.

 

Hope this helps

Dieter

HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 3 of 11 (522 Views)

Re: Traceroute failed - nnmi 9.2

Thank you Dieter!

 

I have taken into account that the command can only run with sudo or by root.

 

I applied this solution;

https://access.redhat.com/site/solutions/226673

chmod +s /bin/traceroute

 

All this is done on my stageing enviornment to test if it works.

If i try to run the traceroute from either the stage or the production environment, the result of the traceroute is still the same as the result i get in my production environment(were i have made no changes)

 

I tried accessing http://nnmi1.lysetele.net/nnm/cmd

got HTTP Status 404 - /nnm/cmd

 

I can try to do it the way you explained but i think i would do the same thing that my approach dose.

 

Thank you i will get back after i have testet it.

 

 

There has to be something else im missing.

 

 

Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 4 of 11 (518 Views)

Re: Traceroute failed - nnmi 9.2

Hi again,

 

It looks like this worked.

I get a full traceroute now.

 

But the problem that triggered this troubleshooting was that i got this error message when i tryed to display a pathview;

Traceroute failed, node is unreachable; unable to find access router for
node Abb-bjorndal1ar1.lyse.mgmt; Traceroute failed, node is
unreachable; unable to find access router for node
Abb-aseral1ar1.lyse.mgmt

 

and im still getting this error message

 

Any ideas ?

 

 

 

Honored Contributor
dieter boschung
Posts: 236
Registered: ‎12-18-2008
Message 5 of 11 (497 Views)

Re: Traceroute failed - nnmi 9.2

Hi Kristoffers

 

The message you see in the UI is not really from the traceroute binary, but from the PathView mbean. PathView itself does not just run traceroute, but does a series of DB queries, pings and finally a traceroute if required to find the path. You can enable tracing on the path calculation (/opt/OV/support/nnmsetlogginglevel.ovpl com.hp.ov.nms.path FINEST).
Then you will see a trace like this ($NNM_LOG/nnm/nnm-trace.log):

 

<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139)  node = <node> ping failed, cannot find router.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139)  node = <node> trace to node failed.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) node = <node> refine router begins.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) node = <node> no router ip set.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) node = <node> find router by snmp failed.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) node = <node> get node trace begins.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139)  node = <node> ping failed, cannot find router.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) node = <node> traceroute failed.
<time> FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-139) edge 0 = NmsPathEdgeInfo:
..

 

I am afraid, but the current implementation requires ICMP access for PathView to work to find the access gateway to the network where the nodes reside for which you want to run PathView.

 

The failure to find the access gateway does not mean that no path can be found, but it may not be complete.

 

Hope this helps

Dieter

HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 6 of 11 (492 Views)

Re: Traceroute failed - nnmi 9.2

[ Edited ]

Thank you so much for your time Dieter!

 

I have attached the loggs and a screenshot from nnmi.

 

The messages i get is that 

2014-02-28 10:21:12.443 FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-24)  node = 213.167.116.70 not found in topology

2014-02-28 10:21:12.450 FINE  [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-24)  node = sdfnyga1cr1.lyse.mgmt ping failed, cannot find router.

 

this is weird since i can both ping the router and run a traceroute to it, aswell as nnmi has SNMP access to it as you will se in my attachments

 

Edit,

Just on a side note, this was done from our stageing environment where i implementet the traceroute script you suggested.

 

When i try the same node in our production environment;

2014-02-28 10:50:11.186 FINE [com.hp.ov.nms.path.ejb.internal.NmsPathApiBean] (tomcat-exec-80) begin node = sdfnyga1cr1.lyse.mgmt
2014-02-28 10:50:11.190 FINE [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-80) begins.
2014-02-28 10:50:11.192 FINE [com.hp.ov.nms.path.ejb.internal.NmsPathModelBean] (tomcat-exec-80) protected ip not found for address 213.167.116.70.
2014-02-28 10:50:11.208 FINE [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-80) node = 213.167.116.70 in topology and is a router
2014-02-28 10:50:11.209 FINE [com.hp.ov.nms.path.ejb.internal.NmsPathEdgeInfoBean] (tomcat-exec-80) node = sdfnyga1cr1.lyse.mgmt is a router.

Honored Contributor
dieter boschung
Posts: 236
Registered: ‎12-18-2008
Message 7 of 11 (450 Views)

Re: Traceroute failed - nnmi 9.2

Hi Kristoffers

 

Is the ip-address 213.167.116.70 managed in NNMi, does it have a status or is it set to 'NoStatus' and isn't monitored? Go to the IP-Addresses tab and check status of above address.

 

On your second system, the code seems to find the router and identifies it as such, does that system still print the header in pathview telling that traceroute failed?

 

 

I think this needs further investigation, best would be you open a case with HP support for this problem for further analysis.

 

Kind regards,

Dieter

HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 8 of 11 (438 Views)

Re: Traceroute failed - nnmi 9.2

Both systems should be identical as it is just a copy.

 

ip-address 213.167.116.70 is the management address for the router and can be found on Lo0 interface.

This address is not managed, but it is the address that nnmi uses to communicate with the router or switch.

 

 

 

On our second system the header in pathview dose not appear for this ip.

 

Thank you I will open a support case with HP

Ill mark your script as the solution for the traceroute problem, thank you so much!

 

 

Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 9 of 11 (417 Views)

Re: Traceroute failed - nnmi 9.2

Some futher investigation here,

 

If the node i am trying to run a pathview on is not in the "Default Tenant" it will not work.

 

In our system we have many different tenants mapped up to security groups so almost none of our nodes has the "Default Tenant"

 

I have opened a suppot ticket on this.

Honored Contributor
dieter boschung
Posts: 236
Registered: ‎12-18-2008
Message 10 of 11 (407 Views)

Re: Traceroute failed - nnmi 9.2

Thanks Kristoffers

 

"If the node i am trying to run a pathview on is not in the "Default Tenant" it will not work."

 

That is a good hint and starting point for the investigation of the case. Do you use Tenants to implement Overlapping Address Domains or do you use the Tenants to implement operater access levels?

 

As a hint, you may have to guide PathView by providing connectivity information in the file /var/opt/OV/shared/nnm/conf/PathConnections.xml ; Use the online help and search for "Configure a Path View Map" for more information about PathConnections.xml and a few usage examples.

 

Kind regards,

Dieter

HP Support
If you find that this or any post resolves your issue, please be sure to mark it as an accepted solution.
Advisor
Kristoffers
Posts: 14
Registered: ‎02-27-2014
Message 11 of 11 (402 Views)

Re: Traceroute failed - nnmi 9.2

Thanks for the tip!

 

We have 30 different partners with network equipment to separte this equimpment we created tenants for all our partnes and mapped that tenant to a security group so when the tech for the partner logs into nnmi he will only see the nodes for the securitygroup the partner belongs to.

 

So when we add a new node to the system i will load a seed with the initial discovery tenant as that partnes tenant.

 

I think thats the only purpose for the tenants in our system.

 

So the case i have with HP now asks the question,

Is there a way to solve this problem without changeing the tenant for all our nodes ?

 

I was reading the help pages and found that there will be other problems aswell here, like creating a layer2 map with nodes beloning to different tenants will not work either.

 

You have been very helpful Dieter thanks a bunch!

 

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.