DECnet IV and duplicate physcail addresses (300 Views)
Reply
Esteemed Contributor
Jess Goodman
Posts: 387
Registered: ‎09-21-2004
Message 1 of 11 (300 Views)
Accepted Solution

DECnet IV and duplicate physcail addresses

I am running DECnet IV on Alpha VMS 7.3-2 (I'd rather not get into pros/cons of OSI/phrase V)

I need to change the interface that DECnet is running on for all 21 nodes in my cluster without doing a cluster reboot and, if at all possible, without a rolling reboot either.

I have set up one node as a level 1 router with active circuits/lines on both the old and new lan segments (vlans).

I also moved the active circuit/lines for two other nodes. They can talk to each and to the rest of the cluster via DECnet or IP.

Only then did I realize that these three nodes now have two active NICs that appear to have the same physical MAC address (due to the way DECnet IV changes the mac address to be based on DECnet address).

Even if I did a rolling reboot, I would still need the router node for the duration of the reboots , and there is no way that I know of to prevent a routing node from having duplicate mac addresses.

Device PrefCPU Medium/User Version Link Speed Duplex Auto BufSize MAC Address Type Description
------ ------- ----------- ------- ---- ----- ------ ---- ------- ---------------- ------------ -----------
EWA0 Ethernet X-79A1 Up 1000 Full Yes 1500 AA-00-04-00-B3-05 UTP DEGXA-TB
00-23-7D-A9-4C-5F (default)
EWC0 Ethernet X-79A1 Up 1000 Full Yes 1500 AA-00-04-00-B3-05 UTP DEGXA-TB
00-1F-29-32-D5-39 (default)

This will obviously cause duplicate entries in an ARP table:

$ PING 172.20.90.35
PING 172.20.90.35 (172.20.90.35): 56 data bytes
64 bytes from 172.20.90.35: icmp_seq=0 ttl=64 time=0 ms
64 bytes from 172.20.90.35: icmp_seq=1 ttl=64 time=0 ms
64 bytes from 172.20.90.35: icmp_seq=2 ttl=64 time=0 ms
64 bytes from 172.20.90.35: icmp_seq=3 ttl=64 time=0 ms


----172.20.90.35 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max = 0/0/0 ms
'AX40$ PING AX35
PING ax35 (10.1.4.65): 56 data bytes
64 bytes from 10.1.4.65: icmp_seq=0 ttl=64 time=0 ms
64 bytes from 10.1.4.65: icmp_seq=1 ttl=64 time=0 ms
64 bytes from 10.1.4.65: icmp_seq=2 ttl=64 time=0 ms
64 bytes from 10.1.4.65: icmp_seq=3 ttl=64 time=0 ms


----ax35 PING Statistics----
4 packets transmitted, 4 packets received, 0% packet loss
round-trip (ms) min/avg/max = 0/0/0 ms
'AX40$ UCX SHOW ARP
Cnt Flags Timer Host Phys Addr
1: UCS 520 10.1.4.25 aa-00-04-00-9f-05
2: UC 3 10.1.4.65 aa-00-04-00-b3-05
3: UC 14 10.1.5.66 00-0d-93-9c-71-d6
4: UC 0 10.1.14.10 00-90-7f-33-56-ce
5: UCS 255 10.20.100.250 00-18-8b-53-bd-5f
6: UC 1 10.20.100.254 00-24-51-96-84-00
7: UC 31 172.20.13.12 00-24-51-96-84-00
8: UC 18 172.20.90.35 aa-00-04-00-b3-05
9: UCS 281 172.20.90.254 00-24-51-96-84-00
10: UCS 400 192.168.248.111 00-11-43-d1-bb-46
11: UCS 236 172.20.75.235 00-00-f8-07-3c-57
12: UCS 281 172.20.75.254 00-24-51-96-84-00

What problems, if any, could duplicate DECnet-based mac addresses cause?
I have one, but it's personal.
Please use plain text.
Honored Contributor
Hoff
Posts: 4,896
Registered: ‎01-29-2006
Message 2 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

The IVADDR invalid media address error you report (which is largely irrelevant to the question, and widely discussed over the years) results from having multiple colliding DECnet addresses and multiple NICs active on each.

As for the general requirements, when you change the DECnet Phase IV host address, then you must also change change the SCSSYSTEMID parameter, and when you change SCSSYSTEMID you either then need to change both SCSSYSTEMID and SCSNODE (to a wholly new and unique address and name pair) or you need to reboot the whole cluster at once.

Both of these get discussed all the time, so I expect you already know this (because you've already Googled for the errors and the requirements and the basic situation) and you're likely now looking for some form of confirmation of this behavior.

So here you go...

Yes.

You need to reboot the whole cluster.

The pairing of the node name and the SCS address are remembered by all nodes, and for the lifetime of the cluster. Change only one of the two, and the host will not be permitted to (re)join the cluster. So you need change both, or reboot everything.

Or you need wholly new host names to be paired with the new addresses at least until you can reboot the whole cluster. Or until a migration over to that DECnet Phase That Shall Not Be Discussed.

And FWIW, DECnet Phase IV does not require a router for local Ethernet-based operations, so you can safely reboot without that being around temporarily. End nodes on a broadcast network can communicate without a level one router available.
Please use plain text.
Esteemed Contributor
Jess Goodman
Posts: 387
Registered: ‎09-21-2004
Message 3 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

Hoff,

Please reread my O.P. No where in it do I say that I wish to change a DECnet address, or any SCSSYSTEMID value. I did not mention an IVADDR error either.

What I am changing is which NIC is the active circuit/line for all the nodes. Since the old and new nics are on separate VLANs, I need DECnet routing until all the nodes have been moved.
I have one, but it's personal.
Please use plain text.
Honored Contributor
Hoff
Posts: 4,896
Registered: ‎01-29-2006
Message 4 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

Ok.

Anyway.

What problems, if any, could duplicate DECnet-based mac addresses cause?

DECnet Phase IV won't start on the LINE.

End-nodes only have one active path at a time.

End-nodes don't need a router.

Phase IV cannot have multiple connections to the same LAN or VLAN.

Connect to each via telnet, and SET and DEF the old CIRCUIT and LINE to OFF and the new CIRCUIT and LINE to ON.

Ah, well. Time for me to exit ITRC, I suspect.
Please use plain text.
Esteemed Contributor
Jess Goodman
Posts: 387
Registered: ‎09-21-2004
Message 5 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

These interfaces also have TCP/IP configured on them. I am worried that there could be a problem caused by the duplicate entries for my nodes in the ARP tables of systems with IP connections to these nodes.

So far I haven't seen any problems on these three nodes, but they do very little IP. Some of my others nodes use IP for important applications.
I have one, but it's personal.
Please use plain text.
Esteemed Contributor
Colin Butcher
Posts: 351
Registered: ‎11-28-2003
Message 6 of 11 (301 Views)

Re: DECnet IV and duplicate physcail addresses

Multiple interfaces (NICs) with the same MAC address are fine, provided that all such NICs are in different VLANs (or on separate LANs).

The ARP cache will map the IP address to the MAC address of the NIC in the VLAN that the IP V4 subnet is mapped to. It doesn't matter if it's the same MAC address as another NIC, provided that it's in a different VLAN (and implicitly a different IP V4 subnet).

However, you'll find it much easier to give up with DECnet Phase IV and migrate to DECnet-Plus, either with the Phase IV compatible transport, or using DECnet over IP. It lets you control which interfaces you change the MAC address on, among many other useful things.

Cheers, Colin (http://www.xdelta.co.uk).
Entia non sunt multiplicanda praeter necessitatem (Occam's razor).
Please use plain text.
Honored Contributor
Robert Gezelter
Posts: 2,367
Registered: ‎10-30-2003
Message 7 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

Jess,

Is ARP being bridged between the two VLANs?

If ARP is not being bridged, then there should be no problem with MAC addresses appearing on different VLANS.

You will need to use routers between the VLANs for both DECnet and IP.

- Bob Gezelter, http://www.rlgsc.com
Please use plain text.
Respected Contributor
Bob Blunt
Posts: 314
Registered: ‎05-01-2003
Message 8 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

Jess, what the others have said is good stuff. But you have to remember that, like Hoff said, you can't change your SCSSYSTEMID or SCSNODE on the fly to match a new address (IF you had any desire to and it sounds like you don't).

The only gotcha? DECnet Phase IV HAS, must, better, needs to start FIRST. DECnet Phase IV grabs the interface and makes sure that it has the ability to fiddle with the MAC address. The only way I know to get a duplicate MAC is to have duplicate DECnet Phase IV addresses in your config. And, just for fun, IF you have a duplicate SCSSYSTEMID then (if memory serves) you're not going to get that system/root to boot into a cluster AND in order for DECnet Phase IV to start the SCSSYSTEMID and the converted Phase IV address have to match.

So what happens IF you manage to start two nodes on one network with the same DECnet address? First one starts fine, second one fails with an invalid address error. As long as the nodes in question are sharing the same physical or virtual wire DECnet will, within reason, prevent you from having two nodes with duplicate addresses.

There is NO relationship between the original MAC address on a NIC and the DECnet Phase IV-massaged MAC address. DECnet doesn't care when it sets that address what it was because it changes it to what it wants. Specifically that would be:

AA-00-04-00-xx-yy. Swap xx and yy so you get a hex number yyxx and convert to decimal. Integer divide by 1024 to get the area. Multiply the number you got for the area by 1024 to get the area base and subtract that from the complete number that you converted from hex to get the node's address in the area. To generate the decimal number that gets converted to hex multiply the area times 1024 and add the node. For example: 33.883 33*1024+883= 34675 in hex is 8773. Flip the bits which should give you a DECnet Phase IV MAC address of AA-00-04-00-73-87.

Now having gone through all that what might happen with IF somehow you managed to get duplicate DECnet Phase IV addresses? In all honesty a lot depends on who is playing the game and what they "do" in your network. For instance if you had a node (magically called node "A") that lives and does real work as a VMS host in your network and somehow someone sneaks in a PC and manages to set it up with a duplicate DECnet Phase IV address (DECnet named "PC") you might get a valid connection to "A" one time and then get a rejection error from trying to connect to *what you think is node "A"* the next time. The exact error could depend on whatever object or resource you're trying to touch on node A. Could be FAL, CTERM, a specific task, etc. If A normally supports that task and the PC temporarily claiming to be "A" grabs that request you'll get an error.

While it is good to have a central clearinghouse of nodenames and addresses in a Phase IV network mistakes CAN (and have) happened. Even if you're sharing the remote node database across all the nodes in a cluster they'll all claim that 33.883 is node A but once the packets hit the wire, especially on Ethernet, if node PC sees the request first and responds first then you'll get an error.

DECnet Phase IV is supposed to check to make sure that noone else responds to an address before it starts and if a response is received then it *isn't supposed to start*. Situations can happen where this isn't guaranteed, mistakes do happen. Not often, but sometimes they do. Only one node responds to action directed at the address at a time so, in general, you get an error if the wrong node responds.

A lot of thought and planning went into trying to make Ethernet safe for DECnet. Problems can occur and finding them can be a bear. Getting different responses from retrying some network activity repeatedly is one way to check. If, by some chance, you manage to get more than one node using the same MAC address defined by DECnet Phase IV then other protocols may behave strangely or not work at all. I'm mainly looking at this from the perspective of DECnet Phase IV's behavior.

bob
Please use plain text.
Esteemed Contributor
Jess Goodman
Posts: 387
Registered: ‎09-21-2004
Message 9 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

Bob G. and Colin,

Thanks for the info. I have gone ahead and moved the active DECnet line/circuit to the "new" interface for the rest of my cluster. I have also configured TCP/IP on the same interfaces.

I don't yet have any important IP traffic on this new subnet, but I am now confident that there will be no problems when I do. As you two implied, the only systems that can see the duplicate MAC address in their ARP table are my VMS systems, since they are the only boxes that have NICs on both subnets.

Bob B., because your response was so well written I gave you three points for it, even though it had nothing to do with my problem and I am quite familiar with the rules you mention. It might be useful to others who google upon it.

Jess
I have one, but it's personal.
Please use plain text.
Advisor
STEPHEN CONWAY_1
Posts: 20
Registered: ‎09-27-2006
Message 10 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

> Bob B., because your response was so well written

But it could be improved. The 'DECNET must start first' rule is relaxed in the version of TCPIP that supports clusters over IP (I don't remember the version info). If you have an IP cluster IP must start first. When (if) DECNET starts later IP will roll with the punch, get the new MAC address, send out a gratuitous ARP, etc.
Please use plain text.
Esteemed Contributor
Colin Butcher
Posts: 351
Registered: ‎11-28-2003
Message 11 of 11 (300 Views)

Re: DECnet IV and duplicate physcail addresses

You might find this technical journal article of use for some of the background:
http://h71000.www7.hp.com/openvms/journal/v5/decnet.pdf

Cheers, Colin.
Entia non sunt multiplicanda praeter necessitatem (Occam's razor).
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation