Re: Seeking advice on improving RSTP performance for a simple network (1484 Views)
Reply
Frequent Advisor
Michael BUTOW
Posts: 52
Registered: ‎11-29-2004
Message 1 of 8 (1,587 Views)

Seeking advice on improving RSTP performance for a simple network

Hello all,

 

I have a network setup as illustrated in the attached diagram, which is actually an "internal LAN" for a system consisting of two physical chains. The two chains are linked via fiber optics on the mini-GBIC ports of the 4 x 2810-24G switches.

 

For the sake of my question, please assume that all the indicated links are required for redundancy purposes, and I cannot change the physical setup in a way that loses redundancy.

 

My problem is finding the optimal spanning tree configuration for the 2810-24G switches that will lead to a minimal reconfiguration time for the internal network in case of failure of either of the switches.

 

The software application running on the nodes 1-10 cannot cope with a long outage of the internal LAN.

I am looking at getting the observable outage, e.g. in terms of UDP echo requests not getting through, to as little as possible.

 

So first, I would like to ask the experts what is possible. Under 1 second? Under 3 s?

 

Secondly, how do I get there... let me tell you where I am.

 

I have disabled admin-edge and auto-edge on the interfaces between the switches, i.e. where the "patch cables" are connecting the switches inside a chain, and on the fiber trunks.

LACP is active on the fibers and on the patch connections.

RSTP has been forced via the CLI on all switches.

point-to-point mac has been forced on all the switch-to-switch interfaces.

Hello time has been set to 1 second.

loop-protect is disabled everywhere, and admin-edge + BPDU protection is set on all the ports leading to the nodes (non-switch-to-switch ports).

Additionally, I have

  spanning-tree config-name "INTERNAL_LAN"
  spanning-tree config-revision 1

 

and

  vlan 1
     name "DEFAULT_VLAN"
     untagged 1-24
     ip address 192.168.7.40 255.255.255.0

and


no stack auto-join

All switches are on default STP priority (32768) as a matter of choice, to ease things for future maintenance operations.

 

I am finding the interruptions when I power off the root switch too long for my liking.

I will provide more detailed outage statistics in a follow-up to this, but for the moment, does anyone have suggestions for some obvious performance tuning to RSTP that I might have overlooked, or anything I am doing wrong?

 

Thanks in advance,

Michael

Frequent Advisor
Michael BUTOW
Posts: 52
Registered: ‎11-29-2004
Message 2 of 8 (1,544 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hmm, no-one replying?

Maybe my post was too complex, but don't be afraid to share your thoughts.

 

Here is some data on the outages I get while running a peer-to-peer ping (so from each node in one chain to its opposite partner) while power-cycling the root bridge of the spanning tree.

 

I have attached the same as a graph.

 

The first column is the detected gap (in millisecond), as estimated from the 50-ms pinging interval.

The remaining 10 columns indicate how many occurrences of this gap duration were detected on any node (column 1 is node1, etc.)

 

100 5 8 12 5 13 2 14 5 9 9
150 0 2 3 1 1 1 0 0 2 1
200 3 2 0 0 2 1 2 1 0 0
250 1 1 1 2 1 0 0 1 1 0
300 1 2 1 2 1 2 1 0 0 1
350 4 4 3 3 2 2 3 3 2 3
400 3 1 2 1 4 2 2 1 4 1
450 0 0 0 0 1 0 0 1 1 0
500 0 3 0 0 0 2 1 1 0 1
550 2 2 2 3 3 4 2 3 3 3
600 2 2 2 1 1 1 3 0 1 1
650 1 0 0 0 1 0 0 0 0 0
700 1 0 0 0 0 0 0 0 0 0
1000 0 1 0 0 0 1 0 0 0 1
1050 1 1 1 1 1 1 1 1 0 1
1100 0 0 0 0 0 0 0 0 1 0
1150 0 1 0 0 0 0 0 0 0 1
1200 0 0 0 0 0 1 0 0 0 0
1850 0 1 0 0 0 1 0 0 0 1
4000 1 0 1 1 1 0 1 1 1 0

Trusted Contributor
Antonio Milanese
Posts: 104
Registered: ‎06-23-2006
Message 3 of 8 (1,512 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hello,

 

first let me say that you can read a really enlightening recap on RSTP convergence on the "master" Lapukhov blog:

 

http://blog.ine.com/2010/04/05/understanding-stp-and-rstp-convergence/

 

anyway RSTP fast convergence rely essentially on active handshake exchanges and port types/roles and only on  "unlucky and corner cases" on RSTP timers 3xhello and/or MaxAge..even more.. fast convergence is fastest when you have a direct p-t-p RP link failure since loss-of-carrier immediately trigger the sync process ..on your topology you have the direct p-t-p links so sub sec triggering is possible but you also have:

 

a) the root bridge "on ring"

b) a lot of blk ports with equal path cost/pri (i assume the you haven't changed those..)

c) LACP on p-t-p link (more processing before FWD)

d)  and even worse each brigde with default pri

 

the latter is the worst 'coz when an already unfortunate root failure happen you've a count-to-infinity during root bridge election since a lot of inferior BPDU are moving along to regular handshakes.

 

THIS IS ULTRA BAD since on each "syncing" RSTP invalidate MAC address tables and put non edge ports on discard..

 

To make the story short build a virtual "inner/primary" (ring) and "outer/secondary" (a cross) path topology using stp path pri AND SET a left/right bridge priority so you dont have to wait for a full election based on lower macs.

 

Regards,

 

Antonio

 

 

 

 

 

 

 

Valued Contributor
Helper
Posts: 142
Registered: ‎10-19-2010
Message 4 of 8 (1,484 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hi,

 

Based on the last reply from Antonio, i would like to suggest you two things.

 

- First modify the Bridge Priority of all your switches, to have a predicted and fatest topology convergence. And to avoid rogue switches to become root bridge or create root election on your topology never use default parameter.

Depending on your logical or routing design :

2810 sw1 = 0 or 2810 sw1 = 0

2810 sw2 = 1 or 2810 sw2 = 2

2810 sw3 = 2 or 2810 sw3 = 1

2810 sw4 = 3 or 2810 sw4 = 3

All other nodes (switchs) = 5

 

- Secondly, simplify your topology. I think they are two many links between Chain A and Chain B. Remove the links from sw3 to sw2 and sw4 to sw1.

If possible you will be able to add a second link between sw1 and sw2 (or sw1 to sw 3 and sw2 to sw4 depending on your design) to reduce the case of STP convergence when you loose this inter-switch connection. You need to use a Link Agregation then no spannning-tree convergence will occur when one of the link will fail, your logical connection will be made through a TRUNK.

 

Bye,

Valued Contributor
Helper
Posts: 142
Registered: ‎10-19-2010
Message 5 of 8 (1,482 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hi,

 

Sorry but i forbade to comment some points of your configuration descriptions.

LACP is active on the fibers and on the patch connections.

>>> If you have only one link it is not necessary. This is another L2 protocol with no effect on spanning-tree. It should be disabled if not used.

 

RSTP has been forced via the CLI on all switches.

>>> they are no difference between RSTP and MSTP if you do not used the vlan to instance mapping of MSTP. but it is not problem. Only one thing, config-name and config revision are MSTP options.

 

point-to-point mac has been forced on all the switch-to-switch interfaces.

>>> i don't understand this statement, why and which features is used for that ?

 

Another suggestion, if you are using triple ports for ISL, something interesting is to avoid the autonegociation process to established the link ASAP, so you should force the port configuration to operate at 1000+duplex auto (interface xx speed-duplex auto-1000).

 

Bye,

Frequent Advisor
Michael BUTOW
Posts: 52
Registered: ‎11-29-2004
Message 6 of 8 (1,449 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hello Antonio,

 

thanks for your suggestions, they look really helpful. As I am not on-site anymore, I will have to forward your suggestions for the client to look into.

 

I have previously looked at the PDF from Petr Lapukhov - it is really an excellent document but I could not extract more configuration hints from it.

 

Some questions on your hints

a) root bridgle "on ring" : are saying that because of the more ring-shaped topology of this net, (R)STP is perhaps not able to perform as best it could (i.e. it is more naturally suited for a "tree"-like topology)?

 

b) I got 3 BLK ports when I simulated it with the simulator at

http://www.iro.umontreal.ca/~kropf/ift-6052/exercices/applets/applet1/

Results diagram attached.

 

c) we disabled LACP on a test run (after my original post), but did not find any improvement (outages still so long that application noticed).

 

d) Yes, we left this to make things easier for maintenance, but we did discuss (and not test yet) this point.

It is interesting that you say this could have greatest effect.

I thought that the defaults (use MAC address of bridge to determine priority) would not have much overhead.

 

I think I will take your ideas and perform simulation, as I don't have access to a real rig anymore.

This (http://sourceforge.net/projects/rstplib/ ) looks very useful in that regard.

 

Many thanks again for your advice.

 

Regards,

Michael

Frequent Advisor
Michael BUTOW
Posts: 52
Registered: ‎11-29-2004
Message 7 of 8 (1,442 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hello Helper,

 

thanks for your explicit recommendations. Together with those from Antonio they form a consistent picture of how I could possibly improve the performance.

 

About the "point-to-point-mac" setting:

I decided to set that based on the advice related to this option in the Procurve Management and Configuration Guide, which says: "This parameter is used to tell the port if it is connected to a point-to-point link, point-mac such as to another switch or bridge or to an end node (force-true)."

I set it to true because I saw some other forum posts or discussions (cannot recall where) that said this could possibly reduce time for a link to recover. Also, I looked at this document:

http://h17007.www1.hp.com/docs/interoperability/Avaya/ConfiguringRSTPwithAvaya.pdf

which mentions "Directly connected trunk ports between switches are explicitly configured as point-to-
point for fast convergence and as non-edge ports." This seemed to affirm what I read elsewhere about the point-to-point-mac being helpful on the switch-to-switch links.


It is interesting that you raise the issue of auto-negotiation. Previously, I thought it cannot be disabled in 1000Base-T, but reading the description about the "auto-1000" it seems you are right, that it does at least avoid the speed sensing, and for Gigabit fiber-optic ports there is an option "1000FDx" to nail it down, according to the manual.

 

Thank you again, your posts have been helpful to me.

 

Regards,

Michael

Trusted Contributor
Antonio Milanese
Posts: 104
Registered: ‎06-23-2006
Message 8 of 8 (1,434 Views)

Re: Seeking advice on improving RSTP performance for a simple network

Hello,

 

>a) root bridgle "on ring" : are saying that because of the more ring-shaped topology of this net

well the big difference in RSTP is that EVERY switch ACTIVELY partecipates (trought handshakes/agreements) on topology convergence sending BPDU and not only passively learning from root TC/TCN mechanism...

so when directly connected p-t-p link goes down will be fast notified sending BPDU to other alive neighburs switches and triggering a vector propagation of TC..the problem arises when you've multiple blocked DP beside RP/AP and when you have

a) an indirect RP path failure near root
b) RP+AP failures
c) root failure

therefore in yours "root on ring" topology you have b+c and thus a full split brain since every switch must react to RP+AP failure syncing using both cached, and STALE, topology info and actively sending handshakes to neighbors paths pretending to be the root but a lot of that BPDU are inferior ones since first everyone must agree on WHo is the new root..and degeneratin in a count-to-infinityscenario

>b) I got 3 BLK ports when I simulated it with the simulator at
>http://www.iro.umontreal.ca/~kropf/ift-6052/exercices/applets/applet1/
>Results diagram attached.

ok but this is from a STP simulator and RSTP is a different beast..here fewer redundant paths means faster convergence time expecially when the root is inside the ring; anyway if you still want a full mesh you should easy the RP path selection using path cost to build a two "virtual" primary/secondary above physical ones

>d) Yes, we left this to make things easier for maintenance, but we did discuss (and not test yet) this point.
>It is interesting that you say this could have greatest effect.

yes root election could be particular tricky

below additional readings to learn more on this particolar RSTP matter

http://blog.ine.com/2009/09/07/rstp-and-fast-convergence/
http://www.journaldatabase.org/articles/drstp_simple_technique_for_preventing.html

May I suggest you to use a GS3 lab to simulate this case

Regards,

Antonio

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.