Re: is Virtual Connect stacking required? (2635 Views)
Reply
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 1 of 33 (3,594 Views)

is Virtual Connect stacking required?

is Virtual Connect stacking required?

<SHORT>

Is there any requirement other than possible
data flow that forces the need for stacking?
(e.g., required for firmware or config updates).

For example, say we have
VC1,2 for strictly production/client network,
with uplinks going to production backbone,
and
VC3,4 are using their own uplinks for isolated iSCSI traffic.

In this scenario, there will be no data packet flow
from VC3,4 out thru VC1,2.

So why would a vertical stacking be necessary?


Further,
Is horizontal stacking used for anything other than
possible standby to active data flow?

In an active/active setup,
with separate uplink sets for "left" and "right" traffic flow,
it seems that horizontal stacking would not be necessary.

In a VC-Flex10, this would free up X7 and X8,
allowing the use of all 8 uplinks for data packets.
((VC 1/10G enet are always
  horizontally stacked via port X0
))

</SHORT>




<LONGER>

Current setup

VC1 1/10G Enet ......... VC2 1/10G Enet
  used for production network into
  Vmware ESX hosts.

VC3 1/10G Enet ......... VC4 1/10G Enet
  used for iSCSI for Vmware ESX hosts

VC1 - VC3 - vertically stacked via CX4
VC2 - VC4 - vertically stacked via CX4


VC1 & VC2 use their other CX4 uplink
to connect to the Cisco backbone.

******
We are replacing the VC1&VC2 with new
VC-Flex10 modules.
******

The Flex10 VCs only have ONE CX4 connection,
which would be needed to stack vertically,
if stacking is required.
I was hoping NOT to stack vertically, thereby
freeing up the CX4 for additional 10G uplink.
((The backbone switches have an SFP slot open, to which
   we will also be uplinking via "SFP cable"s.
))

</LONGER>


tks
bv

"The lyf so short, the craft so long to lerne." - Chaucer
Respected Contributor
Psychonaut
Posts: 214
Registered: ‎08-31-2011
Message 2 of 33 (3,583 Views)

Re: is Virtual Connect stacking required?

I see no reason for vertical stacking in your configuration.  

 

I would leave 7 & 8 connected - or at least 8.  The horizontal link will keep traffic internal to the chassis for any server to server traffic - rather than going up to the switch and back.  By leaving both you gain a 20Gb pipe for side to side traffic.  

Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 3 of 33 (3,570 Views)

Re: is Virtual Connect stacking required?

Thanks, Psyco
(that is your nickname, right ?>)

*********
1)
*********
>I see no reason for vertical stacking in your configuration.

I don't either, but my question was kinda,
"Is that absolutely true?"

I.e., is the vertical stack absolutely only for
vertical module-to-module vNet data traffic,
where uplinks from VC1,2 are being shared by
VC3,4 to get to the external network.
  ((E.g., a blade might be dual-homed, with, say,
        NICs 1,2 teamed in VLANx
    & NICs 3,4 teamed in VLANy.
  ))

I hear that you are saying "yes",
but do you know for certain ;>)


*********
2)
*********
>The horizontal link will keep traffic internal ...


Ahhh, yes!!
I was imagining traffic always within one VC
  from BladeX/NIC1
  to      BladeY/NIC1
.

!!!!
And
!!!!
I just realized that I have misconfigured
a vMotion network.

I configured two separate vNets
     vMotion-left
&  vMotion-right
basically just to be consistent with all
the other vNets defined in left/right pairs
in an Active/Active design.


I was forgetting, that  traffic might need to be
  from BladeX/NIC1
  to      BladeY/NIC2
.
E.g., in vMotion from BladeX to BladeY,
if
  BladeX wants to send Vmotion out NIC1
but
  BladeY/NIC1 is down
(unlikely but I suppose possible)
then the transfer cannot happen.

I was wondering why HP showed just ONE vMotion
vNet and put both NICs in it.


So, horizontal link will stay.
((And a redesign of the vMotion vNet!
))




tks

"The lyf so short, the craft so long to lerne." - Chaucer
Respected Contributor
Psychonaut
Posts: 214
Registered: ‎08-31-2011
Message 4 of 33 (3,564 Views)

Re: is Virtual Connect stacking required?

[ Edited ]

Just don't call me Francis :)

 

For that vertical link, with your config I don't see a need to have it.   If you were sharing uplinks between those interconnects or if you want the two networks to talk to each other then yes.  But since those are two separate networks with their own uplinks it isn't needed.  

 

Actually let me amend that the only thing that comes to mind would be if that iSCSI network is entirely isolated that link could be used to possibly tunnel through for management or monitoring.

Honored Contributor
The Brit
Posts: 1,291
Registered: ‎06-18-2007
Message 5 of 33 (3,558 Views)

Re: is Virtual Connect stacking required?

One thing which you may be overlooking.

 

VC Manager only runs on MOdule 1 or Module 2 (not 3 or 4).

 

Without stacking links, VCM knows nothing about 3 or 4, or anything connected to, or passing through 3 or 4.

 

You will not be able to configure the ports in modules 3 and 4, through VMC, and I also suspect that you will have problems with your profiles if VCM cannot talk to Modules 3 & 4.

 

The only way for VCM to talk to modules other than 1 and 2, is through stacking links.

 

If the stacking links are absent...

 

????

 

Dave.

Respected Contributor
Psychonaut
Posts: 214
Registered: ‎08-31-2011
Message 6 of 33 (3,551 Views)

Re: is Virtual Connect stacking required?

"With VCM 3.10 and higher, the primary modules can be in bays other than 1 and 2."

 

With that in mind, unless those 4 interconnects are in the same domain you should be able to break that connection as long are you are current on your firmware.

 

 

Honored Contributor
The Brit
Posts: 1,291
Registered: ‎06-18-2007
Message 7 of 33 (3,549 Views)

Re: is Virtual Connect stacking required?

You may be right about VCM being able to run on other modules than 1, or 2, however I still contend that without vertical stacking links, VCM will not be able to communicate or configure ports on modules which are not vertically connected (stacked).

 

Could  be wrong (which is why we invested in a Test environment).

 

I would be interested in the results of working with unstacked modules.

 

Dave.

Honored Contributor
The Brit
Posts: 1,291
Registered: ‎06-18-2007
Message 8 of 33 (3,547 Views)

Re: is Virtual Connect stacking required?

Sorry, just read your second sentence.    This implies that you need to run two separate VC domains within the same enclosure, i.e. to instances of VCM.    (I dont see any immediate objection to this, although I am a little concerned about server profiles spanning to different domains).    

 

Dave.

Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 9 of 33 (3,542 Views)

Re: is Virtual Connect stacking required?

Wow,
I have to start typing faster.

I was in the middle of a long-winded reply addressing what you guys just discussed in the last hour!

Actually, I'm MAJORLY concerned about the server profile point in the two-domain scenario.

But, hey Dave, *you're* the one with the test chassis !>).

VC 3.15 Setup and Install Guide says
  "All Virtual Connect Ethernet modules within the
   Virtual Connect domain must be interconnected.
  "
But next sentence says:
  "Any combination of 1-Gb and 10-Gb cables can be
   used to interconnect the modules.
  "
So, how much Meta traffic could there really be?

In the particular config I discussed, where I wanted to free up the CX4 on Flex-10 VC1,2, I guess I could just use a 1Gb link to VC3,4 ....
assuming that stacking *is* required.


Let us know what you find out with your test system, Dave ;>)


tks

"The lyf so short, the craft so long to lerne." - Chaucer
Honored Contributor
The Brit
Posts: 1,291
Registered: ‎06-18-2007
Message 10 of 33 (3,538 Views)

Re: is Virtual Connect stacking required?

To be honest Bob, unless you can wait a couple of months, my workload doesn't give me time to do any of the testing you are hoping for.

 

I have to say though, the more I think about it the more objections rise.    For example, how would two VC domains interact with your OA.    Specifically, when you first insert a VC module into the enclosure, the OA detects it and creates a link on the left-hand navigation bar.    If you create two VC domains, then the OA would have to provide two links, or else how would it know which VC domain you want to connect to.

 

Having said that, it is still the profile issue that stops me.   I suppose it is feasible to split your servers so that they only use either the onboard nics via IC Bays 1 and 2, or they only use Mezzanine card in Mezz 1 to communicate with Bays 3 & 4.    You would have to disable the onboard nics on these servers to stop them trying to link up to bays 1 & 2.

 

the servers can only have one profile, and the profile could only exist in one VC domain.    Each domain would have to have its own set of WWD's and MAC's.     (I dont think you mentioned whether your storage is internal or external, FC or SCSI), but you would have the same issues with FC VC modules (if you are using them.)

 

Bottom line is:   I dont really know if this is doable, however it is sufficiently scary to make me stay away.     I guess there would have to be an over-powering reason to put yourself out there.     You need to do a "cost-benefit analysis",   just to be sure that what you gain on the "swings" you don't lose on the "roundabout".

 

Dave.

 

Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 11 of 33 (3,537 Views)

Re: is Virtual Connect stacking required?

BTW,
I was able to imagine a design wherein the VCM on VC1 *could*, *theoretically*, know about the other VCs via the management network and the OA, which could communicate the VC arrangement to all the VCs.

So, that's how I came up with "no stacking required."
But, absent a "Theory of Operation" document ...


Pyscho's new scenario seemed to imply setting up the domain with stack-links and then breaking the stacking.

I'm wondering about just having two, mutually unaware domains from the get go.

Primary VCM in VC1 and another Primary VCM in VC3.

The OA mgt page probably would only have a link for the VC1 VCM, but we could connect directly to VC3 which should be running VCM, since it doesn't know about VC1,2


But still worried about the profiles.
I'm not even sure how the profiles get applied.
Thru the ILO?


bv

"The lyf so short, the craft so long to lerne." - Chaucer
Honored Contributor
Bart_Heungens
Posts: 690
Registered: ‎10-26-2005
Message 12 of 33 (3,526 Views)

Re: is Virtual Connect stacking required?

Mmm not really sure about what is mentioned above...

 

If there are no stacking links vertically the VCM running on modules 1 and 2 cannot communicate with the others and cannot configure them? Sorry but this is not true...

 

Simple example: how do U explain that in slot 3 and 4 are FC VC modules? There is no stacking possible between FC and Ethernet and yes U can configure them.

 

I have 27 enclosures at my customer site with 6 VC modules: 4 ethernet in 1, 2, 5 and 6 and FC modules in 3 and 4.

 

Without the stacking cables between 1 and 5 and another one between 2 and 6, I can configure all 6 modules... For VCM this is 1 VC domain...

 

Only: without the stacking cables U will get an error stating that this is not an ideal situation... Because with VC I can perfectly say that the onboard NIC 1 (physically connected to module 1) can get out with its data thru the uplinks on module 2, 5 or 6 (if I want), via the stacking cables...

 

Plenty of examples are mentioned in the VC Ethernet cookbook... Please check that one...

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 13 of 33 (3,520 Views)

Re: is Virtual Connect stacking required?


>> VC-FC
Good catch, Bart!!
Of course.

So, the Domain mgt Meta traffic must be occurring on the OA mgt network.
The "two-domain" scenario is unnecessary and out.
Although, I'm not sure about MAC tables.
Again, there's just not enough Theory of Op info.


>> Without the stacking cables between 1 and 5
>> and another one between 2 and 6,
The last phrase is a little ambiguous.
So, just to be clear, you did mean by that:

"No stacking cables between 1,5
  and
  no stacking cables between 2,6.
"
I.e, no vertical stacking at all between the VCs within a single enclosure.
(I assume, "Yes", since that's what the thread is about ;>)

So, what caused you not to use vertical stacking in your environment?
Or did you only try it out for curiosity.


>> ... VC Ethernet cookbook
That is a fine document, but it does address the issue of vertical stacking within an enclosure at all.
All the examples have just two VCs in each enclosure.



tks
bv

"The lyf so short, the craft so long to lerne." - Chaucer
Honored Contributor
Bart_Heungens
Posts: 690
Registered: ‎10-26-2005
Message 14 of 33 (3,516 Views)

Re: is Virtual Connect stacking required?

Hi,

 

All modules can communicate with each other through the signal midplane... However there is always communication via the OA...

 

2 VC domains in 1 enclosure is indeed not possible...

 

In my case Yes I have stacking cables between modules 1 and 5 and another one between 2 and 6... What I meant to explain is that, even without those cables, I can manage all modules... Via the OA and signal pidplane...

 

I know 1 scenario where the customers has all client and management trafic on modules 1 and 2, and only iSCSI trafic on modules 3 and 4 via the additional mezzanine card... Since it is a complete seperate type of trafic, stacking is not needed...

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 15 of 33 (3,508 Views)

Re: is Virtual Connect stacking required?

Thanks, Bart.

The separate iSCSI which you mentioned is the exact scenario that I will be dealing with, so I'm going to try it.

We will be doing a Flex-10 upgrade for VC 1/10G Enet modules in bays1,2 and will have a downtime window to be able to play with it --
basically just disconnecting the current vertical CX4 stacking and see what happens.


>> All modules can communicate with each other through the signal midplane..

My understanding is this:

VCs only communicate *horizontally* via the midplane and lacking vertical stack cables, cannot communicate  vertically.

The VC 1/10G Enet has hidden X0 *always* connected horizontally across midplane.
VC-Flex10s use X7 & X8 horizontally, but if an SFP is installed in either or both ports, then the port(s) are disabled horizontally.

Do you disagree?



bv

"The lyf so short, the craft so long to lerne." - Chaucer
Honored Contributor
Bart_Heungens
Posts: 690
Registered: ‎10-26-2005
Message 16 of 33 (3,506 Views)

Re: is Virtual Connect stacking required?

Hi,

 

Correct. In the older modules the X0 was hardwired through the midplane and could not be changed.

 

In the Flex modules the customer can choose for himself where he wants to make the interconnection. Inside or outside the enclosure.

 

Best practice is however using the internal connections so that the external switches are not (ab)used for trafic that can perfectly be handled inside...

 

There is indeed a difference in data paths and mgmt data paths ... VC itself for its config goes thru the OA, 'real' data goes directly via the signmal midplane to the servers...

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Esteemed Contributor
Bob_Vance
Posts: 602
Registered: ‎06-13-2000
Message 17 of 33 (3,502 Views)

Re: is Virtual Connect stacking required?

>> There is indeed a difference in data paths and mgmt data paths ... VC itself for its config goes thru the OA

 

This is the crucial idea and was the nub of my original query.

 

I'm going to leave this open until I have done my upgrade and will post results thereafter.

 

bv

"The lyf so short, the craft so long to lerne." - Chaucer
Respected Contributor
Psychonaut
Posts: 214
Registered: ‎08-31-2011
Message 18 of 33 (3,493 Views)

Re: is Virtual Connect stacking required?

Bart, that is some great info - I was looking to white board this out a little today and take back some of what I had suggested earlier. But it looks like you guys have hashed it all out.
Honored Contributor
Bart_Heungens
Posts: 690
Registered: ‎10-26-2005
Message 19 of 33 (3,490 Views)

Re: is Virtual Connect stacking required?

That's the reason why this forum exist... To share this kind of information.

 

Do not forget to assign kudo's, it is also appreciated... ;-)

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Occasional Visitor
Brunot38
Posts: 3
Registered: ‎01-07-2013
Message 20 of 33 (2,699 Views)

Re: is Virtual Connect stacking required?

Hello,

 

how did the tests go without vertical stacking ?

 

I have a similar setup ( that I am explaining in a  new post " HP Virtual Connect - Design with no vertical Stacking" )

 

I would really like to know if a setup withoug vertical stacking is working. It would be much appreciated as I am stuck on this..

 

Thanks

 

Brunot38

 

Valued Contributor
SwisspostIT
Posts: 181
Registered: ‎02-20-2012
Message 21 of 33 (2,671 Views)

Re: is Virtual Connect stacking required?

Hi everyone,

 

since today I'm also using VC Modules (had Pass-Thru until now) in my Lab.

Know I've a question regarding the stacking links:

 

Did I understand it right that if you use 2 identical VC Modules (I use VC FlexFabric 10Gb/24-port Modules) on Bay 1 & 2 there is no need to use a cable as stacking link since the Modules should have the stacking link through the Enclosure backplane?

 

Because the Virtual Connect Manager shows under Stacking Links the connection status as "FAILED" and Redundancy Status as "No Redundancy".

 

 

Thanks and Regards,

Ville

Honored Contributor
Jarkko K._1
Posts: 1,204
Registered: ‎01-29-2007
Message 22 of 33 (2,635 Views)

Re: is Virtual Connect stacking required?

Hi, check your FW levels. You might be affected with DNS bug. But Yes, no extra stacking links needed (there are internal cross-connections).
Remember to give Kudos to answers! (click the KUDOS star)
Honored Contributor
Bart_Heungens
Posts: 690
Registered: ‎10-26-2005
Message 23 of 33 (2,634 Views)

Re: is Virtual Connect stacking required?

Hi,

 

Correct if you have 2 similar modules in bay 1 and 2 you do not have to do the stacking...

 

However:

Are these modules the only modules in your enclosure? Or are there other modules inserted?

And if there are only 2 modules, did you populate in those 2 modules the ports X7 and X8? Because if you put SFP modules in X7 and X8, the internal crosslink is disabled, since these are shared ports.

Also: what is your firmware level?

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Valued Contributor
SwisspostIT
Posts: 181
Registered: ‎02-20-2012
Message 24 of 33 (2,627 Views)

Re: is Virtual Connect stacking required?


Bart Heungens_1 wrote:

And if there are only 2 modules, did you populate in those 2 modules the ports X7 and X8? Because if you put SFP modules in X7 and X8, the internal crosslink is disabled, since these are shared ports.

 


This was the problem! Thank you! :-)

 

Regards,

Ville

Occasional Visitor
GIGA-IT
Posts: 1
Registered: ‎07-09-2014
Message 25 of 33 (1,017 Views)

Re: Virtual Connect stacking error

Can anyone help me with this issue

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.