Re: 4 x NIC's in a ALB or 10GB fibre? (473 Views)
Reply
Advisor
AlanByrneIT
Posts: 17
Registered: ‎11-30-2012
Message 1 of 6 (504 Views)

4 x NIC's in a ALB or 10GB fibre?

Hi, we are planning to increase our Network RAID SAN cluster (all 15k disk) with additonal SAN nodes which have 4 x 1GB NICs onboard.

 

Our current SAN nodes in the cluster connect to the network via active-passive 10GB SFP connections.

 

To continue using fibre for the new nodes we'll need to make a significant purcahse of Cisco equipment.

 

Does anyone know if a 4GB ALB bond would ever be satuared and cause performance issues?

 

I would only ever consider using the 4 x 1GB NIC's if I was sure it would not cause performance issues on the cluster?

 

Thanks

 

 

Please use plain text.
Frequent Advisor
Tedh256
Posts: 57
Registered: ‎11-29-2011
Message 2 of 6 (486 Views)

Re: 4 x NIC's in a ALB or 10GB fibre?

I would be shocked if you were able to saturate  4 bonded 1 Gb links. Of course it "depends" on many factors, not least of which is the workload characteristics, number of workloads, etc ...

 

But - most companies using 10GB don't even use a full 1/4

 

I'm not sure how you would test it, though, and I'm anxious to see whether anyone responds with testing suggestions ...

Please use plain text.
Honored Contributor
oikjn
Posts: 756
Registered: ‎07-20-2011
Message 3 of 6 (476 Views)

Re: 4 x NIC's in a ALB or 10GB fibre?

Is your existing SAN a storevirtual system or is it something else?  What bandwidth are you currently using and are you expanding because you need more throughput or because you need more capacity? 

 

If the new nodes are really a new storevirtual system, I would count on a max throughput of the san to be equal to 1/2 * 4GB (assuming LCAP bonding of each node) * # nodes in cluster.  That should be your worste case bandwidth (100% write since NR10 requires writing and comminication between nodes as well...  100% read would be simply 4GB * # nodes).  In addition, this SAN is greatly helped by using the HP DSM for MPIO with windows if you can possibly use it.  The benefit becomes greater the more nodes you have.

 

Short answer is, you probably will probably not notice a difference against 10Gb if you can LCAP bond each node, but the first thing I would check would be to make sure you aren't actually saturating your 10Gb already!

Please use plain text.
Advisor
AlanByrneIT
Posts: 17
Registered: ‎11-30-2012
Message 4 of 6 (473 Views)

Re: 4 x NIC's in a ALB or 10GB fibre?

Hi, I ran some analysis last week on the 4 node NR10 SAN cluster I'm referring to and each node reports as below:

 

Ave B/s- 12, 000, 000

Max B/s- 60, 000, 000

 

From my calcualtions, that works out as Ave 11MB/s and Max  57MB

 

The max figure would saturate a 1GB connection so I guess a 4GB ALB would be suffice?

 

 

Please use plain text.
Honored Contributor
oikjn
Posts: 756
Registered: ‎07-20-2011
Message 5 of 6 (461 Views)

Re: 4 x NIC's in a ALB or 10GB fibre?

saturation depends on IO size, but in theory 1Gb link is good for about 80MB sustained typically.  either way, you are nowhere near 4Gb let alone 10Gb...  I think you are fine w/ sticking w. the 1Gb x4 plan.

Please use plain text.
Regular Advisor
Sbrown
Posts: 120
Registered: ‎06-25-2013
Message 6 of 6 (391 Views)

Re: 4 x NIC's in a ALB or 10GB fibre?

you will run into congestion with gigabit.

I just upgraded months ago from gigabit.

supermicro 48port 10gbe with 4 x 40gbe $12,000

Old Dell BE2 emulex nic dual SFP+ 10gbe $75

Tripplite DAC SFP+ $25 each - no SFP+ needed with DAC cable.

Now I will move to 40gbe mellanox connectx-3 $300 nic with $100 QSFP cable with storevirtual VSA.

You will never hit congestion from vsa to vsa or VSA to HOST! never!

Remember there is a ton of node to node communication in cluster that if congested will bring the whole network to its knees!

I promise you will be so happy with 10gbe !!
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation