Problems with OA 4.11? (948 Views)
Reply
Occasional Advisor
Stuart Cohnen
Posts: 9
Registered: ‎02-15-2011
Message 1 of 15 (948 Views)

Problems with OA 4.11?

I recently upgraded all 6 of my C7000s from OA 4.01 to OA 4.11. 

 

A week later, two of them, at different times seem to have lost access to the blades (I have 16 in each) and then reacquires it back within 10 seconds.

The alert mail shows this for each blade:

EVENT (30 Mar 13:31): Blade in bay 8 status has changed to: Failed.

 

Blade, "CUTTER", has changed from OK to Failed.

Only to get it right back

 

EVENT (30 Mar 13:32): Blade in bay 8 status has changed to: OK.

 

Blade, "CUTTER", has changed from Failed to OK.

 

This happened again this weekend,  once per blade server, a couple hours apart. Its always the same bladecenters.

The individual blades stay up.  This seems to be an ILO connectivity issue, but I don't know why. Any ideas and help is greatly appreciated.

Please use plain text.
Occasional Advisor
Stuart Cohnen
Posts: 9
Registered: ‎02-15-2011
Message 2 of 15 (921 Views)

Re: Problems with OA 4.11?

Update: It happened again this morning:  All 16 blades report the same..

 


Apr  1 06:30:34  OA: Management Processor on Blade 8 appears unresponsive.
Apr  1 06:30:34  OA: Management Processor on Blade 9 appears unresponsive.
<snip>
Apr  1 06:30:59  OA: Management Process on Blade 4 appears responsive again.
Apr  1 06:30:59  OA: Management Process on Blade 2 appears responsive again.

<etc>

Please use plain text.
Trusted Contributor
Ken Krubsack
Posts: 346
Registered: ‎10-20-2009
Message 3 of 15 (919 Views)

Re: Problems with OA 4.11?

Stuart,

 

Which ILO generation (2, 3, 4) and what version of ILO firmware on them?

 

Ken

Please use plain text.
Occasional Advisor
Stuart Cohnen
Posts: 9
Registered: ‎02-15-2011
Message 4 of 15 (913 Views)

Re: Problems with OA 4.11?

ilo2 are at 2.23

and ilo3 are at  1.55, 1.65 or 1.70 

thanks

 

Please use plain text.
HP Pro
scharchouf
Posts: 296
Registered: ‎03-07-2013
Message 5 of 15 (907 Views)

Re: Problems with OA 4.11?

send me a showall for all c7000, in private message ;)

I am an HP employee
Was the post useful? Click on the white KUDOS! Star
Please use plain text.
Honored Contributor
Bart_Heungens
Posts: 683
Registered: ‎10-26-2005
Message 6 of 15 (904 Views)

Re: Problems with OA 4.11?

I can tell you that, since I did the update through Oneview to SPP 2014.02 on my blade enclosure, I have had similar things twice in 3 weeks... No further messages behind. 3 Gen8 blades in the enclosure all updated to latest SPP... If I receive the message again I will post the exact information...

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Please use plain text.
Respected Contributor
Psychonaut
Posts: 212
Registered: ‎08-31-2011
Message 7 of 15 (869 Views)

Re: Problems with OA 4.11?

I am seeing the same thing on the two chassis worth of BL465c Gen8 blades that I've updated to 2014.02.  I left the OA at 4.01.  It looks like it's something with the 1.40 Firmware.

Please use plain text.
Occasional Advisor
Stuart Cohnen
Posts: 9
Registered: ‎02-15-2011
Message 8 of 15 (865 Views)

Re: Problems with OA 4.11?

HP tells me that I have to run SP2014.02 on all my blades and my problems will go away... Humm.

 

Please use plain text.
HP Pro
scharchouf
Posts: 296
Registered: ‎03-07-2013
Message 9 of 15 (861 Views)

Re: Problems with OA 4.11?

or those who have problems, sent me by private message, showAll report

I am an HP employee
Was the post useful? Click on the white KUDOS! Star
Please use plain text.
Honored Contributor
Bart_Heungens
Posts: 683
Registered: ‎10-26-2005
Message 10 of 15 (858 Views)

Re: Problems with OA 4.11?

Stuart,

 

My entire enclosure is running 2014.02 (OA, VC and blades) and I see these issues as well....

 

 

Kr,

Bart

--------------------------------------------------------------------------------
If my post was useful, clik on my KUDOS! "White Star" !
My blog: http://blog.bitcon.be
Please use plain text.
Occasional Advisor
Barmaley
Posts: 6
Registered: ‎05-02-2012
Message 11 of 15 (791 Views)

Re: Problems with OA 4.11?

I have lost iLO access (server unresponsive) to almost all servers in 20 x c7000 enclosures.

 

Different servers (BL460c/BL490c/BL2x220c from G1 to G7).

iLO 2.23

OA 4.01/4.11/4.20

 

It's happened after unsucceful network firmware upgrade (with SPP 2014.02) on some BL460c G1 servers.

NIC configuration data lost (including MAC addresses, etc), NIC adapter is disabled.

 

This is known bug of SPP 2014.02.

http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/HP-Proliant-DL380-G5-NIC-s-not-found-after-fi...

 

Looks like, corrupted NIC's have send some incorrect packets or MAC addresses conflict, which almost destroys all management network!!!

 

I can recover servers only by issues "reset server" command on OA modules. But i can't hard restart for hundreds of servers!

 

Servers which was not restarted by "reset server" still "Critical error"/"Unknwon" state.

 

Please use plain text.
Occasional Advisor
Barmaley
Posts: 6
Registered: ‎05-02-2012
Message 12 of 15 (784 Views)

Re: Problems with OA 4.11?

Fantastic!

All iLO 2 interfaces in our management vlan is stuck!

Even non blades servers!

 

# hponcfg
HP Lights-Out Online Configuration utility
Version 4.3.0 Date 12/10/2013 (c) Hewlett-Packard Company, 2014
ERROR: Error communicating with ILO
ERROR: Unable to communicate with the Management Processor.

 

I can't restart iLO 2 without completely remove power from the servers!

 

p.s. iLO 3 was not affected.

Please use plain text.
HP Pro
scharchouf
Posts: 296
Registered: ‎03-07-2013
Message 13 of 15 (756 Views)

Re: Problems with OA 4.11?

Hi All

 

OA v4.11 and v4.20 contain an OpenSSL version that has the vulnerability for  Heartbleed

 

iLOs are NOT vulnerable as they don't use SSL/TLS libraries that contain the TLS heartbeat extension BUT, we are receiving reports that the script that test for the HeartBleed bug is causing iLO2 to stop responding and the blades have to be refused to recover iLO2 functionality.  

 

I am an HP employee
Was the post useful? Click on the white KUDOS! Star
Please use plain text.
Occasional Visitor
bsodenkamp
Posts: 1
Registered: ‎04-08-2014
Message 14 of 15 (606 Views)

Re: Problems with OA 4.11?

I was being flooded with lots and lots of these messages until I turned SIM off.  Then they all stopped.  Working with HP Support now to be able to use SIM again.

 

Ben

Please use plain text.
Respected Contributor
Psychonaut
Posts: 212
Registered: ‎08-31-2011
Message 15 of 15 (537 Views)

Re: Problems with OA 4.11?

I upgraded the two chassis that I was seeing the alerts from to firmware 4.21 and it looks like that finally stopped the "Blade, "xxxx", has changed from Failed to OK." flapping.

Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation