OAs reboot on their own after 4.21 upgrade (2220 Views)
Reply
Highlighted
Occasional Contributor
Posts: 2
Registered: ‎12-23-2013
Message 1 of 14 (2,220 Views)

OAs reboot on their own after 4.21 upgrade

We have several HP C7000 BladeSystem enclosures and recently upgraded all of the Onboard Administrators (OAs) to 4.21 from 4.01.  After the upgrade, the OAs on 2 of our enclosures reboot on their own every 33 minutes exactly, sending out email alerts and causing unneeded alarm.

We contacted HP about the issue but they just keep scratching their heads.  Their only recommendation was to downgrade the OAs on these two enclosures back to 4.01 and then upgrade to 4.12 before upgrading to 4.21.  So we gave this a try... back at 4.01 the OAs never rebooted on their own... at 4.12 the OAs started rebooting again every 33 minutes and sending alerts and this also occured once we upgraded to 4.21 again.

We noticed in the OA System Log that just before the reboot occurs the following events are logged:

     - OA: DHCP Monitor: DHCPD or RADVD is not running. Restarting OA.
     - OA: Onboard Administrator is rebooting

All OAs (those affected and those that are not) are at hardware version B1 and part number 412142-B21.  Has anyone seen this issue before or have any suggestions on what can be done to correct it?  We'd like all of our OAs to be at the same firmware version (4.21) across the board.

Advisor
Posts: 15
Registered: ‎12-08-2009
Message 2 of 14 (2,126 Views)

Re: OAs reboot on their own after 4.21 upgrade

we got smth strange with OA 4.21 too
in about week after upgrade, SIM opened new case on OA's  without any understandeble info
ping of OA's shown packet loss (~10%), or even stop
on OA all blades shown critical, but acutally blades and interconnects worked
physicaly blades was "red" too

after reseting of OA's via web - we lost web access, only cli, and instable
changing of Active OA  few times lead to nothing


we unpluged both OA's, pluged OA1 - same story...
unplug OA1 - plug OA2, same story...

downgrade fw on OA1, while OA2 was unplaged - nothing
reseat one of blades - it started to show gren "front LED" but iLO, and interconnects - no ping no access

power off\on enclosure - iLO's started to ping and became accessible
in Encl settings - EBIPA settings were "default" so no IP's set etc
but iLO's shown IP and was accessible, interconnects - not

we set IP's to interconnects and OA's manualy via ebipa - and finaly got access to interconnects...


now evr works on 4.21 - hope there won't be any troubles anymore


Occasional Visitor
Posts: 2
Registered: ‎05-27-2014
Message 3 of 14 (2,082 Views)

Re: OAs reboot on their own after 4.21 upgrade

We are seeing the exact same thing.  Just started after the update to 4.21.  Has anyone given you an answer as to how to stop this?

 

Occasional Visitor
Posts: 2
Registered: ‎05-27-2014
Message 4 of 14 (2,079 Views)

Re: OAs reboot on their own after 4.21 upgrade

To add to it, our hardware version on the OAs are H0 and F0, but exact same behaviour - every thirty-three minutes, reboot with the same final error log message.

Occasional Visitor
Posts: 1
Registered: ‎09-14-2014
Message 5 of 14 (1,758 Views)

Re: OAs reboot on their own after 4.21 upgrade

Issue fixed by Version:4.30 (9 Sep 2014)

 

Problems Fixed

General

  • In rare cases, the OA may fail to successfully identify server blades that have been either physically inserted into the enclosure or have been reset by executing the OA CLI RESET SERVER command.  When this issue occurs, the affected server blades are misidentified as an unsupported blade type.
  • Restoring or configuring OA settings from a saved configuration script may fail to apply all network settings on the Standby OA module. The failing configuration script commands return the following error: “An error occurred while communicating with the other Onboard Administrator.” 
  • The RSA key for SSH is not properly exchanged with the Standby OA during an OA failover event when Enclosure IP Mode is enabled. After an OA failover event, when logging into the OA, SSH clients using the Enclosure IP address to access the Active OA may display an RSA key fingerprint warning message or “man-in-the-middle” security breach warning message.  This issue only occurs on OA 4.1x or OA 4.2x versions of OA firmware when the OA has been factory reset or new SSH keys were manually generated.  
  • The OA periodically reboots with the following entry logged in the OA system log:“OA: DHCP Monitor: DHCPD or RADVD is not running. Restarting OA.” 
  • In an IPv6-only management network environment with the VLAN feature enabled, the CONNECT SERVER OA CLI command fails to connect to the server blade serial console with an error displayed similar to the following: Connecting to bay 1 ... iLO failed to respond: Interrupted system call (4)

 

Super Advisor
Posts: 493
Registered: ‎01-20-2011
Message 6 of 14 (1,398 Views)

Re: OAs reboot on their own after 4.21 upgrade

My OA is on version 3.71, so can i just skip and update the firmware directly to version 4.30 with no issue or downtime ?

HP Pro
Posts: 134
Registered: ‎08-23-2011
Message 7 of 14 (1,375 Views)

Re: OAs reboot on their own after 4.21 upgrade

Hi,

 

Yes you can directly go to version 4.30.
For the onboard administrator updates you do not require any downtime for the enclosure and that they can be performed any time.

 

 

Thanks & regards,
I work for HP
A quick resolution to technical issues for your HP Enterprise products is just a click away HP Support Center Knowledge-base
See Self Help Post for more details
______________________________________________________
Was the post useful? Say thanks by clicking the white KUDOS Star!

Super Advisor
Posts: 493
Registered: ‎01-20-2011
Message 8 of 14 (1,366 Views)

Re: OAs reboot on their own after 4.21 upgrade

Thanks Lincy.

Does updating the VC modules firmware using HP SPP bootable can also cause any outage to the network or fibre channel traffic on the enclosure ?
HP Pro
Posts: 134
Registered: ‎08-23-2011
Message 9 of 14 (1,340 Views)

Re: OAs reboot on their own after 4.21 upgrade

Hi,
onboard administrator updates most of the time you do not require any downtime for the enclosure and that they can be performed any time. If you want to update the virtual connect, the first thing to do is run a health check. Once you validated that the virtual connect is healthy, you also need to make sure that your models are set up in a redundant configuration.

Thanks & regards,

Lincy

 

Thanks & regards,
I work for HP
A quick resolution to technical issues for your HP Enterprise products is just a click away HP Support Center Knowledge-base
See Self Help Post for more details
______________________________________________________
Was the post useful? Say thanks by clicking the white KUDOS Star!

HP Pro
Posts: 134
Registered: ‎08-23-2011
Message 10 of 14 (1,339 Views)

Re: OAs reboot on their own after 4.21 upgrade

Hi,

 

Blog on - Minimizing Downtime when Deploying Firmware and Driver Updates

http://h30507.www3.hp.com/t5/Coffee-Coaching-HP-and-Microsoft/Minimizing-Downtime-when-Deploying-Fir...

 

https://www.youtube.com/watch?v=DcHPe30Gz2s

 

 

Thanks & regards,
I work for HP
A quick resolution to technical issues for your HP Enterprise products is just a click away HP Support Center Knowledge-base
See Self Help Post for more details
______________________________________________________
Was the post useful? Say thanks by clicking the white KUDOS Star!

Super Advisor
Posts: 493
Registered: ‎01-20-2011
Message 11 of 14 (1,327 Views)

Re: OAs reboot on their own after 4.21 upgrade

Wow,

I never knew that there is a validate option in HP Blade VC manager page. So is that test safe to be executed without causing any downtime to the production environment ?

Because I need to know if the VC module has been configured to be redundant or not.
HP Pro
Posts: 134
Registered: ‎08-23-2011
Message 12 of 14 (1,303 Views)

Re: OAs reboot on their own after 4.21 upgrade

Hi,


No this will not cause downtime.

Healthcheck Command - List the state of modules in the enclosure. This command is useful if an Administrator needs to know if performing an action, such as a failover or a VC or OA firmware upgrade, would cause an outage.

 

VSCU user guide - https://h20565.www2.hp.com/hpsc/doc/public/display?sp4ts.oid=3794423&docId=emr_na-c04384256&docLocal...

 

http://h20564.www2.hp.com/hpsc/doc/public/display?docId=mmr_kc-0108485

 

 

Thanks & regards,
I work for HP
A quick resolution to technical issues for your HP Enterprise products is just a click away HP Support Center Knowledge-base
See Self Help Post for more details
______________________________________________________
Was the post useful? Say thanks by clicking the white KUDOS Star!

Super Advisor
Posts: 493
Registered: ‎01-20-2011
Message 13 of 14 (1,291 Views)

Re: OAs reboot on their own after 4.21 upgrade

Many thanks for the helpful assistance and guidance Lincy.

I really appreciate your reply.

Have a good day.
Super Advisor
Posts: 493
Registered: ‎01-20-2011
Message 14 of 14 (693 Views)

Re: OAs reboot on their own after 4.21 upgrade

Version 4.40 has just been released

http://h20564.www2.hp.com/hpsc/swd/public/detail?sp4ts.oid=3188477&swItemId=MTX_dc5dc1fa9051424caa14...

 

Has anyone tried to upgrade into that version yet ?

The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.