Problem After Patching 11.23 HPUX IA (249 Views)
Reply
Honored Contributor
likid0
Posts: 977
Registered: ‎03-01-2005
Message 1 of 2 (249 Views)
Accepted Solution

Problem After Patching 11.23 HPUX IA

[ Edited ]

Hi,

 

I have quite a mess here. Story short: before we patch we split our mirror disks(we are no using drd :**), then we patch and after a weak if everything is ok we merge the lvols.

 

What happened here is that we split the mirror disks all ok, and checked our setboot order so we booted from the normal lvols(not the split _b lvols),and patched fine.

 

The problems is after the patch reboot, the primary disk didn't boot:

 

Starting: HP-UX Primary Boot: 0/0/0/3/0.6.0


(C) Copyright 1999-2006,2009 Hewlett-Packard Development Company, L.P.
All rights reserved

HP-UX Boot Loader for IPF -- Revision 2.030

Press Any Key to interrupt Autoboot
\EFI\HPUX\AUTO ==> boot vmunix -lq
Seconds left till autoboot - 10 9 8 7 6 5 4 3 2 1 0

AUTOBOOTING...Could not open file vmunix to check for a directory
Could not open vmunix

 

It also couldn't open the backup kernel,etc.

 

So our guy booted from the backup disk,it all look good, so he tought it was ok.

 

But the problem we have now is that the vmunix kernel we booted from is different from the new patched vmunix kernel we have in lvol1.

 

So we get the usual :

 

#dmesg

Aug 20 17:35
Magic number wrong (namelist mismatch?)

 

The thing is, what is the best way to get the primary boot(vmunix) working so we don't have to patch again?, or is it better to boot the old disk,with old lvols, and start patching all over again.

 

Here is info on the current state:

 

> setboot
Primary bootpath : 0/0/0/3/0.6.0
HA Alternate bootpath : 0/0/1/1/0/4/1.5.0
Alternate bootpath : 0/0/1/1/0/4/1.5.0  --> currently booted from here

 

> lvlnboot -v vg00
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/dsk/c3t5d0s2 (0/0/1/1/0/4/1.5.0) -- Boot Disk
        /dev/dsk/c0t6d0s2 (0/0/0/3/0.6.0) -- Boot Disk
Boot: lvol1b    on:     /dev/dsk/c0t6d0s2   --> strange ??? lvol1b is on disk c3t5d0
Root: lvol3     on:     /dev/dsk/c0t6d0s2
Swap: lvol2     on:     /dev/dsk/c0t6d0s2
Dump: lvol2     on:     /dev/dsk/c0t6d0s2, 0

 

/> lvdisplay -v /dev/vg00/lvol1 | more
--- Logical volumes ---
LV Name                     /dev/vg00/lvol1
VG Name                     /dev/vg00
LV Permission               read/write  
LV Status                   available/syncd          
Mirror copies               0           
Consistency Recovery        MWC                
Schedule                    parallel    
LV Size (Mbytes)            1008           
Current LE                  63       
Allocated PE                63         
Stripes                     0      
Stripe Size (Kbytes)        0                  
Bad block                   off         
Allocation                  strict/contiguous        
IO Timeout (Seconds)        default            

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV 
   /dev/dsk/c0t6d0s2       63        63       

   --- Logical extents ---
   LE    PV1                     PE1   Status 1
   00000 /dev/dsk/c0t6d0s2       00512 current 
   00001 /dev/dsk/c0t6d0s2       00513 current 
   00002 /dev/dsk/c0t6d0s2       00514 current 

root@appomv2p:/> lvdisplay -v /dev/vg00/lvol1b | more
--- Logical volumes ---
LV Name                     /dev/vg00/lvol1b
VG Name                     /dev/vg00
LV Permission               read/write  
LV Status                   available/syncd          
Mirror copies               0           
Consistency Recovery        MWC                
Schedule                    parallel    
LV Size (Mbytes)            1008           
Current LE                  63       
Allocated PE                63         
Stripes                     0      
Stripe Size (Kbytes)        0                  
Bad block                   off         
Allocation                  strict/contiguous        
IO Timeout (Seconds)        default            

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV 
   /dev/dsk/c3t5d0s2       63        63       

   --- Logical extents ---
   LE    PV1                     PE1   Status 1
   00000 /dev/dsk/c3t5d0s2       00000 current 

 

> vgdisplay -v vg00 | grep -i lvol
   LV Name                     /dev/vg00/lvol1
   LV Name                     /dev/vg00/lvol2
   LV Name                     /dev/vg00/lvol3
   LV Name                     /dev/vg00/lvol4
   LV Name                     /dev/vg00/lvol5
   LV Name                     /dev/vg00/lvol6
   LV Name                     /dev/vg00/lvol7
   LV Name                     /dev/vg00/lvol8
   LV Name                     /dev/vg00/lvol1b
   LV Name                     /dev/vg00/lvol2b
   LV Name                     /dev/vg00/lvol3b
   LV Name                     /dev/vg00/lvol4b
   LV Name                     /dev/vg00/lvol5b
   LV Name                     /dev/vg00/lvol6b
   LV Name                     /dev/vg00/lvol7b
   LV Name                     /dev/vg00/lvol8b

 

/> bdf | grep lvol1
/dev/vg00/lvol1    1032192  314680  712000   31% /stand
/dev/vg00/lvol1b   1032192  133192  892072   13% /mnt

 

/> ls -ltr /stand
total 142368
dr-xr-xr-x   3 bin        bin             96 Sep  8  2004 boot.sys
-r--r--r--   1 root       sys             82 Sep  8  2004 kernrel
drwxr-xr-x   5 root       sys           8192 Sep  8  2004 last_install
-r-xr-xr-x   1 bin        bin        58223248 Jul 10  2006 vpmon
drwxr-xr-x   2 root       root            96 Sep 26  2006 lost+found
-rw-------   1 root       sys            108 Jun  1  2007 .sh_history
-r--r--r--   1 root       sys           3323 Jul  5  2007 system.import
-rw-------   1 root       sys           4176 Jul 10  2007 vpdb
drwxr-xr-x   5 root       root          8192 Jun 19  2011 crashconfig
-rw-r--r--   1 root       sys             42 Apr 15 16:45 bootconf
-rw-r--r--   1 root       sys           3525 Aug 20 03:29 system.prev
-rwxr-xr-x   1 root       sys        60455848 Aug 20 03:53 vmunix
drwxr-xr-x   5 root       sys           8192 Aug 20 03:53 backup
-rw-------   1 root       root            12 Aug 20 04:09 rootconf
lrwxr-xr-x   1 root       root             7 Aug 20 04:09 nextboot -> current
lrwxr-xr-x   1 root       root            15 Aug 20 04:09 system -> nextboot/system
lrwxr-xr-x   1 root       root            14 Aug 20 04:09 bootfs -> current/bootfs
-rw-r--r--   1 root       sys           7652 Aug 20 04:24 ioconfig
drwxr-xr-x   2 root       sys             96 Aug 20 04:24 krs
drwxr-xr-x   5 root       sys           8192 Aug 20 04:24 current
/> ls -ltr /mnt
total 140384
dr-xr-xr-x   3 bin        bin             96 Sep  8  2004 boot.sys
-r--r--r--   1 root       sys             82 Sep  8  2004 kernrel
drwxr-xr-x   5 root       sys           8192 Sep  8  2004 last_install
-r-xr-xr-x   1 bin        bin        58223248 Jul 10  2006 vpmon
drwxr-xr-x   2 root       root            96 Sep 26  2006 lost+found
-rw-------   1 root       sys            108 Jun  1  2007 .sh_history
-r--r--r--   1 root       sys           3323 Jul  5  2007 system.import
-rw-------   1 root       sys           4176 Jul 10  2007 vpdb
-rw-r--r--   1 root       sys           3496 May  2  2009 system.prev
-rwxr-xr-x   4 root       sys        59439896 May  2  2009 vmunix
drwxr-xr-x   5 root       sys           8192 May  2  2009 current
drwxr-xr-x   5 root       sys           8192 Jan 13  2011 backup
drwxr-xr-x   5 root       root          8192 Jun 19  2011 crashconfig
-rw-r--r--   1 root       sys             42 Apr 15 16:45 bootconf
-rw-------   1 root       root            12 Aug 20 01:38 rootconf
lrwxr-xr-x   1 root       root             7 Aug 20 01:38 nextboot -> current
lrwxr-xr-x   1 root       root            15 Aug 20 01:38 system -> nextboot/system
lrwxr-xr-x   1 root       root            14 Aug 20 01:38 bootfs -> current/bootfs
-rw-r--r--   1 root       sys           7652 Aug 20 01:38 ioconfig
drwxr-xr-x   2 root       sys             96 Aug 20 01:41 krs

> bdf | grep lvol
/dev/vg00/lvol3     573440  305136  266400   53% /
/dev/vg00/lvol1    1032192  314680  712000   31% /stand
/dev/vg00/lvol8    5832704 3203104 2612000   55% /var
/dev/vg00/lvol7    6832128 3094968 3708016   45% /usr
/dev/vg00/lvol4    1196032  497248  693864   42% /tmp
/dev/vg00/lvol6    6733824 5719976 1006040   85% /opt
/dev/vg00/lvol5    1048576  148472  893080   14% /home
/dev/vg00/lvol1b   1032192  133192  892072   13% /mnt

 

> kclog
======================================================================

Change to configuration 'current'
at 04:20:21 METDST on 20 August 2013 by root:
Module 'gvid_him_cons' version 0.1.0 changed from unused (explicit) to auto (explicit).

 

> kconfig
Configuration  Title
backup         Automatic Backup
crashconfig    booted from 'import.config'
last_install   Created by last OS install

 

> efi_ls -d /dev/dsk/c3t5d0s1 EFI/HPUX
FileName                             Last Modified             Size
.                                      7/ 9/2007                  0
..                                     7/ 9/2007                  0
HPUX.EFI                               8/20/2013             647263
NBP.EFI                                7/ 9/2007              24576
AUTO                                   7/ 9/2007                 16
crashdump.efi                          6/19/2011             107990
vparconfig.efi                         6/19/2011             101897

 

> efi_ls -d /dev/dsk/c0t6d0s1 EFI/HPUX
FileName                             Last Modified             Size
.                                      4/15/2013                  0
..                                     4/15/2013                  0
HPUX.EFI                               8/20/2013             647263
NBP.EFI                                4/15/2013              24576
AUTO                                   4/15/2013                 17
crashdump.efi                          8/20/2013             107990
vparconfig.efi                         8/20/2013             101897

> efi_cp -u -d /dev/dsk/c0t6d0s1 EFI/HPUX/AUTO .

> cat AUTO
boot vmunix -lq
> efi_cp -u -d /dev/dsk/c3t5d0s1  EFI/HPUX/AUTO .
/> cat AUTO
boot vmunix -lq

/> lifls -l /dev/dsk/c3t5d0s2
volume ISL10 data size 7984 directory size 8 06/10/27 14:23:07
filename   type   start   size     implement  created
===============================================================
ISL        -12800 584     242      0          06/10/27 14:23:07
AUTO       -12289 832     1        0          06/10/27 14:23:07
HPUX       -12928 840     1024     0          06/10/27 14:23:07
PAD        -12290 1864    1468     0          06/10/27 14:23:07
LABEL      BIN    3336    8        0          07/07/09 15:09:12
/> lifls -l /dev/dsk/c0t6d0s2
volume ISL10 data size 7984 directory size 8 06/10/27 14:23:07
filename   type   start   size     implement  created
===============================================================
ISL        -12800 584     242      0          06/10/27 14:23:07
AUTO       -12289 832     1        0          06/10/27 14:23:07
HPUX       -12928 840     1024     0          06/10/27 14:23:07
PAD        -12290 1864    1468     0          06/10/27 14:23:07
LABEL      BIN    3336    8        0          13/04/15 14:57:36


 

Thanks for the help



Windows?, no thanks
Please use plain text.
Honored Contributor
likid0
Posts: 977
Registered: ‎03-01-2005
Message 2 of 2 (191 Views)

Re: Problem After Patching 11.23 HPUX IA

This is how I fixed the problem, just in case it happens to someone:

 

we have split the mirror in vg00, we have patched succesfully.

We have the patched disk /dev/dsk/c2t1d0s2 with the normal lvols, And the backup disk /dev/dsk/c2t0d0s2 with the lvolb's:

root@rx4640:/HP> pvdisplay -v /dev/dsk/c2t1d0s2 | grep lvol1 | head -1
   /dev/vg00/lvol1    19        19 
root@rx4640:/HP> pvdisplay -v /dev/dsk/c2t0d0s2 | grep lvol1 | head -1        
   /dev/vg00/lvol1b   19        19       


Once we reboot the server we can't start from our patched/primary disk:/dev/dsk/c2t1d0s2 , we can't access the /stand partition on the disk, to load the vmunix kernel:

HPUX> boot vmunix
Could not open file vmunix to check for a directory
Could not open vmunix

loading failed

HPUX>

HPUX> ls
no such file or directory
                        
HPUX>


So we boot from our secondary disk:

HPUX> boot vmunix
> System Memory = 4067 MB
loading section 0
.......................................................... (complete)
loading section 1
.............. (complete)
loading symbol table
loading System Directory (boot.sys) to MFS
....
loading MFSFILES directory (bootfs) to MFS
..............
Launching /stand/vmunix
SIZE: Text:29364K + Data:7030K + BSS:5151K = Total:41546K


But the problem we have now is that the vmunix kernel we booted from is different from the new patched vmunix kernel we have in lvol1.

#dmesg

Aug 20 17:35
Magic number wrong (namelist mismatch?)

                   root@rx4640:/> lvlnboot -v vg00
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/dsk/c2t1d0s2 (0/1/1/0.1.0) -- Boot Disk
        /dev/dsk/c2t0d0s2 (0/1/1/0.0.0) -- Boot Disk
Boot: lvol1b    on:     /dev/dsk/c2t1d0s2
Root: lvol3     on:     /dev/dsk/c2t1d0s2
Swap: lvol2     on:     /dev/dsk/c2t1d0s2
Dump: lvol2     on:     /dev/dsk/c2t1d0s2, 0


So we check both our lvol1 partitions and pass a fsck all is ok:


t@rx4640:/> fsck -y /dev/vg00/lvol1
file system is clean - log replay is not required
root@rx4640:/> fsck -y /dev/vg00/lvol1b
fsck: /etc/default/fs is used for determining the file system type
file system is clean - log replay is not required
root@rx4640:/>

So if /stand partition is ok, we need to check serveral things on the disk that doesn't boot, lvol1 is on the first pes on the disk, efi and LABLE file config:

lvol1 is on the first PE's of the disk:

root@rx4640:/> pvdisplay -v /dev/dsk/c2t1d0s2 |grep -E 'curren.*0000 $'
   00000 current  /dev/vg00/lvol1    00000
   00019 current  /dev/vg00/lvol2    00000
   00275 current  /dev/vg00/lvol3    00000
   00310 current  /dev/vg00/lvol4    00000
   00323 current  /dev/vg00/lvol5    00000
   00325 current  /dev/vg00/lvol6    00000
   00698 current  /dev/vg00/lvol7    00000
   01086 current  /dev/vg00/lvol8    00000

EFi is ok:

root@rx4640:/> efi_ls -d /dev/dsk/c2t1d0s1 EFI/HPUX
FileName                             Last Modified             Size
.                                     12/31/1997                  0
..                                    12/31/1997                  0
HPUX.EFI                               8/22/2013             644703
NBP.EFI                                8/22/2013              24576
AUTO                                   8/22/2013                 12

total space 523251712 bytes, free space 508829696 bytes

We now compair the label between the disk that doesn't boot:

root@rx4640:/> lifls -l /dev/dsk/c2t1d0s2
volume ISL10 data size 7984 directory size 8 06/04/19 14:02:34
filename   type   start   size     implement  created
===============================================================
ISL        -12800 584     242      0          06/04/19 14:02:34
AUTO       -12289 832     1        0          06/04/19 14:02:34
HPUX       -12928 840     1024     0          06/04/19 14:02:34
PAD        -12290 1864    1468     0          06/04/19 14:02:34
LABEL      BIN    3336    8        0          12/08/22 22:05:04

and the one that does:

root@rx4640:/> lifls -l /dev/dsk/c2t0d0s2

volume ISL10 data size 7984 directory size 8 06/04/19 14:02:34
filename   type   start   size     implement  created
===============================================================
ISL        -12800 584     242      0          06/04/19 14:02:34
AUTO       -12289 832     1        0          06/04/19 14:02:34
HPUX       -12928 840     1024     0          06/04/19 14:02:34
PAD        -12290 1864    1468     0          06/04/19 14:02:34
LABEL      BIN    3336    8        0          08/06/15 17:09:50

As you can see the label created times don't match, so we can bet that the label on disk /dev/dsk/c2t1d0s2 is bad, so the this is how can we recreate the label, now that we have split the disks, and we can't do a lvlnboot -vR /dev/vg00, there is and easy way copy the LABEL file from the good disk to the patched bad disk:
root@rx4640:/> mkboot -el /dev/dsk/c2t1d0
root@rx4640:/> mkboot -a "boot vmunix -lq" /dev/dsk/c2t1d0
root@rx4640:/> lifrm /dev/dsk/c2t1d0s2:LABEL
root@rx4640:/> lifcp -b /dev/dsk/c2t0d0s2:LABEL /dev/dsk/c2t1d0s2:LABEL
root@rx4640:/> lifls -l /dev/dsk/c2t1d0s2
volume ISL10 data size 7984 directory size 8 06/04/19 14:02:34
filename   type   start   size     implement  created
===============================================================
ISL        -12800 584     242      0          06/04/19 14:02:34
AUTO       -12289 832     1        0          06/04/19 14:02:34
HPUX       -12928 840     1024     0          06/04/19 14:02:34
PAD        -12290 1864    1468     0          06/04/19 14:02:34
LABEL      BIN    3332    8        0          13/08/22 22:26:42

we can now reboot our server, and boot from the primary disk:

HP-UX Boot Loader for IPF  --  Revision 2.028

Press Any Key to interrupt Autoboot
\EFI\HPUX\AUTO ==> boot vmunix -lq
Seconds left till autoboot -   9
   Type 'help' for help        

HPUX> ls
AUTO            actual_kernel   backup          boot.sys        bootconf
bootfs          crashconfig     current         ioconfig        kernrel
krs             last_install    lost+found      nextboot        rootconf
system          system.import   system.prev     vmunix                 
                                                     
HPUX> boot vmunix -lq
> System Memory = 4067 MB
loading section 0
.......................................................... (complete)

We can check we have booted from our patched disk and that all is ok, dmesg working again:

root@rx4640:/> ll /dev/dsk | grep $(echo "bootdev/x"|adb /stand/vmunix /dev/kmem | awk '/0x/ {print substr($1,5)}') 
brw-r-----   1 bin        sys         31 0x020002 Apr 20  2007 c2t1d0s2
root@rx4640:/> lvlnboot -v vg00
Boot Definitions for Volume Group /dev/vg00:
Physical Volumes belonging in Root Volume Group:
        /dev/dsk/c2t1d0s2 (0/1/1/0.1.0) -- Boot Disk
        /dev/dsk/c2t0d0s2 (0/1/1/0.0.0) -- Boot Disk
Boot: lvol1     on:     /dev/dsk/c2t1d0s2
Root: lvol3     on:     /dev/dsk/c2t1d0s2
Swap: lvol2     on:     /dev/dsk/c2t1d0s2
Dump: lvol2     on:     /dev/dsk/c2t1d0s2, 0

root@rx4640:/> dmesg | tail -1
SCSI: Ultra160 Controller at 0/1/1/1: Warning: Data transfer rate stepped down for target 14. Now operating at 40 MB/s (Ultra2 Narrow). Possible causes are improper termination, improper cabling, or malfunctioning hardware.

Now we can merge our lvolb, when we check the patches are working ok.

Windows?, no thanks
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation