Re: SAR -d issue with storage (100 Views)
Reply
Visitor
Gigi_
Posts: 3
Registered: ‎02-13-2014
Message 1 of 5 (146 Views)

SAR -d issue with storage

Hello forum, 

I'm newbie on this topic and this forum.

 

I've an issue (?) with my system and/or IO monitoring. In particular my system is an HP Superdome2 with HP-UX 11.31 OS.

In order to monitor IO usage we utilize SAR, out-of-the-box with the OS.

What we are noting during the last days is a quite strange SAR -d behaviour. That is two or three times per day the output of SAR -d is so far away from the physical range normal IO behaviour: we have r+w/s near 35 million and blks/s over 650 millions.

 

The SAR version is:

/usr/sbin/swlist -l file |grep -i "/usr/sbin/sar"
  OS-Core.SYS2-ADMIN: /usr/sbin/sar                    
  PHCO_42120.SYS2-ADMIN: /usr/sbin/sar     

Below there is a sample of the described output:

<CMD-015;20140210120209;xxx;/usr/bin/sar -d 30 1 >

HP-UX xxx B.11.31 U ia64    02/10/14

12:02:09   device   %busy   avque   r+w/s  blks/s  avwait  avserv
12:02:39   disk12    1.13    0.50      38     667    0.00    0.35
           disk13    0.00    0.50       0       0    0.00    0.15
           disk14    1.00    0.50      34     648    0.00    0.36
          disk305    0.00    0.50       0       0    0.00    2.33
          disk306  100.00    0.50  480183 10214909    0.00    1.83
          disk307    0.00    0.50       0       0    0.00    2.53
          disk308    0.03    0.50       0       0    0.00    4.54
          disk309    0.03    0.50       0       0    0.00    5.64
          disk310    0.00    0.50       0       1    0.00    1.18
          disk311    0.13    0.50       0      11    0.00    3.98
          disk312    0.07    0.50       0       4    0.00    4.46
          disk313    0.00    0.50       0       0    0.00    2.51
          disk314    0.07    0.50       0       1    0.00    7.97
          disk315    0.03    0.50       0       0    0.00    2.38
          disk316    0.07    0.50       0       9    0.00    9.98
          disk317    0.10    0.50       1      98    0.64    7.49
          disk318    0.03    0.50       0       0    0.00    7.23
          disk319    0.00    0.50       0       0    0.00    2.37
          disk320    0.00    0.50       0       0    0.00    0.92
          disk321  100.00    0.50   59491 7240524    0.01    5.02
          disk322    0.00    0.50       0       0    0.00    2.35
          disk323    0.03    0.50       0       0    0.00    2.45
          disk324    0.03    0.50       0       0    0.00    4.55
          disk325    0.03    0.50       0       0    0.00    2.30
          disk326    0.00    0.50       0       0    0.00    2.39
          disk327    0.03    0.50       0       0    0.00    2.08
          disk328    6.23    0.51     128    1275    0.01    0.79
          disk329    0.00    0.50       0       0    0.00    2.74
          disk330    0.00    0.50       0       0    0.00    1.82
          disk331    0.03    0.50       0       0    0.00    2.73
          disk332    0.03    0.50       0       0    0.00    1.98
          disk333    0.03    0.50       0       0    0.00    1.99
          disk334    0.00    0.50       0       0    0.00    2.80
          disk335    0.00    0.50       0       0    0.00    2.97
          disk336    0.17    0.50       2      64    0.00    2.50
          disk337    1.03    1.31      16     115    0.61    1.18
          disk338    0.03    0.50       0       0    0.00    3.80
          disk339    0.00    0.50       0       0    0.00    3.88
          disk340    0.00    0.50       0       0    0.00    2.03
          disk341    0.03    0.50       0       0    0.00    4.28
          disk342    0.03    0.50       0       0    0.00    5.40
          disk343    0.03    0.50       0       0    0.00    5.21
          disk344    0.00    0.50       0       0    0.00    2.33
          disk345    0.00    0.50       0       0    0.00    2.17
          disk346    0.00    0.50       0       0    0.00    2.40
          disk347    0.03    0.50       0       0    0.00    5.47
          disk348    0.00    0.50       0       0    0.00    3.49
          disk349    0.00    0.50       0       0    0.00    5.41
          disk350    0.03    0.50       0       0    0.00    2.13
          disk351    0.00    0.50       0       0    0.00    8.02
          disk352    0.03    0.50       0       0    0.00    8.38
          disk353    0.00    0.50       0       0    0.00    3.83
          disk354    0.00    0.50       0       0    0.00    4.16
          disk355    0.03    0.50       0       0    0.00    5.39
          disk356    0.00    0.50       0       0    0.00    2.25
          disk357    0.00    0.50       0       0    0.00    2.09
          disk358    0.00    0.50       0       0    0.00    2.13
          disk359    0.00    0.50       0       0    0.00    4.26
          disk360    0.00    0.50       0       0    0.00    3.60
          disk361    0.03    0.50       0       0    0.00    5.20
          disk362  100.00    0.50   50281 17233818    0.02    4.46

 My question is: it could be a SAR bug for this version ? Otherwise could I investigate more then storage and hd ?

 

Thanks in advance

Please use plain text.
Honored Contributor
likid0
Posts: 979
Registered: ‎03-01-2005
Message 2 of 5 (116 Views)

Re: SAR -d issue with storage

Hi,

 

Check if you have the latest sar cumulitive patch for 11.31.

 

There are several bugs fixed related to "-d"

 

http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/mostViewedDisplay?javax.portlet....

Windows?, no thanks
Please use plain text.
Visitor
Gigi_
Posts: 3
Registered: ‎02-13-2014
Message 3 of 5 (115 Views)

Re: SAR -d issue with storage

Hi likid0,

the current cumulative patch installed is PHCO_42120: it seems to be the last one realeased by HP. 

Moreover I've verified that this patch could solve so many issues with sar -d, it introduces new problem with sar indeed.

Another server with the same OS version, patch and application doesn't give this kind of problem.

It is very frustrating

 

Luigi

Please use plain text.
Honored Contributor
likid0
Posts: 979
Registered: ‎03-01-2005
Message 4 of 5 (111 Views)

Re: SAR -d issue with storage

If you have the latest and greatest in sar cumulative patch, I would try and compair the sar results with another tool, if you have glance or Performance Agent, leave them running and compair the results with sar outputs.

Windows?, no thanks
Please use plain text.
Visitor
Gigi_
Posts: 3
Registered: ‎02-13-2014
Message 5 of 5 (100 Views)

Re: SAR -d issue with storage

Hi,

unfortunately I don't have another tool to monitor IO resource, and in general all resoruces. In any case I can stated that there is a inaccuracy issue related to SAR output values, these values are not physical acceptable for any storage devices: IOPS 10 million and Throughput over 250M MB/s.

 

I suppose that there is an issue concerning SAR and/or OS. For that I'm asking if anyone also did face this situation ?

Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation