Re: Memory usage discrepancy (292 Views)
Reply
Advisor
karthikbalu
Posts: 20
Registered: ‎10-16-2012
Message 1 of 8 (318 Views)
Accepted Solution

Memory usage discrepency

Please help me understand where my memory has been consumed?

 

I have 16GB of physical memory and the OS is 11iv2.  From basic analysis i can see  ~7gb of memory has been consumed by my app ad OS. But i wonder where the memory is taking... is that chache? buffer? how this can be analyzed?

 

 

ProcList CPU Rpt  Mem Rpt  Disk Rpt            NextKeys SlctProc   Help     Exit
 Glance C.04.70.000             08:52:36            ia64       Current  Avg  High
-----------------------------------------------------------------------------------
CPU  Util                                                                    |  0%    1%    3%
Disk Util   F                                                               |  2%    2%    3%
Mem  Util   S                 SU                        UB     B  | 99%   99%   99%
Networkil   U             UR                R                      | 62%   62%   62%
-----------------------------------------------------------------------------------
                                   PROCESS LIST                        Users=    1
                         User      CPU %    Thrd Disk        Memory       Block
Process Name     PID     Name   ( 400% max) Cnt  IOrate      RSS/VSS      On
--------------------------------------------------------------------------------
siebmtsh                 2987 sadmin         0.0    28   0.0   930.8mb  980.9mb SLEEP
siebmtshmw         13101 sadmin        0.0    33   0.0   756.9mb  800.6mb SLEEP
siebmtshmw         10176 sadmin        0.0    21   0.1   715.8mb  784.6mb SLEEP
siebprocmw          2912 sadmin           0.0     8   0.0   637.0mb  666.4mb OTHER
siebmtshmw          2942 sadmin         0.0    33   0.0   446.3mb  488.5mb SLEEP
siebmtshmw          3055 sadmin         0.0    30   0.0   310.8mb  387.1mb SLEEP
siebmtshmw          3081 sadmin         0.0    26   0.0   292.9mb  374.4mb SLEEP
siebmtshmw          3121 sadmin         0.0    63   0.0   273.8mb  330.9mb SLEEP
siebmtshmw          3041 sadmin         0.0    13   0.0   269.9mb  316.6mb SLEEP
siebmtshmw          3098 sadmin        0.0    42   0.0   268.6mb  330.1mb OTHER
siebprocmw          2901 sadmin        0.0     8   0.0   268.0mb  297.5mb SLEEP
siebmtshmw          3070 sadmin        0.0    26   0.0   257.9mb  333.2mb SLEEP
siebmtshmw          3129 sadmin        0.0    62   0.0   255.1mb  306.2mb SLEEP
siebmtshmw          2956 sadmin        0.0    32   0.0   244.0mb  288.9mb SLEEP
siebmtshmw          3103 sadmin        0.0    32   0.0   243.5mb  288.6mb SLEEP
siebmtsh                 2902 sadmin        0.0    10   0.0   223.1mb  252.3mb OTHER
siebmtshmw          2922 sadmin        0.0    15   0.0   187.9mb  229.8mb SLEEP
siebmtshmw          2923 sadmin        0.0    13   0.0   187.1mb  228.2mb SLEEP
siebmtsh                 2911 sadmin        0.0    28   0.0   161.6mb  186.6mb SLEEP
siebprocmw          3020 sadmin         0.0     7   0.0   152.2mb  378.7mb OTHER
siebmtsh                2836 sadmin         0.0    54   0.0    90.0mb  143.6mb OTHER
vxsvc                      2332  root              0.0    18   0.0    54.3mb   84.2mb SLEEP
siebmtshmw          3029 sadmin        0.0    16   0.0    34.5mb   63.4mb OTHER
 
ProcList CPU Rpt  Mem Rpt  Disk Rpt            NextKeys SlctProc   Help     Exit
 
 
###############################
:#./kmeminfo
tool: kmeminfo 8.00 - libp4 9.306 - HP CONFIDENTIAL
unix: /stand/current/vmunix 11.23 64bit IA64 on host "xxxxxxx"
core: /dev/kmem live
link: Sun Feb 20 15:19:30 GMT 2011
boot: Thu Nov 28 14:27:43 2013
time: Tue Feb 18 08:54:54 2014
nbpg: 4096 bytes
 
 
----------------------------------------------------------------------
Physical memory usage summary (in page/byte/percent):
 
Physical memory          =  4186552   16.0g 100%
Free memory                 =    41978  164.0m   1%
User processes              =  1991841    7.6g  48%  details with -user
System                             =  2133805    8.1g  51%
  Kernel                            =  1542320    5.9g  37%  kernel text and data
    Dynamic Arenas        =  1089010    4.2g  26%  details with -arena
      M_TEMP                    =   920887    3.5g  22%
      vx_global_kmcac   =    37664  147.1m   1%
      spinlock                     =    28923  113.0m   1%
      vm_pfn2v_arena  =    16630   65.0m   0%
      VFD_BT_NODE       =    11096   43.3m   0%
      Other arenas           =    73810  288.3m   2%  details with -arena
    Super page pool       =   152101  594.1m   4%  details with -kas
    Static Tables             =   228158  891.2m   5%  details with -static
      pfdat                         =    98122  383.3m   2%
      nbuf                          =    66304  259.0m   2%  bufcache headers
      vhpt                          =    32768  128.0m   1%
      text                           =     8226   32.1m   0%  vmunix text section
      bufhash                    =     8192   32.0m   0%  bufcache hash headers
      Other tables           =    14545   56.8m   0%  details with -static
  Buffer cache               =   591485    2.3g  14%  details with -bufcache
  UFC meta mrg            =        0     0.0   0%
  UFC file mrg                =        0     0.0   0%
 
 
#ipcs -mob
IPC status from /dev/kmem as of Tue Feb 18 08:55:00 2014
T            ID               KEY        MODE        OWNER     GROUP NATTCH      SEGSZ
Shared Memory:
m          0          0x411c09a8 --rw-rw-rw-      root      root      0        348
m          1           0x4e0c0002 --rw-rw-rw-      root      root      2      61760
m          2           0x41200084 --rw-rw-rw-      root      root      2       8192
m       32774  0x06347849 --rw-rw-rw-      root       sys      1      65544
m          7           0x491404b0 --rw-r--r--      root      root      0      22908
 
 
Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,782
Registered: ‎03-06-2006
Message 2 of 8 (315 Views)

Re: Memory usage discrepancy

Your kmeminfo output has the breakdown of where system memory is allocated.

The lines with "details with" show the options to get more details.

Please use plain text.
Advisor
karthikbalu
Posts: 20
Registered: ‎10-16-2012
Message 3 of 8 (310 Views)

Re: Memory usage discrepancy

 

 

Any idea? How to tune this area's??

 

 

System                        =  2153362    8.2g  51%
  Kernel                        =  1575271    6.0g  38%  kernel text and data
    Dynamic Arenas    =  1245145    4.7g  30%  details with -arena
      M_TEMP                =   920883      3.5g  22%
 
Please use plain text.
Honored Contributor
Bill Hassell
Posts: 14,181
Registered: ‎05-29-2000
Message 4 of 8 (292 Views)

Re: Memory usage discrepancy

Check your kernel parameter dbc_max_pct:

 

kctune -s dbc_max_pct
Tunable      Value  Expression  Changes
dbc_max_pct     10  10          Immed

 50% is the (very bad) default for the buffer cache.

 

Please use plain text.
Advisor
karthikbalu
Posts: 20
Registered: ‎10-16-2012
Message 5 of 8 (277 Views)

Re: Memory usage discrepancy

This is what i have for the buffer cache. This value was 20% and i reduced to 15, but still  memory is busy.

 

#kctune -s dbc_max_pct
Tunable Value Expression Changes
dbc_max_pct 15 15 Immed

 

I suspect a memory leak here. Also i wonder how to tune the M_TEMP which is showing on 'kmeminfo'.

 

System = 2095248 8.0g 50%
Kernel = 1543670 5.9g 37% kernel text and data
   Dynamic Arenas = 1086737 4.1g 26% details with -arena
         M_TEMP = 920890 3.5g 22%

 

 

Please use plain text.
Trusted Contributor
Emil Velez_2
Posts: 120
Registered: ‎01-15-2002
Message 6 of 8 (256 Views)

Re: Memory usage discrepancy

Looks like you have a lot of user processes with large amounts of resident memory. Suspect applications allocating memory and either needing it or not freeing it when it does not.
Emil Velez
HP UNIX Certified (CSA, CSE HPUX 11i High Availability) HP Software (Openview) Certified Consultant
Certified HP Instructor, Technical Certified I and II SMB and Enterprise
Master ASE Superdome Solutins

HP Education Services

Ask me about training on Blades, Proliant, HP-UX, ServiceGuard, Polyserve, X9000, Virtual Libraries, and High Availability

internet: Emil.Velez@hp.com
Linkedin: http://www.linkedin.com/in/emilvelez

Please use plain text.
Acclaimed Contributor
Dennis Handly
Posts: 24,782
Registered: ‎03-06-2006
Message 7 of 8 (245 Views)

Re: Memory usage discrepancy

>a lot of user processes with large amounts of resident memory.

 

But karthikbalu was worried about the 8.1g used by "system".

Please use plain text.
Advisor
karthikbalu
Posts: 20
Registered: ‎10-16-2012
Message 8 of 8 (240 Views)

Re: Memory usage discrepency

Issue resolved after installing the below patch and tuning the below param.

 

PHKL_32425 has been released which addresses MEMORY_LEAK.

PHKL_41807 s700_800 11.23 mmap,madvise,mlock,msync, read corruption patch.

 

kctune vxfs_ifree_timelag=-1

Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation