09-12-2013 05:10 AM
In our DB server kcusage for maxfiles_lim always showing more than 400% .
Server Model - rx6600 , OS- 11.31
We are having 2 servers which is configured in RCA Oracle clustering. We set both servers kernel parameters are same, (maxfiles_lim = 4096) But in one server maxfiles_lim always showing more than 400% , in another its showing less than 40%. I am not getting any performance issue on servers.
1. What is use of maxfiles_lim ?
2. How kernel allows to use more than configured settings for maxfiles_lim ?
3. What is the resolution?
09-12-2013 03:17 PM
>1. What is use of maxfiles_lim?
Limit run away processes from opening too many files.
>2. How kernel allows to use more than configured settings for maxfiles_lim?
Possibly root processes can bypass the limit?
09-12-2013 08:57 PM
>1. What is use of maxfiles_lim?
> Limit run away processes from opening too many files.
See I have two servers both are configured in RCA Clustering with Load balancing , So why only one server is showing high utilization of maxfiles_lim ? Why not another server?
09-13-2013 06:35 AM
>So why only one server is showing high utilization of maxfiles_lim?
Are both configured the same are they running the same programs? What is the max usage number instead of the percentage?
09-13-2013 10:52 PM
> Are both configured the same are they running the same programs?
Yes , Oracle 10G is configured & Running on Both Servers with RCA Clustering.
> What is the max usage number instead of the percentage?
Tunable Usage / Setting
maxfiles_lim 49873 / 4096
usage is - 49873
09-14-2013 01:37 PM - edited 09-14-2013 01:37 PM
>Server 1 Usage is - 49873 Server 2 Usage is - 1366
Looks like Server 1 is different, busier or running different applications.
I'm not sure this is worth worrying about.
09-15-2013 09:16 PM
Both the servers are running same applications & I checked Server utilization that is also almost same.
Already informed to my database team they also checking with their end.
Thanks for your replay . If I ll get any update from my team , I ll post.
09-19-2013 06:56 AM
One possibility is that maxfiles_lim may have been configured to a higher value previously (maybe 64k), and the files opened by a process could have grown to that value. This value is cached in a per-process data structure and the process can open files upto that value.
Later on when you changed the maxfiles_lim to a lower value, i.e 4096 it will not apply to processes created prior to the new changed value.
kcusage -t maxfiles_lim can be used to determine which are the top 5 processes and their pids, consuming files.
kclog -n maxfiles_lim will tell when the tunable change was applied.
Maybe you can co-relate the two to find out when the process was created. If it was created prior to the tunable change you have your answer.
09-21-2013 03:31 AM
Thanks for your Replay,
I checked with kcusage -t maxfiles_lim , i got a below O/P
Tunable Usage / Setting Usage Id Name
maxfiles_lim 60209 / 4096
60209 5496 racgimon
60207 4505 racgimon
60205 3938 racgimon
60198 4774 racgimon
1633 3845 diaglogd
kclog -n maxfiles_lim - I am not getting any output , there is no tunable was done previously.
09-21-2013 11:11 PM
The other possibility is that a privileged user has raised hard limit's via the setrlimit() system call.
From the "Restrictions" section specified in the maxfiles_lim man-page (See http://h20000.www2.hp.com/bc/docs/support/SupportM
"Restrictions on Changing
The maxfiles_lim tunable is dynamic (tuning will take effect immediately on the running system).
Dynamic changes affect all existing processes in the system except:
· Processes that have more ﬁle descriptors allocated than allowed by the new limit,
· Processes that have speciﬁcally set their limits through a call to setrlimit() or ulimit()."
The 1st one was ruled out, so it's likely the second one.
From the Security Restrictions section of setrlimit() manpage
Raising hard limits with the setrlimit call requires the LIMIT privilege (PRIV_LIMIT). Processes
owned by the superuser have this privilege. Processes owned by other users may have this privilege,
depending on system conﬁguration."
It's quite likely that the 'racgimon' process you listed out has these privileges and has specifically raised it's hard-limits to a much higher value than the system defined limit. It sounds like it's part of the Oracle RAC suite -- you might probably want to check with your Oracle contacts/forums whether the process really needs those many open files or there is a bug in the RAC suite.
From the HP-UX kernel point of view, things are working as designed.
09-23-2013 11:16 PM
Two days before we done database switch over activity from one server1 to another server2 ( server1 is having maxfiles_max issue ). After switchover the database to another server & i checked the maxfiles usage on server1 its in normal . Now again we return the database on original server1 , After done this activity i checked maxfiles_max its getting reduced on server1 ,
see the output of maxfiles_max -m & -d.
Even i rebooted the server also this problem not solved then how its getting reduced automatically?