05-26-2003 11:22 PM
Oracle and Tru64 manuals say that you must
change the parameter ssm_threshold=0
if you want Oracle yo be NUMA aware.
This parameter can??t be changed in GS1280+
tru64 5.1B (Patchkit 1) !!!
How can I use the NUMA capabilities of Oracle ???
05-27-2003 03:56 AM
o Tru64 System Configuration and Tuning
(System and Network Management Documentation Bookshelf) http://www.tru64unix.compaq.com/docs/pub_page/doc_
o Oracle9i Directed Placement Optimizations (NUMA Systems)
05-27-2003 10:52 PM
Please be more specific...
05-28-2003 07:11 AM
The NUMA capabilities of Oracle where nicely effective on the GS320 (wildfire) with its 4 CPU RADs and relatively big access difference between local and remote memory (3:1)
The GS1280 memory subsystem is much fast to start with, and has a gradually declining memory access speed with 'distance'.
Furthermore, there is only 1 CPU/RAD unless you go for a non-standard firmware setting striping memory in cells, creating 2 cpu rads.
While the kernel should be, and is, paying close attention to numa effects on the Marvel, the impact for Oracle is much reduced.
The main purpose of GH is, as the name implies, Granularity Hint, allowing for a single page table entry to map up to 4MB of memory.
With Tru64 V5.1B there is an equaly effective, and much more flexible solution for that: BIG PAGES. For anyone considering GH, I strongly encourage them to use the big page instead. In SYSCONFIG to set: vm_bigpg_enabled = 1
Unlike GH, big pages apply not just to SHM memory but also toe anonymous mmaps adn code!
Unlike GH big pages just need to be enabled and generaly do not need tweaking and re-tweaking (no reboots!) for changing needs (Oracle SGA sizes changing)
Go big pages and be happy!
05-29-2003 12:10 AM
I really thanks your answer.
As NUMA_ON don??t worked for me, I used big pages, but I thought numa was better.
Now I am safe on this.
Another question is about kernel parameters.
I ajust it like Oracle??s Installation Manual say.
The system has 8 GB (4+4) but when I try to ajust a Oracle SGA bigger than 3 GB (aprox) the db don??t start (insuficient space).
I am sure the problem is some kernel params (data_size or similar). So I ajust these parameters as 'Tru64 System Configuration and Tuning' say and the system hangs.
These are the kernel parameters I configured first time:
Could you tell me the correct values for a
8GB system ?
05-29-2003 07:31 AM
> I used big pages, but I thought numa was better.
> Now I am safe on this.
The two are really independend attributes,
but they come together a little bit because you are using a GS1280.
Internally we have atool to check mapping and granularity hint effectivness.
It looks like:
cda> pmap -gus 147720
rad count percent gh0 gh1 gh2 gh3
0 1 0.00% 1 0 0 0
1 0 0.00% 0 0 0 0
2 0 0.00% 0 0 0 0
3 0 0.00% 0 0 0 0
4 0 0.00% 0 0 0 0
5 289125 100.00% 477 1096 3392 284160
6 0 0.00% 0 0 0 0
7 0 0.00% 0 0 0 0
GH pages total 478 1096 3392 284160
GH pages percent 0.17% 0.38% 1.17% 98.28%
I have asked the developer how we might make that tool accesible to the general public.
Sapw = eager tell teh system to make sure there is room for all in the swap files.
The 'b' partitions tend to be smallish, so I think this is where your limit is.
Solution is to increase swpa space by adding a devices and/or switch to 'lazy' mode.
Be sure to read up in: 'man swapon'
For systems where more than 1/2 the memory is given to Oracle, I woudl suggest to go much lower on the UBC.
Maybe a max/min of 10 over 2 even. For our benchmarks we go down to 2 over 1 , if not file activity is involved.
That's a lot of maxusers for an oracle box....
I'd remcomment staying with (2GB - 4MB) or thereabouts.
Not all software know how to deal with shm chunks grated than 2GB.
05-29-2003 08:04 AM
Thanks your answer again.
/dev/disk/dsk6b partition are bigger
than defaults. I made It 24 GB.
I use the "Tru64 System Configuration and Tuning" manual. It say :
"For ES40 class systems and higher, we recommend setting the attribute to
a value of 8192 (users) or up to the maximum value of 16,384 (users). The
default value is system-dependent."
"We recommend increasing the percentage of physical memory that the UBC
can use at one time to at least 70 percent. The value should not be set to
smaller than 35 percent."
Do you have any example for the parameters:
Another paramter I am not sure is:
(DB_WRITER_MAX_WRITES (default 8192) * DB_WRITER_PROCESSES)
+ (PARALLEL_MAX_SERVERS * DB_FILE_MULTIBLOCK_READ_COUNT) + 10.