Re: GS1280 + Oracle + ssm_threshold (161 Views)
Reply
Advisor
Juan_20
Posts: 24
Registered: ‎12-08-2002
Message 1 of 9 (161 Views)

GS1280 + Oracle + ssm_threshold


Hello

Oracle and Tru64 manuals say that you must
change the parameter ssm_threshold=0
if you want Oracle yo be NUMA aware.

This parameter can??t be changed in GS1280+
tru64 5.1B (Patchkit 1) !!!

How can I use the NUMA capabilities of Oracle ???

Thanks
jm
Please use plain text.
Honored Contributor
Ralf Puchner
Posts: 1,657
Registered: ‎04-22-2003
Message 2 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold

For a description on how to configure GH memory for the ORACLE database please consult the following documents.

o Tru64 System Configuration and Tuning
(System and Network Management Documentation Bookshelf) http://www.tru64unix.compaq.com/docs/pub_page/doc_list.html

o Oracle9i Directed Placement Optimizations (NUMA Systems)
http://docs.oracle.com/HTML_Storage/a90347/appc_tru.htm#637248
Help() { FirstReadManual(urgently); Go_to_it;; }
Please use plain text.
Advisor
Juan_20
Posts: 24
Registered: ‎12-08-2002
Message 3 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


Just these documents are wich I reference.

Oracle??s Metalink say ask HP.
jm
Please use plain text.
Honored Contributor
Ralf Puchner
Posts: 1,657
Registered: ‎04-22-2003
Message 4 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold

Why can't the parameter be changed in 5.1B? The parameter is still available and I've seen Numa-Systems with this version using the parameter.

Please be more specific...
Help() { FirstReadManual(urgently); Go_to_it;; }
Please use plain text.
Advisor
Juan_20
Posts: 24
Registered: ‎12-08-2002
Message 5 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


Kernel tuner show these parameter
un-modificable (grey).

I change the sysconfigtab file manually
and the server don??t boot.

So I suppose the parameter is fixed.
jm
Please use plain text.
Honored Contributor
Hein van den Heuvel
Posts: 6,581
Registered: ‎05-19-2003
Message 6 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


The NUMA capabilities of Oracle where nicely effective on the GS320 (wildfire) with its 4 CPU RADs and relatively big access difference between local and remote memory (3:1)

The GS1280 memory subsystem is much fast to start with, and has a gradually declining memory access speed with 'distance'.

Furthermore, there is only 1 CPU/RAD unless you go for a non-standard firmware setting striping memory in cells, creating 2 cpu rads.

While the kernel should be, and is, paying close attention to numa effects on the Marvel, the impact for Oracle is much reduced.

The main purpose of GH is, as the name implies, Granularity Hint, allowing for a single page table entry to map up to 4MB of memory.

With Tru64 V5.1B there is an equaly effective, and much more flexible solution for that: BIG PAGES. For anyone considering GH, I strongly encourage them to use the big page instead. In SYSCONFIG to set: vm_bigpg_enabled = 1

Unlike GH, big pages apply not just to SHM memory but also toe anonymous mmaps adn code!
Unlike GH big pages just need to be enabled and generaly do not need tweaking and re-tweaking (no reboots!) for changing needs (Oracle SGA sizes changing)

Go big pages and be happy!
Good luck,
Hein.

Please use plain text.
Advisor
Juan_20
Posts: 24
Registered: ‎12-08-2002
Message 7 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


I really thanks your answer.

As NUMA_ON don??t worked for me, I used big pages, but I thought numa was better.
Now I am safe on this.


Another question is about kernel parameters.
I ajust it like Oracle??s Installation Manual say.
The system has 8 GB (4+4) but when I try to ajust a Oracle SGA bigger than 3 GB (aprox) the db don??t start (insuficient space).

I am sure the problem is some kernel params (data_size or similar). So I ajust these parameters as 'Tru64 System Configuration and Tuning' say and the system hangs.


These are the kernel parameters I configured first time:

io:
max_iosize_read=131072
max_iosize_write=131072

vm:
swapdevice=/dev/disk/dsk6b
vm-swap-eager=1


new_wire_method=0
ubc_maxpercent=70
ubc_borrowpercent=10
vm_ubcdirtypercent=90
vm_ubcseqpercent=5
vm_bigpg_enabled=1
proc:
per_proc_stack_size=33554432
per_proc_data_size=201326592

max_proc_per_user=1024
max_threads_per_user=4096
maxusers=8152

ipc:
shm_max=4278190080
shm_mni=256
shm_seg=128

vfs:
fifo_do_adaptive=0

inet:
udp_recvspace=65536
udp_sendspace=65536

rt:
aio_task_max_num=16474

Could you tell me the correct values for a
8GB system ?

Thanks
jm
Please use plain text.
Honored Contributor
Hein van den Heuvel
Posts: 6,581
Registered: ‎05-19-2003
Message 8 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


> I used big pages, but I thought numa was better.
> Now I am safe on this.

The two are really independend attributes,
but they come together a little bit because you are using a GS1280.

Internally we have atool to check mapping and granularity hint effectivness.
It looks like:

cda> pmap -gus 147720
rad count percent gh0 gh1 gh2 gh3
0 1 0.00% 1 0 0 0
1 0 0.00% 0 0 0 0
2 0 0.00% 0 0 0 0
3 0 0.00% 0 0 0 0
4 0 0.00% 0 0 0 0
5 289125 100.00% 477 1096 3392 284160
6 0 0.00% 0 0 0 0
7 0 0.00% 0 0 0 0
GH pages total 478 1096 3392 284160
GH pages percent 0.17% 0.38% 1.17% 98.28%

I have asked the developer how we might make that tool accesible to the general public.


> vm:
> swapdevice=/dev/disk/dsk6b
? vm-swap-eager=1

Sapw = eager tell teh system to make sure there is room for all in the swap files.
The 'b' partitions tend to be smallish, so I think this is where your limit is.
Solution is to increase swpa space by adding a devices and/or switch to 'lazy' mode.
Be sure to read up in: 'man swapon'


> new_wire_method=0
> ubc_maxpercent=70
> ubc_borrowpercent=10

For systems where more than 1/2 the memory is given to Oracle, I woudl suggest to go much lower on the UBC.
Maybe a max/min of 10 over 2 even. For our benchmarks we go down to 2 over 1 , if not file activity is involved.


> max_proc_per_user=1024
> max_threads_per_user=4096
> maxusers=8152

That's a lot of maxusers for an oracle box....

> ipc:
> shm_max=4278190080

I'd remcomment staying with (2GB - 4MB) or thereabouts.
Not all software know how to deal with shm chunks grated than 2GB.

hth,
Hein.
Please use plain text.
Advisor
Juan_20
Posts: 24
Registered: ‎12-08-2002
Message 9 of 9 (161 Views)

Re: GS1280 + Oracle + ssm_threshold


Thanks your answer again.

/dev/disk/dsk6b partition are bigger
than defaults. I made It 24 GB.

I use the "Tru64 System Configuration and Tuning" manual. It say :

"For ES40 class systems and higher, we recommend setting the attribute to
a value of 8192 (users) or up to the maximum value of 16,384 (users). The
default value is system-dependent."

"We recommend increasing the percentage of physical memory that the UBC
can use at one time to at least 70 percent. The value should not be set to
smaller than 35 percent."

Do you have any example for the parameters:
per_proc_data_size
max_per_proc_address_space
etc.

Another paramter I am not sure is:
aio_task_max_num=16474 ==
(DB_WRITER_MAX_WRITES (default 8192) * DB_WRITER_PROCESSES)
+ (PARALLEL_MAX_SERVERS * DB_FILE_MULTIBLOCK_READ_COUNT) + 10.

Thanks
jm
Please use plain text.
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation