Forum .LRN Q&A: Re: Server recommend for dotLRN - with 80.000 users-

Collapse
Posted by Denis Roy on
When you are planning your hardware setup, it very much depends on the number of concurrent users. Having a lot of data in your database doesn't really matter if there are only say 20 users online at the same time.

Since Malte is referring to AIESEC, let me give you some more concrete info:

We have 120K registered users. Only about 20K are active since it is also an alumni database. Our webserver is a Dual-P3 with around 933 Mhz each and 2 GB RAM. Our database server is a Dual-Opteron system as Malte already mentioned.

With this setup we support about 70 concurrent users on OACS 4.6.3 and .LRN 1.0 without any noticable negative impact on performance. When more users try to access the website, performance will decrease considerably. I found out that the webserver is the bottleneck because even when the website is almost not responding anymore at all (around 90 concurrent users) the database is still doing fine. Testing a second webserver at the same time was showing great performance on the second webserver.

We are currently installing a load-balanced solution which will make us very flexible on the frontend (webserver) side. It's just a matter of time until the database will become the bottleneck.

If anyone is interested, I will let you know about the results of our load-balancing and how many users we can handle with it.

Collapse
Posted by Andrew Piskorski on
Dennis, yes, please do let us know how the load-balancing work turns out! These sorts of hands-on reports always make useful background knowledge.

How well are the OpenACS server cluster procs working out for you so far?

Of course, if your site is growing you probably want to add the front-end load-balancing anyway, but in the near term it might be simpler just to swap out that aging dual P3 for a new machine.

Collapse
Posted by Cato Kolås on
We have around 18000 registered users, but max unique logins during a fortnight is close to 10000.

Specs:
webserver (~.LRN 1.0/OACS 4.6.3):
dual Xeon CPU 2.8 GHz, 3 GB RAM (dell 2650)

oracleserver (8.1.7.3):
dual Xeon CPU 2.8 GHz, 4 GB RAM (dell 2650)

(Both also has hyperthreading turned on and a dedicated 1 Gbit network between the two)

On heavy loads we find that it's the oracleserver that has the highest load. I'm wondering if the is anything more to gain by tweaking the aolserver-config:

ns_param  maxconnections    50
ns_param  maxdropped        0
ns_param  maxthreads        50
ns_param  minthreads        10
ns_param  threadtimeout      120
...
ns_section ns/db/pool/pool1
ns_param  connections        6
...
ns_section ns/db/pool/pool2
ns_param  connections        16
...
ns_section ns/db/pool/pool3
ns_param  connections        5

Some pics:
http://elg.uib.no/~edpck/logins-week.gif
http://elg.uib.no/~edpck/load-week.gif (green: webserver, blue: oracleserver)

All feedback appreciated

Cheers

Collapse
Posted by Andrew Piskorski on
Cato, my guess is you have your AOLserver threadtimeout set much too low, which is worth experimenting with and changing.

Why do you have so many more connections in your 2nd database pool than in your first? That seems weird.

On both machines, leaving the P4 hyperthreading turned on is also probably a bad idea unless you have a kernel (Linux 2.6.x, I think) which distinguishes between real and virtual/hyperthread processors.

Collapse
Posted by Cato Kolås on
<blockquote> Cato, my guess is you have your AOLserver threadtimeout set
much too low, which is worth experimenting with and
changing.
</blockquote>

ok, will do. thx.

<blockquote> Why do you have so many more connections in your 2nd
database pool than in your first? That seems weird.
</blockquote>

have turned on logging when pools are empty:
ns_param  WarnEmpty          true
- and based on the numbers of warnings tried to adjust the settings.

Collapse
Posted by Nima Mazloumi on
Hi Cato,
can you tell me what tool you use to log the load on the webserver and the database?

Greetings,
Nima

Collapse
Posted by Cato Kolås on
|Hi Cato,
|can you tell me what tool you use to log the load on the
|webserver and the database?

It's mrtg with a custom script to gather information about the load from the two machines (runs uptime via ssh connections and greps for the load).

Collapse
Posted by Nima Mazloumi on
Hi Denis,
I am really interested to learn more about your load-balancing solution.

Greetings,
Nima