Forum .LRN Q&A: Re: Help Needed in Setting up .LRN to Scale

Posted by Carl Robert Blesius on
Just so people know: our Sun server is serving pages at an acceptable  rate again (and we did not have to make any hardware changes). After a lot of painful detective work what actually solved our performance problem was some fine tuning of .LRN sql queries in Oracle (in addition to some fine tuning of Oracle itself).

We will make sure to get any changes that where made on our server back into the source and into the documentation as soon as they have been peer-reviewed.

Thanks for all the help everyone (special thanks to Janine/Sloan/Collaboraid/Dirk).

Will post the details soon here.

Posted by Lars Pind on
We should mention also that switching to AOLserver 4 improved performance by about 30% as well.
Posted by C. R. Oldham on

Did you come from 3.3, or one of the 3.5 series?


Posted by Lars Pind on
From 3.3.

just a short report regarding current performance status at University of Heidelberg.
On 21th October we moved AOLServer from our Sun, which hosted Webserver and Oracle at that moment, to a linux box, so our current infrastructre is:

- Front-End:
SuSe Linux 9.0
Processor: Athlon 1.8 GHz
Memory: 1.5 Gb
Network: Ethernet, 100Mb/s
DB-Client: Oracle
AOLServer: 4.0.8 (nsopenssl v3_0beta23 / tcl. 8.4.4)
OpenSSL: OpenSSL 0.9.7d 17 Mar 2004
MTA: Postfix 2.0.14

- DB-Server (as mentioned above):
Solaris 2.8 on Sun Fire 280r
Memory: 2 Gb
Disks: 2 * 36 Gb, 1 * 200 Gb raid
Network: Ethernet, 100Mb/s
DB-Server: Oracle 8.1i (patched)
Additional software running until end of year: webct

- Some facts about our dotLRN-installation:
40282 ACS-Users
2580 dotLRN-Users
22 class instances (current semester, about 40 total), 10 communities / 72 subcommunities
162641 ACS-objects, 114108 ACS-permissions and 11129 fs-objects

- Performance relevant AOLServer configuration parameters:
maxthreads 25
minthreads 20
(Changed minthreads != maxthreads, because there seem to be still some memory leakage issues)
threadtimeout 3600
stacksize 512 Kb
db-pool connections: first 20, second 10, third 5
keepalivetimeout 5
maxkeepalive 100
maxconnections 100

Some objective measurements of page response times before and after migration (calculated by

- /dotlrn
before: 2894 ms
after: 841 ms (internal measure by developer support: 80-100 ms)
- /dotlrn/calendar/cal-item-new
before: 2380 ms
after: 633 ms
- /dotlrn/manage-memberships:
before: 3597 ms
after: 1834 ms
- /dotlrn/classes/3520praktischeinformatik/3520urztest/3520urztest/
before: 7575 ms for class start page and 4589 ms for class file-storage
after: 3093 ms for class start page and 1335 ms for class file-storage
      (1694 ms for class start page after removing subgroup- and homework-portlet!)

Alltogether, each page became about twice faster, although there might be still enough things to tune...
(E.g.: For very large query result sets AOLServer seems to need much more (exponential) time to parse the template via templating system compared to very fast output by manual ns_write commands.)

Oh, there's something unnecessarily O(n^2) or worse in the Templating System Tcl code? That's bad. Martin, could you please give more details on exactly what pages and queries demonstrate that, so that someone will be able to track it down?