Forum OpenACS Q&A: System setup for medium/heavy load

Posted by Peter Alberer on 09/17/02 12:13 AM

I am planning a new system setup (especially the hardware config) for a rather large e-learning system of an austrian university. In the moment there are about 700 users in the system and it will get probably around 2000 additional users over the next few weeks. The moment we are using now is one dual-processor intel machine with 4GB Ram to run postgres 7.2, and openacs4.

Currently the content is mostly interactive learning materials (online quizzes, exams, textbooks). To be able to support additional features in the future (most important dotLRN) and have acceptable response times even in weeks before exams (were ~60% of all students do practice) we think that we will need a somewhat more sophisticated system setup.

Currently i am thinking of using 2-3 slim web servers and one big fat database server. My questions are:

What would be a reasonable configuration for the db-machine? (i would like to stay with intel cpus) Should i go for raid5 or for several raid 1+0 combinations and split db-files on these? Any recommendations for #cpus or ram?
What do i have to look for when using more than one webserver for the system? What kind of problems do i have to deal with when they all write to the same db? I saw that there is some kind of clustering support within openacs that deals with util_memoize but did not find too much documentation.
In general i would be very glad to hear about the setup of others with similar basic conditions.

Many TIA

2: Response to System setup for medium/heavy load (response to 1)

Posted by Jonathan Ellis on 09/17/02 01:14 AM

I had to work HARD, caching-wise, to get the nsd cpu usage up to where now it's maybe 20-30% of the total. This is with gzip compression on. I'd guess unless you're serving much more image-intensive pages, you don't really need to buy more boxes for the nsd end. But since you don't say what kind of cpu usage you're seeing currently, that's just a wild guess. :0

My datapoint: carnageblender serves about 250k pages on a good day. I've seen it do 300k w/o breaking a sweat on my dual-1GHz p3 box with 1GB RAM. I would bet the current setup could handle at least twice what it currently does. Past that, I could get a few more percent from caching less-important stuff, but to get past 1M I'd need better disks. I've already split the WAL to its own disk, which helps, but let's face it, 5400 RPM ide disks aren't where it's at. :)

My feeling is if you can optimize to where you can stay on a single (possibly bigger) box, it will be worth it to avoid the admin headache of multiple points of failure.

3: Response to System setup for medium/heavy load (response to 1)

Posted by Jun Yamog on 09/17/02 04:48 AM

Hi Peter,

Here is a bit of doc about that util_memoize thingy.

https://openacs.org/doc/server-cluster.html

Also you can look at the /admin/site-map of your OpenACS under ACS Kernel in section server-cluster.

Based from our load of our servers. Normally aolserver, OpenACS, PG, etc only needs memory to be fast. Not much CPU util. RAID 5 will always be the slowest among RAIDs due to the computation overhead which is made signicantly low by the hardware RAID. RAID 1 is good for reads but bad for writes since you have to write twice. Also RAID 1 is a waste of space. RAID 0 is good on read and write at the expense of less reliability.

If you will use a web farm then those web farm does not need any fast I/O as long as they have enough RAM to cache the files. Since its a web farm you don't need the realiablity, so RAID 0 is very much preferred. Redundancy is taken cared of in the machine level rather than the hard disk level. For the db server a fast I/O (10K or 15K rpm disks, with as much RAM on the RAID controller) is a must, also a reliable storage. RAID 0+1 is a good one if you have the money. But normally I just pick RAID 1 since reliability for me is higher on the list than performance. There is also less likely that such a site will ever tax today's hardware. Unless your site is really really big there is not much in buying the greatest hardware. Just buy what is easy to administer and reliable. Not much worries about performance.

Oh yeah, if you go for a web farm its better to have them mount their OpenACS to a central server. Most likey this will be the db server. That way when you update the sources of the web farm you just update it on one copy.

4: Response to System setup for medium/heavy load (response to 1)

Posted by Gregor Obernosterer on 09/18/02 03:20 PM

Dear Peter, first of all, this is really exciting that also the WU is considering to use DOTLrn. I myself are a student at the TU and together with a computer science professor we founded a company in Vienna three years ago now, focusing on Open Source and IT-Security. For db-driven sites we mainly use OpenACS. Maybe you pass by our stand at the IFABO. You find us under CSO Lanifex GmbH in the company register.

If you need help in setting up the system on site we would definitely like to help, together with Tilmann and Collaboraid.

Cheers, Gregor

5: Response to System setup for medium/heavy load (response to 1)

Posted by Peter Alberer on 09/18/02 04:12 PM

Hi Gregor,

nice to hear from you. I hope that i will have time to visit the Ifabo. As far as dotLRN is concerned i think we will be able to use some of its features already in this semester. A little bit of help in tuning the system could definitely be fine :)

ciao, peter

6: Response to System setup for medium/heavy load (response to 1)

Posted by Carl Robert Blesius on 09/19/02 11:20 AM

This is good news!
I assume WU = Wirtschaftsuniversität Wien.

We are planning on doing the same thing here in Heidelberg (2-3 slim web servers in front of one big fat database server). Looking forward to see how things progress. Gregor, I hope you all are working on getting the Technische Universität Wien to think about dotLRN as well. 😉