OpenACS Monitoring

When running an OpenACS site a couple of packages come in handy to monitor your site and see why and where bottlenecks are located. To use them though, you need to install AOLserver with libthread support. If you are lazy as I am, just download the install script from http://cognovis.de/entwickler/tips/aolserver/aolserver45/ and get started. This will make sure that you have all the necessary ingridients installed.

Then go to your systems administration at /acs-admin and install two packages:

• xotcl-request-processor
• monitoring

The first one gives you an overview over the system performance, while the latter allows you to scan the error.log for errors and have them send to you via e-mail.  A little bit more detail:

Monitoring

The monitoring package has a couple of nice things which help you monitor you website. First and foremost you can use it to get the error logs. This is HIGHLY recommended, as your users will not report errors to you. They will just complain internally and not use your site anymore. We learned this the hard way.

Once you have monitoring installed (and mounted at e.g. /monitoring) go there and edit the parameters. There you can define who should get the error reports send and how often they are send out. NOTE: They will be send out with every restart of the server, so this gives you a pretty good idea if your server has restarted.

You can also get the TOP reports (if you have a high load on the system), which is something I usually don't use, but maybe it is of importance for you.

Last but not least (for me at least) you can view which scheduled procs are running on your server and when they are executed next time. This is really helpful to understand what is going on on your site of the things you cannot see.

XoTCL Request Monitor

The request monitor (automatically mounted under /request-monitor) will give you a performance overview of your site, how many users are online, what are the average page load times ....

If you look at the overview page you will first be delighted to see that the average times are so low. This is misleading as the quick fetches to ".css" files and so on are counted into the equation as well.

More accurate is the aggregated stats view, which allows you to see which URLs are called most often on your site and how expensive they are to load. It is a good idea to have the most often called pages be *fast*.

One thing we do as well is to run the "last 100 request" page on a continuous basis in a browser window. It refreshes every 60 seconds and if you get the sorting right you can actually get a good overview of the activity on your site (e.g. we order by execution time, so we always see how long users are waiting for their pages and if this number goes up considerably, well, then we know we have to act!).

Keepalive

Let me say first, anyone running an OpenACS site without daemontools and keepalive should be send to a very unpleasant place. Keepalive is the only thing preventing you from being called in the middle of the night "our site is down". Why ?

Assume your database restarts for whatever the reason. AOLserver will restart immediately (if you have daemontools), but maybe the database takes longer to come up again, so you aolserver is showing "Installation Error, check your database connection" page to your users.

Or your web server has used up too much memory (for whatever the reasons) and connections are rare (e.g. if someone is trying an attack on your site). Then keepalive will see that the server is down and issue an automatic restart. And notify you about this via e-mail. If you are smart you could actually set it up in a way that, if keepalive has to restart three times within e.g. 15 minutes, then you are send an SMS and the infernal ringtone wakes you up.

Where to configure keepalive? /web/yoursite/etc/keepalive.