Forum OpenACS Q&A: memory usage

Collapse
Posted by robert parker on
Hi

The memory usage of nsd according to ps seems very large (and possibly growing). Currently nsd shows as 120MB which is nearly half the allocation permitted by my web hosting contract. In the past few days the hosting has been charging me over-usage penalties. The mystery is that I have not changed any config or added any new long running processes and, up until few days ago I never had any over-usage penalties.

So I am looking around for causes and how I can fix this. Is it normal for nsd to have an RSS of 120MB ? I only re-started it yesterday, a few hours ago it was 115MB.

I have looked in the nsd logs and there is hardly any traffic, and no sign of any processes "spinning" - top shows nsd at 0.1% CPU

Since this hasn't been a problem before I never checked the rss values when things were running smoothly so I dont know what is "normal".

Is there some known incompatibility between versions that causes large memory usage?

FreebSD 7.3
OpenACS 5.2.3
postgres 7.4.29
aolserver (it's so long since I installed OpenACS I cant remember what version got installed, how can you determine aolserver version?)

thanks
Robert

Collapse
2: Re: memory usage (response to 1)
Posted by Dave Bauer on
What are your minthreads and maxthreads settings in the aolserver config.tcl?

Since it creates a full Tcl interpreter with every procedure defined (all the OpenACS ones plus any custom code you have) the footprint increases for each thread you have defined.

If you have very little traffic you might be able to decrease this but, in my experience., 120mb is pretty small for an aolserver process.

Collapse
3: Re: memory usage (response to 1)
Posted by Andrew Piskorski on
Robert, has your traffic gone up? That would be one likely reason why your AOLserver process is now tending to use more RAM than it used to.

Dave is right, 120 MB is pretty small, and is a tiny amount of RAM these days. Keep in mind that some parts of AOLserver were purposely designed to be somewhat wasteful of memory in order to run faster.

But if you need it small, cutting down your number of connection threads to a minimum is probably the main way. Also configure things so that idle connection threads will eventually exit, rather than hanging around taking up RAM. You might also try un-installing any OpenACS packages (and thus their Tcl code) that you don't actually need.

The main thing though, is find out where your AOLserver process is actually using its RAM... Depending on what applications you're running, it could be various different things. Then hit a development server with a similar load, verify that it's using up RAM in a similar way, so you can then measure how much you save without taking down and/or waiting for traffic on your Production site.

Collapse
4: Re: memory usage (response to 1)
Posted by Patrick Giagnocavo on
The sad reality is that you will need more.

You may as well "bite the bullet" now and either look for a new host or upgrade to their larger hosting plan, perhaps one with 512MB RAM.

In AOLserver's defense, though, it can handle a lot of traffic and usage without growing much more; it is just that its initial footprint is larger.

Collapse
5: Re: memory usage (response to 4)
Posted by robert parker on
I changed :
maxconnections from 100 down to 50
maxthreads 10 down to 6
minthreads from 5 down to 3

After a restart of aolserver :
at 01:00 the rss was 104656,
at 08:40 the rss was 119880,
at 20:33 the rss was 132588

In that time there have been no more than 130 page requests

The environment has not changed recently, so I'm struggling to explain why I'm getting over-usage now. Maybe in the past the hosting went down frequently enough to mask the issue with over usage.

thanks,
Robert

Collapse
6: Re: memory usage (response to 1)
Posted by Torben Brosten on
Robert,

Also look at the Postgres usage. Some of the caching is with the pg process. Restarting nsd would not clear the pg caches; restarting pg would..

For reference, I admin sites using aolserver 4.5.1, pg 8.2 and freebsd 7.3 (OpenACS 5.5+head). Here are some RSS sizes.

a site using caching with ecommerce:
397968 with pg RSS ranging from 1152 to 126736 (5 near 126k size, 6 averaging about 3k)

a site using multiple host node mappings:
424968 with pg RSS ranging from 7708 to 47420 (6 near 40k, 2 averaging 8k)

an unpopular site:
89944 with pg RSS ranging from 0 to 6936 (2 near 5k, 4 near 0 )

an artist website:
215152 with pg RSS ranging from 55144 to 68656 (fairly evenly spread)

cheers,

Torben