Forum OpenACS Q&A: Need some tips about memory allocation

Hello,

I recently put a formerly phisical server into a virtual machine. Right now I can give to it only 1280MB of RAM, where the former server could enjoy a full 2GB, working without issues.

The server hosts 2 OpenAcs instances on Naviserver, both built from installation script like 1 month ago. The OS is Debian Wheezy and the db is Postgres 9.1 obtained from repositories.

When the server starts, memory consumption is fine and the websites are very responsive. During the day though, seems like nsd "gets comfortable" into memory, to the point it is not possible to call external commands anymore (my application needs to do this all the time to generate pdfs). ns_proxy is in place and seems to work correctly.

I had already discussed with Gustaf in this forum about memory allocation in tcl. I've tried different approaches, like turning off the zippy allocator and switching to different malloc alternatives, but the problem seems to persist.

The question is: is there a way to keep the ram size required for nsd to a minumim, maybe through the use of some aggressive allocator you could point me to? Is there some technique I could apply to configurations or tcl scripts to limit memory consumption?

Something that comes to my mind is that some autocompletion webservice we use extensively into forms makes use of tDom, but never delete the documents they create... could that cause a memory leak?

I am ready to give up and just get some more RAM, but first I was curious to explore different tweakings.

Collapse
Posted by Claudio Pasolini on
$doc delete is your friend
Collapse
Posted by Antonio Pisano on
Thanks Claudio. And do you think not deleting a dom tree could cause a leak?
Collapse
Posted by Dave Bauer on
Yes it does cause a memory leak.

Since the interpreter continues after a request, you need to free the documents. The thread cleanup does not take care of them.

Collapse
Posted by Gustaf Neumann on
The recommended way to work with tDOM is to use the idiom
    dom parse $xml doc
instead of
    set document [dom parse $xml]
since the former deletes the tdom-managed memory automatically, when the variable doc is freed, the latter not. If doc is not a namespaced variable, aolserver/naviserver will automatically clean it up after the request. In the second variant, you tell tDOM that you want to manage the lifetime of the documents yourself.... and you have to clean it, otherwise you are facing a memory leak. For details, see [1]

To check potential leaks, use in ds/shell

info command ::dom*0x*
If this command returns non-empty, you have likely leaking tDOM nodes and/or tDOM documents.

-g

[1] http://tdom.github.io/dom.html

Collapse
Posted by Brian Fenton on
Hi Antonio

slightly off-topic, but we're doing something very similar to what you describe (exec calls to generate PDFs). Would you mind briefly describing your ns_proxy setup? I'd be very interested in that.

thanks
Brian

Collapse
Posted by Antonio Pisano on
Well, up to now my system was running on Aolserver 4.5.1 without any ns_proxy in place. Memory was not a concern, I guess because there have always been enough RAM for the nsd process to fork...

On current configuration I am running on out of the box setting for Naviserver and OpenAcs installed from the two automatic scripts provided here.

About Aolserver, I found out that version 4.5.1 provided by Debian disables ns_proxy because of a well known segfault problem which has never made it to the upstream source. This is the reference to a Google group discussing the issue: https://groups.google.com/forum/#!topic/aolserver/KBE8UySmdH4

I think nothing special should be configured on a fresh Naviserver instance. If you are running on Aolserver you will need either to patch the ns_proxy module or be sure to have enough memory, if you require to 'exec' frequently.

Collapse
Posted by Antonio Pisano on
Thanks for the pointers guys! I revised the code to eliminate the leaks.

I have also managed to give an extra 256MB to the machine and all went smooth today... I guess this fixed the issue.

Collapse
Posted by Brian Fenton on
Thanks Antonio!