Forum OpenACS Development: Re: maxidle and maxopen
once again thanks a lot for your reply.
This is what I read on the Aolserver documentation:
MaxIdle - The maximum length of time in seconds that a database connection within this pool can remain open and idle. The default setting causes connections that are idle for 10 minutes to be closed. Note that MaxIdle will not have an effect if it is equal to MaxOpen. Setting MaxIdle to 0 makes it impossible for database handles to become stale, unless the MaxOpen time expires.
MaxOpen - The maximum length of time in seconds that a database connection within this pool can remain open. It is recommended that MaxOpen be a multiple of MaxIdle; otherwise, MaxOpen may be off by as much as (MaxOpen mod MaxIdle). The default setting causes all connections to be closed after one hour, regardless of activity level. Database pool connections will then be opened again as needed up to the value of the Connections parameter. Setting MaxOpen to 0 makes it impossible for database handles to become stale when they are not idle.
Setting MaxIdle or MaxOpen to 0 does not (DOES NOT) correspond to using their default values.
Cheers and thanks,
- handles are not used for some time (unless maxidle is 0), or
- handles are already open for a long time (unless maxopen is 0), or
- handles are stale So, setting these values to 0 disables time-specific closing of handles, which corresponds to the intention of the old default value of "1000000000". The old value can lead to some strange hanging problems due to to integer overflows on 32bit unixes, where the added value overshoots "Unix End of Times". So, 0 should be used to reach the no-close effect.
For details, see: http://dossy.org/archives/000319.html
thanks a lot for your reply.
Some few weeks ago I published a post about having some memory allocation problems with aolserver/openacs. Initially I thought these problems were due to the aolserver/TCL allocator (i.e. the standard mechanism, or Zippy - the default one, or VTMalloc). But then I observed that the majority of the menory was consumed by PostgreSQL connections. By setting the parameters MaxIdle and MaxOpen to their defualt values (600 and 3600) I managed to fix this problem.
I would recommend to update accordingly these parameters in the standard startup file "congig.tcl" which gets shipped together with the OpenACS distribution(s). This way newbies like me won't fall again in this memory problem and invest a lot of time and resources trying to solve it.
Thanks a lot in advance,
this is very interesting. I'll try this on my configurations and check, how the memory behavior changes...
I'll be very interested in knowing your results.
BTW: While you're working on this issue, I noticed that changing the "connections" parameters from 5 to 1 might also help in reducing the memory occupation.
Cheers and thanks,
Btw, reducing connections from 5 to 1 might reduce memory, but the same time your are reducing greatly the ability to run db-queries in parallel. If you have multiple http requests running, and all of these require handles from the pools, and you have ony 1 connection configured, the queries are sequentialized, since each query has to wait until a db connection is available. This means that he waiting time for handles will go up. If most of your http requests use the database the value for maxthreads must be aligned with connections. On our main site, we have 90 connections configured.
I do agree that the change in the "connection" value reduces the ability to run db-quesires in parallel.
Cheers and thanks,
there are no significant changes on my sites. I altered the configuration between Sat. and Sun. On Sat we had 421370 hits, on Sun 488751. The chart shows that the growth of the virtual size is happening with and without the MaxIdle/MaxOpen = 0 settings: http://media.wu-wien.ac.at/download/nsdsize-week-2007-03-26.png
Btw, the M should be a G in the chart. Also my other installation did not show improvements.
How did you measure savings? Is this maybe Windows specific?
ns_section ns/db/pool/pool1 ns_param maxidle 60 instead of 1000000000 ns_param maxopen 360 instead of 1000000000 ns_param connections 1 instead of 5 PID MINFLT MAJFLT VSTEXT VSIZE RSIZE VGROW RGROW MEM CMD 1/11 4143 17341 0 1K 51036K 33860K 51036K 33860K 3% nsd 4146 10226 0 2140K 135.0M 27376K 135.0M 27376K 2% postmaster 4147 1204 0 2140K 132.8M 4964K 132.8M 4964K 0% postmaster 3874 58 0 2140K 131.8M 3084K 0K 4K 0% postmaster 4148 638 0 2140K 132.3M 2716K 132.3M 2716K 0% postmaster ATOP - icaro 2007/03/22 17:44:27 600 seconds elapsed PID MINFLT MAJFLT VSTEXT VSIZE RSIZE VGROW RGROW MEM CMD 1/7 4143 10433 0 1K 66576K 47568K 15540K 13708K 4% nsd 4192 69619 0 2140K 134.9M 110.0M 134.9M 110.0M 9% postmaster 4203 998 0 2140K 132.6M 4144K 132.6M 4144K 0% postmaster 3874 59 0 2140K 131.8M 3084K 0K 0K 0% postmaster 3878 7 0 2140K 6472K 892K 0K 0K 0% postmaster ATOP - icaro 2007/03/22 17:54:27 4143 2374 0 1K 66576K 47560K 0K -8K 4% nsd 4239 1039 0 2140K 132.6M 4312K 132.6M 4312K 0% postmaster 3874 47 0 2140K 131.8M 3084K 0K 0K 0% postmaster 3878 5 0 2140K 6472K 892K 0K 0K 0% postmaster 3877 0 0 2140K 7304K 712K 0K 0K 0% postmasterAfter the first set of data I stayed put (i.e. I did not touch/used the system) and as you can see in the following sets of data that PostgreSQL released some memory (have a look at the VSIZE value). When I was using timeouts with values of 1000000000 or of 0 the memory did not get released by PostgreSQL.
Yes it is true, I put some very short timeouts. But I did that to observe some changes in the allocated memory as soon as possible.
Hope it helps,