Forum OpenACS Development: Re: Is it time for NaviServ Again?

Collapse
Posted by Tom Jackson on
Jon,

An AOL guy added a quick fix, a new file to handle the default threadpool. I have modified this which allows you to specify pools for url patterns. I'm posting my file here, which should be placed in the shared tcl library directory (usually in [ns_info home]/modules/tcl). Below this is an example taken from the server.log file on startup.


# [ns_info home]/modules/tcl/pools.tcl

set cfgsection "ns/server/[ns_info server]"

set minthreads [ns_config $cfgsection minthreads 0]
set maxthreads [ns_config $cfgsection maxthreads 10]
set maxconns [ns_config $cfgsection maxconnections 0]
set timeout [ns_config $cfgsection threadtimeout 0]

ns_pools set default -minthreads $minthreads -maxthreads $maxthreads -maxconns $maxconns -timeout $timeout

ns_log Notice "Default Pool: [ns_pools get default]"

# Setup optional threadpools

set poolSection $cfgsection/pools

set poolSet [ns_configsection $poolSection]

if {"$poolSet" ne ""} {

    set poolSize [ns_set size $poolSet]
    for {set i 0} {$i < $poolSize} {incr i} {
        set poolName [ns_set key $poolSet $i]
        set poolDescription [ns_set value $poolSet $i]
        set poolConfigSection "ns/server/[ns_info server]/pool/$poolName"
        set poolConfigSet [ns_configsection $poolConfigSection]
        if {"$poolConfigSet" eq ""} {
            continue
        }
        set poolMinthreads [ns_config $poolConfigSection minthreads $minthreads]
        set poolMaxthreads [ns_config $poolConfigSection maxthreads $maxthreads]
        set poolMaxconns   [ns_config $poolConfigSection maxconnections $maxconns]
        set poolTimeout    [ns_config $poolConfigSection threadtimeout $timeout]
        
        ns_pools set $poolName -minthreads $poolMinthreads -maxthreads $poolMaxthreads -maxconns $poolMaxconns -timeout $poolTimeout
        ns_log Notice  "$poolName Pool: [ns_pools get $poolName]"
        set poolConfigSize [ns_set size $poolConfigSet]
        for {set j 0} {$j < $poolConfigSize} {incr j} {
            if {[string tolower [ns_set key $poolConfigSet $j]] eq "map"} {
                set mapList [split [ns_set value $poolConfigSet $j]]
                set poolMethod [lindex $mapList 0]
                set poolPattern [lindex $mapList 1]
                ns_pools register $poolName [ns_info server] $poolMethod $poolPattern
                ns_log Notice "ns_pools registered $poolName [ns_info server] $poolMethod $poolPattern"
            }
        }
    }
}

ns_section ns/server/jnm
        ns_param maxconnections 5
        ns_param maxdropped 0
        ns_param maxthreads 5
        ns_param minthreads 5
        ns_param threadtimeout 120

ns_section ns/server/jnm/pool/fast
        ns_param maxconnections 100
        ns_param minthreads 2
        ns_param maxthreads 10
        ns_param threadtimeout 120
        ns_param map {GET /*-thumb.jpg}
        ns_param map {GET /images/*-thumb.jpg}

ns_section ns/server/jnm/pools
        ns_param fast {fast pool}

ns_section ns/server/tutos
        ns_param maxconnections 6
        ns_param maxdropped 0
        ns_param maxthreads 6
        ns_param minthreads 6
        ns_param threadtimeout 126

ns_section ns/server/tutos/pool/default
        ns_param maxconnections 99
        ns_param minthreads 4
        ns_param maxthreads 11
        ns_param threadtimeout 123
        ns_param map {GET /}
        ns_param map {POST /}

ns_section ns/server/tutos/pool/fast
        ns_param maxconnections 101
        ns_param minthreads 3
        ns_param maxthreads 11
        ns_param threadtimeout 125
        ns_param map {GET /*-thumb.jpg}
        ns_param map {GET /*-thumb.gif}

ns_section ns/server/tutos/pools
        ns_param default {default pool}
        ns_param fast {fast pool}

------------

During startup, the following is printed to the error.log file:


[-main-] Notice: Default Pool: minthreads 5 maxthreads 5 idle 0 current 0 maxconns 5 queued 0 timeout 120
[-main-] Notice: fast Pool: minthreads 2 maxthreads 10 idle 0 current 0 maxconns 100 queued 0 timeout 120
[-main-] Notice: ns_pools registered fast jnm GET /*-thumb.jpg
[-main-] Notice: ns_pools registered fast jnm GET /images/*-thumb.jpg

[-main-] Notice: Default Pool: minthreads 6 maxthreads 6 idle 0 current 0 maxconns 6 queued 0 timeout 126
[-main-] Notice: default Pool: minthreads 4 maxthreads 11 idle 0 current 0 maxconns 99 queued 0 timeout 123
[-main-] Notice: ns_pools registered default tutos GET /
[-main-] Notice: ns_pools registered default tutos POST /
[-main-] Notice: fast Pool: minthreads 3 maxthreads 11 idle 0 current 0 maxconns 101 queued 0 timeout 125
[-main-] Notice: ns_pools registered fast tutos GET /*-thumb.jpg
[-main-] Notice: ns_pools registered fast tutos GET /*-thumb.gif

It is hard to imagine why this happened, the attitude is a little difficult to digest: don't upgrade. Maybe the important point is that it took a year for anyone to notice, so the default behavior must be pretty good.

Collapse
Posted by Malte Sussdorff on
Being blatantly ignorant, how would you include your script in the default config.tcl? I think this might be a worthy general addition in OpenACS, if only for the fact that more people read about it :-). And we could easily link it if it is residing in OpenACS' /etc directory.

As for switching to Naviserver: If anyone can provide an install script for Naviserver just like the one I have for AOLserver 4.5 (http://cognovis.de/developer/en/aolserver_install), maybe more would be willing to try.

Collapse
Posted by Don Baccus on
I hope you realize the one you're using for 4.5 doesn't actually work as you expect?

That's the bad news Jon's referring to (4.5 ignores the min/maxconnections parameters).

Collapse
Posted by Don Baccus on
That reply was to malte (we need to upgrade this forums instance so we actually display this flat forum in flat form!)
Collapse
Posted by Malte Sussdorff on
If 4.5 ignores that setting completely, even more reason to make this a) Known within OpenACS and b) provide the patch along with it. I did not run in this problem so far as I never checked these parameters and therefore have no expectations about them whatsoever.
Collapse
Posted by Tom Jackson on
This isn't really in response to any of the above, just an update based upon further analysis and testing.

Here is the deal: AOLserver 4.5 should probably only be used by OpenACS installations using a single virtual server per nsd instance. Threadpools are shared among virtual servers, so you can only set one 'default' threadpool. It is currently unknow if data or code could migrate from one virtual server to another, which would be very bad. But essentially the very well thought out threadpool functionality which was available in AOLserver 4.0.10 has been replaced with broken code and configuration in a truely unbelievably stupid move. Of course this was backed up with no documentation or notification, and the developers at AOL don't even appear to understand how the new code works or doesn't work or how it should be used.

Malte: the code above should just be ignored. It should have been placed in a file pools.tcl as I indicated. I only posted it here so that anyone who didn't watch the AOLserver list would be able to get it without hunting around in the cvs on sourceforge. But it doesn't work due to broken C code.

I don't think this bodes well for the future of AOLserver since we have a combination of secrecy, poor decisionmaking and ass covering at AOL. We can probably add to that no accountability, since I'm sure there is nothing we can do about it. At least tomorrow is Friday!

Collapse
Posted by Andrew Piskorski on
Tom et al., although I can't really disagree with your complaints about the AOLserver 4.5 development, communication, etc., I think you may be overreacting a bit. So, they committed some likely ill-thought out code, and then eventually released it before it was ready, all without nearly enough communication and review. Bad, yes, but it does happen, and shouldn't be an insurmountable problem - just an obnoxious one.

Also, it's not like us other folks in the AOLserver community jumped all over the problem first thing! E.g., I'm still using 4.0.10, haven't touched it in ages, and have barely looked at the 4.5 code, even though it was released back in June 2006, more than a year ago!

Btw, Tom, thanks for discovering and really digging into these 4.5 problems.

I also wonder how much of the premature 4.5 release was driven or abetted by internal turmoil at AOL, which the AOL guys never talk about publically on the AOLserver list. It sounds to me as if most of the AOLserver developers - including Jim Davidson - are now employed at other companies doing other things, and the few that remain at AOL are probably no longer working directly on AOLserver related projects at all. I think an AOL employee alluded to this recently on the list by saying something like, "If it wasn't for the [open source] community, AOLserver would have been dead two years ago."

Also, Tom, re. your "I'm sure there is nothing we can do about it." comment - that seems too negative to me! There is certainly something we could do about it - we could "simply" invest the time to figure out and commit changes to the code! (We could; I'm not saying we should have to, and the wasted effort in straightening out someone else's broken new code when the old was perfectly good is always annoying.) There are many non-AOL folks who have CVS commit access to AOLserver on SourceForege, note, even if hardly any of them ever use it.

Collapse
Posted by Tom Jackson on
Andrew,

Please refer to my first comment in this thread. Until yesterday I pretty much agreed that I was overreacting. But after further discussion and analysis, it is obvious that the best case situation is that the AOL developers are just bad programmers. Worst case is of course that they are just pretending to care about the larger AOLserver community. It is time to stop speculating and demand change which we can verify, or just stop listening to the BS. Why waste another second working on a dead-end? This is the issue. These things waste everyone's time, and if mere ignorance is the reason AOLserver didn't die two years ago, it is little consolation. Do they take everyone for complete idiots? Apparently.