Forum OpenACS Development: Is it time for NaviServ Again?

Collapse
Posted by Jon Griffin on
In light of the recent revelations on the aolserver forums, does it make sense to start using naviserver?

The attitude on the aol list is just like the bad old days.
Lets make changes, don't ask the community, implement them, don't document them.

If the community complains, well, tell them tough S*&* and don't use, after all, we changed it and we don't even use it.

Anyone wonder why LAMP kicks our ass?

Collapse
Posted by Tom Jackson on
Jon,

An AOL guy added a quick fix, a new file to handle the default threadpool. I have modified this which allows you to specify pools for url patterns. I'm posting my file here, which should be placed in the shared tcl library directory (usually in [ns_info home]/modules/tcl). Below this is an example taken from the server.log file on startup.


# [ns_info home]/modules/tcl/pools.tcl

set cfgsection "ns/server/[ns_info server]"

set minthreads [ns_config $cfgsection minthreads 0]
set maxthreads [ns_config $cfgsection maxthreads 10]
set maxconns [ns_config $cfgsection maxconnections 0]
set timeout [ns_config $cfgsection threadtimeout 0]

ns_pools set default -minthreads $minthreads -maxthreads $maxthreads -maxconns $maxconns -timeout $timeout

ns_log Notice "Default Pool: [ns_pools get default]"

# Setup optional threadpools

set poolSection $cfgsection/pools

set poolSet [ns_configsection $poolSection]

if {"$poolSet" ne ""} {

    set poolSize [ns_set size $poolSet]
    for {set i 0} {$i < $poolSize} {incr i} {
        set poolName [ns_set key $poolSet $i]
        set poolDescription [ns_set value $poolSet $i]
        set poolConfigSection "ns/server/[ns_info server]/pool/$poolName"
        set poolConfigSet [ns_configsection $poolConfigSection]
        if {"$poolConfigSet" eq ""} {
            continue
        }
        set poolMinthreads [ns_config $poolConfigSection minthreads $minthreads]
        set poolMaxthreads [ns_config $poolConfigSection maxthreads $maxthreads]
        set poolMaxconns   [ns_config $poolConfigSection maxconnections $maxconns]
        set poolTimeout    [ns_config $poolConfigSection threadtimeout $timeout]
        
        ns_pools set $poolName -minthreads $poolMinthreads -maxthreads $poolMaxthreads -maxconns $poolMaxconns -timeout $poolTimeout
        ns_log Notice  "$poolName Pool: [ns_pools get $poolName]"
        set poolConfigSize [ns_set size $poolConfigSet]
        for {set j 0} {$j < $poolConfigSize} {incr j} {
            if {[string tolower [ns_set key $poolConfigSet $j]] eq "map"} {
                set mapList [split [ns_set value $poolConfigSet $j]]
                set poolMethod [lindex $mapList 0]
                set poolPattern [lindex $mapList 1]
                ns_pools register $poolName [ns_info server] $poolMethod $poolPattern
                ns_log Notice "ns_pools registered $poolName [ns_info server] $poolMethod $poolPattern"
            }
        }
    }
}

ns_section ns/server/jnm
        ns_param maxconnections 5
        ns_param maxdropped 0
        ns_param maxthreads 5
        ns_param minthreads 5
        ns_param threadtimeout 120

ns_section ns/server/jnm/pool/fast
        ns_param maxconnections 100
        ns_param minthreads 2
        ns_param maxthreads 10
        ns_param threadtimeout 120
        ns_param map {GET /*-thumb.jpg}
        ns_param map {GET /images/*-thumb.jpg}

ns_section ns/server/jnm/pools
        ns_param fast {fast pool}

ns_section ns/server/tutos
        ns_param maxconnections 6
        ns_param maxdropped 0
        ns_param maxthreads 6
        ns_param minthreads 6
        ns_param threadtimeout 126

ns_section ns/server/tutos/pool/default
        ns_param maxconnections 99
        ns_param minthreads 4
        ns_param maxthreads 11
        ns_param threadtimeout 123
        ns_param map {GET /}
        ns_param map {POST /}

ns_section ns/server/tutos/pool/fast
        ns_param maxconnections 101
        ns_param minthreads 3
        ns_param maxthreads 11
        ns_param threadtimeout 125
        ns_param map {GET /*-thumb.jpg}
        ns_param map {GET /*-thumb.gif}

ns_section ns/server/tutos/pools
        ns_param default {default pool}
        ns_param fast {fast pool}

------------

During startup, the following is printed to the error.log file:


[-main-] Notice: Default Pool: minthreads 5 maxthreads 5 idle 0 current 0 maxconns 5 queued 0 timeout 120
[-main-] Notice: fast Pool: minthreads 2 maxthreads 10 idle 0 current 0 maxconns 100 queued 0 timeout 120
[-main-] Notice: ns_pools registered fast jnm GET /*-thumb.jpg
[-main-] Notice: ns_pools registered fast jnm GET /images/*-thumb.jpg

[-main-] Notice: Default Pool: minthreads 6 maxthreads 6 idle 0 current 0 maxconns 6 queued 0 timeout 126
[-main-] Notice: default Pool: minthreads 4 maxthreads 11 idle 0 current 0 maxconns 99 queued 0 timeout 123
[-main-] Notice: ns_pools registered default tutos GET /
[-main-] Notice: ns_pools registered default tutos POST /
[-main-] Notice: fast Pool: minthreads 3 maxthreads 11 idle 0 current 0 maxconns 101 queued 0 timeout 125
[-main-] Notice: ns_pools registered fast tutos GET /*-thumb.jpg
[-main-] Notice: ns_pools registered fast tutos GET /*-thumb.gif

It is hard to imagine why this happened, the attitude is a little difficult to digest: don't upgrade. Maybe the important point is that it took a year for anyone to notice, so the default behavior must be pretty good.

Collapse
Posted by Malte Sussdorff on
Being blatantly ignorant, how would you include your script in the default config.tcl? I think this might be a worthy general addition in OpenACS, if only for the fact that more people read about it 😊. And we could easily link it if it is residing in OpenACS' /etc directory.

As for switching to Naviserver: If anyone can provide an install script for Naviserver just like the one I have for AOLserver 4.5 (http://cognovis.de/developer/en/aolserver_install), maybe more would be willing to try.

Collapse
Posted by Don Baccus on
I hope you realize the one you're using for 4.5 doesn't actually work as you expect?

That's the bad news Jon's referring to (4.5 ignores the min/maxconnections parameters).

Collapse
Posted by Don Baccus on
That reply was to malte (we need to upgrade this forums instance so we actually display this flat forum in flat form!)
Collapse
Posted by Malte Sussdorff on
If 4.5 ignores that setting completely, even more reason to make this a) Known within OpenACS and b) provide the patch along with it. I did not run in this problem so far as I never checked these parameters and therefore have no expectations about them whatsoever.
Collapse
Posted by Tom Jackson on
This isn't really in response to any of the above, just an update based upon further analysis and testing.

Here is the deal: AOLserver 4.5 should probably only be used by OpenACS installations using a single virtual server per nsd instance. Threadpools are shared among virtual servers, so you can only set one 'default' threadpool. It is currently unknow if data or code could migrate from one virtual server to another, which would be very bad. But essentially the very well thought out threadpool functionality which was available in AOLserver 4.0.10 has been replaced with broken code and configuration in a truely unbelievably stupid move. Of course this was backed up with no documentation or notification, and the developers at AOL don't even appear to understand how the new code works or doesn't work or how it should be used.

Malte: the code above should just be ignored. It should have been placed in a file pools.tcl as I indicated. I only posted it here so that anyone who didn't watch the AOLserver list would be able to get it without hunting around in the cvs on sourceforge. But it doesn't work due to broken C code.

I don't think this bodes well for the future of AOLserver since we have a combination of secrecy, poor decisionmaking and ass covering at AOL. We can probably add to that no accountability, since I'm sure there is nothing we can do about it. At least tomorrow is Friday!

Collapse
Posted by Andrew Piskorski on
Tom et al., although I can't really disagree with your complaints about the AOLserver 4.5 development, communication, etc., I think you may be overreacting a bit. So, they committed some likely ill-thought out code, and then eventually released it before it was ready, all without nearly enough communication and review. Bad, yes, but it does happen, and shouldn't be an insurmountable problem - just an obnoxious one.

Also, it's not like us other folks in the AOLserver community jumped all over the problem first thing! E.g., I'm still using 4.0.10, haven't touched it in ages, and have barely looked at the 4.5 code, even though it was released back in June 2006, more than a year ago!

Btw, Tom, thanks for discovering and really digging into these 4.5 problems.

I also wonder how much of the premature 4.5 release was driven or abetted by internal turmoil at AOL, which the AOL guys never talk about publically on the AOLserver list. It sounds to me as if most of the AOLserver developers - including Jim Davidson - are now employed at other companies doing other things, and the few that remain at AOL are probably no longer working directly on AOLserver related projects at all. I think an AOL employee alluded to this recently on the list by saying something like, "If it wasn't for the [open source] community, AOLserver would have been dead two years ago."

Also, Tom, re. your "I'm sure there is nothing we can do about it." comment - that seems too negative to me! There is certainly something we could do about it - we could "simply" invest the time to figure out and commit changes to the code! (We could; I'm not saying we should have to, and the wasted effort in straightening out someone else's broken new code when the old was perfectly good is always annoying.) There are many non-AOL folks who have CVS commit access to AOLserver on SourceForege, note, even if hardly any of them ever use it.

Collapse
Posted by Tom Jackson on
Andrew,

Please refer to my first comment in this thread. Until yesterday I pretty much agreed that I was overreacting. But after further discussion and analysis, it is obvious that the best case situation is that the AOL developers are just bad programmers. Worst case is of course that they are just pretending to care about the larger AOLserver community. It is time to stop speculating and demand change which we can verify, or just stop listening to the BS. Why waste another second working on a dead-end? This is the issue. These things waste everyone's time, and if mere ignorance is the reason AOLserver didn't die two years ago, it is little consolation. Do they take everyone for complete idiots? Apparently.

Collapse
Posted by Nima Mazloumi on
Maybe Vienna has tried out naviserver?
Collapse
Posted by Gustaf Neumann on
Nima yes,

i did a port of openacs to naviserver in the past, and i will try again, when a new release is out. In naviserver, there is much progress in various areas, and the active developers are changing the structures where existing ones are not satisfying. This is a good thing, but quite different from the traditional "stability" of aolserver (which has it advantages as well). Fact is, the naviserver deviates more and more from aolserver. I have stopped to catch up with the changes for the time being and wait for the next release.

On the other hand, i would not overweight the relevations about aolserver 4.5 config right now. It is out since a year or so, and many people are using it happily in their environments (true, mostly without virtual servers). This configuration issues can be fixed with moderate effort.

Collapse
Posted by Andrew Piskorski on
Jon, I believe Gustaf was trying out Naviserver with OpenACS last year sometime. If I remember right, his conclustion at the time was that it looked promising, but the code was changing too fast for him to want to try and keep up with it, easier to wait and re-evaluate once Zoran, Stephen, and Vlad slowed down in their changes. Perhaps that's happened now, I don't know - I'm sure the three Naviserver guys would be happy to fill you in, though.
Collapse
Posted by Jon Griffin on
Andrew,
You have been around long enough to remember the other times this BS happened. It is possibly worse now.

I am not even sure anyone is working on AOLServer at all anymore. The attitude of the developers sucks, code is getting worse. Sound familiar.


There was a lot of discussion internally about whether or not to support virtual servers going forward.

----------------
NaviServer has been sitting at 4.99.1 on sourceforge for many months, but I admit I don't know enough about current plans to make a decision.

I will try to contact Vlad and see what the plans are. At least it is an open development environment with a plan going forward. With OACS support, it could be even better and it is not like they have some fly by night developers that got mad and forked.

Collapse
Posted by Vlad Seryakov on
The reason it is being in 4.99.1 for too long time is just we wanted to include all documentation and then real life took over and we've been discussing release for the last couple months already, there are still some things to be complete but overall it is very stable. I use it daily in multiple installations and it is being used in many production sites around the world. Ask Zoran, he uses it for their product for a long time already. Tcl API does not change, the one of the major goals is to keep backward compatibility, may be not with everything but at least on Tcl level.

As for going in different direction than AS, that is true, we ported some stuff from AS 4.5 like new adp parser, pools, Tcl initialization but adjusting to our vision. Overall i think it is much better server than AS 4./4.5, it has support for many many features, a lot of modules and integrated many commands so no need for extra modules like nscache, nsha1, upload statistics for huge uploads, writers support for delivering huge files without using conn threads, embedded virtual hosting, no need for config or server restart to add new virtual hosts.

Also, docs are mostly up-to-date, for quick overview see:

http://www.crystalballinc.com/vlad/software/naviserver//index.oss?&tab=docs

Sorry for advertising, i did not post here for a long time.

Collapse
Posted by Tom Jackson on
Vlad,

Thanks for posting. My concerns were not intended to cause a switch to NaviServer, I don't know anything about it. I am interested in the seriousness of the developers, and AOLserver stock just went way down for me. Is it possible for you to describe here what is NaviServer? I will follow your link, but what is the diff with AOLserver and why did OACS need to be ported to NaviServer? Sorry, but porting sounds like a major process.

Collapse
Posted by Vlad Seryakov on
No porting is needed, i am using apps there i was running on AS 3.5, then 4.0. Some API changed but mostly old Tcl API is the same, we just have many new commands.

Naviserver is fork of AS 4.0.11 but this year we ported some stuff from AS 4.5, mostly pools, tasks, new ns_http command but NS has new ns_https/lightweight nsssl module.

As for seriousness, Zoran relies its business on it, i am using it as swiss knife and actually at my new company we are using it for backend/server parts, i am not sure how Stephen relies on it. And, this is opensource project, we do not have many commiters but this is because of lack of popularity, we never intended to fight for aolserver developers or lure them from AS.

Collapse
Posted by Gustaf Neumann on
Well,

concerning "no porting is needed": well, naviserver 4.99.1 did not work out of the box with openacs due to interface changes between aolserver and naviserver. See for a summary, what was necessary for getting naviserver 4.99.1 to work with openacs:
https://openacs.org/forums/message-view?message_id=351115
see for example these two changes.
http://cvs.openacs.org/cvs/openacs-4/packages/acs-tcl/tcl/request-processor-procs.tcl?r1=1.82.2.3&r2=1.82.2.4
http://cvs.openacs.org/cvs/openacs-4/packages/acs-tcl/tcl/utilities-procs.tcl?r1=1.83.2.4&r2=1.83.2.5
Also openacs-4.5 required a few changes in openacs when it came out.

With the various head versions of naviserver, i had in the past problems with the loading semantics, ns_cache and encodings. All of these problems are fixable to work with minimal invasive changes with openacs, but i don't expect the naviserver head version to work out of the box right now.

Concerning seriousness: i have no doubts about seriousness of the naviserver team, naviserver is here to stay.

Vlad, is there a reason that the nice documentation is your site and not on the naviserver sourceforge site?

Collapse
Posted by Vlad Seryakov on
SF CVS and wiki is sooo slowww i could not stand it anymore an d put a copy of files into my personal web site. Plus, i am running naviserver, so using PHP wiki to promote naviserver/Tcl is not the best option in my opinion.

We've decided to minimize maintenance efforts still to use SF and CVS but i am personally do like it but this is in no way some kind of fork or anything pulling from the project, just faster access.

Collapse
Posted by Carl Robert Blesius on
Would just like to share (with permission) a calm constructive post from the slightly more heated parallel AOLServer thread. It is by John Buckman who uses the server for some great heavily trafficked production websites (e.g http://magnatune.com/ and http://www.bookmooch.com/).

&lt;CopyPaste&gt;
Begin forwarded message:

From: John Buckman
Date: August 3, 2007 1:24:03 AM EDT
To: AOLSERVER at LISTSERV dot AOL dot COM
Subject: [AOLSERVER] learning from naviserver
Reply-To: AOLserver Discussion

In no way do I want to be king, and in an effort to calm things down a bit, let me say that I'm really, really happy with aolserver, and the last major release had amazing things in it. My only real nit is that so many cool things are not-so-well-documented, but hell, it's open source. I just spot things in the text files, and then figure it out by reading the C code, which is often commented, but always clean and readable.

On the subject of cool things and not-so-well-documented, I'd like to bring up Naviserver (you can find it on sourceforge, it's an independent fork of aolsever, at http://naviserver.sourceforge.net). Their fork of aolserver has an insane number of changes to it, and lots of great ideas (look at http://naviserver.cvs.sf.net/naviserver/naviserver/ChangeLog?view=markup). There's a handful of developers working on it, and it seems like a real hotbed of innovation.

I've not used naviserver, though I evaluated it seriously, because it has so many differences to aolserver, almost all undocumented, that it was really hard for me to get up to speed to, and I found it less reliable than aolserver, probably because of all those innovations. I kind of like the slower pace of aolserver, I can actually run a production web site on it. This is not meant as an insult to Vlad and the other Naviserver developers, I'm just pointing out how the two development communities differ.

However, I was wondering if we should perhaps look at merging back some of the best things in naviserver, back into aolserver. In fact, maybe we should treat the aolserver/naviserver split like ubuntu treats its two releases, and recognize that naviserver as an innovative, highly chaotic playground, and merging back in the best ideas from it back into aolserver after a long delay (6 months to a year) once each feature has settled down a bit and we can evaluate whether, in hindsight, it really was a good idea and the way it was implemented turned out well.

I'd be game to go through the naviserver changelog in the future, and be part of a discussion of what is in there that we might want to merge back into aolserver.

-john
&lt;/CopyPaste&gt;

Like most of us here who run production websites I am mainly interested in just installing AOLServer or Naviserver and forgetting about it b/c it does what it is supposed to do for a busy site without much tweaking at all (which has been my experience so far). Whichever way we go, I hope we continue to see sensible defaults and minimal configuration requirements in the future.

Collapse
Posted by Eduardo Santos on
Just a little observation about this: shouldn't we consider for real porting OpenACS for Apache? I know it's an old discussion, but assuming all these problems, I guess we should think about it.
Collapse
Posted by Nima Mazloumi on
Good question: but how much would it cost, who could do it, what would it take?
Collapse
Posted by Nima Mazloumi on
Oh. And is it worth it?
Collapse
Posted by Don Baccus on
Sure, go for it. Tell us when you're done.

Do you have any idea how large this task is?

Collapse
Posted by Steve Manning on
I can't see how you could realistically switch to Apache without a lot of grief. OACS relies on the TCL and database functionality that AOLserver provides. NaviServ might be the answer though. No offence intended to the AOL bods but it would probably be easier to sell to some people if the AOL name wasn't there - it does seem to have a certain stigma.

- Steve
Collapse
Posted by Patrick Giagnocavo on
Every consultant/programmer company I am aware of that does OpenACS or .LRN work, does not even mention AOLserver when talking to customers.

They just say, "application server that works well behind an Apache proxy, just like Java servlets do" with the option of running the application server without Apache.

Dumping the AOL name would be a good idea however.

Collapse
Posted by Don Baccus on
Ah, well, looks like I'm about to be banned from the AOLserver mailing list. NaviServ's starting to look attractive after all :)
Collapse
Posted by Carl Robert Blesius on
You are the master of flames Don (and have a sailors tongue to boot).

If we did decide to move to Naviserver at least all the ns_* commands would start to make sense again. After that we could think about rebranding OpenACS. ;)

Collapse
Posted by Titi Ala'ilima on
Yeah, but then how would we explain all the "ad_*" commands?
Collapse
Posted by Don Baccus on
You are the master of flames Don (and have a sailors tongue to boot).
Read on, Carl. Apparently Nathan's swallowed a nice dose of Shut The Fuck Up and Dossy and I are having a constructive discussion.
Collapse
Posted by Tom Jackson on
Somehow I thought that I was the one put on probation, Don is playing Mr. Nice Guy.

Anyway, another update about the state of the code and a possible solution.

Although the ns_pool C code is messed up and creates server wide threadpools (crossing virtual server boundaries), this is easily fixed. A few internal C APIs need changing along with a struct for the Pool, which needs an additional pointer to the virtual server. I'm not a C guru, but with a little direction I think I could get the configuration from 4.0.10 to work in 4.5. The Tcl API probably doesn't need change to existing commands, although a few query subcommands would be helpful.

Collapse
Posted by Don Baccus on
Somehow I thought that I was the one put on probation, Don is playing Mr. Nice Guy.
You know, I almost posted to the list commenting on how I was being Good Cop, you were being Bad Cop, so why I was being banned?

Oh, yes, I "swore" by suggesting Dossy say "I fucked up", which as it turned out didn't offend Dossy at all ...

Harumph.