Forum OpenACS Q&A: nsjava staus

Request notifications

Collapse
Posted by Dan Wickstrom on
Don has requested a status update on nsjava, and I'm posting it in a separate thread, so I don't clutter up the dotLRN thread with an off-topic discussion.

Background

nsjava was originally developed as part of the opeancs 3.2.x porting effort, and it was quite simple in that it only supported the calling of Java Static methods from Tcl. The Aolserver DB api was also exposed to user-defined java classes through a set of JNI classses. This worked well enough for its intended purpose of porting the Webmail package which made extensive use of Java inside of Oracle.

Extensive testing with various jdk's revealed the flakiness to which Don and Michael have alluded. I had stablity problems with most vendors except for Blackdown's 1.1.8 on Redhat and Sun's 1.1.8 jdk on Solaris. In addition, it was nearly impossible to get nsjava to build on any other linux distro other than Redhat. This seemed to be due to pthread library compatiblity problems between the pre-build jdk distro's and the various pthread library versions distrubuted with various linux vendors OS's.

After the openacs 3.2.x porting effort was complete, I spent some time integrating TclBlend with nsjava, and I released this as an Aolserver module called nsjavablend. This module provided all of the features of TclBlend within Aolserver, but it suffered from several short-comings of the TclBlend design. TclBlend was designed as single-threaded app, so there were several design conflicts when it came to converting it to a multi-threaded app. Extensive testing of nsjavablend revealed several memory leaks, which were quite difficult to track down. It seemed that the reference counting of java objects wasn't completely correct.

In addition, cleanup of the Tcl interpreter would trigger a call to the JVM after thread-detach which would crash the server. I found a work-around for this problem, but it involved fooling the TLS cleanup routine, so that thread detach wasn't done until after the interpreter had been destroyed.

Once the Openacs-4 porting effort started, I found little time to work on or support nsjava, so I put it aside and I didn't bother to much with trying to develop it further.

Once the Openacs-4 porting effort was completed , I started thinking about how to fix nsjava(blend). Since I couldn't find a simple way to track the reference counts of java objects in TclBlend, I started from scratch and reimplemented the TclBlend core functions using nsjava as a starting point. This worked well and to save development time I re-used TclBlend code whenver I could. The end result is that I have reimplemented a sub-set of TclBlend's functionality in nsjava, and I no longer have the reference counting and thread-detach problems that I had when I integrated TclBlend with nsjava.

Current State

nsjava is currently at version 1.0.4, and it supports the following sub-set of TclBlend's commands:

nsjava::new : Allocate Java objects.
nsjava::call : Invoke static Java methods.
nsjava::field : Get and set Java field values.
nsjava::instanceof : Determine if an object is an instance of a Class.
nsjava::info : Introspect Java objects and classes.
nsjava::null : Get a handle to the null Java reference.
nsjava::isnull : Test a handle to see if refers to null.
nsjava::lock : Lock a Java object so that it can not be garbage collected.
nsjava::unlock : Unlock a Java object that had been locked previously.
nsjava::throw : Throw a Java exception.
nsjava::try : Catch a Java exception.
nsjava::cast : Cast a Java object from one type to another.
java::import : Import a Java class name so that fully qualified name need not be used.
javaObj : Invoke instance methods on an object.
javaArrayObj : Invoke methods on an array object.

In addition, I have ported the TclBlend test harness to work with nsjava, and I have ported almost 600 individual tests which exercise almost all aspects of the nsjava implementation.

Limitations

The nsjava module is currently limited to one tcl interpreter per aolserver thread. Tclblend users are used to creating additional interpreters, but doing so in nsjava/aolserver will cause unpredictable results, and it will probably crash the server. Using multiple Tcl interpreters within a single thread is not a common practice in Aolserver, so I don't see this as a big limitation.

db queries in user-defined java methods are not yet supported in the startup thread. This is due to Aolserver blocking signals in the startup thread, which makes it necessary to run the JVM in a separate thread. I have started on a work-around to mitigate this problem, and it will be available in the next release. Once this change is made, nsjava will pretty much be feature complete. nsjava does not affect the normal operation of Aolserver, so it is still possible to perform db queries from Tcl in the startup thread.

TODO

  • Add db support for java methods in startup thread (in-progress)
  • Compatiblity testing - ensure that nsjava builds and runs on same platforms that are supported by Aolserver. (currently builds and runs on various jdks for Redhat Linux and Solaris). I need access to other platforms to complete this or volunteers for testing would also help.
  • Load testing
  • Extend test harness for more comprehensive testing coverage. (Done)
  • Add support for passing binary data to/from java via Tcl
  • Add db support for oracle
  • Implement jdbc interface on top of aolserver api? Not sure if this is important. Might be important to people coming from the java world.
  • All-in-all, I'm pretty happy with the progress I've made so far. I find that it is quite easy to write nsjava scripts in Tcl, and it makes it quite easy to integrate available java libraries with Aolserver. The discussions on dotLRN have gotten me thinking on how to take advantage of all of the Apache-java work that has already been completed, where they have already implemnted most of the web-services componenets that we need for extending dotLRN. Things like XML-RPC, SOAP, WSDL are already available for Apache, and I'm guessing that it wouldn't be too difficult to make them work on Aolserver/nsjava.

    More information can be found at the nsjava project page.

Collapse
2: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Things like XML-RPC, SOAP, WSDL are already available for Apache, and I'm guessing that it wouldn't be too difficult to make them work on Aolserver/nsjava.
Yeah that's exactly right ... thanks for the update, Dan.

I'm *so* glad you're on the TAB!

Collapse
3: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
did you fix the problem with tcl's exec?
Collapse
4: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Actually, I've forgotten exactly what the problem was.  Could you refresh my memory or point me to a thread where it was discussed?
Collapse
5: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
IIRC, tcl uses sigchld to watch for exec exit. this interrupt is also used by some (all?) jvms. (I was using Sun's on linux x86.)

When I did my update to nsjava 0.0.10 I added an exec replacement based on jacl's. (Modified to return a String and otherwise simplified. Worked for me.)

You might also want to have a look at the arg parser I rewrote -- the code is much simpler and allows arbitrary JVM options to be passed in. IMO this is more flexible than hardcoding e.g. how "classpath" is specified, which is a good example because it changed from 1.1 to 1.2.

Collapse
6: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
I'll take a look at your exec replacement.

As far passing in arbitrary options, nsjava already supports this.  Classpath is hardcoded, but it is also possible to set aribitrary java options using the ns_param JavaOption keyword.  You would do something like:

ns_param  JavaOption foo=bar

Collapse
7: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
hmm...  I don't remember that letting you specify the -X options, for instance...
Collapse
8: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Hmmm... I'm not sure why you would want to print help on non-standard options from nsjava, but yes, you're right the JavaOption feature only allows you to specify system properties (it prepends -D) to each passed in option. I can fix this, by removing the -D prepend and passing the options in directly. You would then end up with something like:

ns_param JavaOption "-Dcompiler=none"
ns_param JavaOption "-X"

Collapse
9: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
the main reason wasn't to print help on those options but to use them :P

e.g. logging verbosity was a group of -X options in the jvm I was using at the time.

Collapse
10: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Okay, that makes sense - yet another thing that is non-stadard between jdks.
Collapse
11: Response to nsjava staus (response to 1)
Posted by Carl Robert Blesius on
Dan,

would we be able to access the Common Service APIs that are being
developed for the Open Knowledge Initiative using nsjava?
http://web.mit.edu/oki/specs/index.html

Collapse
12: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
OKI doesn't seem to be close to being complete, but as far as I can tell, it should't be a problem to access the common service apis.  One thing I noticed is that they're assuming a JDBC interface for DB connectivity (no surprise), so it wouldn't be possible to use aolserver's efficient db interface.  This shouldn't be a problem, as the DB could still be accessed using JDBC through a TCP interface - not quite as efficient, but still a workable solution.
Collapse
13: Response to nsjava staus (response to 1)
Posted by John Norman on
Simply implementing the OKI authentication API would be valuable to us.
Collapse
14: Response to nsjava staus (response to 1)
Posted by Alfred Essa on
Dan, Thanks so much for your work on this. Implementing OKI authentication for dotLRN sooner rather than later would be a big win for us and for OKI. I will ask Andrew Grumet to start looking into this right away and then take his findings to the dotLRN Technical Advisory Board.
Collapse
15: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
Dan, this is fantastic.

In addition to SOAP and OKI, we might want to start looking for WebDAV libraries that could be interfaced with file-storage. AFAIK, a WebDAV interface would enable folks who use KDE, Mac OS X, and Win2K/XP to mount file-storage directories on their desktops as if they were folders on a local drive.

Jakarta Slide has WebDAV built in but I have no idea how hard it would be to adapt their code. The same holds true for the W3C's Jigsaw server. You can get a list of all Open Source WebDAV implementations at http://webdav.org/projects/#opensource.

Collapse
16: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
That's an interesting idea.  In addition, webdav could probably be used to access stuff in the CR, since the CR uses a file-system metaphor.

With just a quick glance, it looks like the jigsaw sever could be integrated with nsjava, but it would have to listen on a separate port.  Even so, there could still be a tie in with openacs since db access is possible from nsjava user-defined java classes.

One thing I'm noticing now that I've started considering various applications to use with nsjava, is that it would be dead-simple to add new features/libraries, if nsjava had servlet support...

Collapse
17: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
Well, we should only resort to using Java if a Tcl alternative is not available or achievable, IMHO.

SOAP is already available for Tcl. There's no reason to use nsjava for SOAP in OpenACS unless you already have some Java app that you need to talk to or something of that sort.

WebDAV seems like is very close with the VFS extension for Tcl 8.4 (see http://mini.net/tcl/2466 and http://sourceforge.net/projects/tclvfs/). You already *can* mount WebDAV shares with Tcl VFS (see http://mini.net/tcl/3203 for an example of using Tcl VFS to mount Apple iDisk shares, which are webDAV shares).

While nsjava is exciting and great, we should focus on our core technologies and use Java and something available through nsjava when Tcl is not there yet (or for compatibility/interoperabilty) and even so, we should make efforts to help with Tcl equivalents.

Collapse
18: Response to nsjava staus (response to 1)
Posted by Dave Bauer on
I think Roberto is right. TclSOAP works fine in AOLserver once you work around the package require problem. It supports XMLRPC and SOAP with the same API, so its very nice to export web services.

So we should focus on finding and using as much existing tcl code as we can before we look for Java solutions. At least for code that will become part of OpenACS.

If a client needs something and nsjava makes it possible to deliver on-time, use it.

Collapse
19: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Well ... does someone want to whip up some TclSOAP examples, then?  All we're interested in is solutions and if nsjava delivers them, and we can't get them any other way, we're probably going to want to take this option.

Though I don't want to require nsjava for a standard OpenACS install, no way!  Add-on packages are a different story, though.

Collapse
20: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
I tried integrating webdav into aolserver/nsjava and it basically works. To do it, I copied the aolserver directories into the jigsaw install directory, and edited the classpath to include the jigsaw jar files. I then started aolserver and opened an nscp window and started jigsaw as follows:


1021 nsadmin@localhost:~/webdav >telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
login: nsadmin
Password: *****
 
Welcome to nsjava running at /home/nsadmin/jigsaw/Jigsaw/bin/nsd (pid 8849)
AOLserver/3.2+ad12 (aolserver3_2_ad12) for linux built on Jan 30 2001 at 20:36:35
CVS Tag: $Name: aolserver3_2 $
nsjava:nscp 1>  set args [nsjava::new {String[]} 2 {-root /home/nsadmin/jigsaw/Jigsaw/Jigsaw}]
nsjava0x1e5
nsjava:nscp 2> nsjava::call org.w3c.jigsaw.Main {main String[]} $args
 
nsjava:nscp 3>         

Which resulted in the following output in the log file:

[20/Aug/2002:23:40:22][8849.8201][-nssock-] Notice: nssock: listening on http://localhost:8000 (127.0.0.1:8000)
[20/Aug/2002:23:40:22][8849.8201][-nssock-] Notice: nssock: accepting connections
[20/Aug/2002:23:40:25][8849.9226][-nscp:1-] Notice: nscp: connect: 127.0.0.1
[20/Aug/2002:23:40:28][8849.9226][-nscp:1-] Notice: nscp: logged in: 'nsadmin'
[20/Aug/2002:23:40:28][8849.9226][-nscp:1-] Notice: registerTclJavaFunctions: registering commands
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: attach to JVM
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: attach to JVM worked
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: save env 140526272 tid = 9226
loading properties from: /home/nsadmin/jigsaw/Jigsaw/Jigsaw/config/server.props
Jigsaw[2.1.2]: serving at http://localhost.localdomain:8001/
*** Warning : JigAdmin[2.1.2]: no logger specified, not logging.
JigAdmin[2.1.2]: serving at http://localhost.localdomain:8009/                                                       

So at this point I basically have a webserver running inside a webserver. I'm able to get pages from aolserver using port 8000, and I'm able to get pages from 8001 using jigsaw.

I then proceeeded to download WebDAV client called DAV Explorer, and voila, I was able to browse files under the jigsaw pageroot. This would need some additional work to make it useful, but using this approach, your probably 95% way to having webdav support in aolserver without really even doing anything.

Collapse
21: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
Wow Dan this is great.

Atleast there is a way to reuse a lot of work that was done in Java.  Since ns_java is a container for a JVM right?  In theory it would be possible to run a servlet container like resin or tomcat.  Is that correct?

Collapse
22: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Yup... we've used an arrangement with a tomcat engine in it.....
Collapse
23: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
So, all of a sudden, we basically have WebDAV, SOAP, and OKI about 80-90% ready to integrate with OpenACS/dotLRN.

Damn.

Regarding the idea of using WebDAV as an interface to the CR, this could be part of the solution to the CMS UI problem. In an ideal world, content creators and editors would just mount the relevant folders on their desktops, work with the files as normal, check them in via WebDAV, and they're done.

If you think about it, this would be a pretty cool interface for CVS too. I think several different Open Source projects are working on just this idea.

Collapse
24: Response to nsjava staus (response to 1)
Posted by Jon Griffin on
Does webdav support encrypted connections or only plain text ftp?
Collapse
25: Response to nsjava staus (response to 1)
Posted by Carl Robert Blesius on
We have a Zope server that talks WebDAV over https (the Zope server is behind Apache with mod_proxy and mod_ssl)
Collapse
26: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
WebDAV is an http-based protocol, so I'm guessing that it should be easy to do encryption. You can find the standard specifications here:

http://webdav.org/specs/

Collapse
27: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Yes, it seems pretty easy to reuse java development using this approach.  My thinking is that pretty much any java application could be integrated into aolserver without too much trouble.  To make it really useful, it would be desirable for nsjava to have a servlet interface similar to what is implemented in nstomcat.  That way aolserver and tomcat could share the same port, and the integration would be tighter instead of the current approach of running two web servers side-by-side on different ports.  One caveat though, is that nsjava has never, to my knowledge, been used in a production environment.  I'm glad to support it, but my time is limited, so I wouldn't currently bet the farm on this approach until nsjava has been proved out further.

For my day job, I use Clearcase for version control and it works as you describe.  On Unix, I create a view definition for the particular branch that I want to work on, and from then on it looks like a regular shell interface.  So I can use bash to see my files in version control, and I can edit them using any text editor - very nice.  On windows, the clearcase views just show up as a regular folder.  The views are shared between both environments, so I have a common view regardless of where I'm working.  Clearcase is expensive and a bear to adminstrate, but well worth it for large projects.  I think using webdav to create a clearcase type interface to the CR would be awsome.

Collapse
28: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
Subversion uses webdav to access it's filesystem.

talli

Collapse
29: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
There's an Open Source project called Subversion which aims to replace CVS and which uses WebDAV: http://subversion.tigris.org/

It would be great if somebody could give it a try and report back on whether it's worth thinking about.

Collapse
30: Response to nsjava staus (response to 1)
Posted by Jon Griffin on
My biggest problem with subversion is its absoulte dependence on Apache. I admit I didn't look any farther when I saw
 Apache as network server
, so maybe it will work with another server. If not it is one more administrative headache to deal with.
Collapse
31: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Roberto, I think you're confusing the issue here by saying that vfs could provide webdav support for aolserver. I could be wrong, but a brief skimming of the vfs examples indicates that vfs can act as a webdav client, but it can't act as a webdav server, which is what we really want.

Here are some of the examples I found by following the links that you provided:



    vfs::urltype::Mount ftp
    file copy ftp://foo.bar.com/pub/Readme .
    file copy ftp://user:password@foo.bar.com/private.txt .

or 

    vfs::urltype::Mount ftp
    set image [image create photo -file ftp://ftp.ucsd.edu/pub/alpha/tcl/alphatk/alphatk.gif]
    pack [label .b -image $image]

or 

    vfs::zip::Mount foo.zip foo.zip
    cd foo.zip ; glob *

or 

    vfs::urltype::Mount ftp
    set listing [glob -dir ftp://ftp.scriptics.com/pub *]

or 

    vfs::urltype::Mount http
    set fd [open http://sourceforge.net/projects/tcl]
    set contents [read $fd] ; close $fd

If you evidence to the contrary, that vfs can act as a webdav server, then I would be interested in seeing it.

Even so, vfs would be nice if we had webdav support in aolserver, because you could then remotely mount the CR on your local file system and directly edit content using your favorite text editor instead of some crappy web interface.

Collapse
32: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on

Dan wrote:

Even so, vfs would be nice if we had webdav support in aolserver, because you could then remotely mount the CR on your local file system and directly edit content using your favorite text editor instead of some crappy web interface.

Exactly! This would help solve a big part of the CMS problem. I have long believed that a good Open Source CMS solution should use a combination of WebDAV and a lightly customized version of Mozilla Composer on the client side for all content creation and editing, leaving version control, workflow, publishing, and (to a certain degree) templating on the server side. This would greatly simplify the server-side UI design requirements while simultaneously enabling users to draw on the full range of desktop software functionality that they're used to having when they create documents. As a bonus, if users prefer a closed source client-side authoring app (e.g., Dreamweaver), they could do that too and still use WebDAV for seamless integration with the CMS.

Collapse
33: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
Dan,

I wasn't trying to say that VFS had a complete WebDAV solution for Tcl. I tried to point out that part of it (client) is there, and that with some effort, the rest could be made to be there.

I'm sure Java wasn't born with a WebDAV server built for it.

I just think that if doable (and a webdav server in Tcl is doable) we should try to focus on Tcl, at the risk of undermining confidence in our own technologies.

I'm not saying that we should be totally adamant about Tcl, but we should make a good effort to use it.

Collapse
34: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
I don't understand your point. Saying that vfs only has client support for webdav doesn't mean you're half way there. It means you haven't started yet when it comes to providing webdav support for Aolserver. The client support is of little use and means nothing when it comes to integrating webdav in aolserver. There are plenty of webdav clients around, so vfs contributes nothing new other than a way to access a webdav server from a tcl script. Sure, somebody could write a nsdav module, but I don't see anybody stepping forward to do it. Are you going to write it? Myself, I'm not so interested in developing nsdav. Sure, I'd like to see somebody do it, as I definitely think that it has a lot of potential. In general, the aolserver development community lacks critical mass for bringing forward new technologies. I hate to say it, but Zope had webdav support before we started porting 3.2.x.

If we can leverage code from other opensource projects then, why not? The point isn't that a webdav server couldn't be developed in TCL, I'm sure it could, but rather that tools like nsjava might provide an easy way to integrate webdav support into aolserver.

I just don't understand your bias against nsjava. After all, it is an TCL extension module. In the examples I've posted, I haven't posted a single line of java code. It's all been TCL. The purpose of nsjava is provide an interface to existing java packages from TCL. nsjava is a tribute to the extensiblity of TCL. If weren't for the tcl object type system introduced in TCL 8.0, nsjava probably wouldn't be feasible. Using nsjava does nothing to undermine confidence in our technologies, as it is implemented using key parts of the technologies that we use in openacs. The fact that webdav support is implemented in java is of little consequence, as the interface to it will be in TCL, and as a developer using it, you wouldn't have to get involved in modifying the java code that implements the webdav server.

Collapse
35: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
From a marketing perspective, nsjava is actually a huge win. And the issue isn't the viability of Tcl so much as it is the long-term viability of AOLServer, IMHO. The AOLServer community is just way too small to keep up with the likes of Apache or even Zope in terms of integrating new technologies. It happens to have started with a huge lead on some issues, so it has managed to stay competitive on things like performance and stability, but the new stuff isn't coming on nearly as fast.

What nsjava does, as Dan points out, is it enables AOLServer to leverage work being done in other Open Source communities. And it does it in a way that lets OpenACS developers keep using Tcl. As a bonus, we get to be able to tell folks who have already made big investments in Java apps that AOLServer can play nice with their existing toys.

Collapse
36: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
The news that we have webDAV support in AOLserver now is brilliant, with or without Java. I agree with you guys that any CMS solution via the browser is ultimately a horrible hack when compared with using a desktop solution. I too have been chasing this dream of accessing the CR with desktop tools via webDAV.

I believe, though, that a C module, namely moififying mod_dav, is far more desireable than any of these approaches. The reason being that it seems mod_dav is more supported than any of the tools that have so far been mentioned and many other tools and client will probably be developed to use its features. An example is Subversion which depends heavily on mod_dav and mod_dav_svn. Unfortunately, a link that purportedly explains this is busted, but visit subversion.tigris.org and there should be some info somewhere.

nsjava, though, is a major win, and I would like to thank Dan for taking the technical challenge and delivering. It's great to see you more active Dan!

talli

Collapse
37: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
I have zero experience in ns_java except compiling it a long time ago.  I was supposed to post something on this thread about keeping an open mind to other technologies the other day.  Although I avoided it.

Of course I don't want that we increase our dependency with other techs.  But I am here primarily because OpenACS stack of technology works.  Not because its tcl, aolserver, etc.  Although currently this are the technology that work.

But I think if Dan's ns_java enables a developer like me to reuse or integrate other open source java application then its good.  Even though there is a tcl equivalent.  You have to admit most of jakarta's and apache-xml project have matured.  Why not just make use of xerces or xalan?

I think ns_java has a great potential for us to make use of the big investment the java community has made.  Tcl, java, aolserver, tomcat, etc. whatever... its good as long as it works.  Of course OSS is prefered.

Collapse
38: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on

Talli wrote:

I believe, though, that a C module, namely moififying mod_dav, is far more desireable than any of these approaches. The reason being that it seems mod_dav is more supported than any of the tools that have so far been mentioned and many other tools and client will probably be developed to use its features. An example is Subversion which depends heavily on mod_dav and mod_dav_svn. Unfortunately, a link that purportedly explains this is busted, but visit subversion.tigris.org and there should be some info somewhere.

This brings up an idea that Lars floated in passing on another thread--namely, figuring out a way to be able to tap into Apache mods directly. Is it utterly crazy to contemplate writing nsmod_? Forget about the resourcing for the moment; is it technically feasible/adviseable?

Collapse
39: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Thanks Talli.  It's funny that you should mention mod_dav, as I had a similar thought, since I just downloaded it earlier today and looked at the source code.  I was hoping, as you suggested, that it might be possible to easily convert mod_dav to nsdav.  Unfortunately, apache internals seem to way different than aolserver internals, and I don't think it would be a trivial undertaking to convert the module.  In fact, I think that it might be easier to start from scratch when developing a nsdav module.
Collapse
40: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
Dan, were you looking at the Apache1.x or the Apache2 DAV stuff? I would imagine that the 1.x stuff would be a nasty hack, but I've heard that the 2.0.x version may have some good potential.

talli

Collapse
41: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Well I started here which is a link for version 2 stuff, but when I follow the links, I end up here and the page states that the current version is 1.0.3, and that it has been tested against apache 1.3.x. Are you sure there is a version of mod_dav for apache 2.0?
Collapse
42: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
Dan, I think I might be misleading you a little. Apache2 has the DAV stuff built in, so they don't really have mod_dav for 2.0.x. It may have to be ripped out of the core code. That being said, it may be a lot cleaner than the mod_dav stuff. Maybe.

talli

Collapse
43: Response to nsjava staus (response to 1)
Posted by Todd Gillespie on
Just a few quick notes:

Apache2 ships DAV as a core module.  Get the Apache2 tarball and look in modules/dav.  I believe it is compiled in by default.  The Apache2 crew is very bullish on DAV.

The Apache2 internals look very different from AOLserver, but they are not that different in practice.  Much of the difference is cosmetic, and much more is simply things missing - for instance, there is no (yet) DB connection pooling API in Apache2.  There are some scary things in terms of data structures (Apache2 uses some pool & table structs I haven't quite approved of yet), allocation methods (Apache2's approach to controlling memory leaks is a pool allocation scheme, so when the webserver wants to clean up a thread, it just nukes the pool; reminiscent of process clean-up), and there are some options not available in AOLserver, mostly more places in the request cycle to funge with things, and the ability to filter inputs & outputs from each and every module.  Despite all that, I do not think it wise to re-implement everything.  It is not, in any sense, a representative sample, but when I moved mod_scheme to ns_scheme, I think I changed 10 lines of code.  I should however note that 1) I wrote the original code, and 2) I wrote it to be portable; but it *is* possible.

Certain parties have been goading me into working on an ns_dav built from mod_dav components.  I'll add more later today when I have the time.

Collapse
44: Response to nsjava staus (response to 1)
Posted by Don Baccus on
I'm curious about what others think about a couple of possible issues.

Java isn't "true" Open Source (though programs written in Java frequently are).  We've done a lot of work to make sure that we provide a 100% Open Source OpenACS 4 solution.  Of course, we also provide an alternative based on proprietary Oracle.  My main concern isn't to quibble over whether or not "free as in beer" is better or worse than "free as in speech".  My concern's more practical - can third parties like us bundle up a JVM in a "download an instakit" package, or are potential users going to have to download a compatible JVM themselves?

Because that kind of packaging - download a big chunk 'o stuff from openacs.org, wave a magic wand, and poof! you have a (dotLRN/dotWRK/dotWHATHAVEYOU) up and running - is something we've talked about pushing for vertical apps the community develops.

I don't know the answer to this, which (of course) is why I'm asking.

Then there's also the issue of competion for computing resources on the webserver machine itself.  Heavy use of Java pieces means yet another consumer of resource potentially on the scale of AOLserver and  the RDBMS.  I've also seen some interesting information about the scalability of one JVM due to lock contention issues (shown me under an NDA) so there's another tuning issue to worry about if you're going to run a large-scale service.

Before anyone says "yet another reason to support Apache" let me point out that the same issue exists if you couple a JVM to Apache.

This not only leads to potentially more complicated sysadmin requirements but leaves us open to statements like "if we just use a 100% Java solution life will be much easier".

Having said all this ... the fact is that the idea of being able to rapidly integrate with WebDAV and the like is *extremely* attractive.  I'm not knocking the attractiveness but ... let's think about the possible headaches involved with being too quick to decide that integration with Java services is the right long-term solution.

I don't think there's any doubt that in the short-term this is going to be the right solution for some of our integration problems, though.

If I had to vote today I'd probably vote for a quick push to get TclSOAP integrated (DaveB and Roberto seem very close and I gave them some hints on how to work around the "package requires" issue that might be useful), because it avoids all of the potential hassles that being dependent on having a compatible JVM around might lead to.

On the other hand, Dan's right that having a WebDAV client doesn't bring one close to having a WebDAV server and nsjava seems like the obvious short-term solution to that.  I'm glad he's looking at mod_dav (is the Zope WebDAV support Open Source and not Java-based?  Is this possibly another place to poach code?) because the long-term appeal of this should be obvious, too.

There's no doubt that a lot of Java stuff is out there and there will be more and more as each day goes past.  Being able to rapidly integrate is a big plus despite the potential hassles.

Collapse
45: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Todd and I were posting at the same time. After reading his note ... all together, now:

Goad! Goad! Goad!

Collapse
46: Response to nsjava staus (response to 1)
Posted by Todd Gillespie on
Put that down.  You could someone's eye out with that thing.
Collapse
47: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
I don't know if it's even allowed, but I can't imagine trying to make an  "instakit" verticle app that includes a jdk.  There are just too many variables with different operating systems and vendors, and different jdk vendors and versions.  In such a scenario, potential users would have to download a compatible jdk, which throws a wrench in the concept of an "instakit".

One thing to consider is implementing such services through a service contract, so that if a native aolserver version of something like webdav does arrive later, then it is easy to make the switch.  We do that now with search services and payment gateways, and I can't imagine that it would be too difficult to do the same with Soap services and possibly webdav.

With regards to the lock contention issue, was that with respect to jvm internal locks or was that due to user-defined locks in application code?

Collapse
48: Response to nsjava staus (response to 1)
Posted by Don Baccus on
OK, so you've confirmed some of my fears regarding packaging :(

As you say, this is a natural fit for service contract based service packages.  Definitely the thing to do.

Collapse
Posted by John Sequeira on
I know this a heretical idea, but I was wondering what people would think about writing the dependency on AOLServer as a web server out of OpenACS, and supporting a pure-TCL FastCGI interface or some type of service contract(?) that would support both.

I asked Michael Cleverly about the feasibility of using his nstcl API for this, and he said the only problem he could see would be blocking on long running queries. I don't think that's too big a deal, because AFAIK you can run a collection of TCL/FastCGI interpreters and load balance between them.

Talli and I just discussed this on IM... Here's the link on my blog for people interested in (perhaps tolerant of?) a more impassioned version of this write-up.

My direct motivation is that I would like to get the MSSQL data model port I did into production. In considering my options, I think that AOLServer particulars are the biggest barrier to that being an easy proposition. Also, before Don begins goading, I'm willing to take a stab at this and see if/what I can get to run under tcl/FCGI, but I would certainly be interested in hearing that I might not be the only one interested in using it when it reaches a useful stage. Also, since I'm not nearly as up on the AOLServer dependencies, is there any reason why this wouldn't work or would be very difficult?

I should also say that the work Dan's done is very cool, and seems highly useful in the near term, esp. for WebDav. I've just rolled out my second webdav-enabled KM systems - clients *love* it. But I'd like to raise this slightly off-topic scenario because longer term I think it addresses a lot more recurring integration/portability issues.

Collapse
50: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
there have been two or three threads on doing a fastcgi interface before, and pretty much everyone's thought it was a great idea.

Petru was working on it at one point, but I don't think he got too far along. I could be wrong, though.

Collapse
51: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Jonathon,

Thanks for bringing that up.  Petru was talking about implementing a fastcgi server interface in AOLServer.  This would require absolutely no rewriting of AOLServer,  but would still place a heavy reliance on dedicated folks like Jamie (who's porting AOLServer 4.x to Win32) to keep it truly portable.

I'm talking about rewriting openacs to separate the web listener/server pieces from the rest of the code base.  In other words,  implement openacs in a pure TCL version,  on top of the existing TCL/FastCGI server implementation.

(see http://www.nyx.net/~tpoindex/tcl.html#Fcgi )

This means extreme portability-- the ability to run anywhere using one of several native FastCGI implementations,  or worst case,  using a CGI->FastCGI bridge if necessary.

It's radical,  but I think given Michael's work referenced above,  potentially realistic.

Collapse
52: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
I'm so far out of my element here it isn't funny, but if Michael's code truly implements all the necessary AOLServer functions, would it be possible to test this out on mod_tcl for Apache 2 (http://tcl.apache.org/mod_tcl/mod_tcl.html) pretty much right now?
Collapse
53: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
Wow John!

nstcl looks very interesting.  By looking at the documentation it seem to implement a lot of the useful api.  It would be great to make use of OpenACS aside from aolserver.

If OpenACS is written against tcl and not aolserver it might be better. Or maybe aolserver specific stuff that is not yet done in nstcl can be more isolated.

Has anybody used nstcl?  Not on openacs but just plain aolserver like coding.

Collapse
54: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Once again I'd just like to throw in my tuppence worth (late again ;)...

The issue of including JAVA in OpenACS to some degree or other is fairly close to my heart ;0)

I personally think, despite the few concerns about 'true' open source, that having a strong Java integration/inclusion element is critical, and gets more so every day.

Don and I were chatting the other day, and I recall highlighting that we as a company (OM) have delivered a number of successful ACS based projects, however *none* of them have been actua websites, in their most common form!

In actual fact it turns out that the OpenACS makes a very good general purpose infrastructure/integration platform.

That being the case, I'm sure you can see that access to Java is pretty critical. Many companies are employing Java more and more as their system development language of choice, and therefore unless we have a good story about Java (even if our product isn't) the better placed we are...

So I think Java, as part of OACS, is a must to some degree.....

As for re-development of Java stuff into TCL.... well, its a nice concept, but in reality by doing so we then lose the benefit of ongoing java projects, supported and developed by others.

So in actual fact I'd like to see us resist converting things to TCL for their own sake and leave it as a 'only when its *really* justified'....

Now, if I could just persuade someone to port AOLServer onto Jacal (TCL implemented in Java).... well... we'd really have *the* killer application. A native Java based OpenACS with TCL still remaining as the primary web devlopment tool.... a lethal combination that would surely offer us access to a much wider user base...

Well, I know its a lot of effort, so its just pipedreaming at the moment. This would probably have to something privately funded.... but keep an eye on this space.... I have ambitions to see that happen...

As many of you will know, Java and TCL came out of the same company (SUN) and to an extent the same group of people, and the intention was always that TCL and Java would be complementary tools....

Combining both under AOLServer would surely realise a worthy goal..?

Collapse
55: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Michael F.,

In considering a port to mod_(d)tcl,  you have to confront the same major issues of moving to FastCGI:  separating AOLServer idioms (esp. those not covered by nstcl) from the OpenACS source code.  Once you've removed them and are in the land of pure TCL,  you're not looking at a lot of work to go with mod_tcl.  I'm still getting a feel for what the big crucial AOLServer dependencies are (hence my post),  so while I don't think it'd be a huge amount of work,  I also don't think it falls under the 'right now' timeframe (as much as I'd like).

Simon,  the architectural separation I propose is a very important step when considering the use of a Java/TCL implementation.  The more AOLServer-isms sprinked in your code,  the more work you'd sign up for such a reimplementation.  The decoupling of logic and webserving is much closer to the servlets approach, AFAIK.  Also,  the KM/webdav implementation I referred to was done using Intraspect/JPython.  The combination of scripting and java was pretty slick.

FWIW,  here's a writeup I found on fastCGI/TCL.

http://philip.greenspun.com/wtr/cheap-tcl.html

Collapse
56: Response to nsjava staus (response to 1)
Posted by Don Baccus on
So in actual fact I'd like to see us resist converting things to TCL for their own sake and leave it as a 'only when its *really* justified'....
Unfortunately the packaging issue is extremely real for many people. Making our core technology dependent on java would be a mistake if that can't be solved. SOAP's too important, if WebDAV gets blended into our CMS efforts (which sounds like a good idea) it too will be too important. If we have such a CMS solution we'll want people to be able to download it and fire it up, right out of the box, not dick around with configuring Java pieces. This isn't a matter of "rewriting for the sake of rewriting".

On the other hand offering the capability to blend easily with Java in order to integrate with less critical bits of technology is really great, yes indeed.

As far as programming out AOLserver dependencies and moving to a pure Tcl-based solution ... why do we want to move to a technologically inferior platform? There's no way we're going to match the AOL folks in building an independent platform that delivers high performance.

Running AOLserver natively under windows isn't an impossibility, there's already an OpenACS site in Ireland that runs natively under windows. Jamie plans to bundle it up when AOLserver 4.0 comes out, I believe. If he doesn't have time then surely the time involved in getting this packaged won't be huge.

The problem then is writing a MSSQL driver for AOLserver. Since MSSQL is based on Sybase and since the AOLserver Sybase driver is essentially an ODBC driver, I don't think this should be terribly difficult either.

And porting the toolkit to another RDBMS is not a trivial undertaking. The work I describe above will be a piddling fraction of a total porting effort.

AOLserver is the most solid piece of technology in our platform, we shouldn't be in such a rush to dump it.

Collapse
57: Response to nsjava staus (response to 1)
Posted by Andrew Piskorski on
John, Michael Cleverly nstcl is indeed very cool, but given that it's a pure Tcl emulation layer using soley Tcl lists and the like to implement APIs which, in AOLserver, are implemented in C, I would be surprised if it's performance even approachs the native AOLserver implementation. Not that that's necessasrily a show stopper - after all, if there was sufficient interest, some of nstcl's guts could be replaced with C code, etc.

And as long as we're talking about interoperability and code reuse:

It would be very nice to refactor the real AOLserver connection pooling database and driver code into a library that could be embeded nicely into tclsh as well as AOLserver. Or into other C code for that matter, R or Scsh, for instance. That's something I've thought about, but so far never looked into to try to decide if it's feasible (and I'm not familiar with the AOLserver database driver internals anyway).

For that matter, Aapache 2.0 still doesn't have database connection pooling. Maybe a dual-mode AOLserver + Apache hacker like Todd could give it to them, I dunno.

Collapse
58: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
Looks like my server was bouncing all of yesterday's mail, so I'll reply to a couple things here:

Dan wrote: "I just don't understand your bias against nsjava. After all, it is an TCL extension module."

I don't have any bias against nsjava. I've said it many times that it is great, in several posts here. Maybe I haven't been able to make my intended point come across, perhaps due to a language barrier.

I'll say it again: I think nsjava is great, and it's great to have Java integration, and it's great to be able to give our users that functionality, and I think your work is great, and you are a great hacker. And although I think it's overhyped and misused in some places, I like Java.

What I was trying to say was basically what Don said regarding packaging and integration of important things. Hope this will clear things out.

re: nstcl, it's very nice. I'm suprised to see the surprise of some here who hadn't heard of it yet. I've used it in some instances for stand-alone Tcl scripts.

re: separating OpenACS from AOLserver, I think it's possible, but it'd take resources. I remember a little over a year ago the AOLserver folks wanted to make AOLserver a shared library that could just be emdedded anywhere.

If that happenned, then you could just do whatever with it. Make an Apache module, make a java interface through JNI (?), etc.

re: Jon Sequeira's SQL Server port. I heard you had finished the OpenACS kernel port, which is interesting. Perhaps you would like to write something about what you learned during the data model port (we've talked about starting an OpenACS Systems Journal on the new website).

I read that MS had changed its EULA on the .NET line of products to not allow GPL'd software to be used/developed. I don't know if that applies to the new SQL Server, however.

Collapse
59: Response to nsjava staus (response to 1)
Posted by Andrew Piskorski on
Incidentally, while supporting more databases (e.g, MS SQL Server) for OpenACS is very cool from a gee-whiz technical achievement point of view, in practice, it's hard for me to imagine why anyone would do it for any practical technical (as opposed to marketing) reason.

OpenACS has Oracle, the market and capability leading closed-source database. OpenACS has PostgreSQL, the market and capability leading open-source database. OpenACS already has the two best databases from both worlds, why would anyone want to invest a huge effort to instead use an inferior database?

The only entirely rational technical reason I can imagine is someone with a large committment to a legacy app using a different RDBMS (Sybase or MS SQL Sever, say), and their new OpenACS installation must, for some hypothetical reason, directly run OpenACS in the same database as their legacy app. Which sounds pretty far fetched to me... Particularly since most organizations integrating OpenACS into a large legacy app would probably insist on using an entirely serparate database installation for OpenACS, be it Oracle, Postgres, Sybase, or whatever.

Or perhaps there's the, "Our vast huge company uses only database X, we don't want to have to even think about ever supporting any other database" argument. But supporting a port of OpenACS sounds like much more work than simply learning to use Postgres, or even Oracle for that matter. So is that a valid or a specious argument?

John, you know more about this than me. As long as we're already off-topic, I'm curious, does some big organization porting OpenACS to a new database really make sense from a hard-headed IT management point of view? Or is it solely a potentially critical tool to get in the door and pitch the OpenACS solution to, ah, less than entirely hard-headed IT managers and staff? Or somewhere in between?

Collapse
60: Response to nsjava staus (response to 1)
Posted by carl garland on
Just some quick notes on things that don't look to be in nstcl
  • No nsvs
  • No Registered filters
  • No Scheduledprocedures or startup/shutdown
  • No internal handling of persistent db connections using shared pool in thread safe manner and not sure if any part of api thread safe being that it is a shell extension
  • No comparable nsd config file exists for shell
  • No ns_conn structure
  • What about modules that are required how would you handle ns_cache, etc.
  • While I greatly admire nstcl I don't think people should be too optimistic about how *easy* that approach may be and I'm not sure the end result would be even on a Java level.
Collapse
61: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
Don, two points:
  • I see you and Roberto saying "we need WebDAV in tcl or C" but I don't see anyone saying "sounds like fun, I'd love to do it." It's not a small amount of work, as you're aware.
  • I think you're making a mountain out of a molehill here with the nsjava install/config issue. Anyone who an could install OACS in its current form can install nsjava.
  • (Okay, 3 points.) Lots of projects bundle a jre with their software, including free projects. The JRE license explicitly allows this. (See this page.) It would be a LOT easier to add nsjava to the hypothetical InstallMagic package than to rewrite WebDAV et. al.
Collapse
62: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
"I see you and Roberto saying "we need WebDAV in tcl or C" but I don't see anyone saying "sounds like fun, I'd love to do it." It's not a small amount of work, as you're aware."

I think you are mistaken. I didn't say we _NEED_ WebDAV in Tcl or C, I said "it would be better".

And about someone saying "I'll do it", you're right, nobody has claimed it. As in most open source projects, the huge vast majority are just takers. They take the product, use it, don't volunteer to do a thing, and then come to the mailing lists and demand developers to do things for them.

I wish OpenACS was different and we had lots of people voluntarily stepping up for small tasks, of which there are plenty.

Collapse
63: Response to nsjava staus (response to 1)
Posted by Dan Wickstrom on
Jonathan, nsjava or any other app that embeds a jvm must use a jdk not a jre.
Collapse
64: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Roberto,

I think the lame EULA applied to dev tools.  I used emacs and perl/dbi to do the kernel port,  so I think it's safe :-).
I'd be happy to write something up for oASJ at some point.

Andy,

Database and webservers are peculiar beasts.  Organizations would much prefer to not support many of them.  Most of the serious environments I've coded in have centralized monitoring apparutus (HP OpenView, CA's TNG, NetIQ, etc).  It's a big deal in $$$ and time to add a new set of infrastructure that has to be monitored and patched.  Way more if you're looking at some type of common fault tolerant architecture (w/replication, failover) in the db tier.  Doing that several different ways gets complicated and expensive quickly.

I'm not saying MSSQL is a better database,  it's just better if you've already bought it :-)  Anyone with a big investment in MSSQL behind their web site would prefer new code and data sit side-by-side,  and not have a new set of integration issues to contend with.  There are a lot of sites like this,  I would argue more than any other platform right now ( I think Oracle wins in $$$, not installed instances,  + MS has a free version of SQL Server )

OpenACS is the best thing out there for a stand-alone CMS or community-based corporate site.  But it has the potential to do a lot more,  if it could leverage the hooks developed by the vastly larger TCL community in terms of data access etc.

As Don pointed out,  the db driver and apache stuff *can* be done,  but my point is that it *has* been done,  and it's never really gotten critical community mass to be supported.  We can keep trying,  a new nsd module,  a new AOLServer port, as Don suggests.  These are useful endeavors,  and will probably succeed for awhile,  but ....  It seems just plain unnecessary to continually require people to fire up their c compilers and keep current with their respective projects external dependencies (apache, sql server, iis, Windows + openacs and aolserver)  over and over again.  I think FastCGI gets us out of this rut.

Carl

THANK YOU!  That kind of feedback was what I was looking for.

FastCGI supports filters, and multiple persistent interpreter instances,  each of which can support a persistent connection.  I *think* that minimizes the threading and pooling issues.

ns_conn shared data is my biggest concern,  but I think FastCGI's session affinity might address it.  In essense,  you're getting the same architecture as a web farm (multiple discrete load-balanced instances),  but at the interpreter level instead of the web listener level.  If you have a performance problem,  you just add more interpreters,  potentially on other machines.  This means that you take a hit having in-memory multiple caches,  but since it's the same issues w/a web farm it doesn't seem too bad to me.

I think ns_cache (is that required?), other required modules (?), config files and scheduling stuff would likely remain platform specific.  That doesn't seem like too big a deal.

Collapse
65: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
... missed that, sorry.

2nd try: debian includes the ibm 1.1 jdk, which IIRC works with nsjava.  The 1.3 appears to have more restrictive licensing, a la Sun.

Collapse
66: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
Jonathon,

*I* am saying that we need webdav, and I've been working hard to support and to get people to support its development. As Roberto said, this is an open source community which also means that things just take longer to develop.

However, I do hope that webdav will be coming along very soon, preferrably as some work that is a port of Apache2 DAV.

talli

Collapse
67: Response to nsjava staus (response to 1)
Posted by Don Baccus on
<blockquote><i> 2nd try: debian includes the ibm 1.1 jdk, which IIRC works with nsjava. The 1.3 appears to have more restrictive licensing, a la Sun.</i></blockquote>
We're not just supporting Linux, though.  Much of this thread - John's whole thrust, for instance - centers around improved support for Windows.  Does Win2K come with a jdk installed?
<p>I'm not making a mountain out of a molehill when saying that packaging concerns are important.  There are people in our community who are very strongly wedded to the concept of being able to point to a single spot on our openacs.org (or dotLRN or dotWRK) site and pull down one set of packages that install and start up automatically.
<p>Look what Peter Marklund did with TclWebTest to implement an autoinstall of dotLRN (after you've manually installed the substrate)?
<p>People are looking in this direction because it is important to them.  People want to simplify the installation process, not make it more complex.
<p>Now if a compatible JDK existed on every platform we intend to support, my argument goes away.  And when that becomes true I'll stop making it.
<blockquote><i> As Don pointed out, the db driver and apache stuff *can* be done, but my point is that it *has* been done, and it's never really gotten critical community mass to be supported. We can keep trying, a new nsd module, a new AOLServer port, as Don suggests. These are useful endeavors, and will probably succeed for awhile, but .... It seems just plain unnecessary to continually require people to fire up their c compilers and keep current with their respective projects external dependencies (apache, sql server, iis, Windows + openacs and aolserver) over and over again. I think FastCGI gets us out of this rut. </i></blockquote>
So your argument is that FastCGI will never change and we'll never have to maintain any FastCGI-based code?  Is this really true?
<p>Unless we were to drop AOLserver entirely - replacing our existing platform technology with something inferior in the name of compatibility - supporting two platforms means a lot more work for the community.  Ongoing work.
<p>If AOLserver for windows hasn't gotten a lot of community support, maybe most of the community at the moment doesn't care about windows support?  I posted that the native PG for windows folks want testers and didn't get a single response here that I'm aware of, which rather surprised me.
<p>Why would community support for a FastCGI effort fare better?
<p>Why is dumping AOLserver a good thing?
Collapse
68: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Don,

You're characterizing my suggestion as dumping AOLServer.  That's not really necessary.  Webware, for example, supports a native apache run-mode (mod_webware),  but also does CGI and FastCGI if you're not an Apache disciple or don't want to run it as a module.  OpenACS could have a reference implementation in AOLServer exactly the way it does now,  but by implementing future functionality in a way that minimized native AOLServer API dependencies not covered by MC's nstcl (or not easily ported to nstcl),  and considering back-ports of portable code into the source tree,  you could make life easy for OpenACS/FastCGI porters to keep up.  This is quite similar to the whole DB-API thing,  now that I think about it,  but I'm pretty sure it'd be asking a lot less.  Unlike data access,  the vast majority of scripts do not call into non-portable AOLServer APIs,  right?  (Non portable as in beyond simply getting query params)

Also,  for better or worse,  FastCGI hasn't evolved in about 5 years,  that I know of.  Their external dependencies (apache., perl, tcl) have evolved,  but you have a larger community committed to supporting it,  and they've historically done a decent job of it.  Plus the run-everywhere CGI->FastCGI bridge probably hasn't needed updating in a *long* time.

On the Windows stuff in general,  communities need to be bootstrapped,  to achieve critical mass and be viable - the usual catch-22.  Pg/Win32, AOLServer/Win32, etc never got there.  Do we need to make a community forming itself around those elements a precondition for what I expect to be a much broader acceptance and deployment of the toolkit?

You're point about it being more work is well taken.  I was hoping to get some help doing the homework of how much work it would be (and I got some),  but I'll go off look into it some more.  My instinct is that Michael's work makes this plan doable,  but we'll see.

Collapse
69: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
Simon wrote: " As many of you will know, Java and TCL came out of the same company (SUN) and to an extent the same group of people, and the intention was always that TCL and Java would be complementary tools.... "

Although John Ousterhoust worked at Sun Microsystems for some years, Tcl did not come out of Sun. It was created much before that when Ousterhoust was a professor at UC Berkley. So Sun had nothing to do with the initial development of Tcl AFAIK, although I think Sun engineers worked on it while Ousterhoust was there.

Collapse
70: Response to nsjava staus (response to 1)
Posted by Jonathan Ellis on
Heh...  when you argue with me, Don, you take the position that Windows is a first-class platform to OACS.  When you argue with John you say it's not.  In the same post.  Which is it, really?

The smell here is an awful lot like a bad case of NIH syndrome. :(

Collapse
71: Response to nsjava staus (response to 1)
Posted by Jamie Rasmussen on
I'm planning on being an early adopter of native Win32 PostgreSQL once it is ready, but I unfortunately don't have time to devote to testing it right now. I'm sure most people here are similarly busy - I don't think many people are testing AOLserver 4 on the various platforms either, despite the fact that it has been in beta for many months. I know that the PostgreSQL folks were planning on making a GUI installer for it, which should be very helpful.

I'm still planing to release and maintain an installer for AOLserver 4 for Win32 shortly after it goes final. Keeping up with the server core isn't that difficult, but trying to keep up with all of the modules is (more information on this). A good example would be nsjava - I have no idea what it needs, if anything, to make it work on Win32, I just haven't had time to look. All of the modules currently needed for OpenACS are there though. Anyone who wants to try it out today can get the installer from http://empoweringminds.mle.ie/openacs/installer/. I'm using a newer pthreads-enabled version of the CVS HEAD on my test machine, but I haven't made an installer for it yet. There are also a few fixes needed for OpenACS that I haven't had time to write them up yet. (I'm seeing a pattern develop here. :-)

I think John is right about the AOLserver community needing a bootstrap. (Not just AOLserver on Windows.) Several people in the AOLserver community have been discussing this lately, and I think the pieces are slowly starting to come together. I think you'll see some AOLserver advocacy work start in conjuction with the 4.0 release.

Collapse
72: Response to nsjava staus (response to 1)
Posted by Todd Gillespie on
This thread is getting out of control.

Responding to Andrew's connection pooling questions; I ran through a couple designs to poach ns_db code into an external lib; currently I'm leaning towards building on top of libdbi to start from a layer that doesn't have Tcl_DString types through everything. I plan on using such for both Apache2 and Gauche, which is a Scheme with a lot of overlap with scsh, but does Unicode and native threads. On some layer I want to build a similar system to UncommonSQL, a Common LISP database interface, which has the best OR mapping system I've ever seen, as well as the only decent query writer, and add some extra intelligence to the drivers to automate the type casting. Then roll that back into [mod|ns]_scheme, replace all this Tcl nonsense, and watch my lab by destroyed by OACSers rampaging with torches and pitchforks... oh, excuse me, I seem to be getting ahead of myself here.

I will post the details in a new thread, but here's the gauntlet: I will be writing an nsd module to handle WebDAV, with extensive poaching from the DAV testbed, mod_dav.

I'm going to avoid the rest of this thread.

Collapse
73: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
It seems like we've surfaced a bunch of short- and long-term issues here that are all getting mashed together. The short-term issues are somewhat easier: For example, if Jamie thinks he'll be able to get AOLServer 4 final running on Windows, that's clearly the best short-term solution for Windows compatibility. (In fact, it may be the *only* short-term solution for Windows compatibility.) Everybody also seems to agree that using Java packages in the short-term for stuff where Tcl code doesn't exist isn't a bad idea. This is a bit trickier, since at least some here are concerned with dependencies, but the service-contract approach sounds like a strategy that people agree will work for handling this.

I'd like to float one more solution that I'm explicitly tagging as short-term. Regardless of the long-term viability of *replacing* AOLServer with nstcl on FastCGI, might this be a shortcut for getting mod_AOLServer on Apache? If nstcl has all the necessary features, and if it can run without too much work using mod_tcl, it seems to me that we could get Apache compatibility relatively quickly. We could at least try to benchmark the performance...

Longer term questions are murkier. Can AOLServer be made more portable to a range of operating systems? Can it be made more marketable? Can it be replaced? If so, with what? Is Java a part of the long-term solution or not? And if so, how? Rather than argue these issues with no data, we should lay down a plan for exploring our options. Advocates for each of these paths should come up with a plan to explore the options and see if objections can be answered. The most attractive plans will most likely be the ones that attract resources to carry them out.

Collapse
74: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Heh... when you argue with me, Don, you take the position that Windows is a first-class platform to OACS. When you argue with John you say it's not. In the same post. Which is it, really?
There are two observations:
  1. People talk about wanting to support windows as a first-class platform. This desire seems real.
  2. Despite all the talk, no one has done the work of making windows a first-class platform. Talk is cheap ...

It's not really a contradiction to argue that we want it to be a first-class platform in the future while pointing out that at present no critical mass has formed around the idea. It's not the technology that's in the way, it's the fact that the vast majority of us here love Unix and are only interested in Windows as a way of widening acceptance of our platform. Will we reach critical mass if we don't have a significant number of community members who love Windows?

Michael ... as far as operating system support, we support Linux, Solaris, FreeBSD, HP-UX, SGI, OS X (eventually with Oracle as well as PG) and using Jamie's work and either Oracle or the upcoming PG native Windows release, Win2K/NT.

All natively.

So we have all significant platforms covered, don't we?

I'll admit to a bias. I hate the notion of adding extra complexity, not to mention push a technically inferior platform to clients, to fix a shortcoming in marketing. Why not get better at marketing, instead?

Collapse
75: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
Don, my comment about supporting other operatings systems was as much a response to Jamie's comment (above) that AOLServer 4 has not been tested much on other platforms as it is about the Windows issue. I understand that, compared to the Windows problem, supporting other *nix platforms is a lot easier. But there's always going to be a resource question.

I agree completely with your emphasis on marketing better, BTW. It's not at all clear to me that AOLServer is *inevitably* going to fall behind. At the moment, though, we have two big obstacles in the way of a good future for the platform. The first is marketing. I agree that this may not be a huge issue. The second, though, is that fact that AOL is (understandably) focused on keeping the platform best for *them*, and their needs are somewhat different than those of the rest of the community. Now, if marketing goes well enough, then the second problem may turn out to be solvable too in one way or another. But until these obstacles are actually overcome, I think the community would be wise to actively investigate long-term alternatives, even if those alternatives are clearly technologically inferior...today.

Collapse
76: Response to nstcl (response to 1)
Posted by Michael A. Cleverly on
I didn't design nstcl with any intent to replace AOLserver (now or in the future). I wrote nstcl because I like to use the AOLserver/(O)ACS-idioms in the work I do (both professionaly and just for fun) in Tcl/Tk (but outside of AOLserver).  It's allowed me to be much more productive in areas outside of web development.  Even when a script doesn't do anything database related, just being able to use ad_proc in tclsh is a big win.

The first version of nstcl came about because I found the pgtclsh database extensions interface to be so much more painful than the AOLserver database api's that I'd come to know & love over the previous three years, so I created wrapper procs to emulate the AOLserver API.  From there it grew to support Solid, and then Oracle, plus other things like ns_sendmail, ns_hrefs, etc.

With the latest release came templating, a general code cleanup, actual documentation (I was getting sick of Kriston redesigning the AOLserver.com site and breaking all the existing link/URLs every couple of months), a test suite, and more supported database extensions.

I don't have any experience using a threaded version of tclsh, so I can't even begin to comment on the thread safety of nstcl.  In general I think it should be no more and no less than any other library of pure-Tcl code.  Another question would be the underlying thread-safety of the database extensions themselves, I suppose.  IIRC Oratcl 4.0 claims to be thread-safe now, whereas previous versions weren't.  I have no idea about pgtclsh and the others.

John emailed me and asked if I saw any obvious reasons that would completely rule out it's use with FastCGI.  I don't see any issues that are insurmountable.

Obviously nstcl would need all the AOLserver API's related to HTTP & HTTPS requests/connection handling.  My working version of nstcl 1.1  adds some additional non-connection oriented commands: the ns_schedule_proc and ns_config/ns_param set of commands, for instance.

I hope that anyone who decides to persue this path would _fully realize_ the speed difference between a C implementation (AOLserver) and anything written in pure Tcl (nstcl).  That said, however, for low-end/low-volume sites, I imagine speed wouldn't be an issue.

BTW, there's at least one area where nstcl already outshines AOLserver/(O)ACS: migrating data from two different types of databases (Oracle to Postgres, for example), because nstcl's database api doesn't force you to fall back on the AOLserver ns_db api for the second database. :^)

Collapse
77: Response to nsjava staus (response to 1)
Posted by Don Baccus on
<blockquote><i>But until these obstacles are actually overcome, I think the community would be wise to actively investigate long-term alternatives, even if those alternatives are clearly technologically inferior...today.
</i></blockquote>
The problem is that the suggestions are most likely to be technologically inferior <b>forever</b>.  As Michael Cleverly points out , a Tcl-based solution is never going to approach the speed of a C-based solution (AOLserver).  nstcl today solves perhaps a quarter or a third of what would be needed to support a poorly performing FastCGI solution.
<p>mod_nsd will never be part of the Apache mainstream, leaving us with maintenance issues similar to our having to support AOLserver on windows. Note that we <i>have</i> AOLserver on windows in preliminary form while mod_nsd hasn't been maintained for years, now, and moving it to Apache 2.0 will be a big job.  An argument that supporting mod_nsd will be a bigger headache than supporting AOLserver on Windows  is easy to make, given that it fits today's situation.
<p>It's tempting to try to fix marketing problems by changing technology, just as it's tempting to fix technical problems by changing one's marketing approach.
<p>ArsDigita tried to fix their marketing problems by dumping ACS Tcl and rewriting it in Java.  ArsDigita no longer exists.  The problem wasn't the technology, it was the fact that the team in charge at the end probably couldn't sell snowcones in the Sahara.
Collapse
78: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
Hi,

I think we have some negative feelings about Java due to aD switching from Tcl to Java.  I hope this gets reduced.

On my personal experience I believe that my learning curve to CCM was made even harder due to the negative feedback on this community about it.  I am not defending CCM, all I am saying is that its not as bad as the posts that I have seen on OpenACS.

I will post some nice stuff that I have found about CCM.  In case you think its not good for you then its not.  I am posting it here because I think it was good for me and someone might see that its good for them too.  Of course CCM is still less productive for me than OpenACS.  But there are the nice things about it.

- Given the fact that I started to really learn CCM on my own and had nobody to turn to (no bboards) I was able to learn it more or less.  Since Java had some tools and I was able to pick up one.  Which is eclipse and I was able to look at the whole code base with help from an IDE.  I think I got more mileage from an IDE than if I just used emacs and grep.  So as a newbie I had more tools and help from other non CCM based but Java based resources.

- Since CCM is in Java there are some tools that helped me like the debugger.  It sure helped me see what is really happening to the code.  In fact I recently downloaded tclpro just to see if I can make it run and use a debugger.  Ofcourse since tcl is so simple ns_log and printf debugging is something that is good enough.  But for some reason I miss having a debugger.  I guess its been a long time for me since I used a debugger in C, C++ and Java.

- Even though CCM is thicker than OpenACS since its very hard to access the database directly (there is no db api aside from using jdbc).  But its actually closer to the data model of ACS.  For example when you get a new User and save it it will actually insert rows on acs_objects and users table.  Its because the java class User inherits from acs_objects java class.  So the Java classes in CCM is closely mapped to the data model.  So if I inherit User into FooUser in java.  I then create a foo table on the db.  Making a new instance of FooUser will put rows in foo, users and acs_objects table.  Also by getting FooUser(object_id) it selects that user from the database.  Changing the attribute of FooUser.name actually corresponds to the column in the db.  You can also define methods that needs custom sql queries like foo.getAll for performance reasons.

- Since things are more rigid and less flexible there is a more uniform way of doing things.  I guess its because you are more retricted.  Being a beginner that is a big plus for me because after a while there is a pattern that develops.  Of course once you are an expert it might hinder you... I don't know since I still consider myself a beginner in CCM.

- Since the java classes closely models the data model and have a meta data associated to it.  The create, delete, insert, etc. done the same way.  There is always a retrieve and save method.  Much like what OpenACS have in create and delete plsql.  But this is done on the super class Data Object class.  So Users, etc. does not need to implement this stuff unless they need to.  Since things are more or less uniform there is already a move to use the metadata to create the tables.  So porting to a new database may become trivial in the future.

- its nice to see that CCM uses a lot of java libs for their xml, crypto and authentication.

On some extent the restart of server everytime may not be true.  Since this was always been posted on the bboards I did not do to much work on it.  But recently I tried to use hot swapping and class reloading.  So while in the debugger I was able to change my code without a restart.  Of course big changes of code needs a restart.  But I think more advance webapps like Websphere is able to handle this.  I guess this is the one of the things you pay for to IBM.  But I am not sure about this as this info pertains to other Java code.

To summarize, CCM has still a lot more problems but its not as bad as what most people think of.  Of course OpenACS still have the general advantage.  Please let us try to look at CCM in a better light.  Also it may not be the failure of aD has a lot to do with their decision to use Java.  The same way that it is not true that if aD was successful a lot of it will attributed to Java. Of course the marketing heads will screw a lot of the truth.

I hope I did not move the discussion to any side my intention is to put it more in the middle ground.

Collapse
Posted by John Sequeira on
This recent performance comparison is one of the articles that got me thinking about FastCGI as on option for long term, low-maintenance OpenACS portability. My feeling from reading this and all the other papers on FastCGI performance is that though it's probably inferior to AOLServer's well-tuned embedded interpreter like Don says, it's not by any amount I'd spend time worrying about. Remember, Scalability (inferiority?) means speed (i.e. c vs. TCL) _and_ throughput. FastCGI might lag in the former relative to AOLServer, but I believe middle-tier scale-out gives it a clear advantage in the latter.

I think Don's point about the additional amount of work to core OpenACS development is a much better one to consider. If I felt FastCGI support would be as much work as, say, the database abstraction layer (XQL files etc), I never would have brought it up. I'll try to do some prototyping to address that more concretely. My guess is that without service contracts in place or some sort of agreed upon convention for abstracting web serving and app logic, you're still talking about minimal patching/overloading to move the native AOLServer TCL code to run under FastCGI. Therefore I think keeping a fork in sync without distracting the main OpenACS developers is do-able. It would be nice to have people thinking about writing more portable code to support that, but I don't see this as an either-or proposition.

Collapse
80: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Just to re-clarify me earlier posting....

I do not in any way shape or form, recommend or support any move from AOLServer

AOLServer is what brought me to OpenACS in the first place (so much for having poor marketing value).

AOLServer is still the best thing out there, when one considers all aspects of the function of a web server

TCL is an excellent tool for web development, and again A primary motivator for us.

All I really meant was that the OpenACS should offer a way to intergrate Java components/legacy systems, that is simple and clearly documented, and that we should highlight this capability.

All this talk of moving to other web server etc is counter-productive.

Surely we'd all be better off just imporving AOLServer itself?

Cheers

Simon

Collapse
81: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
Yup that pretty much sums my opinion too.
Collapse
82: Response to nsjava staus (response to 1)
Posted by Janine Ohmer on
I have to admit that I'm occasionally tempted by thoughts of running with Apache, simply because AOLserver has been a hard sell for us at times.  But I usually change my mind when I start thinking of how much work it wouuld be, and that it's likely that even with all that hard work the performance still wouldn't be as good as what we have now.

The OpenNSD project was started about a year and a half ago to try to address some of these issues;  unfortunately it ran out of steam.  It was born out of frustration with the folks at AOL, who were very rude and condescending to community members who dared to contribute patches or even point out bugs.  But the very act of creating OpenNSD was apparently the kick in the pants they needed, and attitudes improved which took away some of the urgency behind the project.  Then we started thinking we should wait for 4.x to come out... who knew it was going to take this long???

It might be worth considering reviving that project, if only because the name OpenNSD a) doesn't mention AOL and b) makes it clear that it's an Open Source project, not one dependent on the whims of a large corporation.

Collapse
83: Response to nsjava staus (response to 1)
Posted by Jon Griffin on
Janine,
I really think we should rethink OpenNSD. When I was at LinuxWorld with Talli and Roberto one of the biggest objections to OpenACS was AOLServer. I too, have been waiting and waiting for 4.0 and it doesn't look like much is happening.

I would be willing to put some time into OpenNSD if the community thought it was worthwhile to go forward with it again.

Maybe we need another thread on this topic.

Collapse
84: Response to nsjava staus (response to 1)
Posted by Louis Zirkel on
Janine and Jon,

I too have inquired as to the status of OpenNSD on a number of occassions over the past year or so and there seems to have been a complete lack of any interest in moving this project forward. I feel that as good as AOLserver is, there is still a great stigma with having the letters A-O-L attached to your webserver of choice.

I for one would love to see the OpenNSD project go forward so that a great technology would have the chance to see the light of day against those who are too closed minded to get past the stigma.

I've done some work to keep the Solid database driver up to date as we use it here at my place of employment and I've never gotten any response out of any of the AOL folks with regards to patches I've submitted to them, and last I knew they had dropped support for Solid altogether. I would like to keep that driver maintained against an OpenNSD webserver base personally.

Collapse
85: Response to nsjava staus (response to 1)
Posted by Janine Ohmer on
Ok...

I will start an OpenNSD thread and see if we can revive the thing.  However, I need a few day's breathing room - I can't get involved in anything else until I get the 4.6 release status page done! :)

Collapse
86: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Maybe I just need to write Kriston another private e-mail :)
Collapse
87: Response to nsjava staus (response to 1)
Posted by Janine Ohmer on
Go, Don, go! :)

BTW, I meant to mention that at least part of the delay in 4.0 has been that AOL has been converting sites like MapQuest over to AOLserver and I assume they are waiting to release until they have finished fine-tuning it for their own needs.  In the long run this is a good thing, since it helps to be able to say "no, it's not a dead product, AOL is using it for all these big sites and is still converting *to* it, not away from it".  But still, the lack of information is unsettling.

Collapse
88: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Remember, Scalability (inferiority?) means speed (i.e. c vs. TCL) _and_ throughput. FastCGI might lag in the former relative to AOLServer, but I believe middle-tier scale-out gives it a clear advantage in the latter.
Why would you believe this? The middle-tier technology in AOLserver is almost entirely written in C, after all. That's one of the major advantages of AOLserver's "I'm really an app server not just a web server" design.

Besides which, the FastCGI model gets its fast throughput compared to the mod_perl approach primarily for one reason. Here's a quote from the paper you reference:

Globally initialized objects, for example database connections, are thus persistently maintained.
The FastCGI example connects to PG once, while the mod_perl script connects every time the script is run.

So the benchmark is to a large extent measuring the performance impact of implementing a persistent connection mechanism.

Which AOLserver has built in, of course.

I believe there are persistent db pooling APIs for Perl these days, no? The benchmark would be far more interesting if the FastCGI and mod_perl examples both used one of these modern perl db APIs.

One other area where FastCGI wins over mod_perl is that the script is only compiled once (this is why the db connection is only built once). AOLserver caches the compiled bytecode from libraries and script pages therefore has a similar advantage over mod_perl ...

This does tell me that a fast-performing mod_nsd would have to implement connection pooling (not just the db api) and would need to cache compiled bytecode just like AOLserver does if it would have any chance of performing equivalently. Adding to the complexity of the task.

BTW I'm ignorant of mod_perl so am basing my observations on the paper you've so kindly referenced. If the author's wrong, so am I, I'm just assuming he knows what he's talking about. Based on his comments, though, I see nothing to suggest that the FastCGI approach would yield higher throughput than a properly written AOLserver application.

Collapse
89: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Hmmm...looking closer I see the sample code they include isn't one of the actual benchmarks they ran.  In fact the benchmark sources aren't included (I'm sure they were in the original dead trees version of this paper).  But the comments at the end of the paper imply that no connection pooling was done in the mod_perl benchmark.  It would be nice to have the actual benchmark sources to see if they're as skewed as the example makes one fear they may be.

One of the comments at the end of the paper supports the experiences of some folks here: "In our humble opinion, servlets and Java are not very well suited to quick site development, a field in which PHP excels."

Yet they reported that PHP suffers, "as rumored", from scalability problems, at least in their benchmarking.

Maybe that's a marketing ... pardon me, evangelism ... message to build on - "OpenACS combines the ease of the scripting approach to site building with enterprise scalability!"  (ugh!)

And they do love PostgreSQL, they have that in their favor!

Collapse
90: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Don,

Yeah - the bytecode compilation will be a big issue. This is biggest porting issue I've identified, and probably beyond my TCL-hacking skills. However, I am in touch with David Welton, author of mod_dtcl (which caches bytecode, as does mod_perl.), who offered to help me with this. I may have to forego FastCGI short term and just stick with mod_tcl for now if bytecode-caching stretches my scripter-brain too much (even with David and Michael C's help.) The rest of the issues (nsv's, adp, filters, scheduled procs, connection pooling) aren't too bad or have prior art to pull from that I believe does fall within my abilities.

Listen, you've been hammering away on performance, but I maintain that raw performance is a straw man w.r.t. a portability strategy. It appeals only to (some) coders, not the people you'd want to target with a refined marketing campaign. I studied engineering in school, and can appreciate well-architected software as much as the next guy. But, seriously, how many times has anyone been in a meeting where somebody says, Darnit I'd like to go with that apache|iis|netscape web server, but it just doesn't do enough pages-per-second for my taste. Hands up, how many current OpenACS users actually *need* it's capacity to fill a T-1 on a $1K modern server? I worked at a site last year with 10 million unique visits a month running on Cold Fusion 4 and IIS. Awful, ugly, terrible architecture, but moving to something else so they could have a fractionally smaller web farm was pretty low on their list of priorities. I bet that's true for almost all sites with any sort of legacy dynamic code (it has been in every case I've seen). OpenACS is a great sell if you've got no legacy code/db's to integrate with. Isn't that everyone's experience when they go pitching it to clients? A portability strategy that addresses this would be a big win.

From my blog:

Also, the longer-term back-of-my-head plan which I'm guilty of not voicing more clearly is that I *want* a simple, unified, high-performance AOLServer stack to remain the preferred platform. I want to be able to deliver to clients a solution that runs on their preferred infrastructure, but has compelling benefits when run on it's native reference platform. That's exactly the kind of platform advocacy Microsoft is pursuing by pushing Rotor, and not-yet-suing Mono. They know both are going to suck, where suck means lags the Windows version by a good two years. But they can point to it, and it will allow them to attract a few fence-sitting *nix types to their camp. And maybe convince a few universities that they can write courses about their stack without making MS endow too many chairs.

People will build on Mono. Their code will work. And the clients who use that software will become potential customers of Microsoft's .NET Server stack (where the real money is).

I view the OpenACS/FastCGI project the same way. The customers that adopt it, who aren't ready to rip-and-replace their current infrastructure, will have one more excellent reason to consider doing so in the future.

Collapse
91: Response to nsjava staus (response to 1)
Posted by Andrew Piskorski on
John, your business arguments mostly make sense to me, but I'm confused about one thing. You seem to be talking about being able to eliminate AOLserver entirely from the OpenACS application stack and replace it with mod_nsd, mod_tcl, mod_dtcl, or something like that, with this something maybe running under FastCGI, maybe not. Why?

Why wouldn't you just want to implement Patru Paler's idea and extend AOLserver so that it can run as an app server under FastCGI, and use Apache, IIS, Zeus, or whatever other webserver you want to talk to AOLserver over the FastCGI protocol?

Yes, you're still running AOLserver, but solely as an "application server", which I believe just captured all the business/marketing benefit from your arguments. Now, it's not certain that adding FastCGI support is technically feasible. There are concerns about AOLserver's lack of a protocol module, for example, which were brought up in the thread above. But assuming it can in fact be done, would this approach of using AOLserver solely as an app server not make more sense than eliminating it altogether?

Or are you saying that some customer is really going to be happy running Apache + mod_fastcgi + mod_our_kludgy_replacement_for_AOLserver, but refuse completely to run Apache + mod_fastcgi + AOLserver? Or what am I missing here?

Collapse
92: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Andrew,

I think Petru's AOLServer-as-FastCGI server would be fine at a lot of places,  most in fact.  I'm not arguing that mod_crummy would be better technically :-).  There are two reasons why what I propose,  while more work for less performance (*ugh* gotta work on that sales pitch),  is desirable from a big-picture perspective:

The AOLServer dependency limits you with respect to platform (db/os) support and tcl standard package support.  A c-based fastcgi->nsd bridge still has a bunch of dependencies that come along for the ride,  which the community hasn't historically been able to maintain.

If you want to bring OpenACS to the Apache and Win32 communities,  I'm arguing that you'll get over the hurdle of convincing people you'll be around for the long haul if your only portability dependency were TCL,  you had native TCL db support,  and you had the exponentially larger TCL community supporting you.

Collapse
93: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
John wrote:

<p><i>"The AOLServer dependency limits you with respect to platform (db/os) support and tcl standard package support."</i>

<p>I don't see why. AOLserver has a database abstraction layer that supports several databases. Tcl standard package support is being worked on for 4.0, and currently works if you just put a small hack in your startup files. The only "dependency" I see here is the operating system, and that also is being worked on. FastCGI is probably just a matter of grabbing the Windows version (I assume it has a Windows version).

<p>
<i>A c-based fastcgi->nsd bridge still has a bunch of dependencies that come along for the ride, which the community hasn't historically been able to maintain. </i>

<p>But then what makes you think that you'll be able to maintain a completely different piece of software that nobody is familiar with, namely mod_crummy, that has no strong backer behind it? At least with AOLserver you can point to AOL.

Collapse
94: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
I forgot to add that regarding "the exponentially larger TCL community supporting you", the Tcl Developer Exchange (www.tcl.tk) runs on AOLserver (on ACS 3.x in fact). AOLserver is mentioned often on comp.lang.tcl (http://groups.google.com/groups?hl=en&lr=&ie=ISO-8859-1&q=aolserver&meta=group%3Dcomp.lang.tcl.*)

I talked to David Welker (hope I spelled that right) at LinuxWorld, and they (the Apache Tcl project) were interested in hooking up AOLserver as an application server to Apache.

So as far as the Tcl community supporting AOLserver, I don't think it's fair to imply that AOLserver does not have the Tcl community support (maybe Michael (Cleverly) will want to comment on this since he follown comp.lang.tcl very closely).

Collapse
Posted by Michael A. Cleverly on
The more I think about what John Sequeira wants to try, the more the idea intrugies me. The more I find myself thinking that it could very well be workable, and that it could be beneficial, and not in anyway detract or take away from OpenACS or AOLserver.

Let's first segment the universe into a couple of buckets:

  1. People who've seen the light & run OpenACS (means they run AOLserver already)
  2. People who run the worlds best webserver (AOLserver) but don't currently use OpenACS
  3. People who aren't running OpenACS, aren't running AOLserver, but are willing to switch to AOLserver to run OpenACS
  4. People who aren't running OpenACS, aren't running AOLserver, and for whatever reason (real or imagined) aren't (yet) willing to leave foo brand webserver
  5. People who aren't running OpenACS, aren't running AOLserver, and would never want to
Let's assume for a moment that someone manages to implement every single one of the Tcl API's that AOLserver offers, that OpenACS uses, using some other Tcl-enabled webserver. For sake of discussion, let's call this pseudo-nsd.

What does pseudo-nsd offer to each of our five groups? For groups 1, 2, 3, and 5, absolutely nothing. Group 5 could care less about anything. Group 3 is ready to move to the canonical webserver for OpenACS. Group 2 and Group 1 are already running the best of the best.

It's only the 4th group of people who might even care. Let's imagine they're interested enough to try OpenACS, get some experience under their belt, test drive it, kick the tires, see if it fits their needs/lifestyle. Pseudo-nsd would (if it plugged into their current server, and for some it wouldn't) give them a way to try things out. Play around. Get some experience.

Would it be as fast as AOLserver, as efficient in its use of memory & CPU cycles, as rock solid? No, it almost certainly wouldn't. In comparison AOLserver would be the luxury hotel. Pseudo-nsd would be the cheap motel.

If group 4 used pseudo-nsd, fell in love with OpenACS, then they might very well be persuaded to upgrade to the Real Deal(tm). Or, they might find that their existing hardware supports whatever volume it is they are getting adequately. Maybe they don't end up liking things at all. Is that a loss? No. By definition these were people who were curious enough about OpenACS only to try it, but not invested even enough to test-drive it under AOLserver. That they didn't find OpenACS to be a good fit for them is no great loss.

Writing pseudo-nsd should be as simple as re-implementing every single one of the AOLserver Tcl API's used by OpenACS. The same commands, taking the same parameters, and returning the same results given the same inputs. Not a single line of OpenACS code should need be modified to accomodate pseudo-nsd.

Would the above be simple? Obviously, it would take some work. There might be some hidden insurmountable barrier preventing it from happening. But, I don't think so. I think it's possible...

Let's assume for a moment it is possible. Is it something the OpenACS core team should work on? Absolutely not. Does it preclude others from resurrecting mod_aolserver or from Petru hacking AOLsrever itself into the FastCGI framework? Absolutely not. Does it preclude work on nsjava? Nope. Does it preclude people from renewing work on OpenNSD so that those who see A-O-L and dismiss us out of hand maybe at least give us a second look? No, not at all.

This, to me, sounds like the proverbial open source itch. John's obviously interested in giving it a shot. I'm inclined to help him. Do I run Apache? No, the only three webservers I've ever run are AOLserver, mathopd, & TclHTTPD. Do I have a need that pseudo-nsd would solve? I don't. Do I think it sounds like an interesting challenge where even if it ultimately didn't pan out I'd at least learn some new things? Yes, I do.

Collapse
96: Response to nsjava staus (response to 1)
Posted by carl garland on
I tend to agree with almost all of John recent post with the exception that initially I think it may pull some of the development away from AOLserver community. Of course developers can work on what interests them and this temporary sidetrack may end up bringing more people to the community and have a better long term effect on AOLserver. As far as protocol support I know through talking with Scott G. that one of his primary goals in the upcoming months (after 4.0) release is too take the AOLserver core and make it so *much* more. Adding additional protocol support is one of his major projects I think he is mainly waiting on the 4.0 release. Of course it would be nice if more of the AOLserver coders were in on this thread but I think the OpenACS community is close to its own release and I hesitate even posting thinking it may draw away attention that may be spent elsewhere currently. Don't want to be a distraction.
Collapse
97: Response to nsjava staus (response to 1)
Posted by carl garland on
I meant I agree with Michaels post above mine not John ... although I'm sure I probably agree with him too.
Collapse
98: Response to nsjava staus (response to 1)
Posted by Lars Pind on
The compelling thing for me about the thought of replacing AOLserver with Apache or other mainstream web servers would be to reap the benefits of big mainstream support:

1) Bugs are likely to be fixed more quickly

2) We're likely to get access to things like DAV, SOAP, and whatever the next cool technology is going to be.

3) It might make it easier to run OpenACS alongside some other technology, something that I believe people will want.

4) Running under Windows

5) Easier installation, because people are likely to have Apache running already.

That's what I'd be interested in, anyway.

/Lars

Collapse
99: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Hi Lars,

Bit surprised to hear you say that, and I thought I might respond to a couple of points.

1) Bugs are likely to be fixed more quickly
Urm... I don't think there's much evidence for that.In fact you might argue that the larger a community, the more difficult its is to get certain changes/fixes in. And, given the larger number of people and usage, it may make some fixes inappropriate, even if you need them desparately. Secondly, surely its more to do with the actual number of bugs! In my experience, AOLServer has been much more reliable than many of the alternatives, and therefore the actual bug fixing requirements are smaller. Perhaps this is a benefit of the fact that AOLServer is essentially maintained by commercial interests. I could argue this leads to higher quality and less bleeding edge, buggy code.....

2) We're likely to get access to things like DAV, SOAP, and whatever the next cool technology is going to be.
Again, there is some truth to this, but I *really* don't see what the problem is. SOAP is just not that complicated! I think the reason its not yet been implemented is that no-one has a burning need for it. If we'd *ever* come across a customer who wanted it, we'd have built it!... I do take your point, but I don't agree its all that critical

3) It might make it easier to run OpenACS alongside some other technology, something that I believe people will want.
Urm.. which tecnologies? It quite happily runs alongside most of the things I can think of

4) Running under Windows
Exactly who is this requirement for? Firstly the OpenACS is an excellent architecture for runing websites that don't go down. If someone is dumb enough to run it on windows, then they are obviously not all that interested in reliability/performance. Also there are many good alternative for windows.. Why bother with yet anouther one? The rest of the World is moving to Linux (my God even Deutsche Bank are moving their entire infrastructure to it)... and I personally think the pressure for Windows Server based web stuff will diminish. And against the effort of porting it... who's actually going to end up using it?

5) Easier installation, because people are likely to have Apache running already.
Apache is more difficult to install. Because you oftend end up having to bolt on many extenmsions and configuration items its doubly troublsome.. I really can't think how on earth you can make AOLServer installationb much simpler... I mean we're talking about a webserver/application server here... its not Notepad. I don't think a quick compile and a single config file are *really* all that difficult.

And I also notice you avoiding mentioning all the major *advantages* of AOLServer that we'd essentially lose!

I understand what your getting at, but dropping AOLServer is not the solution. The better solution is to make AOLServer compete!.

Plus... and this is the really worrying thing.... if this community were to ditch AOLServer, become equally windows focussed (with all the concessions that brings), I, my company and I daresay many others would abandon the community like rats off a ship! We'd just end up taking a cut, forking the code base and continuing on...

Trying to make OpenACS a panacea for everyone, for all things is a mistake. We'd be better of focussing of the what it does really well, and leveraging that to give it an advantage over the alternatives..

After all, some want Ford Prefects, some want Ferraries and some just want skateboards.... If we tried to turn our Ferrari into a Station Wagon with a Turbo... we'll get exactly what we deserve ;)

Cheers Simon

Collapse
100: Response to nsjava staus (response to 1)
Posted by John Sequeira on
And I also notice you avoiding mentioning all the major *advantages* of AOLServer that we'd essentially lose!

At the risk of speaking for Lars, that not our bag, baby.

Let me give you an example that might help out. I've written my last few big projects in perl because it ran where I needed it to and it talked to the db's I needed to talk to and it was quick to develop in. In perl land, code can be easily ported between plain old CGI.pm (the 'gold standard'), CGI::Fast (FastCGI), ASP on Apache *and* IIS (using ActiveState's Perlscript w/Win32::ASP and/or Apache::ASP) and mod_perl.

The existence of all the other deployment options in no way threatens CGI.pm. It made it healthier, in the way that open source projects with more eyes focused on them are always healthier. mod_perl did not threaten CGI.pm, or cause CGI.pm adopters to lose benefits, did it? Even borg-ifying Perlscript didn't threaten it. CGI.pm folks just went on making their stuff extremely useful, and the rest of the deployment options that adhered to its API just kept on benefitting. I see the same benefits for the OpenACS community. [ Aside: Does anyone not want the ability to deploy your application at a plain-vanilla ISP? ]

Also, something else to think about: beyond writing scripts that can be deployed pretty much wherever I or my client wants webserver/os/db-wise, perl lets me easily write scripts that I debug from the command line, using a line debugger. For some problems, I even fire up OpenIDE, a visual debugger.

Pseudo-nsd (tks, Michael) will provide the same benefits. It will make writing installers, packaging OpenACS apps, and debugging them a lot simpler, even if the deployment target is the gold standard, AOLServer.

Collapse
101: Response to nsjava staus (response to 1)
Posted by Michael Feldstein on
If John and Michael want to work on nstcl/pseude-nsd, I think that's great. I don't imagine a mass migration from AOLServer but, as Michael points out, it may very well facilitate migration *to* AOLServer. For the same reasons, I think it's fantastic news that the mod_tcl owners are interested in building a bridge to AOLServer. What can we do to facilitate that happening?

I agree with all of Lars' points about what's desireable for the platform in the long-run but, as Don points out, we might be able to achieve the goals that Lars enumerates by growing the AOLServer community (which would also, BTW, make a potential OpenNSV fork more viable should that be necessary).

It's a good idea to look at AOLServer alternaatives as ways of easing people into the AOLServer community in the short-run. If, in the long-run, one of these projects leads down a path to a platform that more of the hard-core community can look at and be tempted by it, that would be great. But we don't have to assume that will happen in order to see the value in investigating AOLServer alternatives.

Collapse
102: Response to nsjava staus (response to 1)
Posted by defunct defunct on
John,

I understand you point, but... well its got nothing to do with OpenACS or AOLServer......

I don't have your requirements. I write Db-backed stuff that wouldn't run a aplain vanilla ISP anyway, and if you really want that level of simplicity, what do you want AOLServer/OpenACS for anyway.....?

Collapse
103: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Writing pseudo-nsd should be as simple as re-implementing every single one of the AOLserver Tcl API's used by OpenACS
Another way of saying this is that implementing pseudo-nsd should be no more difficult than rewriting AOLserver from scratch.

Leaving you with another piece of software no one's heard of being maintained by people with a budget significantly smaller than AOL's.

Collapse
104: Response to nsjava staus (response to 1)
Posted by Don Baccus on
For me, the main issue is dilution of resource. My guess is that John's not willing to work with the existing technical envelope anyway, so any effort he puts into mod_nsd is not going to keep us from solving real problems anyway, because he won't work on them regardless. So, John, go for it. I just don't want anyone doing useful work getting diverted from the very serious technical issues we need to solve over the next several months if we're going to succeed.

But when I read things like this ...

[ Aside: Does anyone not want the ability to deploy your application at a plain-vanilla ISP? ]
I don't see an argument for pseudo-nsd

I see an argument for rewriting OpenACS as a LAMP application. Simon hints at the difficulty above, i.e. when it comes to running OpenACS at a plain-vanilla ISP, AOLserver is not your only problem. How many plain-vanilla ISPs offer Oracle for $19.95/month? Most only offer MySQL (I personally would shun any ISP that forces me to use Windows as the platform). Your plea to migrate from AOLserver will be followed soon by one to support MySQL if you follow your argument to its logical conclusion, because most ISPs won't run PostgreSQL. At most you're going to be given a choice between LAMP or 2K/IIS/MSSQL or perhaps 2K/Apache/MSSQL.

I co-lo my own box for $100/month and have full control over it at my vanilla ISP. That's starting to look overpriced, Cathy Sarisky is offering hosting on a remote box that costs her about the same but which includes rebooting and other low-level service. And she didn't have to buy her own box like I did.

If I have a client that can't afford that kind of solution, I know they can't afford me. So, frankly, no ... I personally don't care

And I don't see why anyone would.

Let me give you an example that might help out. I've written my last few big projects in perl because it ran where I needed it to and it talked to the db's I needed to talk to and it was quick to develop in.
In other words OpenACS would've brought you no advantage even if it were written in Perl and supported whatever platform you were interfacing to.

OpenACS is not and never will become a panacea for every web job out there. The push towards vertical apps like dotLRN is in fact a push away from the vision of being all things to all people. We've been focusing effort on making OpenACS a great platform for developing applications like dotLRN.

The great interest that dotLRN is triggering tells me that this focus is one that might play well in the marketplace.

Meanwhile, what message is made if we implement mod_nsd or some other hack to get rid of AOLserver.

Even the OpenACS people realize AOLserver sucks, because they're trying to abandon it for Apache?
That's not a great message to bring to the world when any stable alternative is months away at minimum. It also opens the door for competitors to say "they were wrong about AOLserver, they're wrong about Tcl and PostgreSQL, too".

Really ... if we're going to abandon rock-solid, easy-to-install technology like AOLserver, why not just go the whole nine yards and rewrite everything as a LAMP application? We'd be immediately far more popular than we are today. If ultimate popularity, not technical excellence, is our goal, why not just choose the most popular platform out there and run with it?

Collapse
105: Response to nsjava staus (response to 1)
Posted by John Sequeira on
Don,

3 replies:

1)
    Cygwin != Unix
      Wine != Windows
pseudo-nsd != AOLServer

In the cases above, how has the emulation layer detracted from the main effort? How has the emulation layer been anywhere near the effort involved in the original?

2)
Hey Red Hat, why are you supporting Cygwin if Linux is so great?
Hey Microsoft, why are you supporting Rotor if Windows .NET is so great?
Hey OpenACS, why are you supporting other web servers if AOLServer is so great?

Don - "Go for it"
3) OK.

Collapse
106: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Let's first segment the universe into a couple of buckets:

   1. People who've seen the light & run OpenACS (means they run AOLserver already)
   2. People who run the worlds best webserver (AOLserver) but don't currently use OpenACS
   3. People who aren't running OpenACS, aren't running AOLserver, but are willing to switch to AOLserver to run OpenACS
   4. People who aren't running OpenACS, aren't running AOLserver, and for whatever reason (real or imagined) aren't (yet) willing to leave foo brand webserver
   5. People who aren't running OpenACS, aren't running AOLserver, and would never want to 
This is good, because Michael Cleverly's making a marketing argument (as is Lars, essentially). Moving away from AOLserver only makes sense as a fix for our presumed inability to market the existing platform (I'm assuming Dan's nsjava efforts and other efforts to improve our ability to quickly integrate with new technology continues to move forward).

So the real issue, given that those in group four above are the only who would be impacted by some sort of kludged-up AOLserver platform, is "how badly do we want these people?"

Put crassly, "how much are we willing to pay to get these people?"

Where the resource being consumed isn't necessarily money but volunteer developer resources as well.

Meanwhile, we have folks like Ben Koot pointing out that in many places OpenACS 4 still sucks. (my words, not his, but it's an excellent example of the lack of polish in OpenACS 4).

We're arguing about whether or not to devote resource to something like a brand new mod_nsd while the toolkit itself still is absolutely horrible in some places and moderately messed up in many others? I'm having a real problem understanding the prioritization that this implies. We need to sharpen our focus, folks, and fix it before we spend too much time worrying about where it will run.

Collapse
107: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Urm.. John...

I don't understand that... We're not supporting any other web servers are we???? (probably because AOLServer is so great... :-s)

And although I agree Wine, Cygwin etc are as you say.. so what? Whats your point..... You;d have to ba half cooked to want to run any kind of serious site involving any of those anyway?

Sorry for being dumb... but I don't understand you post?

Collapse
108: Response to nsjava staus (response to 1)
Posted by defunct defunct on
On the marketing point.... yes I agree that up until now perhaps AOLServer has not necessarily been good from a marketeers perspective... (although I don't know why I'm excepting it... it was AOL/TCL that brought us to this community in the first place!)

However, that only emphasises we need to do a better marketing job... and as Don points out *the* best way to do that is get the current OpenACS working well....

Beleive me... when it comes down to it, if I can developer faster, better and with more stability and give my customers what they want... well.... the windows/apache etc argument just sorta disappears!

Why does everyone in computing these days keep banging on about conversion and symmetry? Has anyone out there forgotten that these are 'web sites' were building? Given a user can't tell the difference from their perspective then why the over focus on platform????

Anyone who turns down a competitive bid simply because they 'like windows and have never heard of AOLServer' is a pillac anyway!  And I'll be jiggered if I want to spend any time making their life easier (sorry... personal rant)..

And... aren't we on the verge of the new web evolution? i.e. Web Services? Aren't we all supposed to be moving to a heterogeneous environment of cooperating services and common interfaces?

These platform/software/porting argument belong back in the early nineties......  Can't we settle for an OpenACS that does what its supposed to do, but which retains facilities for integrating other technologies on an 'as-needed' basis. (this is pretty much what we have now.

Collapse
109: Response to nsjava staus (response to 1)
Posted by Michael A. Cleverly on
I don't believe that OpenACS coders should spend time or resources on bootstraping OpenACS on a non-AOLserver platform, and I said as much in my post. That said, however, I don't see the need to be hostile to people who might have a different itch to scratch, especially if they're not already dedicating large amounts of their personal life to active OpenACS toolkit coding already.
Collapse
110: Response to nsjava staus (response to 1)
Posted by Michael A. Cleverly on
When I hear someone say "marketing argument" I tend to think of compromises to be buzzword compliant. My interest in pseudo-nsd isn't to be buzzword compliant as much as I'm interested in the technical challenges of interoperability--could OpenACS be bootstrapped into another enviroment without requring any changes to OpenACS? Exploring that question is of interest to me. <shrug>
Collapse
111: Response to nsjava staus (response to 1)
Posted by Don Baccus on
    Cygwin != Unix
Cygwin sucks so bad that both the Apache and PostgreSQL folk felt compelled to write native Windows versions of their applications.
      Wine != Windows
And I've never gotten it to work well with anything I've needed unless I've had a Windows partition with the real Windows DLLs available (which defeats the purpose for those who wish to avoid buying a Windows license), and even then it couldn't handle IE. I bought VMWare so I could do what windows work I need to do on my Linux machine because Wine sucked so bad. I understand it's slowly getting better, though.
pseudo-nsd != AOLServer
This in and of itself doesn't prove a hack effort won't suck as bad as Cygwin or Wine but it doesn't prove it won't, either.
2)
Hey Red Hat, why are you supporting Cygwin if Linux is so great?
Hey Microsoft, why are you supporting Rotor if Windows .NET is so great?
Give me one percent of the resources of either of those companies and my objections disappear like magic.

On the other hand, give me one percent of the resources of either of those companies and I can get you a solid AOLserver running under Windows with good java integration to a wide variety of technologies, and have enough budget left over to properly market the thing.

Hey OpenACS, why are you supporting other web servers if AOLServer is so great?
We aren't, because thus far we haven't proven we have the resources to properly tackle the job at hand much less take on significant new work that has dubious technical merit. Though I'm flattered that you seem to think our humble project is in the same league as either of those companies.
Don - "Go for it"
3) OK.
Can't stop you from doing what you want to do, even if I wish you'd help us fix our core technology instead.

Collapse
112: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Technical challenges and itches to scratch  are fine... I certainly don't have a problem with it....

If you really enjoy this kind of thing from a purely pedagogic or explorative perspective thats ok too...

Buty to be honest thats a fairly hackery thing too.... everyone wants to play with different stuff cos its a damn site easier than actually doing the basics...

There's a good (paraphrased) Japanese saying...

'Anyone can draw a Demon, but drawing dogs is incredibly hard'

All I'm saying is that the community doesn't *need* what your proposing, nor is there very meuch general benefit.... therefore (other than for your own interest) whats the point??

Just think how much discussion we might have got done on Testing for example ;)!!! Far less intersting, but much more important.. (testing is the dog in this one ;)

Collapse
113: Response to nsjava staus (response to 1)
Posted by Don Baccus on
That said, however, I don't see the need to be hostile to people who might have a different itch to scratch, especially if they're not already dedicating large amounts of their personal life to active OpenACS toolkit coding already.
Well, John's initial posts were along the line of rewriting the toolkit to eliminate dependencies on AOLserver. Integrating such work into our mainstream releases would impact the project and require resource from folks like me (responsible for making those releases work), Simon (responsible for making sure they're tested), etc.

There's a serious possibility that performance under AOLserver might be negatively impacted.

If the community supports repackaging for other platforms there's going to be consumption of resource there, too, along with documentation needs.

Note that we're not doing a bang-up job in the above areas as it is.

If John were simply announcing that he intends to scratch his itch for his own pleasure I wouldn't object in the least. I'm not objecting to your doing the same, right?

But John's first post made it clear that he was suggesting a new direction for the community to take (rewrite OpenACS to remove AOLserver dependencies), not simply that he intends to pursue a personal project for personal pleasure.

That's a very different thing. If we were to take that path, it would require our resources as well as his. You're not asking for our resources. See the difference?

For this reason I'm far more interested in mod_nsd as a potential "solution" (this presumes a problem exists but for the sake of argument I'll accept that one does). This wouldn't require rewriting of the toolkit therefore wouldn't impact us. My attitude towards such a project would be simple: you're on your own. I'd suggest we host it here but folks working on it would be responsible for their own hacking, documentation, testing and packaging and I would fiercely fight to keep existing OpenACS resources involved in working on OpenACS issues instead.

Because there are still so damned many of them ...

Collapse
114: Response to nsjava staus (response to 1)
Posted by Talli Somekh on
When I sell the OpenACS, I spend most of my time extolling the virtues of AOLserver. The reason is that it I can point to the fact that the largest ISP in the world uses it, that my developers love building tools with it and that it *works*. I also find that I can put together a very easy sales pitch around it, which I shared in another thread (and plan on writing up more formally for the community).

That being said, I think John and Michael C's ideas are good for a few reasons, partly because I have been speaking with John a lot about it offline so I understand his needs better but also because I think it will be a very useful experience even if the tool isn't all that's cracked up to be.

They are both long time members of the community and contributors. Michael's been around since the beginning and even turned Roberto on to the ACS. John has contributed the VMware install and has worked extensively with the workflow package. He will be contributing a paper on his work eventually and also a package for document peer review.

What I think that John is looking for, and I think that some other users and potential users may appreciate, is using the datamodel and code that is present in the oacs within the context of legacy systems or new applications where the platform has been specified even *prior* to the application specifications have been written down.

Now, I agree with Simon that this is the worst case and it's reasonable to reject those clients out of hand. But it's also reasonable to accept them if they are willing to pay you full cost for the death march :)

John's initial suggestion may have been a bit too harsh, but I think that probably everyone has thought at one point or another that it life would be easier if we were using Apache rather than AOLserver, if only from a marketing perspective.

I also think that John's idea of a tool that works but is not the canonical tool is a good one. Some people might use the nstcl stuff with apache or IIS, realize that they are only getting 20% of the functionality and administrative ease of AOLserver and will be interested in switching. This could bring in much new blood to the community.

So I think that this project is very worthwhile, but I do agree with Don that it should remain peripheral to the core community at least until it proves that it will have support. If John and Michael build the tool, it works *and* they or others maintain and support it, then it can become a piece of the canonical distribution. Until then, we should consider it valuable and worthwhile community R&D.

talli

Collapse
Posted by Andrew Piskorski on
FYI, I happened to be looking through the AOLserver cvs sources for an unrelated reason, and noticed this in the ChangeLog:

2002-06-19  Jim Davidson <jgdavidson@aol.com> 
    
  * include/ns.h: 
  * nsd/tclinit.c:  Added new Ns_TclCreateInterp() which 
  creates and initializes interps for server and non-server 
  interps (e.g., the config interp).  Interps are now 
  initialized with a call to Tcl_Init() which enables the 
  standard Tcl library procedures (e.g., unknown, http::, 
  etc.). 
   
  * nsd/tclthread.c: 
  * nsd/tclcmds.c:  Thread and cache related commands no 
  longer require a server, exposing them in any interp created 
  with Ns_TclCreateInterp. 
   
  * nsd/server.c:  NsGetServer handles NULL server. 
   
  In general, the changes above are steps towards making the 
  AOLserver API's more useful outside the context of a server. 
  The idea is to enable use of libnsd for other applications, 
  e.g., shells interps. 
 
2002-05-15  Jim Davidson <jgdavidson@aol.com> 
 
  * include/nsdb.h (new):  Moved the NsDb interface from core 
  to new nsdb module.  Simply loading nsdb.so should work as 
  before.  Goal is to enable full replacement of NsDb in the 
  future.

Now, I've no idea how far Jim and Kriston plan to take that, or how soon, but clearly the AOL team has at least some interest and has made at least some progress in refactoring the AOLserver code into libraries that can be used anywhere. And once that's done, it should be very cool indeed...

Collapse
116: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Now this is an idea I can get behind, because it gives rise to the potential of greatly lowering the implementation cost of incorporating the AOLserver API in non-AOLserver environments.

And (see my above posts about resource) best of all someone else, not us, is doing it.

Collapse
117: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
This is what I like about Jim Davidson. Action. About a year ago or so he brought up this idea of making a libnsd (and I think I brought it up on this thread somewhere).

This is the way I would prefer to make things happen for other webservers/architectures. Making a mod_nsd out of libnsd should be significantly easier.

Collapse
118: Response to nsjava staus (response to 1)
Posted by Don Baccus on
Right, and it might even work well.

Of course then we're still vulnerable to the argument that "LAMP folk don't need mod_nsd, and mod_nsd is AOLserver based, and AOL is evil, and no one supports mod_nsd, and ... and ... and ... you're still using PostgreSQL, and almost no one uses that!  If you're not using MySQL you obviously suck!"

Really, you folks who are so concerned about this issue should just pony up the bucks to get a LAMP port.  Then no portion of our platform will be subject to market pressure from the hordes of LAMP implementors, most of whom have no money and websites that suck.

Collapse
119: Response to nsjava staus (response to 1)
Posted by Jun Yamog on
DonB,

Although agreeable to the post above.  It might attract some LAMP people.... hehehe.  Its just too bloody blunt.

Talli,

Paste the above post to your DonB's collection of posts.

Everyone,

Although I am for portability but given the choice to improve or make the toolkit portable.  I will choose to improve since admittedly its still very unpolished.  Although this is just my opinion and my contribution to the toolkit is very small.

Maybe there is the lack of itch or challenge on OpenACS for John and Michael.  Why not the challenge of making OpenACS not portable but have better integration.  Like ns_java, soap, xml, calendar synch, ldap, etc.  Of course this does not get into the itch of John and Michael which is portability.  But it might be close enough.

I think we have more marketing gain if OpenACS has better integration capabilities that having OpenACS being portable.  The same way MS is selling .net against java.

If OpenACS can:

- use users of current corporate systems.  Most likely via ldap

- thick application like Word can save to CR.  Most likely via webdav

- OpenACS can offer its application via web services.  Most likely via SOAP, WDSL, etc.

Although the above did not make OpenACS portable but sure does make it easier to be integrated with.  I believe that will be more compeling reason for users to use openacs than making it portable.

I am presenting this ideas as a possible itch to John or Michael who are exceptional developers.  Some of ideas has been done but little has been an official implementation to OpenACS.  If anybody who can make this integration with good design it has to be John or Michael.  So what do you think?  John and Michael scratches an itch and OpenACS toolkit becomes better at integration.  Win-win, of course if John and Michael will consider this to be their itch.

Collapse
120: Response to nsjava staus (response to 1)
Posted by Don Baccus on
If OpenACS can:

- use users of current corporate systems. Most likely via ldap

- thick application like Word can save to CR. Most likely via webdav

- OpenACS can offer its application via web services. Most likely via SOAP, WDSL, etc. 
I think this is doable in a six month timeframe with a combination of nsjava, Todd's nsdav work, etc.

And I think we can incorporate many of the basic toolkit improvements (and out-and-out fixes) people have been talking about in the same timeframe.

And we'll have AOLserver 4.0 for windows bundled in that timeframe, as well, I hope.

And hopefully libnsd will be a reality then, too.

And we'll have had six months trying to push dotLRN in the education world, too, and we'll have a much better understanding as to whether or not AOLserver is the major barrier to acceptance there, or in the Windows world, etc.

Six months from now libnsd would potentially make writing a mod_nsd much easier, we'll have a much more solid toolkit that integrates with at least some popular outside services, and all around be on a much stronger footing than we are today.

At that point in time it might well make sense to divert resource to bundle together a mod_nsd from libnsd. Then again, we may learn that AOLserver isn't the marketing horror story folks keep telling us it is. We should be in a better position to make an informed decision.

For me it's primarily a matter of prioritization, timing, and having a better understanding of what we really need to do to get our work more widely accepted.

Collapse
121: Response to nsjava staus (response to 1)
Posted by Jon Griffin on
Is AOLServer going to be at 4.0 soon. I see no indication of that on the website. I would love to know though.
Collapse
122: Response to nsjava staus (response to 1)
Posted by Jamie Rasmussen on
You won't find this on the webpage, but the grapevine says that AOLserver 4.0 will be released shortly after TCL 8.4.0 final is released.  I would expect TCL (currently in beta) to be released soon - I've seen September 5th mentioned.  Probably before the annual TCL/TK Conference anyway (Sept. 16-20).  I also encourage everyone who is interested in the future of AOLserver to join in the weekly AOLserver chats - http://dqd.com/~mayoff/aolserver/weekly-chat.html for more information.
Collapse
123: Response to nsjava staus (response to 1)
Posted by Roberto Mello on
As for the AOL-ness of AOLserver, which some have pointed out as a stigma for adoption of AOLserver, I talked to dozens and dozens of people about OpenACS and AOLserver at LinuxWorld.

I didn't get _one_ single "Ewww, AOL? Blergh" sort of comment. What I did get was _one_ "Tcl? Oh my gawd! I'm a perl guy".

What I also got were questions like "Wow, this is great. Can it interoperate with Apache?", and the occasional "Does it work with MySQL?".

On our "sales pitch" we pitched AOLserver as an application server, not as a web server, which I think is far more accurate, although I usually also noted that AOLserver was also a good web server.

Collapse
124: Response to nsjava staus (response to 1)
Posted by Andrew Piskorski on
What is "LAMP"?
Collapse
126: Response to nsjava staus (response to 1)
Posted by Jon Griffin on
I got plenty of AOL, it must suck. After explanations though they usually would listen.

I still think that while it is great that everyone wants OpenACS to take over the world the reality is it never will. It does something extremely well and most of the world doesn't care.

You have to pick your battles AND pick your clients. I would never try to convert a LAMP shop to OpenACS it is to much work for little return. If they come to me fine, I hate trying to convert people as it usually ends up in a huge waste of everyones time. Apache is a general tool that has been hacked to do anything and everything. They don't do it great, but they do it OK and that is all that matters for the majority of people.

Besides, what percent of the LAMP folks have any budget to pay me? If they are using LAMP they don't have a scalable platform and most likely don't care because the 15 hits they get a day don't stress anything.

MS shops are a little easier to target only because the security is so bad even dumb CEO's don't want to deal with that. They are still a tough sell, just not as tough.

There are plenty of places AOLServer and OpenACS work well and those are the people that are running a community. Your bank doesn't (probably) care about community so they run java apps. Mom & Pop Ecommerce sites don't care about community they run LAMP.

Who does that leave? Governments, NPO's and some other enlightened companies.

The point of this rant; Lets focus on our core strength and something no other project has and that is a community based system. There are still plenty of bugs/UI/enhancements that need work.

That said, I think it is great that java is an option, I think that SOAP needs to be implemented. That is why AOLServer IS an application server (although I could also argue that so is Apache).

Collapse
127: Response to nsjava staus (response to 1)
Posted by Don Baccus on
LAMP = Linux+Apache+MySQL+PHP ... sorry, I thought folks here would be familiar with the acronym.
Collapse
128: Response to nsjava staus (response to 1)
Posted by defunct defunct on
As Roberto correctly says, the real problem is one of presentation.....

We also communicate AOLServer, more as an application server. In fact, if you're up against real PRINCE II wielding, MS obsessed uber-designer types, you can even juts about get away with describing OpenACS as a Multi-tier Application Server ... Arf... always amuses me that one..

Collapse
129: Response to nsjava staus (response to 1)
Posted by defunct defunct on
Actually, just to add to Jon's list of potential targets for OpenACS its also worth remembering the most lucrative/opportune area.

Even aD recognised that the real bucks flow from corporate communities, mainly intanets, but also many other internal business functions, particularly those with a B2B element, such as CRM....

This is a wide open space full of grade one shite products... Remedy, ACT, Lotus Notes..... an so on....

Also (and of course we're living proof of this) the mobile arena (particularly in Europe) is begging to be exploited.

There is a *huge* lack of good server software and content delivery in this area, and given the Telco recession, its a great time if you happen to have an open source, low cost, low risk solution.

In fact, I'm rapidly coming to the conclusion as I think earlier post re-enforce, that OpenACS is *least* suitable for standard web-sites. It may even be worth highlighting this in the new web site.

Perhaps we ought to discourage adoption by people in this latter category. After all they are almost always going to be able to say.. well my LAMP is faster/easier/better blah blah.....

Anyway, not wishing to go off topic, if people want to make a living/cash out of this product, I'd suggest skewing your focus slightly.