Forum OpenACS Q&A: nsjava status

Things like XML-RPC, SOAP, WSDL are already available for Apache, and I'm guessing that it wouldn't be too difficult to make them work on AOLserver/nsjava.

I'm *so* glad you're on the TAB!

As far passing in arbitrary options, nsjava already supports this.  Classpath is hardcoded, but it is also possible to set arbitrary java options using the ns_param JavaOption keyword.  You would do something like:

ns_param  JavaOption foo=bar

ns_param JavaOption "-Dcompiler=none"
ns_param JavaOption "-X"

With just a quick glance, it looks like the jigsaw sever could be integrated with nsjava, but it would have to listen on a separate port.  Even so, there could still be a tie in with openacs since db access is possible from nsjava user-defined java classes.

One thing I'm noticing now that I've started considering various applications to use with nsjava, is that it would be dead-simple to add new features/libraries, if nsjava had servlet support...

So we should focus on finding and using as much existing tcl code as we can before we look for Java solutions. At least for code that will become part of OpenACS.

If a client needs something and nsjava makes it possible to deliver on-time, use it.

1021 nsadmin@localhost:~/webdav >telnet localhost 9999
Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
login: nsadmin
Password: *****
 
Welcome to nsjava running at /home/nsadmin/jigsaw/Jigsaw/bin/nsd (pid 8849)
AOLserver/3.2+ad12 (aolserver3_2_ad12) for linux built on Jan 30 2001 at 20:36:35
CVS Tag: $Name: aolserver3_2 $
nsjava:nscp 1>  set args [nsjava::new {String[]} 2 {-root /home/nsadmin/jigsaw/Jigsaw/Jigsaw}]
nsjava0x1e5
nsjava:nscp 2> nsjava::call org.w3c.jigsaw.Main {main String[]} $args
 
nsjava:nscp 3>         

Which resulted in the following output in the log file:

[20/Aug/2002:23:40:22][8849.8201][-nssock-] Notice: nssock: listening on http://localhost:8000 (127.0.0.1:8000)
[20/Aug/2002:23:40:22][8849.8201][-nssock-] Notice: nssock: accepting connections
[20/Aug/2002:23:40:25][8849.9226][-nscp:1-] Notice: nscp: connect: 127.0.0.1
[20/Aug/2002:23:40:28][8849.9226][-nscp:1-] Notice: nscp: logged in: 'nsadmin'
[20/Aug/2002:23:40:28][8849.9226][-nscp:1-] Notice: registerTclJavaFunctions: registering commands
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: attach to JVM
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: attach to JVM worked
[20/Aug/2002:23:40:41][8849.9226][-nscp:1-] Debug: NsJava_GetThreadEnv: save env 140526272 tid = 9226
loading properties from: /home/nsadmin/jigsaw/Jigsaw/Jigsaw/config/server.props
Jigsaw[2.1.2]: serving at http://localhost.localdomain:8001/
*** Warning : JigAdmin[2.1.2]: no logger specified, not logging.
JigAdmin[2.1.2]: serving at http://localhost.localdomain:8009/                                                       

So at this point I basically have a webserver running inside a webserver. I'm able to get pages from aolserver using port 8000, and I'm able to get pages from 8001 using jigsaw.

I then proceeded to download WebDAV client called DAV Explorer, and voila, I was able to browse files under the jigsaw pageroot. This would need some additional work to make it useful, but using this approach, your probably 95% way to having webdav support in AOLserver without really even doing anything.

http://webdav.org/specs/

talli

 Apache as network server

Dan wrote:

Even so, vfs would be nice if we had webdav support in AOLserver, because you could then remotely mount the CR on your local file system and directly edit content using your favorite text editor instead of some crappy web interface.

Exactly! This would help solve a big part of the CMS problem. I have long believed that a good Open Source CMS solution should use a combination of WebDAV and a lightly customized version of Mozilla Composer on the client side for all content creation and editing, leaving version control, workflow, publishing, and (to a certain degree) templating on the server side. This would greatly simplify the server-side UI design requirements while simultaneously enabling users to draw on the full range of desktop software functionality that they're used to having when they create documents. As a bonus, if users prefer a closed source client-side authoring app (e.g., Dreamweaver), they could do that too and still use WebDAV for seamless integration with the CMS.

If we can leverage code from other opensource projects then, why not? The point isn't that a webdav server couldn't be developed in Tcl, I'm sure it could, but rather that tools like nsjava might provide an easy way to integrate webdav support into AOLserver.

I just don't understand your bias against nsjava. After all, it is an Tcl extension module. In the examples I've posted, I haven't posted a single line of java code. It's all been Tcl. The purpose of nsjava is provide an interface to existing java packages from Tcl. nsjava is a tribute to the extensibility of Tcl. If weren't for the tcl object type system introduced in Tcl 8.0, nsjava probably wouldn't be feasible. Using nsjava does nothing to undermine confidence in our technologies, as it is implemented using key parts of the technologies that we use in openacs. The fact that webdav support is implemented in java is of little consequence, as the interface to it will be in Tcl, and as a developer using it, you wouldn't have to get involved in modifying the java code that implements the webdav server.

I believe, though, that a C module, namely modifying mod_dav, is far more desirable than any of these approaches. The reason being that it seems mod_dav is more supported than any of the tools that have so far been mentioned and many other tools and client will probably be developed to use its features. An example is Subversion which depends heavily on mod_dav and mod_dav_svn. Unfortunately, a link that purportedly explains this is busted, but visit subversion.tigris.org and there should be some info somewhere.

nsjava, though, is a major win, and I would like to thank Dan for taking the technical challenge and delivering. It's great to see you more active Dan!

talli

Talli wrote:

I believe, though, that a C module, namely moififying mod_dav, is far more desireable than any of these approaches. The reason being that it seems mod_dav is more supported than any of the tools that have so far been mentioned and many other tools and client will probably be developed to use its features. An example is Subversion which depends heavily on mod_dav and mod_dav_svn. Unfortunately, a link that purportedly explains this is busted, but visit subversion.tigris.org and there should be some info somewhere.

This brings up an idea that Lars floated in passing on another thread--namely, figuring out a way to be able to tap into Apache mods directly. Is it utterly crazy to contemplate writing nsmod_? Forget about the resourcing for the moment; is it technically feasible/adviseable?

talli

talli

Java isn't "true" Open Source (though programs written in Java frequently are).  We've done a lot of work to make sure that we provide a 100% Open Source OpenACS 4 solution.  Of course, we also provide an alternative based on proprietary Oracle.  My main concern isn't to quibble over whether or not "free as in beer" is better or worse than "free as in speech".  My concern's more practical - can third parties like us bundle up a JVM in a "download an instakit" package, or are potential users going to have to download a compatible JVM themselves?

Because that kind of packaging - download a big chunk 'o stuff from openacs.org, wave a magic wand, and poof! you have a (dotLRN/dotWRK/dotWHATHAVEYOU) up and running - is something we've talked about pushing for vertical apps the community develops.

I don't know the answer to this, which (of course) is why I'm asking.

Then there's also the issue of competion for computing resources on the webserver machine itself.  Heavy use of Java pieces means yet another consumer of resource potentially on the scale of AOLserver and  the RDBMS.  I've also seen some interesting information about the scalability of one JVM due to lock contention issues (shown me under an NDA) so there's another tuning issue to worry about if you're going to run a large-scale service.

Before anyone says "yet another reason to support Apache" let me point out that the same issue exists if you couple a JVM to Apache.

This not only leads to potentially more complicated sysadmin requirements but leaves us open to statements like "if we just use a 100% Java solution life will be much easier".

Having said all this ... the fact is that the idea of being able to rapidly integrate with WebDAV and the like is *extremely* attractive.  I'm not knocking the attractiveness but ... let's think about the possible headaches involved with being too quick to decide that integration with Java services is the right long-term solution.

I don't think there's any doubt that in the short-term this is going to be the right solution for some of our integration problems, though.

If I had to vote today I'd probably vote for a quick push to get TclSOAP integrated (DaveB and Roberto seem very close and I gave them some hints on how to work around the "package requires" issue that might be useful), because it avoids all of the potential hassles that being dependent on having a compatible JVM around might lead to.

On the other hand, Dan's right that having a WebDAV client doesn't bring one close to having a WebDAV server and nsjava seems like the obvious short-term solution to that.  I'm glad he's looking at mod_dav (is the Zope WebDAV support Open Source and not Java-based?  Is this possibly another place to poach code?) because the long-term appeal of this should be obvious, too.

There's no doubt that a lot of Java stuff is out there and there will be more and more as each day goes past.  Being able to rapidly integrate is a big plus despite the potential hassles.

As you say, this is a natural fit for service contract based service packages.  Definitely the thing to do.

Petru was working on it at one point, but I don't think he got too far along. I could be wrong, though.

The issue of including JAVA in OpenACS to some degree or other is fairly close to my heart ;0)

I personally think, despite the few concerns about 'true' open source, that having a strong Java integration/inclusion element is critical, and gets more so every day.

Don and I were chatting the other day, and I recall highlighting that we as a company (OM) have delivered a number of successful ACS based projects, however *none* of them have been actua websites, in their most common form!

In actual fact it turns out that the OpenACS makes a very good general purpose infrastructure/integration platform.

That being the case, I'm sure you can see that access to Java is pretty critical. Many companies are employing Java more and more as their system development language of choice, and therefore unless we have a good story about Java (even if our product isn't) the better placed we are...

So I think Java, as part of OACS, is a must to some degree.....

As for re-development of Java stuff into TCL.... well, its a nice concept, but in reality by doing so we then lose the benefit of ongoing java projects, supported and developed by others.

So in actual fact I'd like to see us resist converting things to TCL for their own sake and leave it as a 'only when its *really* justified'....

Now, if I could just persuade someone to port AOLServer onto Jacal (TCL implemented in Java).... well... we'd really have *the* killer application. A native Java based OpenACS with TCL still remaining as the primary web devlopment tool.... a lethal combination that would surely offer us access to a much wider user base...

Well, I know its a lot of effort, so its just pipedreaming at the moment. This would probably have to something privately funded.... but keep an eye on this space.... I have ambitions to see that happen...

As many of you will know, Java and TCL came out of the same company (SUN) and to an extent the same group of people, and the intention was always that TCL and Java would be complementary tools....

Combining both under AOLServer would surely realise a worthy goal..?

So in actual fact I'd like to see us resist converting things to TCL for their own sake and leave it as a 'only when its *really* justified'....

On the other hand offering the capability to blend easily with Java in order to integrate with less critical bits of technology is really great, yes indeed.

As far as programming out AOLserver dependencies and moving to a pure Tcl-based solution ... why do we want to move to a technologically inferior platform? There's no way we're going to match the AOL folks in building an independent platform that delivers high performance.

Running AOLserver natively under windows isn't an impossibility, there's already an OpenACS site in Ireland that runs natively under windows. Jamie plans to bundle it up when AOLserver 4.0 comes out, I believe. If he doesn't have time then surely the time involved in getting this packaged won't be huge.

The problem then is writing a MSSQL driver for AOLserver. Since MSSQL is based on Sybase and since the AOLserver Sybase driver is essentially an ODBC driver, I don't think this should be terribly difficult either.

And porting the toolkit to another RDBMS is not a trivial undertaking. The work I describe above will be a piddling fraction of a total porting effort.

AOLserver is the most solid piece of technology in our platform, we shouldn't be in such a rush to dump it.

Dan wrote: "I just don't understand your bias against nsjava. After all, it is an TCL extension module."

I don't have any bias against nsjava. I've said it many times that it is great, in several posts here. Maybe I haven't been able to make my intended point come across, perhaps due to a language barrier.

I'll say it again: I think nsjava is great, and it's great to have Java integration, and it's great to be able to give our users that functionality, and I think your work is great, and you are a great hacker. And although I think it's overhyped and misused in some places, I like Java.

What I was trying to say was basically what Don said regarding packaging and integration of important things. Hope this will clear things out.

re: nstcl, it's very nice. I'm suprised to see the surprise of some here who hadn't heard of it yet. I've used it in some instances for stand-alone Tcl scripts.

re: separating OpenACS from AOLserver, I think it's possible, but it'd take resources. I remember a little over a year ago the AOLserver folks wanted to make AOLserver a shared library that could just be emdedded anywhere.

If that happenned, then you could just do whatever with it. Make an Apache module, make a java interface through JNI (?), etc.

re: Jon Sequeira's SQL Server port. I heard you had finished the OpenACS kernel port, which is interesting. Perhaps you would like to write something about what you learned during the data model port (we've talked about starting an OpenACS Systems Journal on the new website).

I read that MS had changed its EULA on the .NET line of products to not allow GPL'd software to be used/developed. I don't know if that applies to the new SQL Server, however.

• No nsvs
• No Registered filters
• No Scheduledprocedures or startup/shutdown
• No internal handling of persistent db connections using shared pool in thread safe manner and not sure if any part of api thread safe being that it is a shell extension
• No comparable nsd config file exists for shell
• No ns_conn structure
• What about modules that are required how would you handle ns_cache, etc.

I think you are mistaken. I didn't say we _NEED_ WebDAV in Tcl or C, I said "it would be better".

And about someone saying "I'll do it", you're right, nobody has claimed it. As in most open source projects, the huge vast majority are just takers. They take the product, use it, don't volunteer to do a thing, and then come to the mailing lists and demand developers to do things for them.

I wish OpenACS was different and we had lots of people voluntarily stepping up for small tasks, of which there are plenty.

I think the lame EULA applied to dev tools.  I used emacs and perl/dbi to do the kernel port,  so I think it's safe 😊.
I'd be happy to write something up for oASJ at some point.

Andy,

Database and webservers are peculiar beasts.  Organizations would much prefer to not support many of them.  Most of the serious environments I've coded in have centralized monitoring apparutus (HP OpenView, CA's TNG, NetIQ, etc).  It's a big deal in $$$ and time to add a new set of infrastructure that has to be monitored and patched.  Way more if you're looking at some type of common fault tolerant architecture (w/replication, failover) in the db tier.  Doing that several different ways gets complicated and expensive quickly.

I'm not saying MSSQL is a better database,  it's just better if you've already bought it 😊  Anyone with a big investment in MSSQL behind their web site would prefer new code and data sit side-by-side,  and not have a new set of integration issues to contend with.  There are a lot of sites like this,  I would argue more than any other platform right now ( I think Oracle wins in $$$, not installed instances,  + MS has a free version of SQL Server )

OpenACS is the best thing out there for a stand-alone CMS or community-based corporate site.  But it has the potential to do a lot more,  if it could leverage the hooks developed by the vastly larger TCL community in terms of data access etc.

As Don pointed out,  the db driver and apache stuff *can* be done,  but my point is that it *has* been done,  and it's never really gotten critical community mass to be supported.  We can keep trying,  a new nsd module,  a new AOLServer port, as Don suggests.  These are useful endeavors,  and will probably succeed for awhile,  but ....  It seems just plain unnecessary to continually require people to fire up their c compilers and keep current with their respective projects external dependencies (apache, sql server, iis, Windows + openacs and aolserver)  over and over again.  I think FastCGI gets us out of this rut.

Carl

THANK YOU!  That kind of feedback was what I was looking for.

FastCGI supports filters, and multiple persistent interpreter instances,  each of which can support a persistent connection.  I *think* that minimizes the threading and pooling issues.

ns_conn shared data is my biggest concern,  but I think FastCGI's session affinity might address it.  In essense,  you're getting the same architecture as a web farm (multiple discrete load-balanced instances),  but at the interpreter level instead of the web listener level.  If you have a performance problem,  you just add more interpreters,  potentially on other machines.  This means that you take a hit having in-memory multiple caches,  but since it's the same issues w/a web farm it doesn't seem too bad to me.

I think ns_cache (is that required?), other required modules (?), config files and scheduling stuff would likely remain platform specific.  That doesn't seem like too big a deal.

*I* am saying that we need webdav, and I've been working hard to support and to get people to support its development. As Roberto said, this is an open source community which also means that things just take longer to develop.

However, I do hope that webdav will be coming along very soon, preferrably as some work that is a port of Apache2 DAV.

talli

You're characterizing my suggestion as dumping AOLServer.  That's not really necessary.  Webware, for example, supports a native apache run-mode (mod_webware),  but also does CGI and FastCGI if you're not an Apache disciple or don't want to run it as a module.  OpenACS could have a reference implementation in AOLServer exactly the way it does now,  but by implementing future functionality in a way that minimized native AOLServer API dependencies not covered by MC's nstcl (or not easily ported to nstcl),  and considering back-ports of portable code into the source tree,  you could make life easy for OpenACS/FastCGI porters to keep up.  This is quite similar to the whole DB-API thing,  now that I think about it,  but I'm pretty sure it'd be asking a lot less.  Unlike data access,  the vast majority of scripts do not call into non-portable AOLServer APIs,  right?  (Non portable as in beyond simply getting query params)

Also,  for better or worse,  FastCGI hasn't evolved in about 5 years,  that I know of.  Their external dependencies (apache., perl, tcl) have evolved,  but you have a larger community committed to supporting it,  and they've historically done a decent job of it.  Plus the run-everywhere CGI->FastCGI bridge probably hasn't needed updating in a *long* time.

On the Windows stuff in general,  communities need to be bootstrapped,  to achieve critical mass and be viable - the usual catch-22.  Pg/Win32, AOLServer/Win32, etc never got there.  Do we need to make a community forming itself around those elements a precondition for what I expect to be a much broader acceptance and deployment of the toolkit?

You're point about it being more work is well taken.  I was hoping to get some help doing the homework of how much work it would be (and I got some),  but I'll go off look into it some more.  My instinct is that Michael's work makes this plan doable,  but we'll see.

The smell here is an awful lot like a bad case of NIH syndrome. :(

Responding to Andrew's connection pooling questions; I ran through a couple designs to poach ns_db code into an external lib; currently I'm leaning towards building on top of libdbi to start from a layer that doesn't have Tcl_DString types through everything. I plan on using such for both Apache2 and Gauche, which is a Scheme with a lot of overlap with scsh, but does Unicode and native threads. On some layer I want to build a similar system to UncommonSQL, a Common LISP database interface, which has the best OR mapping system I've ever seen, as well as the only decent query writer, and add some extra intelligence to the drivers to automate the type casting. Then roll that back into [mod|ns]_scheme, replace all this Tcl nonsense, and watch my lab by destroyed by OACSers rampaging with torches and pitchforks... oh, excuse me, I seem to be getting ahead of myself here.

I will post the details in a new thread, but here's the gauntlet: I will be writing an nsd module to handle WebDAV, with extensive poaching from the DAV testbed, mod_dav.

I'm going to avoid the rest of this thread.

Heh... when you argue with me, Don, you take the position that Windows is a first-class platform to OACS. When you argue with John you say it's not. In the same post. Which is it, really?

1. People talk about wanting to support windows as a first-class platform. This desire seems real.
2. Despite all the talk, no one has done the work of making windows a first-class platform. Talk is cheap ...

It's not really a contradiction to argue that we want it to be a first-class platform in the future while pointing out that at present no critical mass has formed around the idea. It's not the technology that's in the way, it's the fact that the vast majority of us here love Unix and are only interested in Windows as a way of widening acceptance of our platform. Will we reach critical mass if we don't have a significant number of community members who love Windows?

Michael ... as far as operating system support, we support Linux, Solaris, FreeBSD, HP-UX, SGI, OS X (eventually with Oracle as well as PG) and using Jamie's work and either Oracle or the upcoming PG native Windows release, Win2K/NT.

All natively.

So we have all significant platforms covered, don't we?

I'll admit to a bias. I hate the notion of adding extra complexity, not to mention push a technically inferior platform to clients, to fix a shortcoming in marketing. Why not get better at marketing, instead?

The first version of nstcl came about because I found the pgtclsh database extensions interface to be so much more painful than the AOLserver database api's that I'd come to know & love over the previous three years, so I created wrapper procs to emulate the AOLserver API.  From there it grew to support Solid, and then Oracle, plus other things like ns_sendmail, ns_hrefs, etc.

With the latest release came templating, a general code cleanup, actual documentation (I was getting sick of Kriston redesigning the AOLserver.com site and breaking all the existing link/URLs every couple of months), a test suite, and more supported database extensions.

I don't have any experience using a threaded version of tclsh, so I can't even begin to comment on the thread safety of nstcl.  In general I think it should be no more and no less than any other library of pure-Tcl code.  Another question would be the underlying thread-safety of the database extensions themselves, I suppose.  IIRC Oratcl 4.0 claims to be thread-safe now, whereas previous versions weren't.  I have no idea about pgtclsh and the others.

John emailed me and asked if I saw any obvious reasons that would completely rule out it's use with FastCGI.  I don't see any issues that are insurmountable.

Obviously nstcl would need all the AOLserver API's related to HTTP & HTTPS requests/connection handling.  My working version of nstcl 1.1  adds some additional non-connection oriented commands: the ns_schedule_proc and ns_config/ns_param set of commands, for instance.

I hope that anyone who decides to persue this path would _fully realize_ the speed difference between a C implementation (AOLserver) and anything written in pure Tcl (nstcl).  That said, however, for low-end/low-volume sites, I imagine speed wouldn't be an issue.

BTW, there's at least one area where nstcl already outshines AOLserver/(O)ACS: migrating data from two different types of databases (Oracle to Postgres, for example), because nstcl's database api doesn't force you to fall back on the AOLserver ns_db api for the second database. :^)

I think we have some negative feelings about Java due to aD switching from Tcl to Java.  I hope this gets reduced.

On my personal experience I believe that my learning curve to CCM was made even harder due to the negative feedback on this community about it.  I am not defending CCM, all I am saying is that its not as bad as the posts that I have seen on OpenACS.

I will post some nice stuff that I have found about CCM.  In case you think its not good for you then its not.  I am posting it here because I think it was good for me and someone might see that its good for them too.  Of course CCM is still less productive for me than OpenACS.  But there are the nice things about it.

- Given the fact that I started to really learn CCM on my own and had nobody to turn to (no bboards) I was able to learn it more or less.  Since Java had some tools and I was able to pick up one.  Which is eclipse and I was able to look at the whole code base with help from an IDE.  I think I got more mileage from an IDE than if I just used emacs and grep.  So as a newbie I had more tools and help from other non CCM based but Java based resources.

- Since CCM is in Java there are some tools that helped me like the debugger.  It sure helped me see what is really happening to the code.  In fact I recently downloaded tclpro just to see if I can make it run and use a debugger.  Ofcourse since tcl is so simple ns_log and printf debugging is something that is good enough.  But for some reason I miss having a debugger.  I guess its been a long time for me since I used a debugger in C, C++ and Java.

- Even though CCM is thicker than OpenACS since its very hard to access the database directly (there is no db api aside from using jdbc).  But its actually closer to the data model of ACS.  For example when you get a new User and save it it will actually insert rows on acs_objects and users table.  Its because the java class User inherits from acs_objects java class.  So the Java classes in CCM is closely mapped to the data model.  So if I inherit User into FooUser in java.  I then create a foo table on the db.  Making a new instance of FooUser will put rows in foo, users and acs_objects table.  Also by getting FooUser(object_id) it selects that user from the database.  Changing the attribute of FooUser.name actually corresponds to the column in the db.  You can also define methods that needs custom sql queries like foo.getAll for performance reasons.

- Since things are more rigid and less flexible there is a more uniform way of doing things.  I guess its because you are more retricted.  Being a beginner that is a big plus for me because after a while there is a pattern that develops.  Of course once you are an expert it might hinder you... I don't know since I still consider myself a beginner in CCM.

- Since the java classes closely models the data model and have a meta data associated to it.  The create, delete, insert, etc. done the same way.  There is always a retrieve and save method.  Much like what OpenACS have in create and delete plsql.  But this is done on the super class Data Object class.  So Users, etc. does not need to implement this stuff unless they need to.  Since things are more or less uniform there is already a move to use the metadata to create the tables.  So porting to a new database may become trivial in the future.

- its nice to see that CCM uses a lot of java libs for their xml, crypto and authentication.

On some extent the restart of server everytime may not be true.  Since this was always been posted on the bboards I did not do to much work on it.  But recently I tried to use hot swapping and class reloading.  So while in the debugger I was able to change my code without a restart.  Of course big changes of code needs a restart.  But I think more advance webapps like Websphere is able to handle this.  I guess this is the one of the things you pay for to IBM.  But I am not sure about this as this info pertains to other Java code.

To summarize, CCM has still a lot more problems but its not as bad as what most people think of.  Of course OpenACS still have the general advantage.  Please let us try to look at CCM in a better light.  Also it may not be the failure of aD has a lot to do with their decision to use Java.  The same way that it is not true that if aD was successful a lot of it will attributed to Java. Of course the marketing heads will screw a lot of the truth.

I hope I did not move the discussion to any side my intention is to put it more in the middle ground.

I do not in any way shape or form, recommend or support any move from AOLServer

AOLServer is what brought me to OpenACS in the first place (so much for having poor marketing value).

AOLServer is still the best thing out there, when one considers all aspects of the function of a web server

TCL is an excellent tool for web development, and again A primary motivator for us.

All I really meant was that the OpenACS should offer a way to intergrate Java components/legacy systems, that is simple and clearly documented, and that we should highlight this capability.

All this talk of moving to other web server etc is counter-productive.

Surely we'd all be better off just imporving AOLServer itself?

Cheers

Simon

The OpenNSD project was started about a year and a half ago to try to address some of these issues;  unfortunately it ran out of steam.  It was born out of frustration with the folks at AOL, who were very rude and condescending to community members who dared to contribute patches or even point out bugs.  But the very act of creating OpenNSD was apparently the kick in the pants they needed, and attitudes improved which took away some of the urgency behind the project.  Then we started thinking we should wait for 4.x to come out... who knew it was going to take this long???

It might be worth considering reviving that project, if only because the name OpenNSD a) doesn't mention AOL and b) makes it clear that it's an Open Source project, not one dependent on the whims of a large corporation.

I too have inquired as to the status of OpenNSD on a number of occassions over the past year or so and there seems to have been a complete lack of any interest in moving this project forward. I feel that as good as AOLserver is, there is still a great stigma with having the letters A-O-L attached to your webserver of choice.

I for one would love to see the OpenNSD project go forward so that a great technology would have the chance to see the light of day against those who are too closed minded to get past the stigma.

I've done some work to keep the Solid database driver up to date as we use it here at my place of employment and I've never gotten any response out of any of the AOL folks with regards to patches I've submitted to them, and last I knew they had dropped support for Solid altogether. I would like to keep that driver maintained against an OpenNSD webserver base personally.

Remember, Scalability (inferiority?) means speed (i.e. c vs. TCL) _and_ throughput. FastCGI might lag in the former relative to AOLServer, but I believe middle-tier scale-out gives it a clear advantage in the latter.

Besides which, the FastCGI model gets its fast throughput compared to the mod_perl approach primarily for one reason. Here's a quote from the paper you reference:

Globally initialized objects, for example database connections, are thus persistently maintained.

So the benchmark is to a large extent measuring the performance impact of implementing a persistent connection mechanism.

Which AOLserver has built in, of course.

I believe there are persistent db pooling APIs for Perl these days, no? The benchmark would be far more interesting if the FastCGI and mod_perl examples both used one of these modern perl db APIs.

One other area where FastCGI wins over mod_perl is that the script is only compiled once (this is why the db connection is only built once). AOLserver caches the compiled bytecode from libraries and script pages therefore has a similar advantage over mod_perl ...

This does tell me that a fast-performing mod_nsd would have to implement connection pooling (not just the db api) and would need to cache compiled bytecode just like AOLserver does if it would have any chance of performing equivalently. Adding to the complexity of the task.

BTW I'm ignorant of mod_perl so am basing my observations on the paper you've so kindly referenced. If the author's wrong, so am I, I'm just assuming he knows what he's talking about. Based on his comments, though, I see nothing to suggest that the FastCGI approach would yield higher throughput than a properly written AOLserver application.

Yeah - the bytecode compilation will be a big issue. This is biggest porting issue I've identified, and probably beyond my TCL-hacking skills. However, I am in touch with David Welton, author of mod_dtcl (which caches bytecode, as does mod_perl.), who offered to help me with this. I may have to forego FastCGI short term and just stick with mod_tcl for now if bytecode-caching stretches my scripter-brain too much (even with David and Michael C's help.) The rest of the issues (nsv's, adp, filters, scheduled procs, connection pooling) aren't too bad or have prior art to pull from that I believe does fall within my abilities.

Listen, you've been hammering away on performance, but I maintain that raw performance is a straw man w.r.t. a portability strategy. It appeals only to (some) coders, not the people you'd want to target with a refined marketing campaign. I studied engineering in school, and can appreciate well-architected software as much as the next guy. But, seriously, how many times has anyone been in a meeting where somebody says, Darnit I'd like to go with that apache|iis|netscape web server, but it just doesn't do enough pages-per-second for my taste. Hands up, how many current OpenACS users actually *need* it's capacity to fill a T-1 on a $1K modern server? I worked at a site last year with 10 million unique visits a month running on Cold Fusion 4 and IIS. Awful, ugly, terrible architecture, but moving to something else so they could have a fractionally smaller web farm was pretty low on their list of priorities. I bet that's true for almost all sites with any sort of legacy dynamic code (it has been in every case I've seen). OpenACS is a great sell if you've got no legacy code/db's to integrate with. Isn't that everyone's experience when they go pitching it to clients? A portability strategy that addresses this would be a big win.

From my blog:

Also, the longer-term back-of-my-head plan which I'm guilty of not voicing more clearly is that I *want* a simple, unified, high-performance AOLServer stack to remain the preferred platform. I want to be able to deliver to clients a solution that runs on their preferred infrastructure, but has compelling benefits when run on it's native reference platform. That's exactly the kind of platform advocacy Microsoft is pursuing by pushing Rotor, and not-yet-suing Mono. They know both are going to suck, where suck means lags the Windows version by a good two years. But they can point to it, and it will allow them to attract a few fence-sitting *nix types to their camp. And maybe convince a few universities that they can write courses about their stack without making MS endow too many chairs.

People will build on Mono. Their code will work. And the clients who use that software will become potential customers of Microsoft's .NET Server stack (where the real money is).

I view the OpenACS/FastCGI project the same way. The customers that adopt it, who aren't ready to rip-and-replace their current infrastructure, will have one more excellent reason to consider doing so in the future.

I think Petru's AOLServer-as-FastCGI server would be fine at a lot of places,  most in fact.  I'm not arguing that mod_crummy would be better technically 😊.  There are two reasons why what I propose,  while more work for less performance (*ugh* gotta work on that sales pitch),  is desirable from a big-picture perspective:

The AOLServer dependency limits you with respect to platform (db/os) support and tcl standard package support.  A c-based fastcgi->nsd bridge still has a bunch of dependencies that come along for the ride,  which the community hasn't historically been able to maintain.

If you want to bring OpenACS to the Apache and Win32 communities,  I'm arguing that you'll get over the hurdle of convincing people you'll be around for the long haul if your only portability dependency were TCL,  you had native TCL db support,  and you had the exponentially larger TCL community supporting you.

I talked to David Welker (hope I spelled that right) at LinuxWorld, and they (the Apache Tcl project) were interested in hooking up AOLserver as an application server to Apache.

So as far as the Tcl community supporting AOLserver, I don't think it's fair to imply that AOLserver does not have the Tcl community support (maybe Michael (Cleverly) will want to comment on this since he follown comp.lang.tcl very closely).

1) Bugs are likely to be fixed more quickly

2) We're likely to get access to things like DAV, SOAP, and whatever the next cool technology is going to be.

3) It might make it easier to run OpenACS alongside some other technology, something that I believe people will want.

4) Running under Windows

5) Easier installation, because people are likely to have Apache running already.

That's what I'd be interested in, anyway.

/Lars

And I also notice you avoiding mentioning all the major *advantages* of AOLServer that we'd essentially lose!

At the risk of speaking for Lars, that not our bag, baby.

Let me give you an example that might help out. I've written my last few big projects in perl because it ran where I needed it to and it talked to the db's I needed to talk to and it was quick to develop in. In perl land, code can be easily ported between plain old CGI.pm (the 'gold standard'), CGI::Fast (FastCGI), ASP on Apache *and* IIS (using ActiveState's Perlscript w/Win32::ASP and/or Apache::ASP) and mod_perl.

The existence of all the other deployment options in no way threatens CGI.pm. It made it healthier, in the way that open source projects with more eyes focused on them are always healthier. mod_perl did not threaten CGI.pm, or cause CGI.pm adopters to lose benefits, did it? Even borg-ifying Perlscript didn't threaten it. CGI.pm folks just went on making their stuff extremely useful, and the rest of the deployment options that adhered to its API just kept on benefitting. I see the same benefits for the OpenACS community. [ Aside: Does anyone not want the ability to deploy your application at a plain-vanilla ISP? ]

Also, something else to think about: beyond writing scripts that can be deployed pretty much wherever I or my client wants webserver/os/db-wise, perl lets me easily write scripts that I debug from the command line, using a line debugger. For some problems, I even fire up OpenIDE, a visual debugger.

Pseudo-nsd (tks, Michael) will provide the same benefits. It will make writing installers, packaging OpenACS apps, and debugging them a lot simpler, even if the deployment target is the gold standard, AOLServer.

I understand you point, but... well its got nothing to do with OpenACS or AOLServer......

I don't have your requirements. I write Db-backed stuff that wouldn't run a aplain vanilla ISP anyway, and if you really want that level of simplicity, what do you want AOLServer/OpenACS for anyway.....?

But when I read things like this ...

[ Aside: Does anyone not want the ability to deploy your application at a plain-vanilla ISP? ]

I see an argument for rewriting OpenACS as a LAMP application. Simon hints at the difficulty above, i.e. when it comes to running OpenACS at a plain-vanilla ISP, AOLserver is not your only problem. How many plain-vanilla ISPs offer Oracle for $19.95/month? Most only offer MySQL (I personally would shun any ISP that forces me to use Windows as the platform). Your plea to migrate from AOLserver will be followed soon by one to support MySQL if you follow your argument to its logical conclusion, because most ISPs won't run PostgreSQL. At most you're going to be given a choice between LAMP or 2K/IIS/MSSQL or perhaps 2K/Apache/MSSQL.

I co-lo my own box for $100/month and have full control over it at my vanilla ISP. That's starting to look overpriced, Cathy Sarisky is offering hosting on a remote box that costs her about the same but which includes rebooting and other low-level service. And she didn't have to buy her own box like I did.

If I have a client that can't afford that kind of solution, I know they can't afford me. So, frankly, no ... I personally don't care

And I don't see why anyone would.

Let me give you an example that might help out. I've written my last few big projects in perl because it ran where I needed it to and it talked to the db's I needed to talk to and it was quick to develop in.

OpenACS is not and never will become a panacea for every web job out there. The push towards vertical apps like dotLRN is in fact a push away from the vision of being all things to all people. We've been focusing effort on making OpenACS a great platform for developing applications like dotLRN.

The great interest that dotLRN is triggering tells me that this focus is one that might play well in the marketplace.

Meanwhile, what message is made if we implement mod_nsd or some other hack to get rid of AOLserver.

Even the OpenACS people realize AOLserver sucks, because they're trying to abandon it for Apache?

Really ... if we're going to abandon rock-solid, easy-to-install technology like AOLserver, why not just go the whole nine yards and rewrite everything as a LAMP application? We'd be immediately far more popular than we are today. If ultimate popularity, not technical excellence, is our goal, why not just choose the most popular platform out there and run with it?

Let's first segment the universe into a couple of buckets:

   1. People who've seen the light & run OpenACS (means they run AOLserver already)
   2. People who run the worlds best webserver (AOLserver) but don't currently use OpenACS
   3. People who aren't running OpenACS, aren't running AOLserver, but are willing to switch to AOLserver to run OpenACS
   4. People who aren't running OpenACS, aren't running AOLserver, and for whatever reason (real or imagined) aren't (yet) willing to leave foo brand webserver
   5. People who aren't running OpenACS, aren't running AOLserver, and would never want to 

So the real issue, given that those in group four above are the only who would be impacted by some sort of kludged-up AOLserver platform, is "how badly do we want these people?"

Put crassly, "how much are we willing to pay to get these people?"

Where the resource being consumed isn't necessarily money but volunteer developer resources as well.

Meanwhile, we have folks like Ben Koot pointing out that in many places OpenACS 4 still sucks. (my words, not his, but it's an excellent example of the lack of polish in OpenACS 4).

We're arguing about whether or not to devote resource to something like a brand new mod_nsd while the toolkit itself still is absolutely horrible in some places and moderately messed up in many others? I'm having a real problem understanding the prioritization that this implies. We need to sharpen our focus, folks, and fix it before we spend too much time worrying about where it will run.

However, that only emphasises we need to do a better marketing job... and as Don points out *the* best way to do that is get the current OpenACS working well....

Beleive me... when it comes down to it, if I can developer faster, better and with more stability and give my customers what they want... well.... the windows/apache etc argument just sorta disappears!

Why does everyone in computing these days keep banging on about conversion and symmetry? Has anyone out there forgotten that these are 'web sites' were building? Given a user can't tell the difference from their perspective then why the over focus on platform????

Anyone who turns down a competitive bid simply because they 'like windows and have never heard of AOLServer' is a pillac anyway!  And I'll be jiggered if I want to spend any time making their life easier (sorry... personal rant)..

And... aren't we on the verge of the new web evolution? i.e. Web Services? Aren't we all supposed to be moving to a heterogeneous environment of cooperating services and common interfaces?

These platform/software/porting argument belong back in the early nineties......  Can't we settle for an OpenACS that does what its supposed to do, but which retains facilities for integrating other technologies on an 'as-needed' basis. (this is pretty much what we have now.

If you really enjoy this kind of thing from a purely pedagogic or explorative perspective thats ok too...

Buty to be honest thats a fairly hackery thing too.... everyone wants to play with different stuff cos its a damn site easier than actually doing the basics...

There's a good (paraphrased) Japanese saying...

'Anyone can draw a Demon, but drawing dogs is incredibly hard'

All I'm saying is that the community doesn't *need* what your proposing, nor is there very meuch general benefit.... therefore (other than for your own interest) whats the point??

Just think how much discussion we might have got done on Testing for example ;)!!! Far less intersting, but much more important.. (testing is the dog in this one ;)

That being said, I think John and Michael C's ideas are good for a few reasons, partly because I have been speaking with John a lot about it offline so I understand his needs better but also because I think it will be a very useful experience even if the tool isn't all that's cracked up to be.

They are both long time members of the community and contributors. Michael's been around since the beginning and even turned Roberto on to the ACS. John has contributed the VMware install and has worked extensively with the workflow package. He will be contributing a paper on his work eventually and also a package for document peer review.

What I think that John is looking for, and I think that some other users and potential users may appreciate, is using the datamodel and code that is present in the oacs within the context of legacy systems or new applications where the platform has been specified even *prior* to the application specifications have been written down.

Now, I agree with Simon that this is the worst case and it's reasonable to reject those clients out of hand. But it's also reasonable to accept them if they are willing to pay you full cost for the death march :)

John's initial suggestion may have been a bit too harsh, but I think that probably everyone has thought at one point or another that it life would be easier if we were using Apache rather than AOLserver, if only from a marketing perspective.

I also think that John's idea of a tool that works but is not the canonical tool is a good one. Some people might use the nstcl stuff with apache or IIS, realize that they are only getting 20% of the functionality and administrative ease of AOLserver and will be interested in switching. This could bring in much new blood to the community.

So I think that this project is very worthwhile, but I do agree with Don that it should remain peripheral to the core community at least until it proves that it will have support. If John and Michael build the tool, it works *and* they or others maintain and support it, then it can become a piece of the canonical distribution. Until then, we should consider it valuable and worthwhile community R&D.

talli

And (see my above posts about resource) best of all someone else, not us, is doing it.

Of course then we're still vulnerable to the argument that "LAMP folk don't need mod_nsd, and mod_nsd is AOLserver based, and AOL is evil, and no one supports mod_nsd, and ... and ... and ... you're still using PostgreSQL, and almost no one uses that!  If you're not using MySQL you obviously suck!"

Really, you folks who are so concerned about this issue should just pony up the bucks to get a LAMP port.  Then no portion of our platform will be subject to market pressure from the hordes of LAMP implementors, most of whom have no money and websites that suck.

If OpenACS can:

- use users of current corporate systems. Most likely via ldap

- thick application like Word can save to CR. Most likely via webdav

- OpenACS can offer its application via web services. Most likely via SOAP, WDSL, etc. 

And I think we can incorporate many of the basic toolkit improvements (and out-and-out fixes) people have been talking about in the same timeframe.

And we'll have AOLserver 4.0 for windows bundled in that timeframe, as well, I hope.

And hopefully libnsd will be a reality then, too.

And we'll have had six months trying to push dotLRN in the education world, too, and we'll have a much better understanding as to whether or not AOLserver is the major barrier to acceptance there, or in the Windows world, etc.

Six months from now libnsd would potentially make writing a mod_nsd much easier, we'll have a much more solid toolkit that integrates with at least some popular outside services, and all around be on a much stronger footing than we are today.

At that point in time it might well make sense to divert resource to bundle together a mod_nsd from libnsd. Then again, we may learn that AOLserver isn't the marketing horror story folks keep telling us it is. We should be in a better position to make an informed decision.

For me it's primarily a matter of prioritization, timing, and having a better understanding of what we really need to do to get our work more widely accepted.

I still think that while it is great that everyone wants OpenACS to take over the world the reality is it never will. It does something extremely well and most of the world doesn't care.

You have to pick your battles AND pick your clients. I would never try to convert a LAMP shop to OpenACS it is to much work for little return. If they come to me fine, I hate trying to convert people as it usually ends up in a huge waste of everyone's time. Apache is a general tool that has been hacked to do anything and everything. They don't do it great, but they do it OK and that is all that matters for the majority of people.

Besides, what percent of the LAMP folks have any budget to pay me? If they are using LAMP they don't have a scalable platform and most likely don't care because the 15 hits they get a day don't stress anything.

MS shops are a little easier to target only because the security is so bad even dumb CEO's don't want to deal with that. They are still a tough sell, just not as tough.

There are plenty of places AOLserver and OpenACS work well and those are the people that are running a community. Your bank doesn't (probably) care about community so they run java apps. Mom & Pop Ecommerce sites don't care about community they run LAMP.

Who does that leave? Governments, NPO's and some other enlightened companies.

The point of this rant; Lets focus on our core strength and something no other project has and that is a community based system. There are still plenty of bugs/UI/enhancements that need work.

That said, I think it is great that java is an option, I think that SOAP needs to be implemented. That is why AOLserver IS an application server (although I could also argue that so is Apache).

We also communicate AOLServer, more as an application server. In fact, if you're up against real PRINCE II wielding, MS obsessed uber-designer types, you can even juts about get away with describing OpenACS as a Multi-tier Application Server ... Arf... always amuses me that one..