Forum OpenACS Q&A: ANN: Pure TCL OpenACS

Posted by John Sequeira on
I wanted to announce the Portable.NSD project - an attempt to rewrite OpenACS' AOLServer dependencies in pure TCL based on Michael Cleverly's nstcl library. From the project web page

The portable.nsd project will all the OpenACS source code to execute outside the AOLServer environment. We will achieve this portability be redefining the AOLServer TCL API dependencies to support different deployment containers. The container responsibility is to map AOLServer functionality to functionality offered by the environment. Deployment environment include tclsh, tclhttpd, FastCGI, and Apache 2.0 (mod_tcl and Rivet).

The AOLServer functionality we need to implement includes the tcl api, registered procs, scheduled procs, registered filters, and global state (nsv's). AOLServer initializes a single interpreter and then clones it to perform it's multithreaded request processing. Our initial approach will be to treat the application as a single-threaded one, and ignore the issue of multiple interpreters, and the separation of the initializaton and request processing. Eventually we'll figure out some way to deal with singleton initialization behavior and page serving behavior (See the Presentation for a proposed solution).

Note: Portable.nsd is not an attempt to replace AOLServer's role as OpenACS' native deployment environment. It is intended to provide viable options to prospective users constrained to web servers, operating systems, or single-threaded TCL packages not directly or sufficiently supported by AOLServer. The intention is absolutely not to fork OpenACS but to provide an emulation layer that supports 100% of the current and future OpenACS codebase.

Right now, I've implemented nsv_ and ns_config functionality as globals, and I've implemented enough of ns_xml to successfully parse xql files using tDOM. I am currently working on a pure-tcl pg_bind function.

I'm doing the bulk of my work on the bootstrap code using TclPro's visual IDE ( screenshot ), and I'm finding breakpoints and watch variables really useful. I hope my next generally useful deliverable to be a set of scripts which enable 'headless' installation (i.e. no browser required) of OpenACS.

Posted by Ben Adida on

This is very cool. Having more options is always better. You seem to have a
lot of the pieces under control. The only advice I'll give at this stage is not to
ignore the multi-threaded aspect for too long. Certainly tackling the memory-
shared functionality (nsv_ and ns_config) is a solid first step.

Looking forward to seeing how this progresses!

Posted by Dan Wickstrom on
Speaking from some bad experiences in the area, I wouldn't ignore the multi-threaded issue.  It's much easier to incorporate multi-threading from the start than it is to go back and retrofit it later.  As an example, Tclblend was originally implemented as a single-threaded app, and I ran into some really ugly problems when I tried to integrate it in with aolserver.  In the end, it turned out to be easier to rewrite the core functionality than it was to integrate tclblend with aolserver.

In any event, I can't wait to see how your work turns out.  I've always wished that I had a tclsh based interface to openacs.  This could turn out to be quite a useful tool in many respects.

Posted by Andrew Piskorski on
John, nearly all the nsv_* and ns_thread work you need to do, plus more, has already been done. Check outZoran Vasiljevic's 2.4 release of the Tcl Thread Extension:

I looked over the Tcl Thread Extension docs and release notes back in August, but did not actually review the code or try it out. Basically, it sounds a lot like AOLserver's ns_thread and nsv souped up, made more general and powerfull.

E.g.: "thread::send passes a script to another thread", which is something I currently do in AOLserver using ns_cond and an nsv to hold lists of Tcl code to eval. All the tsv:* thread shared variable stuff, like nsv_* but with additional usefull atomic commands like lreplace, lpop, lpush. It apparently does some fancy manipulation of Tcl 8 objects for the sake of efficiency, rather than only using strings. Etc.

And, the release notes say it supports use both as a tclsh and an AOLserver 3.x loadable module. Sounds fabulous!

As far as I know, no one is actually using the Tcl Thread Extension in AOLserver yet, but it would make sense to do so, and I'm told the AOL crew thinks that might be feasible and/or good somewhere down the road.

And of course, for your purposes, emulating nsv on top of the Thread Extension's new tsv:: stuff should be trivial, as tsv does everything nsv does plus more.

Posted by John Sequeira on
To Ben and Dan,

Points taken.  I'm really encouraged that you see something useful dropping out of this.

Multithreading support is not on *my* short list,  because I mostly just want to get to Apache/IIS support quickly and I don't need it if I rely on FastCGI.  Plus ( more importantly ) it's not really in my skill set.

However,  it is both on Michael C's short list and his skill set,  because he has TclHTTPd as his primary deployment environment. If my writings about the project make it seem like your cautions aren't being heeded,  consider it my own bias creeping in.


Thanks for the pointer.  Michael is aware of Zoran's work,  and will be using it as one of our deployment options.  The idea is you set a flag somewhere in your config file,  i.e.

DeploymentEnvironment  (mod_tcl|tclhttpd|fastcgi)

and the wrapper scripts we're writing will take care of mapping aolserver-isms into that environment.  We're targeting singlethread,  multithread and multiprocess at the same time - I'm roughing out globals-based commands just so the code runs in tclsh and Michael's doing the polishing necessary to make an alternate set of commands which will run in more demanding environments.

Also,  I should note that we certainly hope to leverage AOLServer's unbundled libraries at some point (the one's you pointed me to),  but we won't be letting AOL's release schedule slow us up.

Posted by Ola Hansson on
I hope my next generally useful deliverable to be a set of scripts which enable 'headless' installation (i.e. no browser required) of OpenACS.
Peter Marklund of Collaboraid recently wrote a collection of scripts that installs .LRN and OpenACS on Oracle, non-interactively with the help of tclwebtest. Both very cool stuff IMO!

Maybe this is what you want?

I have have added two switches (--postgresql and --oacs-only) to his script so it works on PG and so you dont have to install .LRN if you don't need it.

I've emailed a tarball with my modified version of his scrips to Peter. Let me know if you want a copy.


Posted by russ m on
I may be totally off-beam here, so feel free to slap me if I'm
talking nonsense, but I was under the impression that one of the
goals of AOLServer 4 was to seperate out the AOLServer
application API (libnsd) from the http server. If this is the case,
surely effort would be better spent in moving OpenACS from
nsd331+ad13 to nsd4, and looking at ways to use libnsd from
within the other environments in which we want to run OpenACS.

This is all assuming that AOLServer 4 with a seperate libnsd will
be usable "soon"...

Posted by Talli Somekh on
Russell Muetzelfeldt: "I may be totally off-beam here, so feel free to slap me if I'm talking nonsense"

SLAP! ;)

Just kidding. There's a lot that is worthwhile in what John and Michael are doing. For one, libnsd will be a great thing but it's not quite there yet. In addition, their have preliminarily moved off of nsxml and onto tDOM, got things running in tclsh, etc.So for the pure academic pursuit of building a completely tcl-based system, it's truly worthwhile.

Most likely, it will also continue to be worthwhile even if libnsd comes out as one can imagine how much easier it is to add to and manage a tcl library than a C library, even though JimD is amazing.

So it's energy well spent, IMHO.


Posted by Andrew Piskorski on

Remember back on Aug. 28 I mentioned the AOL crew's apparent plans to refactor much of AOLserver into libraries? Well, here's another item on that from the AOLserver ChangeLog:

2002-10-14  Jim Davidson 

  * include/ns.h: 
  * nsd/config.c: 
  * nsd/tclinit.c: 
  * nsd/tclcmd.c: 
  * nsd/tclsock.c: 
  * nsd/main.c:  Added support for to be loaded 
  into a standard (thread enabled) tclsh or linked into a 
  custom tclsh.  Calling Ns_TclInit from a custom tclsh or 
  loading via the load command will add many 
  non-server AOLserver commands (e.g., adds ns_set but not 

Posted by John Sequeira on
Thanks Andy... pretty darn cool.

I've got some stub functions set up to handle the server commands that this library won't be providing,  and I plan on writing some unit tests to go along with them,  so hopefully my project will still be useful in transforming openacs to other environments when is released.

Posted by Andrew Piskorski on
For those interested alternative web/application servers like TclHttpd and the like, another interesting little option is TWS, from D. Richard Hipp, creator of SQLite. I noticed it due to Nov. 2005 discussion on the SQLite list.
Posted by Guan Yang on
Fwiw, many moons ago I created a Windows port of TWS (based on an old version of TWS).