Forum OpenACS Q&A: portable.nsd updated

Posted by John Sequeira on 03/12/03 01:29 AM

I wanted to update everyone on the status of portable.nsd. I'm happy to report that I've reached the point with the code where more things work than don't and the useful parts are pretty solid. It's works well for running arbitrary OpenACS code outside of AOLServer and should be useful for automating installation and testing. As an example, A script to install OpenACS looks like this:

source pnsd-init.tcl;  # read config file
pnsd::source_openacs
apm_source "$::pnsd::root/packages/acs-bootstrap-installer/installer.tcl"
apm_source "$::pnsd::root/packages/acs-bootstrap-installer/installer/auto-install.tcl"

The web page generation code works so-so, because the AOLServer API is just too broad and interacts with the templating library in subtlely different ways than pnsd/nstcl. For most new pages it works, but for pages that were written years ago, to different standards, they often don't. The release notes contain a list of known limitations.

I believe the biggest barriers to serving up pages via tclhttpd or Apache are currently thread safety and garbage collection. These problems should go away when pnsd can replace the OpenACS dependencies on global arrays and nssets by a slightly more stateless mechanisms (maybe autoload'ing commands and caching nsset's in the database). If someone with C coding skills were motivated to see this happen sooner, I could use some help updating Tom Poindexter's C-based FastCGI extension to work the latest tcl and fastcgi libraries. I'm reluctant to go this route because it requires a compiled extension, but it would be very slick and should have performance in the ballpark of AOLServer.

My original goal was to allow OpenACS to run easily in other web servers. After working on it a bit, I'm pursuing the slightly less ambitious goal of enabling OpenACS code to be easily ported to other web servers. More specfically, if an OpenACS package author want to write code which ran outside of AOLServer, I hope in the future to define a set of guidelines to make the porting process a trivial one.

This release adds query dispatcher support to nstcl, includes a tDOM replacement for ns_xml, and has prototyped tclhttpd and apache/fastcgi implementations.

Feel free to download the latest code and let me know what you think.

2: errata (response to 1)

Posted by John Sequeira on 03/12/03 06:52 PM

To speed up testing, I've cached/serialized the XQL parsing step by reading the queries from a single big data file (xql.dat), and didn't include this in the distribution. So you can either bypass this and parse XQL each time, or create your own xql file. To bypass, uncomment this line in pnsd-init.tcl:

set LOAD_XQL_FROM_CACHE 0

I'll document creating your own xql cache ASAP, but if you look at the code around LOAD_XQL_FROM_CACHE you should be able to figure it out. Sorry for that.

One other helpful example. This code will load the home page... logging in, checking permissions, compiling the adp template, executing the tcl data source file, and echo'ing the HTML to stdout while generating an execution trace to a separate log file. It will run from TclPro:

source pnsd-init.tcl
::pnsd::source_openacs
set ::pnsd::url /
set ::pnsd::querystring ""

#sign myself in as sysadmin
ad_user_login 2537

rp_filter "Because I said so"

#Trace execution of the handler
package require Tclx; cmdtrace on  [open cmd1.log w] 
rp_handler

#echo html to stdout
pnsd::write_html

Also, I'm currently working on converting a client's existing ACS 3.x learning management system (LMS) to run on Apache+pnsd+OpenACS 4.x. The licensing of the completed work is being decided right now, with a strong leaning towards GPL. I should have some more good sample code when that porting job wraps up.

3: Re: portable.nsd updated (response to 1)

Posted by Don Baccus on 03/12/03 07:03 PM

Why is this client so interested in spending extra money to avoid AOLserver when the existing AOLserver solution works so well?

I still can't help but feel this is an incredible waste of resource.

4: Re: portable.nsd updated (response to 1)

Posted by Talli Somekh on 03/12/03 07:09 PM

I don't think that's fair to say at all, Don, but we've had this discussion before...

If nothing else, John's work is good for the open source-ness of it - giving it a run to see if it can be done. But there are real benefits to it.

For instance, Portable.nsd will get us extremely close to an OACS IDE because it can be loaded into TclPro for OACS debugging. So far, in very preliminary conversations, ActiveState was interested in learning more about this approach before John and I got really busy. We do plan on following up.

But I don't see any reason to tag this work an "incredible waste."

talli

5: Re: portable.nsd updated (response to 3)

Posted by Patrick Giagnocavo on 03/12/03 07:27 PM

Actually Don, I think this is tremendously useful.

AOLserver does not run well on every single architecture that TCL runs on, for one.

Further, the Apache/TCL project(s) are not accomplishing anything, at least as far as I can see. There is Rivet, websh, mod_tcl etc. and they are a mishmash of code, each incompatible with each other.

This solution could well become a way for "the OpenACS way" to become a de facto standard for TCL/web app development. Which would be a good thing.

6: Re: portable.nsd updated (response to 1)

Posted by Don Baccus on 03/12/03 09:22 PM

AOLserver 4.0 will support Win32 natively, that's been the big hole.

If people really want an Apache solution to replace the AOLserver-based approach as "the OpenACS way" surely use of libnsd rather than a Tcl re-implementation would be the way to go?

That way folks smart enough to use AOLserver natively would be assured of compatibility with those using an Apache-based solution, because both would be using the same code base.

And some of the kludges suggested above - "cache nssets in the database" (dotLRN uses nssets in every portlet, that idea simply won't scale) - aren't necessary, right? Code would just run ...

7: libnsd (response to 6)

Posted by John Sequeira on 03/12/03 11:15 PM

I like the idea of using libnsd to replace the bits of nstcl that overlap with it. I've downloaded Jamie's binary and played with it at Andy P's suggestion. It looks cool and wouldn't take long to embed in pnsd. It's on my roadmap.

I know that libnsd/Apache integration has been attempted (a month or two ago) and it was not successful. Whether the (threading?) issues get resolved when the AOLServer reaches release status, or breaks on subsequent releases, remains to be seen. If it works, I would love it.

Regardless, since libnsd omit's lots of AOLServer pieces related to the web server, there's a decent amount of glue code that needs to be written to make it work with an arbitrary web environment. It's not complicated code... mostly just parameter marshalling, but someone needs to write it.

Also, regarding caching, let's just say that's in research mode. Scalability is something I've decided to address once basic web serving is functional, and not before. I have plenty of ideas (rdbms-caching, tequila, edge-side-includes), some of which might only be appropriate for low-traffic Intranets, but others just might scale. libnsd on it's own will not solve the problem - it will need to be told when, how and what to garbage collect, for instance. And of course you have the problems of cache synchronization with multiprocess web servers, etc.

Anyway, I had hoped to achieve 100% runtime compatibility with OpenACS, and avoid any but I've realized there's no way I'll do that on my own (even with a huge nstcl headstart). libnsd will go a long way towards helping to achieve this.