Forum OpenACS Q&A: The future of mod_nsd and other web server support

Once mod_nsd 1.3 will be released, the 1.x version will be pretty much in maintenance mode. All new development effort was going to go to version 2.0 (for Apache 2.0), but there might be a better alternative.

Instead of building an Apache ns_* API compatibility layer, I will work on adding FastCGI support to AOLserver. A FastCGI-enabled AOLserver (or OpenNSD) process will behave *exactly* like a "normal" one (since it's the same source code), but will run a FastCGI listener instead of the HTTP one. This means that one can run applications written for the AOLserver API using any web server that supports FastCGI (like Apache, Zeus, IIS and iPlanet).

Comments?

Hold on, my only question is: why didn't we think of this earlier?
Petru, I think this is an *excellent* idea. It makes AOLserver an
application server in the stack, which is *exactly* what we need!
That's the purpose it serves, after all. I'm racking my brain for
potential pitfalls here, but I find no red flags. This sounds like an
absolutely awesome direction. My only questions, and these are
mostly out of ignorance for the current state of FastCGI:

- is the FastCGI spec taken seriously by web server vendors?
Are their implementations of FastCGI decent?

- What other application servers use FastCGI as their primary
interface?

The beauty of this approach is that we can be immediately
compatible with Apache 1.x and 2.x, while letting the Apache
Group handle all of the issues related to their new threaded
environment.

I'm stunned. Hats off to Petru for this approach.

Thanks for your comments.

I only worked with the Apache and Zeus FastCGI implementations so far. Apache's mod_fastcgi is written by the same guys who wrote the FastCGI spec, and it behaved very well for me in production (I used it with Zope). I only did a couple of quick tests with Zeus, but it looks like a solid one too. I don't know much about the state of FastCGI support for IIS and iPlanet.

Other things that talk FastCGI (this is not a complete list by any means):

  • Zope has a FastCGI server integrated.
  • WebWare has a FastCGI adapter (in the default distribution).
  • PHP 4 has FastCGI support in it's SAPI (server API). A funny thing to note is that the Zeus folks did some tests with Zeus+PHP using ISAPI and Zeus+PHP using FastCGI, and they concluded that the FastCGI method is much more stable and robust (see http://support.zeus.com/products/php.html#isapi). Also, benchmarks done by them show that Zeus/FastCGI/PHP is significantly (26%) faster than Apache/mod_php.
This is an absolutely fabulous idea. I think that FastCGI has been very underrated in its potential. Its too bad there's no longer any linked-in support for FastCGI on iPlanet or IIS since Fast Engines was bought out by Adero. See www.fastcgi.com for the sad story. Nonetheless, you can still use the cgi-fcgi program that comes with the FastCGI development kit at www.fastcgi.com if you really want to use those web servers. The author claims that there's very little performance overhead in this configuration but I couldn't find any performance numbers.

Anyway, being able to host an OpenACS site at an ISP running Apache will greatly reduce costs for some(including me) and probably lower the barrier to acceptance of OpenACS too. Trying to get an ISP to compile in a module like mod_nsd into a shared Apache server is an uphill battle in my experience. Getting an ISP to allow a FastCGI program to run will probably be a much easier task.
At this point I am asking myself the same question as Ben: "Why didn't we think of this earlier?". I looked at FastCGI about 2 years ago, but didn't think of Petru's idea. Excellent Petru!
yes, i think petru is very close to stepping into god-like status!

how hard is it to write a fastcgi interface? there are people constantly seeking ways to get it to work with IIS. as a non-techie, i have no ability to determine what it would take to update the Fast Engine IIS extensions.

talli

So, instead of enabling an Apache ns_* API, we will be adding Aolserver to the OpenACS ForOtherServers installation, which will speak to the servers through FastCGI.Talk about cross-platform suppor. Its the AOLserver Virtual Machine! Can performance really hold up? If so, it would seem this tactic might work for some other web dev projects too.
My previous experience with FastCGI was not good, but I think that the problems we encountered can be overcome.

A few years ago I worked at a place that ran FastCGI under Apache. Each CGI script ran in its own process resulting in dozens of processes sitting around, usually doing nothing except consuming memory. Apache queued the requests to each CGI. You would get server busy errors if a CGI was slow and the default queue size was exceeded.

If there is a way to configure Fast CGI so that it can simultaneously attach to each thread in an AOLserver process then these problems won't be encountered.

For the AOLserver+FastCGI setup, AOLserver will be started by the user/admin/whatever, not by the web servers' FastCGI process manager.

The request path will look like this: the front-end web server (Apache, for example) gets and parses the requests, then sends it via FastCGI to AOLserver's request handling thread, from where it gets dispatched to one of the worker threads, then returned back to the web server.

What exactly does this buy you?

...apart from consuming more server resources and being slower, requiring extensive modifications to AOLserver, not enabling easy intallation at the typical ISP, and increasing the maintenance burden of the website operator from one webserver and a database to two webservers and a database..?

Buzzword compliance? Application Server! Three Tiered Architecture!

Great response, Stephen. What exactly does *that* buy us, other than a smart ass comment that doesn't help brain storm what has been one of the biggest problems for the system's acceptance. Petru has come up with an idea that might work, but I guess that you lame reponse has changed my mind.

talli

It sounds similar to the system aolserver uses for virtual hosting with nsunix.  If that's so it probably won't require that extensive of changes to AOLServer.
Sorry Talli, I didn't realise you were so delicate. I did however point out 3 or 4 of the many problems with this solution, apparently you don't see them as significant.

I am still interested to here what this buys you. What is the problem you are trying to solve? The unwritten assumption here is 'get rid of AOLserver', which is fine as far as I'm concerned, but that is not what you are doing. You are just ADDING another layer of complexity.

As far as implementation is concerned, it is not as easy as the existing nsvhr module. AOLserver supports socket drivers; nssock, nsssl, nsunix, but it does not support protocol drivers. You could write a socket driver which reconstructs the http request from the FastCGI stream and lets AOLserver parse it as normal, but it sounds a bit silly to me. In fact, it sounds like your writing a proxy server. It's been done.

Running two webserver is pointless unless you need the features of both at the same time, in which case run two webservers. If you can't afford a second IP address, run squid, mod_proxy or nsvhr, all of which are available right now and require no coding.

I'm not delicate Stephen. I just prefer not reading jerky posts.

Thanks for following it up with something productive. I understand the issues you brought up better now. Appreciate it.

talli

I don't ever plan on running mod_nsd or the FastCGI version because I am comfortable with the current version but I do think the FastCGI does offer some value.  There are ASPs that will not run AOLserver but might run a fastcgi version. There are many developers that dont want to learn another webserver but dont mind learning another module. As silly as this seems alot of it boils down to perception. If we can breakthru the reluctance of acceptance on step at a time once people are crawling it should be easier to show them that its easier to run.  Not to mention that it would give more exposure/mktg by having the functionality. The barrier of entry/acceptance is often taken in steps. This could be a good one... best regards,
If you can run AOLserver alone, this is best, but from the 3850 Apache RPM downloads (almost half of OpenACS downloads), I'd say that the ability to run on other servers will help us grow, right? And let's face it, while many of us need AOLserver scalability for when we hit it big, many smaller client sites do not aim at a level of which performance will be an issue. A whole lot of these sites run on Apache.

Do I smell a flame ? :) I hope not.

Thanks for the comments, Stephen. I have a couple things to address, though:

  • On the technical side, the points made by Stephen are perfectly valid. It's not going to be as easy as nsvhr, because we need a protocol module and AOLserver doesn't have that, so patching in the core server will be needed. I will think of a way to do this as cleanly as possible.
  • Using FastCGI instead of a proxy module in the other web server (because we're talking about using another web server, not about dedicated proxies like squid in reverse mode) has the advantage of being supported by a couple different web servers, and being lower overhead than a proxy.
  • From an architectural point of view, this whole FastCGI stuff is silly indeed. As you said it, it remembers of buzzword compliance and three-tiered architectures.
  • The most important thing is that the FastCGI interface is *not* for those who are already happily running AOLserver. The recommended way of running OpenACS will not change. The fact that AOLserver is a great web server will not change either. This whole "make it usable with Apache" thing is however a very popular request, because many people are reluctant to change their web server. I didn't believe it either, but then I realized that most of people I talked to about (Open)ACS -- even one of my consulting clients -- asked "what is AOLserver, I never heard of it". As Johnny Rocoo mentioned, there are quite a few downloads of the Apache RPMs, and even so I'm sure that at least half of the newbies that download it give up, because mod_nsd kind of sucks right now.
  • Hosting. How many AOLserver hosting companies do you know? I know about 2, though let's suppose there are 5 and we don't know about them. Now, how many Apache hosting companies are there? A couple bazillions.
  • Inclusion into distributions. OpenACS is not included in any distribution right now, exactly because it requires another web server in place of the carefully configured Apache that the vendor ships.
  • Upgrade path. Suppose you're a newbie and you try out OpenACS, using Apache and AOLserver with FastCGI support. It's easy to setup, because you just grab a RPM and install it. After a while, you like OpenACS more and more and build a site with it. Eventually, you'll be willing to run AOLserver, but then all you need to do is change the AOLserver config file to disable the FastCGI stuff and enable the HTTP listener. Voila, you now have a fully working AOLserver with no FastCGI crap.

I'm kind of interested in Petru's use of the term 'protocol module'. That sounds like we could move AOLserver beyond the webserver area and get it toward a more general purpose server. Maybe that is what others here are refering to as an application server? At any rate, http is a nice protocol, but it would be nice to be able to run AOLserver without it.

Right now within AOLserver, you can add modules that listen and handle requests. Is the protocol module going to do this, or will it replace the http nssock module?

I had a conversation recently with a prospective client who was
interested in using the ACS but already had invested
$1,000,000+ in development on an mod_perl-based app. He
wanted to use OpenACS (with many features are largely
orthogonal to his system) without throwing out the investment he
had made in the existing code. At the time (and probably to this
day), the best solution seemed to be to have Apache and
AOLServer running out of different ports on the same machine.

Putting aside the marketing issues for the moment (since this
particular person seemed comfortable with the separate ports
solution), are there any technical advantages to running the
OpenACS system through Apache rather than around it? Would it
make it any easier to start merging the two code bases in the
future, for example?

I think FastCGI would be great.

It would allow one to use Apache's already wonderful virtual hosting features to setup truely virtual openACS sites.

It also gives one the ability to do fancy things with re-write rules with Apache, which AOLServer doesn't do, AFAIK.

Well, I was just reading this old thread again after David Kuczek reminded us about it.

FastCGI sounds like it would be a very cool tool to have in one's AOLserver toolbox. But, I wonder about one thing - the tight integration of the AOLserver API. Would FastCGI allow the all of the ns_ API to work normally? I'm not sure, but presumably, AOLserver API calls like ns_conn could be properly implemented by having Apache (or whatever the main web sever is) pass the necessary information via environment variables or something to the AOLserver FastCGI process?

In other words, are the FastCGI spec and implementations powerful enough to allow the sort of tight API integration with the web server that has always been AOLserver's hallmark? From a brief look at www.fastcgi.com, I think the answer might be yes, but I'm not really sure.

Indeed, the FastCGI protocol passes a bunch of variables down to the application, and I think that most of the stuff needed is in there, though I didn't try to see exactly if (and what) something is missing.