Forum OpenACS Q&A: request for review: TUX and nsd document

I've been playing with TUX2.0, the in-kernel webserver with network throughput stats that make young girls squeal. /. readers may remember TUX eating the lunch of all competitors on the SpecWeb benchmark this July (by a factor of 3 to 4). More importantly, I've been exploring how to use TUX as part of an OpenACS system. Basically, TUX and nsd run in a vhost type arrangement. TUX handles all static requests, and a hacked up version of nsd constructs a dynamic pages and feeds them into the TUX object cache with the tux(2) syscall. The aim with patching nsd is that rather than talk to clients directly, nsd should generate pages as fast as possible, write it out into TUX, and get back to work on querying the DB and evaluating tcl. This will reduce the ability of a few slow connections to choke the whole aolserver process.

The document in question is at http://www.toddbrandlabs.com/code/TUX/. While I finish the nsd patches, would anyone mind critiquing my writing, and suggest how to clarify some points?

Collapse
Posted by Andrew Piskorski on
Interesting. You're trying to do two rather separate things there, though:

One, use TUX as a buffer between AOLserver and the client. If you change AOLserver only such that it returns HTTP responses to TUX rather than to the end client, and you don't change anything else, then I guess it would work without breaking anything else. Sounds nice and clean and simple.

Two, you're also talking about using TUX to "serve static files" and "Pass .tcl and .adp requests through to AOLserver", and that's a significant change in the application logic, not just an under-the-covers implementation enhancement. For example, ACS 4.x does do access control checking on URLs that map to static files. And then there's the Static Pages module that reads in an HTML file and then post-processes it in order to tack dynamic General Comments onto the end! So that fact that a file is "static" (not a script) does not mean that you can just arbitrarily hand over all responsibility for serving it to TUX without breaking the semantics of your application.

So for that more intrusive feature, you definitely need some fairly smart way to tell TUX just what it's allowed to serve directly and what it must allow AOLserver to process, something much smarter than just letting TUX server everything other than *.tcl and *.adp files.

Of course, by the time a site really needs more performance than a user-space web server can give, I imagine you'd have long since separated out your truly static, no-application-logic-involved content (e.g., graphics) from the stuff that needs ACS access control or other AOLserver processing. (In ACS 4.x land, this means hacking/extending the request processor.)

Also, is having TUX serve static files going to be better than just using nsunix for AOLserver? I thought the whole point of nsunix is that it passes files descriptors around rather than copying actual bytes. Or is it that AOLserver/nsunix can't pass the file descriptor to TUX or any other part of the kernal, but must copy the bytes?

Finally, and I admit I'm curious, are you just pursuing this AOLserver / TUX thing as a cool research project, or do you actually have someone who thinks that a user-space web server isn't going to be fast enough for them?

Collapse
Posted by Vadim Nasardinov on

Verily, verily sayeth Andrew unto you:

Finally, and I admit I'm curious, are you just pursuing this AOLserver / TUX thing as a cool research project, or do you actually have someone who thinks that a user-space web server isn't going to be fast enough for them?

My question exactly.

Collapse
Posted by Todd Gillespie on
And Andrew and Vadim did seek to find one who desireth such a server:
Finally, and I admit I'm curious, are you just pursuing this AOLserver / TUX thing as a cool research project, or do you actually have someone who thinks that a user-space web server isn't going to be fast enough for them?
Yeah. Me. And some people I have worked with in the past, who would like to stop buying Suns by the rack.

My question is, when did we stop appreciating the cool hack, and demand cost validation on everything?

In response to Andrew's 2 technical questions: paragraphs 3,4, and 5 in your message can be consolidated into 'do static files pass through the nsd filter system'. I am well aware of the problems faced by this approach, and perhaps it might be best for TUX to pass all requests for simplicity's sake. I am currently doing silly tricks to make TUX pass some static files to nsd, like setting the sticky bit on these files. But for the vast majority of static files, I don't want the ACS security filters on them. I am looking for a general solution, it will probably include some concept of a URL tree in TUX that can be bound and assigned to different daemons.

I'm not really sure where this hostility has come from. I think the general architecture of the TUX subsystem is exceedingly sweet, and I would like to get around people's blind spot where AOLserver is concerned with some insanely fast benchmarks. This is win-win.

Collapse
Posted by Jonathan Ellis on
s/hostility/curiosity/
Collapse
Posted by Todd Gillespie on
Exception.throw("Todd needs more sleep")

please pardon the misunderstanding.

Collapse
Posted by Vadim Nasardinov on
some people I have worked with in the past, who would like to stop buying Suns by the rack.

Surely, they didn't buy those racks for serving static content? Benchmarks that I'm aware of (like the widely publicized Mindcraft benchmark) seem to indicate that any decent web server - user-space or not - can saturate a 100Mbps link when running on a four-processor Intel box. Is that not fast enough for them?? What kind of sites do they run?

Given the current state of hardware, trying to beat the existing web server speeds seems to be a purely academic exercise to me. I see nothing wrong with academic exercises, as long as they are clearly announced as such.

Collapse
Posted by mark dalrymple on
We're probably all talking at cross-purposes here.  The whole TUX thing is pretty neat, and it sounds like Todd has a fun exercise of mixing two pretty neat technologies (aolserver / tux).  For a site that's primarily static content, this could be a win when it starts being network I/O bound, he TUX kernal optimizations can be a big win.  For most sites that (open)acs is targeted for, the work profile is heavily on database / application server: churning through tables of data, running tcl and checking permissions and processing templates and all that fun stuff.  Disk I/O and CPU are the bottlenecks, network I/O is barely a blip on the radar, which is why many folks are probably looking at this project and going "huh??"
Collapse
Posted by Jerry Asher on
I think it sounds pretty interesting.

I've long wanted the time to write a version of nsunix for Apache and Squid, and I've also considered a kernel module version of nsvhr, so that sockets could be read, fiddled with and passed around and then have any user program just think they are getting a regular socket without needing any unix domain socket passing. Not because anyone needs it, but because it would seem to be pretty cool.

If you're looking for URL tree code, there is of course, URL tree code to be harvested within AOLserver, assuming the licenses are compatible.

From my nsvhr/nsunix experience, I think it's a good idea to make your application logic independent of the the vhost/cache technology.  If I understand what everyone has said, then I would like to see (and maybe help out on), a application transparent tux cache.  As other's mention that would probably involve having tux feed everything initially to AOLserver, and having AOLserver fill the cache via your tux syscall.  I would imagine that the soon enough, tux would be serving the static pages out of its cache.

I haven't examined your code, but the other suggestion I have is hacking it in the best way you can to nssock directly.  AOL seems averse to supporting much beyond nssock, and nssock has certainly been well tested -- I can't say the same about the drv.c driver approaches.