Forum OpenACS Development: Re: Varnish HTTP Reverse Proxy for XoWiki on project-open.org - Experiences and Remaining Issues

Hi Frank,

We're using varnish in a completely different (non-openacs, non-aolserver) environment and what we've found is that when it works, it works great, but getting it to that point can be frustrating due largely to the quirky configuration (which has apparently changed a lot with each major release).

An alternate idea: Why not do the page caching completely within aolserver? Set up filters on each end of the request, at the beginning to check if there's a cached response available and at the end to cache the page that was sent. Since you're doing it within the server you have a very flexible and familiar environment to work within to do whatever validation you need or choose to skip.

Here's a little code I put together to do this:

pagecache-init.tcl

ns_log notice "initializing pagecache filters"
ns_cache create pagecache
nsv_set cls pagecache [ns_cls alloc]

ns_register_filter prewrite GET / pagecache_save
ns_register_filter -priority -10 preauth GET / pagecache_check

-----
pagecache-procs.tcl

ad_proc pagecache_save {why} {
    This is the worker proc for a full-page cache.  It's used as a
    very late output filter
} {
    if {[ns_cls get [nsv_get cls pagecache]] eq ""  && [ad_conn url] == "/"} {
        ns_cache set pagecache [ad_conn url] [ns_conn responsecontent]
        ns_log notice "caching [ad_conn url]"
    }
    return filter_ok
}

ad_proc pagecache_check {why} {
    a very early filter for serving pages from cache
} {
    if {[ns_cache get pagecache [ns_conn url] pagedata]} {
        ns_log notice "serving cached page for [ns_conn url]"
        ns_cls set [nsv_get cls pagecache] served
        ns_return 200 text/html $pagedata
        return filter_break
    }
    return filter_ok
}

This setup as written skips all validation for the cached page - no sessions, no logins, no cookies, etc. On the other hand, it's pretty fast. But it's pretty simple to adjust this tradeoff; for example by making the check a postauth rather than preauth you could have user info to check sessions; you're then accepting the cost of checking the user info, but that's likely still much less than the full page cost.

However, there's one big, possibly showstopper caveat: this code relies on functionality that's only available in the bleeding-edge (cvs head) version of aolserver. I certainly wouldn't mind more testing of this functionality and more eyes on the code, but that might not be an acceptable risk for you. But let me know if there's anything I could do to help on this front.

Cheers,
-J