Forum OpenACS Development: Re: Varnish HTTP Reverse Proxy for XoWiki on project-open.org - Experiences and Remaining Issues

> The server gets some 300.000 Google "impressions" per month

This sounds like a strange kind of metric. What do you mean by this? Hits by the google bot? 
E.g. OpenACS.org gets about 670.000 google hits per month, and has rather moderate traffic.

> ...  found the necessary features to work around OpenACS/AOLserver HTTP headers that prevent SQUID from working as a reverse proxy

What does this mean? We use OpenACS on most sites behind a reverse proxy (nginx).

> ... factor 10x speedup in content delivery (~0.04 seconds with VarnishD vs. 0.4 seconds directly from AOLserver)

If your AOLserver takes 0.4 seconds for delivering files, something is really broken in your installation. 
Already many years ago, naviserver reached easily >1200 requests per seconds (these are 0.00083 secs per request), see e.g.:
http://www.mail-archive.com/naviserver-devel@lists.sourceforge.net/msg02038.html

On todays hardware the performance will be certainly much better. If we look at the traffic of our production site, 
and measure just the dynamic pages (no images, css, .js included, which is much faster), the average response time
is between 0.06 and 0.09 seconds on a four year old hardware, with currently more than 2500 users active, 
>12000 currently logged in). Note, that delivering a dynamic page involves typically many SQL queries (often 30+); 
Comparing dynamic pages with the delivery of a cached page without permission checking etc. is comparing apples with pears, 
... but maybe, the application running within aolserver is slow.

Are you running your server in a virtual machine? Maybe the configuration of this machine is in a bad state...

-gustaf neumann
Hi Gustaf,

Thanks for answering!

> strange metric

I was looking at Google Webmaster tools, and that's the metric they are using there.

> nginx

Right, I've seen that one (although never used). Most of ]po[ servers are behind Pound. However, Pound (and nginx AFAI) don't have caching features. So Varnish allows to store the /images/ folder etc. completely in memory.

> broken in your installation

I know that you manage to get really high speed on your IBM hardware, but I've never seen XoWiki pages to come out faster then 250ms on any (Intel) hardware ever, with average speed around 300-400ms. I'm talking about installations directly on hardware, not virtualized environments.

I'm using

time wget http://localhost:30140/xowiki/
for measuring the entire content delivery though, which might include a lot of latency.

We'd be happy to pay your half a day of consulting (your rate whatsoever...) if you'd manage to speedup our standard Intel installation (CentOS 6.2, PostgreSQL 8.4, AOLserver 4.5.1) by a factor 2 or more :-)

Cheers,
Frank

> I was looking at Google Webmaster tools, and that's the metric they are using there.
this is still a strange metric, measuring rather google-visibility than business of the server.

When one compares openacs (with db access etc.) with a server delivering cached pages, one is comparing the computation of pages on the fly and their delivery with the pure delivery of precomputed pages. But that's not comparing aolserver with the caching proxy, but the OpenACS application vs. the proxy. Actually, for dynamic pages, the caching proxy can't do much unless it is skipping cache validation (which would lead to incorrect deliveries, no user tracking etc.).

> I've never seen XoWiki pages to come out faster then 250ms on any (Intel) hardware
Well, on openacs.org with Ubuntu 10.04.4 LTS + pg835 (intel server), i see on such a test 0.101secs real time, and this is with general comments and various includelets in the side-bar.

time wget http://openacs.org/xowiki/aolserver-install
real	0m0.101s
user	0m0.001s
sys	0m0.002s
The total average on openacs.org is 176ms over the last week. One should note that openacs.org is not a tuned site and it is running still a quite old version of OpenACS.

Maybe i'll get in touch when it is less busy on my side to figure out, what makes your site slow ...

all the best
-gustaf neumann

Hi Gustaf,

> skipping cache validation

This is what we're doing for certain pages (CSS, JS, images, ...). We have currently included the actual XoWiki pages as well, because we're using the XoWiki of www.project-open.org basically like a static Web site. However, I'd be delighted if we could remove this setting.

> less busy

That would be great. It's not only one site, but ALL servers I've seen. My offer concerning the consulting time stands :-)

Frank