Forum OpenACS Q&A: Request Processor Performance
We were going to use OpenACS 4.5 for http://bloglog.net, but benchmarks showed that we are only getting ~8 DB PPS whereas we are getting 20+ DB PPS with OpenACS 3.5.
Is permission checking the primary bottleneck? Is there a way to disable permission checking for a page, directory, or file type?
Don's permissions, which should make it into 4.6.1, should help quite a bit.
And why are you looking into 4.5 when 4.6 is out? 4.6 is in much better shape.
Same permissions checking?
Both pages execute the same query (only one query), but the OpenACS 3.5 page doesn't perform permission checking (which is what we want for this page). Ideally, I want to turn permission checking off for certain pages -- has this feature been developed for 4.6?
experience from tuning my homepage indicates that you can cut response times a lot by turning on caching for permissions (an APM parameter of the kernel). You should mount developer support, activate query logging (APM parameter of the package) and restart the server. Figure out which queries are the bottlenecks and consider caching with util_memoize, see this thread:
Recently I had been looking into unneccessary queries that the request processor is making. One such query that you might see is
where host = :host
- not exactly an expensive query but still, it should be cached. The query will be executed if the hostname in your AOLServer config file doesn't match exactly the host header of the request. For my homepage that means the query is executed for marklundweb.com but not for www.marklundweb.com (what I have in the config file).
When running developer support you may see a query (executed twice sometimes) building up the context bar. This query has been replaced with a cached lookup recently. I think this change will be in 4.6.
On the OpenACS development branch (what will be 4.7 eventually) there is an overhead query that is run four times per request to check the locale preference for a user for the requested package. I'm working on adding a cache to elliminate those queries.
Another thing to do is set the performance mode kernel APM parameter to 1. This will reduce the amount of code executed in the request processor on each request.
What is PPS? Are you using Oracle or PG? What kind of hardware are you on?
all unnecessary over head queries invoked on every request that we have found so far have been cached now (including the ones mentioned above such as host-node-map and user locale lookups on 4.7).
If there are any that we've missed, let us know and we'll squash them.
Also I thought I'd added my site node caching of permissions work I did for Greenpeace to OpenACS but apparently I forgot to do so. I will do this soon. It tracks changes to site nodes accurately so if you shut a node off to the outside world by removing the "read" permission it will be honored immediately.
General caching of permissions is a *bad* idea, IMO. Making permissions fast enough to use combined with higher level caching where there's more control is much safer I think.
The new permissions implementation is almost complete for PG, BTW, so there's hope there.
So before I start digging into the PermissionCacheP code myself, what was the conclusion of that discussion? Is PermissionCacheP now robust and a good thing, or are there still problems or concerns?
Here are a couple other threads related to caching of permissions queries, from Feb. 2003 and June 2002
Properly maintaining the cache in that circumstance is very expensive and consequently no attempt is made to do it.
So the short answer is that if you have permissions set up in a particular way on your site it *might* work but but generally there are a lot of cases where the cache is out of sync with the DB. With Don's permissions checking speedup work it is hopefully not necessary for most sites anyway.