Forum OpenACS Q&A: Re: Some thoughts on OpenACS caching

does not scale ...

On large applications of OpenACS (many cache entries, many threads) the performance will degrade substantially.

... and leads to many locks. Flushing of this cache is brute-force

The reasons are wild-card flushes like the ones below.

./new-portal/tcl/portal-procs.tcl: db_flush_cache -cache_key_pattern portal::get_page_header_stuff_${portal_id}_*
./acs-subsite/tcl/application-group-procs.tcl: db_flush_cache -cache_key_pattern application_group_*

The problem is that with a substantial cache (e.g. 100K cache entries) every wild-card flush requires

1. Lock the cache
2. Transfer all cache keys (e.g. 100K) into the Tcl interpreter
3. Iterate over every cache entry and perform a "string match" operation
4. For matching entries, flush these
5. Unlock the cache

The more entries, the longer the lock duration will be. When multiple such locks happen, the second one has to wait until the first one finishes until it gets the lock, Therefore, the waiting times can pile up. In the following figure, t1, t2, t3 ... are mutex lock operations issued from multiple threads.

Note that these locks bring these threads to a full stop, and even fast calls have to wait. I have seen in real world examples, where such locks take up to 100ms or longer, which means that this brings the server to a crawl. It is not unusual that even in actual versions of OpenACS there might be many locks (100+) locks on the db_cache_pool.

The main problem are the wild-card flushes. Even with the cache key that you are using (without wild cards), some other cache flushing operation might cause a long locks and make your requests slow.

The solution is to avoid wild card locks (which is sometimes hard for db-queries) and to use partitioned caches. I will tell probably something about this in the forthcoming OpenACS conference.

Does the util_memoize approach also have this problem?

Yes, pretty much. This applies for wild-card flushes on all ns_caches, also util_memoize, which is a kitchen sink cache. You might have noticed, that the usage of the util_memoize cache was significantly reduced over the last years. It is much better to have multiple specialized caches than one large cache.

sadly our OpenACS version doesn't have this feature.

If you are concerned about performance, upgrading will improve it. Caching is greatly improved over the last years.

Hope, this explains.
-g