Forum OpenACS Q&A: Some thoughts on OpenACS caching

I particularly like the -cache_keys parameter available

Well, the db_interface with "-cache_key" iis not a recommended interface, since this does not scale and leads to many locks. Flushing of this cache is brute-force or not happening at all. In the sketched case - as you noted -, the cache has to be flushed (in the full cluster) whenever a new package is enabled (or installed) or disabled.

If you do not care about getting proper results in these situations, one can use the version below on oacs-5-10, which will be a lock free cache (not flushed as well).

one proc that I noticed gets called a lot ...

Why is apm_get_installed_versions gets called a lot on your site. On stock instances, this is only called by the package installer...

-g

  ad_proc -public apm_get_installed_versions {
     -array:required
  } {
     Sets the current installed version of packages installed on this system
     in an array keyed by package_key.
     
     @param array Name of array in caller's namespace where you want this set
 }  {
     upvar 1 $array installed_version
 
     array set installed_version [acs::per_thread_cache eval -key acs-tcl-apm_get_installed_versions {
        db_list_of_lists -cache_key installed_packages installed_packages { 
            select package_key, version_name
            from apm_package_versions
            where  enabled_p = 't'
        }
    }]
  }

does not scale ...

On large applications of OpenACS (many cache entries, many threads) the performance will degrade substantially.

... and leads to many locks. Flushing of this cache is brute-force

The reasons are wild-card flushes like the ones below.

./new-portal/tcl/portal-procs.tcl: db_flush_cache -cache_key_pattern portal::get_page_header_stuff_${portal_id}_*
./acs-subsite/tcl/application-group-procs.tcl: db_flush_cache -cache_key_pattern application_group_*

The problem is that with a substantial cache (e.g. 100K cache entries) every wild-card flush requires

1. Lock the cache
2. Transfer all cache keys (e.g. 100K) into the Tcl interpreter
3. Iterate over every cache entry and perform a "string match" operation
4. For matching entries, flush these
5. Unlock the cache

The more entries, the longer the lock duration will be. When multiple such locks happen, the second one has to wait until the first one finishes until it gets the lock, Therefore, the waiting times can pile up. In the following figure, t1, t2, t3 ... are mutex lock operations issued from multiple threads.

Note that these locks bring these threads to a full stop, and even fast calls have to wait. I have seen in real world examples, where such locks take up to 100ms or longer, which means that this brings the server to a crawl. It is not unusual that even in actual versions of OpenACS there might be many locks (100+) locks on the db_cache_pool.

The main problem are the wild-card flushes. Even with the cache key that you are using (without wild cards), some other cache flushing operation might cause a long locks and make your requests slow.

The solution is to avoid wild card locks (which is sometimes hard for db-queries) and to use partitioned caches. I will tell probably something about this in the forthcoming OpenACS conference.

Does the util_memoize approach also have this problem?

Yes, pretty much. This applies for wild-card flushes on all ns_caches, also util_memoize, which is a kitchen sink cache. You might have noticed, that the usage of the util_memoize cache was significantly reduced over the last years. It is much better to have multiple specialized caches than one large cache.

sadly our OpenACS version doesn't have this feature.

If you are concerned about performance, upgrading will improve it. Caching is greatly improved over the last years.

Hope, this explains.
-g

My takeaway from your comments is that if I never flush ...

The problem is not a cache flush in general, but wild card flushes (which have to iterate over all the cache keys). Also note, that wild-card flushes on the cache stop as well all other operations on this cache (e.g. the flush from portal-procs above will block as well "your" harmless operation).

... then it's safe to use this caching.

Whatever "safe" means. When something is cached from the database, and the database is updated, but the cache is not flushed, stale results (incorrect) will be reported.

In general, it is better to use specialized caches with precise flush semantics (typically via API), not based on wild-card flushes. Many small caches are also better, since this leads to better concurrency. If there is only one cache, and this cache is locked, then everything comes to a halt. On our LEARN system, we have on average 320 locks per page view, all of these can bring all running threads to a full stop.

that ship has sailed unfortunately.

When your application is performance sensitive, an upgrade will help. Since acs-core is adapted to work with Oracle 19c, upgrading your core should be feasible. This is probably less effort than cherry-picking some changes to improve performance, increase security, etc. Let me know if I can help....

all the best -g