Forum OpenACS Q&A: Some thoughts on OpenACS caching

Hi all

I've been testing some OpenACS caching mechanisms in our product, and thought people might be interested in this. I particularly like the -cache_keys parameter available on some of the db_ procs. Of course, the easy part is caching the database call, the hard part is knowing where to flush the cache.

One proc that I noticed gets called a lot and that isn't using caching is apm_get_installed_versions https://openacs.org/api-doc/proc-view?proc=apm_get_installed_versions&source_p=1&version_id=

Now, db_foreach doesn't support -cache_keys, but I tried a quick rewrite using db_list_of_lists e.g.

ad_proc -public apm_get_installed_versions {
     -array:required
 } {
     Sets the current installed version of packages installed on this system
     in an array keyed by package_key.
     
     @param array Name of array in caller's namespace where you want this set
 } {
     upvar 1 $array installed_version
 
     set lol [db_list_of_lists -cache_key installed_packages installed_packages { 
               select package_key, version_name
               from apm_package_versions
               where  enabled_p = 't'
             }]
             
     template::util::list_of_lists_to_array $lol installed_version    
 }

Some quick timing tests gave an average speed improvement from 90745 microseconds to 603 on our ancient dev server, which is pretty good. However, I'd need to think further about the cache flush. Generally we would never enable/disable packages without a system restart, so in our case, it's probably safe not to worry about flushing the cache.

It may also be preferable to use util_memoize in this case instead of -cache_keys, as there is still the call to template::util::list_of_lists_to_array, but still I thought it was interesting to report.

many thanks for the information - as always, very informative. I don't understand what you meant about "does not scale and leads to many locks. Flushing of this cache is brute-force or not happening at all." If we are not flushing, then this shouldn't be a problem, right? In the case where we are flushing, would it be a reasonable approach to use a different cache pool for each db-* call we want to cache? That will create a different ns_cache under the hood, right?

Does the util_memoize approach also have this problem?

Thanks for the code sample - sadly our OpenACS version doesn't have this feature.

I checked to see why we are running apm_get_installed_versions so much, and it looks like one of my colleagues added that in some custom code, so yes you are correct that it's not stock code.

Brian

My takeaway from your comments is that if I never flush, then it's safe to use this caching.
And also if I do need to flush, that it is best to use a partitioned cache i.e. a dedicated cache pool, and keep it as small as possible.
Also I will take a look at our usage of util_memoize and see if there are any potential issues there.

Upgrading our system would be wonderful, but that ship has sailed unfortunately.

Best wishes
Brian

Brian

Wait, so using db_multirow with -cache_key is not recommended?

Using the -cache_key option with db_* functions is generally problematic because the SQL queries can combine data from multiple tables. When data in these tables is added, updated, or deleted, the cached data can become inconsistent, resulting in discrepancies between cached and uncached queries. The developer is responsible for flushing the cache in these cases, which can be challenging.

If a single cache is used for many queries, it can become overloaded (blocked requests), containing numerous entries, and potentially locking the entire system to a standstill. Wildcard cache flushes are particularly problematic in such scenarios (see above in this thread).

Explicit flushing can be avoided by limiting the cache time (e.g., to 5 minutes) and accept temporary inconsistency, similar to the "eventually consistent" model from high-availability systems. This approach may be suitable for some applications, but not for others, where users expect immediate updates after changing some content.

For certain applications, using -cache_key might be acceptable, but it’s not ideal for building scalable systems. By "scalable," I mean caches with hundreds of thousands of entries, processing hundreds of requests per second, each triggering hundreds of locks, reaching peaks of 500,000 locks per second (measured in real OpenACS applications). Most of these locks are from ns_cache or `nsv' .

In particular, I want to report how old a cached result is. OpenACS doesn’t store that information, so I stored it in an ancillary nsv. But this isn't the best solution...

This is actually problematic - not only performance-wise (since it requires locks for both the cache and the nsv), but also in terms of atomicity, leading to race conditions. If you want to expire old entries, use the -expire option of ns_cache when creating a cache or a single cache entry. If you want to report the time when an entry was added to the user, store a small dictionary containing the timestamp and value instead of the pure value. You can consider using nsv_dict to retrieve just the value.

The recommended scalable approach to caching is to use an API rather than raw SQL queries, which allows full control over update operations. For managing ns_caches, use ::acs::HashKeyPartitionedCache for non-numeric keys or ::acs::KeyPartitionedCache for numeric keys. One can specify the desired number of cache partitions either at creation time, or from e.g. the configuration file (using different sizes for development and production).

Usage example:

::acs::HashKeyPartitionedCache create ::acs::misc_cache \
    -package_key acs-tcl \
    -parameter MiscCache \
    -default_size 100KB

set x [::acs::misc_cache eval -key foo-$id {
    db_string .... {select ... from ... where ... = :id ...}
}]
::acs::misc_cache flush foo-$id

In OpenACS 5.10.1, we have additionally lock-free caches (for per_request_cache and per_thread_cache) with very similar interfaces.

So this NX-based stuff is a backwards compatibility wrapper for old AOLserver ns_cache calls? And OpenACS still uses those old AOLserver-style calls instead of the ns_cache_* API built into NaviServer? Which cache API should I use for future work?

Since OpenACS 5.10.1 requires NaviServer, it’s possible to remove the compatibility wrapper and use the underlying functions for all packages in the oacs-5-10 branch. However, retaining the wrapper still makes sense for legacy or site-specific packages to ease the transition to OpenACS 5.10.1.