The Toolkit for Online Communities
16680 Community Members, 1 member online, 2504 visitors today
Log In Register
OpenACS Home : Forums : OpenACS Improvement Proposals (TIPs) : TIP #57 (Approved): Add caching db_* API (based on Greenpeace/Berkeley work) to 5.2 : One Message

Forum OpenACS Improvement Proposals (TIPs): Re: TIP #57 (Proposed) Add caching db_* API (based on Greenpeace/Berkeley work) to 5.2

One problem with this suggestion is that queries often reference several tables, and that tables can be bashed by triggers or PL/[pg]SQL ...

I really think composite keys, typically based on object_ids, with individual applications being responsible for flushing cached query results when content is updated or added is the only practical approach.  At least I've not been able to think of any practical automagic cache coherence approach that works for general queries.

I'm not sure I understand the concept of composite keys. Why not just attach a list of object_ids to a cached query, and whenever at least one of those ids is mentioned in a flush statement the query is flushed?

Thus the example could look like this:

set cache_ids [list $my_message_id $my_reply_id $my_forum_id $some_other_arbitrary_id ...]

db_1row -cache my_query_name -cache_ids $cache_ids {}

Tilmann, one problem with your scheme is that for a lot of queries you do not know the ids when you do the query (eg, for a thread in forums you know the root message id, but not all the childrens ids).

Another is that with the forums example, adding a post adds an id rather than changing any existing ids so you have to do an explicit flush on the parent_id anyway.

A third really hairy issue is that for queries which involve permissions checks, it would be prohibitive to try to do any real dependency checking; and I given the varying need for security such queries probably should not be cached (or at least there should be some option to disable caching of queries involving such checks). Certainly, the experiences with PermissionsCacheP point to a lot of problems with caching such things.

I also think its really easy to go overboard on dependency checking, I have worked with a couple of different systems that had dependency checking for caching or for lazy evaluation and both had numerous places where checking the dependencies was slower than simply recalculating.

In any case, I definitely approve the tip, although I think it will be a while before we are really happy with how things work.