Forum OpenACS Development: RDBMS distributed transactional cache

Posted by Andrew Piskorski on
Naturally, all cacheing schemes that keep the cache outside the RDBMS - NSV, memcached, whatever - are inherently non-transactional. This is fine in many cases, and is keeps the NSV/cache implementation (but not necessarily the application that uses it!) simple.

What if, instead of the memory cache layer existing as a completely independent application, you turned it into part of the RDBMS? In essence, let the RDBMS keep its own in-memory cache of query results, but let it keep it on many other machines across the network, not just in it's own RAM. Obviously that means the cache servers and the master RDBMS have to exchane lots of short messages frequently in order to keep the network memory cache coherent and transactional, and getting that right would be critical for performance and scalability.

At least without thinking about it in much more detail, it sounds feasible to me. There are also obvious potential synergies with the stuff that the HPC cluster/Beowulf work on, optimized MPI libraries, low-latency high-bandwith networks and network drivers, etc. (Thus see also the somewhat related thread discussing Clusgres, Postgres-R, and the like.)

But, this seems like such an obvious R&D/thesis opportunity that I wonder whether it's already been done. Anyone know of any such projects?