In my view the low-level permissions caching stuff is *extremely* fragile and would second everything Jeff has said.
For one thing if permissions change you're going to get different results from queries that look at permissions directly vs. calls to the Tcl API. And it would be very inefficient to funnel all permissions checking through the Tcl API.
I think the permissions work I'm doing will speed permissions sufficiently so that the one-off perm checks caches by the Tcl API won't really be necessary. I mentioned in another thread that I have safe and consistent caching of site nodes implemented at Greenpeace but simply forgot to migrate it over to OpenACS in time for 4.6 (bozo!)
This removes one perm check hit per page and helps a lot without sacrificing security.
At the higher query level - for instance a query that returns the contents of a folder - intelligent high-level caching is not only much more efficient but could be managed closely by the package in question to ensure consistent results.
Also my results with my partal rewrite of the permissions system are quite promissing, 0.5 seconds to return a folder with 50 files fully checked for permissions (vs. about 33 seconds in the 4.6). That's on a P500 Celeron, on a more modern machine we'd be looking at a couple tens of a seconde for the full query and that's without even trying to juice it further (which can be done, I want to attack file storage with a vengence after 4.7 comes out).
file storage has historically been one of our worst performing packages so if it can be made to be fast, I'd say anything we'll provide can be made to be fast.
OK, Peter, now that I'm scared by the low-level perm caching code that came from SloanSpace V2 I'll try to remember to e-mail you my db_* caching API soon!
