Home
The Toolkit for Online Communities
17309 Community Members, 1 member online, 2228 visitors today
Log In Register
OpenACS Home : Forums : OpenACS Development : util_memoize_flush_regexp does not scale

Forum OpenACS Development: util_memoize_flush_regexp does not scale

Icon of envelope Request notifications

Collapse
Posted by Dave Bauer on
I was looking into slowness working with bugtracker on openacs.org and analyzed the workflow package to figure out why it takes 30 seconds to close a bug.

I tracked it down to multiple calls to util_memoize_flush_regexp in workflow.

That procedure loops over all the util_memoize cache entries, in this case 400,000 of them, and performs several regexp on them.

This is obviously very slow. I don't recommend using this procedure in any busy site.

The 400,000 cache entries only take up 9mb so you can easily get a huge number of cache entries with a relative small total cache size.

Collapse
Posted by Victor Guerra on
What we can do here is to pass a pattern to the "ns_cache names" call in order to get only the entries we want to flush and then iterate over this ones.

The change is easy.. we can try it out and see how it performs in openacs.org

Collapse
Posted by Dave Bauer on
How would that work?

Don suggested using the db_* cache feature which will also greatly simplify the code.

Dave

Collapse
Posted by Dave Bauer on
Interesting. The cache was flushed sometime today and now there are 12,000 entries and the regexp is plenty fast.
Collapse
Posted by Dave Bauer on
So far all the calls to util_memoize_flush_regexp could use ns_cache names *pattern* instead. I am testing if there is a big speed difference.
Collapse
Posted by Dave Bauer on
ns_cache names is 76000 microsecond with 500,000 cache entries and a simple match like "XYZ*"

util_memoize_flush_regexp is 1600000 microseconds with 500,000 cache entries and a simple regexp like "XYZ.*"

So ns_cache names is about 20x faster. There isn't anyplace in the toolkit that a regexp is required and simple match would not work.