Just thought I'd let folks know that Robert Locke, with a teensy bit of help from me, has figured out one of the reasons why openacs.org gets so overloaded despite our relatively light traffic.
It's due to a bug in the query that clears out notifications after they've been sent. My fix to these queries months ago caused the right notifications to be sent (which originally was not the case) but didn't kill them afterwards correctly. I only fixed one of the two relevant queries, it turned out.
Therefore we have accumulated well over 5,000 notifications and about 900,000 notification->user map requests! The mail sweeper query runs for 90 seconds or so and it is this that has caused the postmaster to hog the openacs.org box killing performance of other processes.
Robert's got a fix for the queries that we'll get installed here at openacs.org and of course in the openacs sources, too, in the next couple of days.
This won't fix the slow insert time for forum posts but will lower the overall load on the server considerably.
Good work, Robert!