Forum OpenACS Development: [Open]ACS 4.x bboard scalability

Collapse
Posted by Don Baccus on
There is work to be done here ...

Good news for Oracle fans

Oracle is able to create new bboard messages about five (gulp) times faster than PostgreSQL.

Good news for PostgreSQL fans

PG can build the main summary page for the OpenACS general forum 10% faster than Oracle. It takes a few seconds, slow but down from about 10 MINUTES which was how long it took before I started using my knowledge of PostgreSQL to optimize the queries.

Some of these changes prove to be faster for Oracle, too, as it happens. Though acs_object building is slower than with the Oracle version (largely due to building the tree_sortkeys, I imagine) it's fast enough for all but the most intense web sites.

Bad news for Bboard fans

There is work to be done here ... oh, shit, I just repeated myself.
  • The bboard_forum_message_map should be used to track replies and last reply dates so the individual forum summary pages don't have to build those on-the-fly for every user. After all, they only change when a message is inserted or deleted, not every time a user visits! The penalty should be paid when the forum's altered, not when every visitor visits.
  • The queries to summarize messages by category seem slow and could probably be sped up.
  • As mentioned in at least one other thread, the page that displays a thread hits the database with a query for each post, looking for attachments. It should be possible to fold this into the original query that grabs all the posts in the thread.
  • The datamodel's a bit funky.

Anyone want to volunteer to help make bboard work better?

I've got a list of quick hacks to make the bboard run faster under PG that I'll implement shortly but don't have time at the moment to tackle the more fundamental changes that are needed. Given the fact that bboard is one of the more popular packages in the toolkit, and that we can't migrate openacs.org over to OpenACS 4 until we make it scale better, it would be nice to have these improvements in by beta,

At the moment we're looking at about 5 seconds of RDBMS effort to build the general OpenACS forum summary page on a PIII 500. That sucks. I'm confident we can get that down to around a second, though, which would be acceptable (it's a large forum with about 10,000 messages in a bunch of threads and several categories).