Forum OpenACS Q&A: Response to How would one handle 12,000 db backed request per second?

OK, a simple question: what happens to the existing mainframe installation if you throw 12,000 queries similar to those in question at it?  Does it smile and say "heh, I didn't even have to work up a sweat to handle THAT trivial task!"  or does it leave broken computer parts strewn over the floor as it dies?

In other words, by studying the existing mainframe installation and the throughput it can sustain you can start getting a handle on what would be required to service the workload you anticipate for the web.

If I were in your shoes, I'd do everything I could to understand where  the existing bottlenecks exist on the current installation.  Since the db is likely to be your most difficult bottleneck to solve, a deep understanding of how the current db performs can only help.

When moving to the web, the actual webserver portion can be solved simply by throwing more and more boxes behind a load balancer, each talking to the database server (or server cluster if you're forced to go that route).

As far as how scalable Open Source software is, depends on the software.  We know, for instance, that AOLserver and Apache scale well  enough to serve the busiest sites in the world - proof by example, because they *do* serve some of the busiest sites in the world.

Ditto Linux.

PostgreSQL hasn't been tested in that space.  To be honest, though, I'm not certain that Oracle has been tested in that space IN A WEB ENVIRONMENT.  As Ben said, 12,000 db hits a second is far and above any load placed on a db server by any web site we're aware of.

In the more general sense, several companies - IBM, Intel, etc - have invested in the Open Source Development Lab (OSDL), specifically meant  to be available for testing scalability of Open Source solutions on massive hardware typically not available to Open Source projects (or most software development companies in the closed-source world, for that matter, we're talking BIG IRON).  As it happens it's only a few miles from my house and I had the pleasure of talking to their first technical hire about three weeks ago - he just happened to wander into  my local coffee shop, saw that I was running Linux on my laptop, and struck up a conversation.

They're really just getting up and running, but in the next several months to a year we should be able to get firm answers on some of these questions.  I know that Postgres is high on the list...