Forum .LRN Q&A: AOL performace data
Can anyone provide AOL webserver performance data. I am trying to guestimate the server hardware required to support 100 URLs per second.
It would be useful to know if anyone has experience with blade servers. My idea is to use standard 2GHz processors with say 1GB memory. Assuming this can support 100 URLs per second, we simply add load balancing and another blade server for an additional 100 URLs per second.
It will greatly depend on what those pages in those 100 URLs are doing. If those are simple operations then maybe a single blade will do. Employing caching also helps.
OpenACS does support the load balancing / clustering that you are looking for in adding capacity out of the box. By adding more front end boxes you may run into that the db server will be the bottleneck. So employing caching will be needed when you run into this.
Mark, you need a much more detailed idea about what it is you're application is trying to do. (And you didn't even mention disk IO, which for an RDBMS can be the limiting factor.) As Jun pointed out, serving 100 requests per second could be nearly trivial, or very difficult, it all depends on what those requests are doing. You posted this in the .LRN forum so perhaps you intend to run .LRN?
Modern CPUs, RAM, disks, and networks are so fast that many interesting sites can be run with quite modest hardware. But even much fancier hardware isn't all that expensive nowadays. For more towards the large high-volume end of things, see the Denis Roy's Jan. and Feb. posts with details on the hardware they use for aiesec.net.
I may be a bit out of date, but currently I'd guess that a rather high-end dual SMP system might have two 3+ GHz Intel Xeons (or AMD Athlon MPs or Opterons of similar or better performance), 8+ GB ECC RAM of commensurate speed, and depending on use, 4-12 or more fast disks in a RAID array (more disks is better). I'm not sure what that machine would cost currently but you're not talking tens of thousands of dollars here. Maybe there are some OpenACS sites out there big and complicated enough that a monster dual CPU box like that would be just too wimpy to run the RDBMS, but if there are I haven't personally heard of any of them.
Most sites probably never need hardware anywhere near that. (For example, openacs.org certainly runs on much more modest hardware!) And anyway, if you think you need more than that, my guess is you're probably either Yahoo or AOL or some big corporate entity like that with a massive public website, or (more likely) you have egregiously poorly performing software and just throwing more hardware at it probably isn't a good idea anyway.
Hardware gets faster all the time, but the design of your software is still, and probably always will be, the dominant factor in meeting your performance goals for a large and complex site. If nothing else, Jim Davidson's Feb. 2000 slides on the AOL Digital City site architecture should help show that.
After some test and theories and getting a 5 day trial key of Optimizeit. We where able to pinpoint the problems. With less than 50 lines of code changes. We are now running in less than 200mb with caching... not only that we are about 10x faster than the old code. Now we have a big fat server... so we just decided to turn up the cache :)
Sometimes it pays to get your hands dirty and Open Source is a real bonus. You don't need to wait for the vendor to fix it. Which also reminds me Lars new profiler will likely save someone's big bucks one of this days. :)