How much does it cost to build the infrastructure for Ebays, Yahoo, Googgle, AOL, or Nasdaq. You are talking about that type of traffic and normally their architecture was evolved and continues to be today.
Most of these sites have many specialized features to help optimize them for their particular needs. An open source solution often contains many features that are useful to many but may not be the *best choice* implementation for some. That doesn't mean that bits and pieces of open source solutions could be used in the overall infratstructure.
There are many issues that would need a little more clarification to get better feedback.
- Is this a web application / does it need to be?
-
Are the users of the system required to login? Are the sessions monitored, secured, and timedout?
- Do the users have different areas of
access or is it the same accross the board with no permission checking?
- Will there be images / ads?
- What sort of accounting is needed at what granularity?
The scope and architecture needs to be designed and each piece optimized if possible. If you can separate out a majority of the base functionality
that is across whole application and optimize that then move on to the application itself. You may want to denormalize and partition you data. The modeling itself is probably the biggest aspect limiting your underlying application scalability.
BTW what exactly does champion mean? Potentially fund an architecture that would then be released into the Open Source community? Digital Citys core API that sits on top of AOLserver is probably the most scalable toolkit that works well with the most scalable web server but that toolkit is not open source
As an aside I have been developing a site for the last 16 months that recreates alot of the ACS functionality but without ever having to touch the db/fs. Amadahl law dictates that the scalablilty of any system is limited by its bottlenecks. Determine these bottlenecks and eliminate/optimize as much as possible. I consider any time the server process has to hit the db/fs a bottleneck. There may be times that this is not realistic but a large portion of the sites functionality may be handled inside the server process residing in memory. You do give up a certain things by moving this functionality from the db to memory but hopefully AOLserver is reaching a point where crashes are rare.
I have been working on some articles that show how AOLserver's api can function in this manner and hope to have more time to finish up and post. Remember though that any site requiring your type of performance will require alot of customization.
Best Regards,
Carl Garland
