This topic was discussed over the the aD web/db bboard in April 1999. Basically, there are certain situations where having clusters of cheap Linux boxes, etc. make sense: a Web-based USENET server (like Dejanew, now Google Groups). Google, as most people here probably already know, runs on thousands of Linux boxes to build their search indexes.
Beowulf clusters are most effective when the tasks can be chopped up in nice little discrete and easily manageable units and there isn't much data dependency, and you need mostly CPU cycles.
Summary: it depends on the task you're doing.
Note: I personally have never worked on a web site that ever required more than one CPU, nor do I have any experience with Beowulf clusters. I do have lots of experience with SMP and cc-NUMA boxes (a.k.a., "The Big Iron"), mostly dealing with rendering computer animation though.
