Forum OpenACS Q&A: Building a high-capacity, high-availability OpenACS solution...

BUT - you're proposing building what is quite likely the busiest and most popular Postgres-backed site in the world.  You'll be an early achiever, in other words.  I can't point you to any data from existing OpenACS sites to support the notion that Postgres will give you the 24/7 reliability you need.  Of course, I also can't point you to any data suggesting that it won't.  We just don't have the data.

Now, I want to see this data exist, i.e. busy PG-backed OpenACS sites running out there so we can get a handle on the combination's scalability and reliability, but I can't honestly tell you that you should be the first major Postgres-backed site on the block.  That's something you'll have  to decide.

If the others in your business share your desire to support Open Source, it's probably a reasonable path to take.  If the others are conservative, could care less about Open Source, and want to take the lowest risk path, then Oracle's probably the right choice for you.  It  sounds like money's not an issue...

To put things in perspective, the use of Oracle doesn't guarantee a trouble-free operation, either.  I'm working on a client site using ACS Classic (ACS+Oracle), and Oracle hosed itself last week.  Crashed the database engine on a simple on-line "update", first from Tcl scripts then from SQL*Plus.  Each and every time.  We brought everything down, even rebooted Linux, and the crash remained.  Had to drop tables and restore from a dump, evil evil.  I've not had to do that yet with my Postgres-backed site that's been up for about a year.

For hardware, you might just want to buy a motherboard that supports
1GB RAM or more, and then cache everything you can, counting on that
to get you through any rough spots.  I saw a cheapo $99 motherboard
that supports up to 2GB at my local PC place just today, so they are
out there.  A lot of motherboards only support up to 768MB (3x256MB
DIMMs).

Remember that even if Postgres itself doesn't / can't use all the RAM,
the filesystem caching will still be in effect for reading.

I would think that you should be able to serve a decent amount with
your current machine.  Even if you are saturating a T1 completely, you
are using less than 20MB per minute of disk bandwidth between logging
and reading (10MB internal usage and 10MB reading/logging/db
searching).

If you are serving a "highly graphical" interface, that is a lot of
images, right?  Again, RAM might be the best answer - no need to hit
the disks.

The question of Karl's templating vs manual rewrites makes me ask the
question of how many pages you need?  What I mean is that you seem to
want bboard, ecommerce, homepages as the main modules that need to be
heavily modified for your interface.

Jim Davidson of AOL gave a talk at a Tcl meeting a few months ago on the Digital City solution.  It is run with a surprisingly small number  of servers (when you consider the bazillions needed to run microsoft.com).  They've put a lot of energy into the design, figuring  out what database queries can be cached rather than executed each time, etc.  I think that's the kind of thing you'll need to be looking  at when your site gets busy, because the single database server is probably going to be the bottleneck.  The presentation may be up on the AOLserver site, I'm not sure - if not, Jim's registered at the site so you should be able to find his e-mail address and ask him where it exists.  This is the single most useful thing I can think of, because it goes  into the architecture of Digital City in decent detail.

Postgres 7.1 will have the ability to cache query plans, and if you do  so you'll be able to avoid the overhead of parsing and optimizing queries.  Since newer versions of the ACS will name queries, it should  be possible for us to make use of this feature, though it will be a memory hog.  Along with query caching, there's good potential for increasing the scalability of PG-backed websites down the road.

WHOA!  You really don't want to do this, as you have no claim of data integrity in this case.  -F turns off the fsynch'ing of modified data files after the end of each transaction.  If your system goes down, the database will be toast, guaranteed.

Now ... -F makes no difference for read-only selects, which form the vast majority of ACS queries.  And the ACS does its inserts in transactions so the -F penalty is not nearly as bad as worst-case testing would imply.

So in practice -F is only going to slow up a running ACS installation by, oh, 10% or so? unless people are doing tons of forum postings, etc.  Even then I bet it won't be as much as you fear.

A better approach to speed things up is to move index files to another  disk drive.  Currently this really sucks in PG because you need to move the files by hand and link to them in the database directory, but  people are working on implementing a tablespace-like facility to manage such things from within PG.  This wins for the same reason it wins with Oracle - it minimizes head travel.  Even though hand-moving and hand-linking sucks, it doesn't suck nearly as bad as guaranteeing your database will be permanently hosed due to a crash with -F enabled.

And work is also underweigh to implement write-ahead logging in PG.  This not only increases data integrity, but only the log will need to be fsynch'd at the end of transactions, rather than all datafiles modified by a transaction.  Since rational people will put the write-ahead log on its own platter, this will be a quick operation because the log will be written sequentially and will be the only file  open on the drive.

Of cousre, if you do nightly dumps and are perfectly comfortable with the high probability that you'll probably have to roll back to one of the dumps eventually, the use of -F is just fine.

But, if you're doing financial transactions or keeping data you don't want to roll back to a nightly backup, you certainly don't want to do this.

Actually, hosing isn't 100% guaranteed.  When the operating system does the synch at its leisure (the 30 second standard linux interval),  it writes blocks out in any order it wants to (hopefully optimizing disk access!).

If it writes out the transaction log before the datafiles during this process then you'll have transactions logged as complete without the data really being there.  Not a good time to crash.

I'm sure there are plenty of other ordering issues that come up.

The key is that PG uses fsynch to ensure that datafiles are synch'd before the transaction log.

Of course, if you're minimizing your risks by using a UPS that can tell Linux when to shut down, and are running RAID 1, your odds of not  experiencing a hosed database go up a lot.

And, of course, if you're willing to lose a day's data (going back to your most recent backup), -F might well be worth it.

It just makes me cringe a bit :)

Now, until 6.5 came out, even read-only selects fsynch'd causing the transaction log to needlessly by synch'd, causing one's disk drive to go absolutely nuts under load.  I pointed that out, and it got fixed for that version.  Seems that none of the developers ever used it in our favorite way (i.e. backing a website) and hadn't noticed ...  -F REALLY improved performance for ACS-like scenarios back then!

"(bought cases with lots of fans from rackmount.com)"

My SMP box has three case fans (it came with two), plus disk coolers for the two fast SCSI disk.  Noisy as hell.  I don't care, it's downtown, not in my house!  Fans are something too many people ignore,  aggressive cooling can really increase reliability and extra fans can  save your box when (as is inevitable) a fan dies.  If you only have one fan, and it dies, you're in trouble.

I don't think you can buy i820/i840 with the MTH (SDRAM) anymore, Chris's experience is why Intel recalled all i820/i840 mobos with the MTH and shipped folks boards with RDRAM (including people who bought from sources other than Intel).  Manufacturers should only be offering  RDRAM solutions - which are still expensive (around 2-3x the price of PC100 at the moment).  I was warned off MTH+SDRAM solutions before the MTH bug and subsequent recall event, because their performance sucked compared to GX boards (even with expensive RDRAM, GX boards stack up very well), so I managed to avoid Chris's negative experiences with such boards.  Chris has my sympathy, Intel really screwed up.

I've never used the Intel 810-based board Chris mentions, but my linux development machine at home is built around an equivalent SuperMicro 810E board (the 810E supports 133MHz FSB Coppermines as well as older Katmai PIII and non-E (100 MHz FSB) Coppermines), and I've been very pleased.  810E and its replacement, the 815E (supports PC133 memory, 810E only supports PC100), both have integrated video.  My linux development  machine is "headless" (I have an X-server on my windows box) so I only needed bare-boned video in order to install linux.  The integration saved me $25.  If you're building 1U boxes like Chris, more importantly it saves you a slot for a video card.

The gamers who  run most hardware review sites loath these boards but they're great for building a cheap, reliable machine if you're not planning to devote your life  to playing Quake on it.  These boards only support socket 7, not slot one, though - Celerons, Celeron IIs (Coppermine core), and Coppermine FC-PGA packaged PIIIs (up to about 733MHz are available in socket form, I think, at the moment).  The socket configuration helps keep the resulting machine tiny and tidy.

In short, the quiet ones push more air than the cheap pieces of crap you buy with your imported case anyway. Don't worry about it!

If you want a great fan buy one made in the good ol' US of A (i.e. PC Power & Cooling). There may be exceptions (although I don't know of any).

This isn't a slight on Taiwan, it is simply fact. Taiwan is into mass production and products for the masses. That is ok. PC P&C is into building the best fans,cases, power supplies and etc. and don't market to the Fry's/Consumer crowd.

Well, I wasn't trying to start a bun fight with you by asking you to retract anything... I was merely just trying to ah, educate you by pointing out that the real world is a bit more complicated. You'd understand if you actually worked within the electronics supply/manufacturing industry.

For example, are you certain that the fan you like is in fact made in the USA? Hmmm... nothing on their site tells me that this is the case. But just suppose there is a sticker on it that says it is made in the USA. Well, what about the components? In fact, the content and labor requirements to qualify something as made in the USA are surprisingly complex and low. See http://www.ftc.gov/bcp/conline/pubs/buspubs/madeusa.htm for more information.

The reality of the situation is that almost no manufacturer is vertically integrated enough to make anything in-house these days.

It's really got nothing to do with "mass market" products. Niche products and the components in them are actually more likely to be sourced overseas because the low volumes make it prohibitively expensive to make them in-house. For example, the tooling required for just the fan-blade can easily run to US$20,000. You'd have to sell an awful lot of fans to justify this alone. And this isn't even counting the tooling cost of the connectors, PCB, etc.

Anyway, rather than pollute this thread with something off-topic, I'll stop now. Just email me if you'd like to know the dirty secrets of the electronics supply/manufacturing industry.

I've never had a CPU fan fail catastrophically before. First time for everything, i guess. And it happened within 72 hours of being switched on, so I can't be too bitter.

I am annoyed about the "china-syndrome" series of mishaps that prevented the computer from shutting down automatically when the bad fan was detected. I guess asking windows + my mobo manufacturer's shutdown utility to work properly is too much to ask 😊

It's a good reminder, though, not to leave new hardware unattended.

For starters, the kind of bin-sorting he describes is done for *all* CPUs, otherwise you wouldn't have the number of speed steps available that are available.  Secondly, Intel juggles the numbers released in various speed categories in order to help prop up the price of higher speed parts.  This is one reason they introduced clock-locking, such a  high percentage of certain Pentium vintages ran well at higher-than-rated speeds that remarking became a real problem.  This is one reason why so many of the earlier, lower speed (i.e. 500 MHz) 100 MHz FSB Coppermine run just fine plugged into a 133 MHz FSB motherboard.

I can't think of any particular reason why Intel would want to sell higher-quality parts in retail boxes to people who are frequently going to overclock and otherwise abuse the CPU...that user will buy a CPU every couple of years, while the high-end OEMs will buy CPUs by the pallet-load.