My experience with automated testing hasn't been quite as good as Simon's. It was brought in at a big software company I used to work for, and ultimately the quality of our QA went down, not up. The reasons why could fill a novel; the main ones were a) once an automated test exists, most QA testers will assume the feature has already been tested and give it a cursory glance at best, and b) the quality and coverage of an automated test is only as good as the person who wrote it and the amount of time they had to spend on it. So we quite often had a skimpy automated test hitting the high points, and very little else being tested. We in development found that there were a lot of bugs being found at the last minute, or even worse being found by customers, because they weren't caught by the automated tests.
However, my reason for posting is not actually to argue against automated testing. It obviously works better for Simon than it did for us, and I have no reason to think he can't make a useful initiative out of this. What he is proposing is somewhat different from my experience anyway, since in our case the QA testers were writing the automated tests themselves. Whether or not that is better than having the developers do it, from a testing methodology point of view, is an exercise left to the reader. :)
My point is that this community project is somewhat different from what Open-Msg does as a company. You guys can decide on a strategy and require everyone to abide by it. IMHO the community can't, or at least shouldn't, do that. We can make suggestions of what we think best practices would be, and we can make sure that packages that come with automated tests are marked in some way as having been especially well-tested. But I don't think it's right to say you can't contribute a package unless it includes an automated test, just as I also don't feel we can require all contributed packages to follow the 4.x programming paradigm perfectly. So I don't think that the idea that this should be *required* is a good one - just make it attractive and hope that people will participate.
Now, Simon, you gave us a slightly breathless :) description of all the benefits of automated testing. There must be some drawbacks, no matter how small - how about describing those, too? Right now your story sounds too good to be true, and people aren't likely to buy into it until they know the costs as well as the benefits.
