Forum OpenACS Q&A: State of OpenACS 5

Collapse
1: State of OpenACS 5
Posted by Joel Aufrecht on 08/26/03 02:09 PM
I think OpenACS 5, release management, automatic install, and automated testing are all related, and I would like to solve or untangle them all. The purpose of this post is to lay out perceived problems, get people to agree, disagree, or list their own perceptions; to propose solutions, and get discussion of the best solutions.

Problems

OpenACS is lagging behind in quality control. We have a lot of unfixed bugs; those bugs probably don't represent full coverage because we use only sparse, infrequent manual testing; we are at risk to break a lot of stuff whenever we make architectural changes because we don't have regression testing.

OpenACS has very inconsistent releases. Schedules are very unpredictable, the methodology and people change for each release, and there is no clear, publicized, consistent road-map for future releases and their features.

Solutions

The "best practices" for quality control include:

  • daily build and smoke test. Every day, the current code from CVS is exported and a new system is installed. The system is then tested, including basic functionality for all features. This is unfeasible without automated install and automated test.
  • regression testing. When a developer makes a change, she can run a suite of tests covering existing functionality to ensure that her change has not had unintended effects. Again, this must be automated to be practical.

We need the following changes to OpenACS to support this:

  • Full suite of automated tests
  • Automatic install scripts
  • A dedicated server for the community that does daily build and smoke test.

In order to get full support for automated testing, we need to integrate tclwebtest or similar functionality into the automated-testing module.

In order to get more community involvement with the next release, we need to make it easy for people to participate, especially with testing. This could include:

  • Better documentation for automated testing (tutorial)
  • Ability for community members to create and submit new automated tests and see the results
  • A list of needed automated tests
  • Package maintainers with both responsibility and power to test their packages.

For better release management, we need consistent membership, clear release criteria that we can measure and stick to, and a road map for the future which has funding or time committments from people.

Collapse
2: Re: State of OpenACS 5 (response to 1)
Posted by Jade Rubick on 08/27/03 06:47 PM
Joel, I'm excited to see an organized effort to improve quality control in OpenACS.

I think the documentation portion is really critical. In early versions of your tutorial application, it included references to adding in automated testing. I was excited to see that, and I think once that is fleshed out, it will be a much easier endeavor.

Also, if we require automated testing, then there's more incentive for individuals to do it. Right now, it just seems like a lot of extra work with no real payoff. But if it is documented and everyone is doing it, it becomes much more likely that I'll spend the time to do it.

Thanks for organizing this!