Forum OpenACS Q&A: Package Maturity Standards

Posted by Joel Aufrecht on 01/26/04 08:56 AM

I propose that we add a new piece of meta-data to packages, "Maturity Level." This would be set in the .info file and visibile in the repository, installer, and views of currently intalled packages. We would also show the package author (already in the .info, I think) whereever we show maturity level, so that the user knows who is asserting a maturity level. (Author for core modules should probably "OpenACS release team" or similar.)

Proposed levels:

Level 0. No information available. (default for packages that don't have it set explicitly, and for new contributions)
Level 1. Buggy. Works for some people, has problems that make it unsuitable for general use.
Level 2. Approved Contribution. Documentation exists, no serious bugs found, code has been looked at by 2 parties other than the author (for obvious coding problems, back doors, etc)
Level 3. "Stable" or "Conformant" or some similar word. "Certified". Same as 2, plus i18n support and adherence to current coding standards. No namespace conflicts with other Level 3 or higher packages.
Level 4. "Actively Maintained" ?? Do we need this? Same as level 3 + active maintainer (measured by actual response time to bugs?).

If feedback is positive, i'll TIP it.

2: Re: Package Maturity Standards (response to 1)

Posted by Torben Brosten on 01/26/04 09:13 AM

Suggest adding a reference to the OpenACS version that relates to the maturity level posted. For example, ecommerce package might be level 3 or 4 for 4.6.3, but level 0 or 1 for 5.0.

Hmm... Aren't the packages going to have their own blog soon (ie when openacs.org is updated)? and if so, wouldn't that be a better place to track this?

3: Re: Package Maturity Standards (response to 2)

Posted by Jeff Davis on 01/26/04 09:51 AM

I also think "actively maintained" should be orthogonal to code maturity level; there are plenty of level 1 or 2 things which are actively being maintained and plenty of level 3-ish things that are not and are mostly static (well, although right now I guess nothing would really be level 3 as there are no formally documented coding standards).

4: Re: Package Maturity Standards (response to 1)

Posted by Malte Sussdorff on 01/26/04 01:22 PM

This sounds like a really good strategy. And I agree with Jeff, make "actively maintained" orthogonal, if we have an active maintainer (that might differ from the author).

Furthermore, we should add a filter to the APM, so you can select to only install packages that have a certain maturity level.

5: Re: Package Maturity Standards (response to 1)

Posted by Dave Bauer on 01/26/04 03:02 PM

I like this idea. Let's make sure the Levels are readable in the info file. That is "<maturiry-level>0</maturity-level> won't be very helpful. So we should make sure the content of the tags is informative in itself.

I also agree that the active maintenance should be seperate.

6: Re: Package Maturity Standards (response to 1)

Posted by Caroline Meeks on 01/26/04 04:58 PM

Another useful and even straightforward evaluation would be a checklist of coding standards. This would give the developer an idea of how difficult it will be to customize the code.

Here are some examples:

Uses ad_form? yes/no/occasional/NA
Uses template::list? yes/no/occasional/NA
html in adp files? yes/no/occasional/NA
Queries in xml files? yes/no/occasional/NA

7: Re: Package Maturity Standards (response to 1)

Posted by Jade Rubick on 01/26/04 08:02 PM

These are all excellent ideas.

8: Re: Package Maturity Standards (response to 7)

Posted by Joel Aufrecht on 02/02/04 01:40 PM

Draft #2:

Each package has a maturity level. Maturity level is recorded in the .info file for each major-minor release of OpenACS, and is the appropriate value for that release of the package.

<maturity>
  <platform id="openacs-core" version="5.0">-1 - Incompatible</platform>
  <platform id="openacs-core" version="5.1">2 - Mature</platform>
</maturity>

Maturity level is visible in the Installer and when browsing the repository.

Level -1: Incompatible. This package is not supported for this platform and should not be expected to work.
Level 0: New Submission. This is the default for packages that don't have it set explicitly, and for new contributions. The only criterion for level 0 is that at least one person asserts that it works on a given platform.
Level 1: Immature. Has no open priority 1 or priority 2 bugs. Has been installed by at least 10? different people, include 1 core developer. Has been available in a stable release for at least 1 month. Has API documentation.
Level 2: Mature. Same as Level 1, plus has install guide and user documentation; no serious deviations from general coding practices; no namespace conflicts with existing level 2 packages.
Level 3: Mature and Standard. Same as level 2, plus meets published coding standards; is fully internationalized; available on both supported databases.

9: Re: Package Maturity Standards (response to 8)

Posted by Malte Sussdorff on 02/02/04 02:27 PM

I like the second draft a lot, though I doubt that we can give Level 1 for a lot of packages, if the number of people is higher than 5. Furthermore, with each new release this would mean to switch all packages back to Level 0, as we won't have the ability to go beyond Level 1 before the release. Last but not least, how do people report about an installation of a package.

Could you give your estimation of the maturity of some example non-core packages ?

10: Re: Package Maturity Standards (response to 1)

Posted by Dave Bauer on 02/02/04 03:44 PM

Joel,

Seems like maintaining every acs-core version will be a pain. What is the default is a version is not listed?

Otherwise, I think this is a good idae.

11: Re: Package Maturity Standards (response to 8)

Posted by Bart Teeuwisse on 02/02/04 07:50 PM

Joel,

I like the idea but would change the implementation slightly so that the level # and description are encoded in different XML tags (or attributes). E.g.

<platform id="openacs-core" version="5.1">
  <level descr="Mature">2</level>
</platform>

This encoding style removes the need for parsing the maturity level string in your example and de-couples the description from the level #.

/Bart

12: Re: Package Maturity Standards (response to 9)

Posted by Joel Aufrecht on 02/04/04 10:42 AM

I would estimate that file-storage, photo-album, weblogger, bug tracker, forums are probably level 2 (Mature). I doubt anything is fully Level 3 (Mature and Standard) yet.