Forum OpenACS Improvement Proposals (TIPs): TIP #47: (Implemented) Package Maturity Standards

Proposal

Each package has a maturity level. Maturity level is recorded in the .info file for each major-minor release of OpenACS.

<version ...>
  <maturity>1</maturity>
</version>

Maturity level is visible in the Installer and when browsing the repository.

  • Level -1: Incompatible. This package is not supported for this platform and should not be expected to work.
  • Level 0: New Submission. This is the default for packages that don't have it set explicitly, and for new contributions. The only criterion for level 0 is that at package is uploaded into the repository.
  • Level 1: Immature.
    • Meets Level 0 Criterion
    • Current release has no open priority 1 or priority 2 bugs.
    • Has been available in a final major-minor release for at least one month and major-minor-dot release for one week.
    • All API functions have documentation
    • Source code is available at published location.
  • Level 2: Mature.
    • Meets all Level 1 criteia
    • Has installation guide and user documentation
    • OCT member verifies no serious deviations from general coding practices
    • Has no namespace conflicts with existing level 2 packages
  • Level 3: Mature and Standard.
    • Meets all Level 2 Criteria
    • OCT member verifies code meets published coding standards
    • Contains no non-internationalized text
    • Available on all supported databases.

Reasons

We don't have any objective way to determine the quality of our 100+ packages

Disadvantages

Volunteers will have to do ongoing work to determine maturity.

Implementation

  • The APM needs to parse and display this new information when looking at local packages and when looking at lists of packages to install. A volunteer needs to do this.
  • The repository generator must parse and display this new information so that it is visible when browsing the repository. I will make sure this is implemented.

Changes

Corrected example to match implemented code. JoelA 30 Nov 2004. Old example was
<maturity>
  <platform id="openacs-core" version="5.1">
    <level descr="Mature">2</level>
  </platform>
</maturity>
Collapse
Posted by Malte Sussdorff on
I like this revised version a lot better than the intial draft. Yes, Joel, thanks.

Approved.

Collapse
3: Re: TIP #47: Approved (response to 1)
Posted by Lars Pind on
Approved
Collapse
Posted by Peter Marklund on
Approved
Collapse
Posted by Don Baccus on
What happens if a mature package is hit with a level 0 or 1 bug?  Does it then get bumped back to 0?  Does it then go through the review process steps to get back to being "mature"?  Who actually bumps it back to 0? The bug czar?

I like the idea but am concerned about the implied amount of effort and want to get a better handle on how much is involved.

Collapse
Posted by Joel Aufrecht on
I think it needs to hit the bug hurdles to get promoted in maturity, but it shouldn't in turn get demoted when new bugs are found, unless the bugs go unfixed for a long time or are marked "will not fix".
Collapse
Posted by Don Baccus on
That sounds good to me ...

Approve.

Collapse
Posted by Joel Aufrecht on
Marked approved
Collapse
Posted by Joel Aufrecht on
We're going to implement some simpler XML:
<maturity>1</maturity>
Rationale:
  • We don't have a use for the id field
  • The version field is redundant because it's in the same file already in three places. Instead of having three different blocks with different versions in one file, we should have one block in each different version of the file.
  • The description should be a message key, so that it can be internationalize. We'll code the mappings of integers to message keys in the page code.
Collapse
Posted by Roberto Mello on
Joel,

Since you seem to be revising your TIP, could you please post the modified XML that would have to be implemented?

Also, with respect to the TIP process, does this mean a revote is in order (once your modified version is posted)?

Thanks,

-Roberto

Collapse
Posted by Joel Aufrecht on
The revised XML is in my last post. It's really as simple as that: just
<maturity>1</maturity>
I think we're going to run in changes between the approved TIP and the implementation from time to time. It can happen when the TIP is incompletely specified, which we should fight hard against because incomplete specs usually go hand in hand with underestimated costs and consequences and delayed implementation. It can also happen when there are problems or obvious changes with the TIP that become apparent during implementation.

Here are two options on how to handle this:

  1. Changes must always be posted in the original TIP, and that posted changes are legitimate amendments if they are not vetoed within 1 week. Veto means that the original TIP must stand or be re-TIPped.
  2. Changes must always be posted in the original TIP, and must then be approved according to the same rules as TIPs - two yes and no no in a week, or full vote in two more weeks. Veto or negative vote means the original TIP stands or must be re-TIPped.
Collapse
Posted by Nima Mazloumi on
Too bad that we don't have test cases for each packages. If that would be the case you could automatically set or reset the maturity level depending on the number of cases that run successfully.

Another option would be to include bug tracker in the process so that the number of open bugs influence the maturity level as well.

The danger I see with the proposed maturity level standard is that it lacks of some automation. This will results packages to reach a high level during their life cycle and stay at a high level even if the code base get's messed up over the time.

Therefore I would suggest that the level is set involving the testing results and the bugs in bug tracker.

Collapse
Posted by Alfred Essa on
What needs to happen so that this TIP is actually implemented? And what are the views of the OpenACS Core Team on this issue? Thank you in advance.
Collapse
Posted by Gustaf Neumann on
it would be a great help for the release process, if the package manager could be extended such that it shows prominently the maturity level, allows to show the differences of the locally used packages to the approriate version in cvs, and supports directly a commit for authorized developers after running the regression tests (the tests of the package to be committed and maybe as well the tests of the packages, that depend on that package).

Only packages that pass the tests should go into the release or beta versions, others should go into something like the HEAD version.

The package manager could as well use categories to flag where the packages "belong to" (core, dotlrn, commerce suite, project, ...). This would be an alternative to the discussion, whether the packages directory should be restructured (see 5.3 roadmap discussion). This would help especially beginners to see what's there and what is ready to use for what purpose

Collapse
Posted by Malte Sussdorff on
The first part is easy and hopefully someone with a little bit of time can show the maturity level (actually, isnt this shown already)?

Differences between local and CVS version would work if CVS diff was reliable, but somehow I never got it to show me differences when something changed on the server side. Any pointer, so someone can write a page which takes the package-key, looks for the branch TAG in the CVS directory (or for something in the APM which stores what repository to use) and then execute the CVS diff and display the result on the page?

Automatic commit is nice, same is regression test, would be interesting to see it in action.

As for the categories, this requires a TIP but should be implemented in 5.3. It should be a parameter like ā€œ< belongs_to name=dotlrn version=2.2>ā€. No need to restructure the packages directory then.

Additionally, we should have a switch which allows us to mention if a package uninstalls correctly once installed. Though this would probably a good maturity requirement, the conditions in packages and the linking between packages can sometimes prevent this form happening (Iā€™m just thinking about people uninstalling project-manager if it was running in a mixed .LRN/contacts/invoices environment. Cascading will most likely break).