Forum OpenACS Q&A: Proposed post-5.0 changes to openacs bug system

After we ship 5.0, I would like to make the following changes to the bug system:

Severity

Severity indicates how badly the system is broken if the bug isn't fixed. Severity should be set by the reporter. Currently we have six values for severity:
  1. - Critical
  2. - Major
  3. - Normal
  4. - Minor
  5. - Trivial
  6. - Enhancement
I see three problems with this scheme: first, there are too many values and thus too many unwarranted fine distinctions. Second, there aren't definitions for the values. Third, "Enhancement" is out of place, both because we track that information in "Type" and because it hides the severity of an enhancement.

Instead, I propose these values:

  1. - Critical. System crash, security violation, or data loss.
  2. - No Workaround. Because of this bug, a function cannot be made to work.
  3. - Workaround. The bug interferes with functionality but an alternative
  4. - Cosmetic. The bug does not interfere with functionality.

Enhancement severity is ranked using the same scale, to indicate the consequence of not implementing the enhancement.

Priority

Priority indicates how important fixing the bug is to the maintainers. Currently we have priority 1, 2, and 3. Because Priority only has meaning within a development process, it's harder to pin specific meanings on priority. I propose these:
  • 0: fix immediately. Bug has severe consequences for active users. Example: root security violation in stable branch, uninstallable HEAD during release testing.
  • 1: high. Must be addressed before next release.
  • 2: normal. Should be addressed before next release.
  • 3: low. It would be nice if it were addressed.

Found in Version

HEAD should be in the list. If we have automated weekly tarballs or something similar, they should probably also go in the list.

Type of Bug

currently
  • Bug
  • Todo
  • Suggestion
Todo and suggestion should be consolidated into Task.
Collapse
Posted by Eduardo Pérez on
<blockquote> Severity
Severity indicates how badly the system is broken if the bug
isn't fixed. Severity should be set by the reporter.
Currently we have six values for severity:

    1. - Critical
    2. - Major
    3. - Normal
    4. - Minor
    5. - Trivial
    6. - Enhancement
</blockquote>

As these are equivalent to bugzilla Severity field:
http://bugzilla.mozilla.org/queryhelp.cgi#severity
I take the same definitions.

Have you spoken with bugzilla people about this change?
They may have tried to do the same (and maybe found the change to be an error)

As bugzilla works very good we should follow bugzilla unless we know for sure the change won't be a problem.

I'm saying this because bugzilla seems to have much more installed base and both are GPL.

Collapse
Posted by Tilmann Singer on
It would also be useful to have a field to record the database version, if that is technically possible.
Collapse
Posted by Lars Pind on
I'm fine with all the proposed. Easy to implement using bug-tracker's categories, once openacs.org is upgraded.
Collapse
Posted by Andrei Popov on
But categories aren't that multi-filter-friendly, as in I can't combine them all together, can I?  At least in 4.6.(2? 3?) I can't.  (I know I can do it in logger).
Collapse
Posted by Lars Pind on
You can in 5.0 (HEAD).
Collapse
Posted by Joel Aufrecht on
I've used the smaller list of severities in a number of commercial products with good success, and I don't really see them being used effectively in bugzilla. There are three changes in bugzilla's list and my proposal. The first is basically to eliminate 4 - Minor. What's the difference, exactly, "This is the run of the mill bug." and "Minor loss of function, or other problem where an easy workaround is present." To my mind, nothing. Second is to cut blocker - "blocking" is an attribute of priority, not severity. A trivial spelling mistake, if prominent enough, can be a blocking bug. It's high-priority, but it's not high-severity. Third is to drop Enhancement because we store it in a different field - Bug vs Task.

I would also like to start a discussion about priorities and releases. There are several "things" associated with releases and I want to clarify which are fundamental definitive things and which are dependent. For example, what defines a dot release, the smallest release possible (major.minor.dot), from a cvs tag or just a particular date in the tree? I think the basic definition is that any release, down to a dot release, is part of a tested upgrade path. One part of the definition of a major release might be that we break some backwards compatibility.

Then, what are dependent attributes of a release? A tarball is dependent - we cut a tarball when we do a release, rather than releasing when we cut a tarball. Updating docs is another. The full list of these dependent things is probably the Milestone criteria That is to say, when we hit definition X - maybe just the need to release a new feature that's ready and fully backwards compatible, it's time for a dot-release. Once it's time for a dot-release, we have to do everything on the milestone criteria to complete the dot release. Is there anything on that list that should instead be a definitive thing?

Finally, how does this relate to Priority? Priority is a balance between Severity, urgency, cost, and (frequently volunteer, unpredictable) resources. Should we define priority solely in terms of releases? In that case we might have:

  • Pri 0: fix immediately. Used for things that break HEAD or security patches.
  • Pri 1: must fix before next dot release. Even if the dot release is unrelated, these are the most serious bugs that accumulate and should be fixes quickly.
  • Pri 2: must fix before next minor release, should fix before next dot release.
  • Pri 3: try to fix sooner or later.