Forum OpenACS Development: RFC: Rules for Version Numbering and tagging of non-core packages

Problems I'm trying to solve:

  • How does an OpenACS admin know which versions of which packages are compatible with their version of OpenACS Core? Assuming most admins use the repository, this problem is the same as, how do we build a repository channel with the most recent compatible version of each package?
  • How are non-core packages versioned?

Proposed Rules

  1. I have updated the Release Version Numbering engineering standard to reflect current practice.
  2. The core packages are acs-admin, acs-api-browser, acs-bootstrap-installer, acs-content-repository, acs-core-docs, acs-kernel, acs-mail, acs-messaging, acs-notification, acs-reference, acs-subsite, acs-tcl, acs-templating, acs-service-contract, search, automated-testing, acs-authentication, ref-timezones
  3. All core packages released with the same version number.
  4. The kernel version is the ultimate authority in case of discrepencies.
  5. Core module versions can lag behind the kernel during development but must all be syncronized at each release.
  6. A version number for the core packages should be reflected both in the .info file and via cvs tags.
  7. CVS tags for core packages follow the scheme openacs-x-y-zw (see the Version numbering doc linked above for details). These tags are historical markers and should not usually be moved.
  8. Only core packages get openacs-* tags
  9. All other packages are versioned package-x-y-zw, where x-y-zw are specific to each package and not correlated to any other package or to OACS core. These tags are also historical markers and should not be moved.
  10. The newest version of each package which is compatible with each version of OpenACS since 5.0.0 is tagged openacs-x-y-z-compat. These tags apply only to non-core packages. These tags can be moved as new versions of packages are released, so long as the new packages retain compatibility with the specified version of core.

Implementation

  • We are already following this scheme for core.
  • We needn't bother to retroactively tag anything; the basic goals are satisfied if we simply start tagging new package releases as we go forward.
  • The repository generation code (and package inventory list generation code) needs to know this naming scheme and generate packages accordingly.
What is the reason for CVS tagging non-core packages with "openacs-x-y-z-compat" rather than the same "openacs-x-y-zw" used on the OpenACS core? If I want a particular version of the core plus the latest compatible version of all non-core packages - which is the usual usage case - then it would be a lot simpler if they all use the same "openacs-x-y-zw" tag. In that case of course, the "openacs-x-y-zw" tag indicates something different for core and non-core packages, which might or might not be a perceived problem.
If we use the same tag for core and for compatible, it's easier to confuse the core, historical tags with the per-package floating compatibility tags - we are using the same syntax for two different semantics.  If we do it the other way (the way in the RFC), we keep that distinction clear but broad cvs checkout by version becomes two steps instead of one.
Why don't we use two trees then? One ACS-Core tree, that should get you started so you could install packages from the OpenACS Repository over the web and one for the packages.

For each release we keep a matrix of *reported & tested* package versions (either .info file or cvs tag).

This way we can name the packages any way we want. The Repository will only generate the packages based on the Matrix. The matrix will be filled by OpenACS users.

Which brings up another idea. Automated error reporting by websites. I know, we might as well first get our hands on reporting installations back to the system. But if we'd log all error messages with a package_key and have a sweeper on the error log, to see which packages generate errors, we could automatically report back which package version works with what core version and which does not. Though we'd have to make sure the files have not been changed after the download from the repository. Well, most likely too much work to implement.....

Joel, have you tried the CVS steps in question? Because I have not, but I suspect it is not "two steps instead of one", it might be more like N steps instead of one, where N is the number of non-core packages you care about.

However, as long as somebody's actually tested out using your "openacs-x-y-z-compat" CVS tagging convention, has concluded that in practice it doesn't cause trouble, and has specific instructions and examples of how to use it (which can later go into the docs), then sure, in that case it should be fine. In other words, I think the basic organization and strategy of your CVS tagging conventions above make sense, I'm just questioning whether anyone's checked whether in practice it is convenient and simple to work with, or if it causes extra trouble that might not be worth it.

The current package locations in CVS precisely mirror the locations they need to actually be in the CVS checkout in your filesystem in order to actually use those packages. This is a good thing. Malte, I'm not sure what you mean exactly with your "two trees" proposal (clarify please?), but if you mean changing the CVS repository around in ways which move packages into non-runnable locations, I don't really see why that would be worthwhile.

Oh! An alternate solution might be to just put the "openacs-x-y-z-compat" tag on both the core and non-core packages. I mean, a core package would still have the "openacs-x-y-zw" tag, but at the same location, it'd also have the "openacs-x-y-z-compat" tag. That gives one tag that is identical across all the latest packages compatible with a particular core release, regardless of whether those packages are part of the core release or not.

I'm sorry, Andrew, your alternate solution is simply too obvious to work.  :)
My idea is to use a non-core package repository as we do with dotlrn at the moment. If you want to have more than your usual set of packages, you can either install them using the webinterface or you can do a cvs checkout out of the non-core package repository. Each package could get a tag for each core-version it has been reported to work, so you could say:

cvs -d /non-core-cvsroot -r openacs-5-2 forums

This would allow in the long run to distribute packages not from the cvs repository at OpenACS.org. This might be good, if someone forks a package for a reason, that does not apply to everybody but a certain subset of OpenACS users.

But it does not have to be that way.

Andrew's suggestion is elegant. But there's no reason that the same "standardized" tag couldn't be used across isolated repositories as Malte suggested. And, the same version of a package could be tagged with multiple "compat" tags. Sounds pretty cool to me...
If there's a desire for it, Malte's suggestion sounds basically reasonable, but there are some drawbacks.

If in your packages directory, you have package A checked out from one CVS repository and package B checked out from another, doing CVS operations withing either package A or B will work just fine, but doing any operations across both package A and B at once (e.g., "cvs -nq up") will not work. At least, it didn't work when I just tried it.

The packages directory has a CVS/Root file too of course. Let's say that it points to repository-A, but package-B is checked out from repository-B instead. Well, repository-A doesn't have any packages/package-B directory, so when CVS sees package-B, it just silently skips it. The fact that packages/package-B/CVS/Root points to a different yet valid repository makes no difference.

For any real site you'd already have vendor imported everything into your own CVS repository anyway, so it doesn't matter there. What about OpenACS developers doing stuff that requires co-ordinated change across multiple packages? Malte says dotLrn CVS is set up that way now, so re-factoring and moving code from dotLrn into OpenACS should be a good test case. You guys who've done any of that, are you ok with the way CVS has worked for you there?

Btw, it'd be good to hear first hand experiences with split repositories, but until I do, my gut feeling is to leave as much as possible in the main OpenACS CVS repository unless there's a particularly strong reason to do otherwise. Having everything in one place is nice, and CVS just isn't designed to deal with distributed anything.
My first hand experience from releasing both OpenACS and .LRN, which is on a separate repository on the same box: mildly annoying, not critical. Had to write and debug some extra batch commands to keep it convenient, and it introduces an extra layer of opportunity for error. Would rather not do it without a pressing reason.