Forum OpenACS Development: Re: OpenACS Roadmap 5.3
The current system of piling all things into /packages is no longer appropriate for a system of this complexity, and greatly inhibits integration, release, and regression testing. As Dave noted, 5.2 took a long time, and a smaller set of features might be a better next bite. As well, as becomes clear when searching through past forum entries, the needs and desires for some applications have conflicted with the need and desire for a stable, releasable, progressive codebase. Further, duplication of function in one package to add just a few features has led to abandoned hulks cluttering up the landscape. Just like any good piece of software, eh?
I don't know the size of the original ArsDigita team, though I imagine it was small. The organization of the ACS code certainly looks like it was managed by a tight knit team with high-bandwidth internal communications. Today, however, the team appears to be diverse; in geography, need, and coding style. The current codebase does not serve the product well in that environment.
The problem lies primarily in regression issues. This is an age old problem. Every successful large software project is riven by the need to produce quality code and the need to add features. Stable code is mostly mature code. New features are necessarily less mature than old code. And so it goes.
Fortunately, the solutions are nearly as old and are quite proven: separate compilation and functional segregation. Though a scripted system cannot benefit directly from separate compilation (notwithstanding the inestimable shoulders of the giants of the aolserver, oracle, and postgresql), it can benefit, and greatly, I submit, from functional separation. To that end, I propose a separation of the packages into core, base, system, contrib, test, and site. The migration of packages/* would look like this:
packages/acs* -> packages/core/*
packages/calendar -> packages/base/calendar e.g.
packages/dotlrn-* -> packages/system/dotlrn/*
Putting the packages currently called packages/acs-* into packages/core means that they can be tested as a group and released as a group. If features are necessary for the market acceptance of the system, then features can be migrated to core as they mature. New packages, as they mature and gain acceptance, can be migrated to the feature area.
Now, I'm expressly not suggesting a wholesale change in internal namespaces. While there may be some advantages to that, namespace changes are often hijacked to become architechtural overhauls, and I don't see any advantage worth the risk of that. What I am suggesting is modification to the package manager, so that it looks for packages in an ordered set of subdirectories, perhaps with a control list that an administrator can use to control the order.
The benefit from this will flow from a subsequent decision o update the core and system directories separately from various features, perhaps even from each other. Thus, the ore can be refined and released on a regular basis. Regression testing of feature packages won't hold up the ore. Fixes that address critical issues will not have to be tested against every possible combination of packages, only against those in the system collection. Over time, this will allow the system-core interface to stabilize such that poblems can be readily localized. The result will be that the fix to a problem with a feature will lie within the feature. These benefits, of course, are what are made of changes as I propose. Merely re-arranging the files will do nothing.
Clarity. It is not at all clear for someone not intimately familiar with OpenACS which component belongs where, acs-* aside, of course. And that illustrates the point. How does one signify that a component is now essential to the core function of OpenACS? One cannot rename it, as that will break extant installations. So, one must make a note on a web page. My experince with OpenACS is that it is more difficult to learn the website than the software itself. And, as I imagine that folks would rather work on software than websites, it seems a good way to add clarity for new adopters.
Torben Brosten> The core packages are listed here:...
My point exactly. If you want someone to be able to jump right in and go to work, make things clear in the code and its organization. Lookaside lists are just something else needing updating and inviting forgetting.
Cheers!
But none of that is particularly interesting anyway. What matters, is what do people do with these classifications? How do they help, and what features should be added to what code or applications to enable that? That's what you should be thinking about. The implementation details of mapping packges to various categories are trivial, and best ignored until you know what you want to do with that info.
Andrew Piskorski
all package files would need to be moved both in CVS and in everyone's file systems anytime...
Yes, that's an issue. Filesystem is a tcl script run at upgrade time. CVS is an issue, for sure. Not insurmountable, but not trivial.
Some sort of text or Tcl file mapping each and every package to the "group" that it belongs to seems better to me...
In light of CVS, that seems a good compromise. But see my point on /contrib, below.
FWIW, I presented my view of an implementation, because I feel it very uncultured to present requests of free systems without providing an example. Truth be told, I am often criticized at the office for explaining examples rather than stating requirements.
You are very correct in saying the important thing is what people *do* with classifications. I am motivated by my experiences coming up to speed with ACS over the past six months. It is not at all clear which packages, other than acs-* are stable, etc. There are release numbers in packages, and there is dependency information. However, they don't convey the information that I believe a new programmer would need, namely, what can I count on/what to use now.
To your point, what would I *do* with such information? Or have done.
1) Decouple the core acs functions from the others in the release process, providing greatly reduced loads on the regression testing process, and providing for easier and cleaner releases of core function. Same ease of testing and release goes for other packages and package groups.
2) Provide for ease of importing a new package/packages. Having one dir makes it difficult to import from other CVS repositories.
It was this rooting around moving things from contrib into packages that was the root of my proposal. Moving them into packages breaks the CVS. Sym-linking them seems to work. But there is no way to cleanly deal if OCT moves the package to packages from contrib. If contrib packages can be made to appear to the package manager, then one can easily see how separation would work. The same issue exists if someone else chooses a name for a package that conflicts with a site-local package name.
Site-local, contributed, third-party packages should not clutter up the packages directory. Indeed one can argue they should not share the same namespace. But I said I wasn't arguing for that :)
3) Ease the retirement of deprecated packages, and the introduction of new packages. There are a number of packages in the distribution that are confusing, bulk-mail vs spam, address-book vs. contacts, portal vs new-portal.
How is one to know? Indeed, are they even kept up to date with the release? Should they be?
One can certainly root around in the OpenACS site to find these answers. One can also find conflicting answers. There is nothing so disambiguating as looking at the actual released code and seeing what's where.
Thanks for taking the time to discuss this with me. Obviously, I feel strongly about this, but I also understand the dangers in talking things into the ground. I hope I haven't done *too* much of that.
Cheers!
