Forum OpenACS Q&A: Open Developer Discussions
After some discussions with some other developers, I think it might be
a good idea to explore how the OACS community can improve its approach
to collaborative development.
Right now, I think the OACS community is head and shoulders above any
other free software project that I'm aware of. I'm on the mailing list
of quite a few projects simply to keep abreast of other systems, and I
have yet to come across one with a community that is as knowledgeable
and helpful as OACS. We've got some top of the line hackers here and
we've got a lot fewer arrogant curmudgeons than most communities have
(that's a compliment, btw :)).
That being said, some have pointed out that a lot of development goes
on in private. That means that a lot of the important decisions
regarding infrastructure and architecture are not done terribly
transparently. I am as guilty as anyone else in this regard and have
chosen this approach for many projects, for instance some email
dialogues I've had about the admin UI and others regarding ETP.
However, I think that I made a mistake in these times.
Since we're about to release beta, and many of these discussions
probably took place in private in order to expedite the porting
process, we should seriosly consider making a community wide effort to
make development conversations and decisions more transparent for the
next release or generation of OACS.
I don't think that I'm entirely the right person to propose ways to
facilitate this transparency, but I would like to pose this commitment
to the community. But I'll also make it a policy at Musea to try and
discuss general development approaches more public, especially with
regards to ETP.
Any thoughts or comments would be appreciated.
talli
I love transparency!!! Some time ago the openacs community was discussing transparency alongside this awful quarrel about gatekeepers, contribution etc. so I hope that this is not going to happen again 😊
The project on trimming down OpenACS 4.x could be a great starting point for a *transparency initiative*...
https://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0003za
I believe that transparency and easy contribution are important factors for motivation and innovation (where have I read this 😉...
accomplish but is fairly important in the long run.
In the past few weeks, I've received a few murmurs of
disapproval concerning OpenForce's delay in releasing an initial
version of dotLRN. I suspect more people have this feeling and
are maybe reluctant to mention it. We've been struggling with the
right model for dotLRN because we need to do extremely rapid
development, but we also want community input and
participation (it is an open-source effort). It seemed to us that we
needed a solid base before we could fully open up the process. I
think we're reaching this solid base.
In the next couple of weeks, OpenForce is going to work hard to
reach this higher level of transparency. We're asking for
everyone's patience as we progressively add source code
availability, bug submission mechanisms, and eventually fully-
privileged external contributors. But we will get there, and we
have, of course, complete confidence that with a solid base of
code, complete openness is the way to go.
Regarding dotLRN:
In the past few weeks, I've received a few murmurs of disapproval concerning OpenForce's delay in releasing an initial version of dotLRN. I suspect more people have this feeling and are maybe reluctant to mention it. We've been struggling with the right model for dotLRN because we need to do extremely rapid development, but we also want community input and participation (it is an open-source effort). It seemed to us that we needed a solid base before we could fully open up the process. I think we're reaching this solid base.
http://www.tuxedo.org/~esr/writings/cathedral-bazaar/cathedral-bazaar/x147.html
"Release early. Release often. And listen to your customers."
What would a discussion about open source development be without a reference to ESR?
are the ones that start with a solid base. Linux was not a group
project at first. OpenACS received community interest only once
it was at least somewhat functional. Etc...
But yes, we will definitely release often. We wanted to make sure
we had something workable and usable before we put out any
code. We're almost there. And we're looking forward to
everyone's comments, contributions, and real participation.
As far as Michael's suggestion, perhaps we can reach a compromise. I fully understand that releasing undocumented and unsolid code would slow things down for you and your client. That would not be good. Not only would you have to spend time packaging something buggy, but you'd have to spend time responding to community questions and inquiries. You're on a deadline and the community should respect that.
However, perhaps a compromise would be to release the specs of the new system? Meaning some of the overall architecture documentation, design documents and so on? So the community can take a look and have a better idea of how things are going to look.
For one, this would help a lot of the indy developers and companies out there that are trying to sell dotLRN or OACS with dotLRN improvements to clients (and this is not just a selfish Musea request). If this were possible, I think a lot of people would really appreciate it.
Thanks.
talli
I think sometimes people believe their little problem is not worth posting to the bboard because it is not of general interest.
The bboard and openacs.org should be the central store of OpenACS knowledge so that valuable information is available to the community. It is not as easy to post to the bboard as it is to send an email, but the benefits to everyone involved are worth it.
I appreciate all input from the community. When I am working on something I do like to check if someone is working on a similar problem, or has a better approach than one I have thought of.
The model that I think works well is for interested folks to work together on a design and plan for improvements to a particular bit of the toolkit or a new package, and to actively seek community feedback *before implementation*.
If the group feels comfortable using our forum tools to communicate while working on a design, that's fine, but if not, I won't get bent out of shape as long as the toolkit community at large has the chance to make meaningful feedback at some point.
So I find nothing wrong with the fact that Neophytos and I (to pick one example) are privately discussing the fact that we'd like to move Service Contract descriptions to XML much like package .info files and away from SQL definitions that have to be written separately for Oracle and PostgreSQL. When things move further along, I'll certainly ask Neophytos to make a public proposal so folks can comment, suggest improvement etc.
Likewise I've talked some with other folks about possible improvements to permisisons, workflow and other packages and expect to float some ideas in public after I have some more time to muddle through my own thinking.
We could hold all of our discussions in public - there's been a public thread on redoing the context bar stuff and to use the template's "property" feature to move rendering of headers and the context bar to the site's master template (Carl Coryell-Martin's on the hook for this task as of last night, large quantities of beer will cause folks to foolishly volunteer, remember that!). In truth, though, that discussion's serves as an example as to why I don't get terribly hung up about folks discussing features and changes in private - only a very small number of folks participated in this public discussion.
But ... significant changes or new work should be floated in public before implementation whenever possible. In the case of dotLRN I'm not sure it was really practical to do so, as the work was large, and the product spec client-driven and not really subject to change (Sloan has a rather large existing user base who expect to find familiar UIs and functionality in dotLRN as they now have in SloanSpace).
This is the Gnome hacker summary page I mentioned above. Something like this where the module names link to a summary page with an activity log and module owners would open up a nice window into OACS development, especially as plans for fully-privileged external contributors unfold. Not sure if that's the kind of thing you had in mind, Talli, but I think it would be helpful nonetheless.
The purpose of keeping everything in the open is not to enforce a design by comitee process on developers, but to keep anyone who cares to be, informed, and to educate. Although only five people may contribute to a any one discussion, fifty more are subscribed and read it right away, and many hudreds will read it in the future.
I can't learn anything from the closed discussion about an XML format for describing service contracts, and i've learnt nothing from the .LRN project. Maybe I have knowledge that would have saved days of developer time. Too bad, for both of us.
I don't find it practical to say "let's run to the bboard before inserting any comments about adding features to X while discussing such-and-such topic in private".
Also, how about ideas spawned by a single person? Do I have to post every idea I have before solidifying it into a solid proposal? Yuk. Count me out of that process!
And I think in many cases we're better served by having folks put up a target for comment that's organized logically and reasonably well-written.
What I want to avoid is design lock-in and certainly implementation without public feedback. If we meet that goal (and we haven't always in the past) I really don't see any room for complaint. If the design of the portal package written by OF had been presented before implementation I don't think people would be complaining.
It's sad to think that .LRN is unavailable because Ben was worried that community input and participation would slow his development process, that he was under some obligation to provide stable, bug free code from day one, and that it should come with a bug tracking tool and open access to his CVS repository.
I think the benefits of open source are realised at a much lower level of effort. Make the information available, things will happen. That's how we got started, some guy made the code behind his websites available. No obligation, no expectation, here it is, make of it what you will.
While appreciating your desire for a controlled, thoughtfull approach to making changes to the core OACS Don, what I want to avoid is the feeling that people can't kick around ideas in public, and that the only meaningful contribution that can be made requires a support team behind it.
"unavailable," let me attempt to clarify things. The issue is not
that community input would slow down the development of
dotLRN. The issue is that someone needs to set a direction for
dotLRN before it can take off. That's what we've been doing.
Finding that direction. Defining the initial high-level goals of
dotLRN and taking a first pass at implementation of this. We are
almost there. These decisions cannot be made by committee. Is
dotLRN a course management system? A learning
management system? An extension to OpenACS that everyone
will want to use? *Those* discussions would have slowed
things down.
And don't downplay the role of CVS and bug tracking. One of the
biggest killers of open-source projects is the inability to provide
feedback. Putting source out there isn't enough. Successful
open-source projects have strong leadership, a well-defined
direction, and a solid process for feedback.
So ... if someone e-mails me a new idea in public I'm to tell them to cease and desist and post to the forums? I don't think so. Well, no, I won't do so, actually. That's doesn't fit my idea of how sound development marches forward. People need to have some freedom to work in ways that best suit them. This is true in professional settings as well as a loosely-coupled project like this. If you try to squeeze people into one tightly defined way of working, you'll just drive off a lot of creative people.
Plans must be subjected to public feedback, I've made it clear that I believe that. But idea-thrashing shouldn't be burdened with this requirement.
Ben doesn't mention one important thing in regard to dotLRN, and that's the fact that a lot of the direction setting etc he's talking about has been set in a *contractual* environment. Sloan is driving much of the UI based on their existing system used by a large number of people in the school, and since they're footing the bill that's their privilege.
That doesn't mean that OF couldn't've been more open about things. Publishing an overview of the new portal system early on would've been welcomed by a lot of people, for instance, more than seeing the code (which after all didn't exist in the very beginning).
But any thought that the community rather than Sloan could've driven the dotLRN external design, at least, ignored the fact that Sloan's footing the bill.
Are we to demand that every client-financed project that results in GPL'd packages be planned with community assistance? If we do, I would predict that financing for GPL'd packages would dry up quickly while financing for closed-source packages would increase.
Also, I will be the last one to suggest that a client project necessarily be made totally open just because it's been promised to be released. That screws with deadlines, contractual responsibilities, client expectations, etc. Neither am I suggesting that all development be made totally available, nor should it go on in the bboards only.
But we do need to realize that way too much is going on in the back rooms. There are some consistent and core people, but there are a great many lurkers out there as well. Transparency is not only nice for the core developers to be able to watch as things develop but also provides an in for lurkers either to participate or to test new pieces out.
Allow me to provide an example for how this kind of transparent development can take place without compromising development efficiency from my experience. Dave Bauer and Jun Yamog have taken a great deal of interest in using ETP for their own projects. They contacted Luke Pond with a lot of their fixes, suggestions and questions. As the three of them have poured over the code, decided on improvements and culled a feature set, they've decided to build ETP2.
Before beginning development, Luke wrote up a preliminary design document and posted it on the boards to solicit feedback. If anyone is interested in adding to the list or in helping to develop the package they are free to ask or post suggestions.
In the meantime, Luke, Dave and Jun are working out of a private CVS on Musea's server that was set up for expediency's stake. Once there is an alpha or something it will be merged back into the main CVS. If it were possible to set up a private CVS for small development teams on the OACS box, it would have been there.
IMHO, I think this might satisfy everyone's desire for greater transparency while not limiting anyone's personal work preferences.
As far as dotLRN, Ben, no one is questioning your leadership or your right, along with Sloan, to choose the direction of dotLRN. You've got a lot more invested in its success than anyone else. I want to make it very clear that I and the community recognizes that and does not question it. Particularly since you must deliver a rather focused scope to your client.
Some are only asking that in the future when an opportunity arises like the portal package that you consider strongly to solicit community input before pushing ahead on its development. I think that this will not only benefit your development process and schedule with increased design feedback but will strengthen your position as benevolent leader.
And this should apply to EVERYONE that is building new packages for OACS, just as a courtesy to the rest of the community and in the interest of building a better OACS.
Allow me to provide an example for how this kind of transparent development can take place without compromising development efficiency from my experience. Dave Bauer and Jun Yamog have taken a great deal of interest in using ETP for their own projects. They contacted Luke Pond with a lot of their fixes, suggestions and questions. As the three of them have poured over the code, decided on improvements and culled a feature set, they've decided to build ETP2.Talli, this description exactly fits what I'm talking about. A handful of interested people kicked around ideas, put together a design doc, and posted it to the boards in order to get feedback.Before beginning development, Luke wrote up a preliminary design document and posted it on the boards to solicit feedback. If anyone is interested in adding to the list or in helping to develop the package they are free to ask or post suggestions.
A slightly cleaner way to do it would've been to solicit interest from others first, inviting people with a serious interest in contributing actively to the upgrade to join the effort. They might've gotten a couple of other folks involved at an early who would've been willing to roll up their sleeves and get down to it.
But as you describe it, the process has met my basic bottom-line criteria for sufficient transparency - a proposal is being made and comments solicited before implementation has begun.
I have no, and want no control over anyone elses project. I have been arguing the opposite, and would even take issue with Tali's seemingly benign statement that perhaps Ben could have solicited input on the portals package at an early stage. Ben has the absolute right to full control of all the projects he is working on, and anything that finds it's way to the public is a gift.
What I am saying is that design discussions and code are extremely valuable to more than those immediately involved at that point in time. I also say that developers have to communicate anyway, and that the incremental cost of placing discussion on a public bboard rather than in an email is nill. If you agree with me on those points, then I hope you'll also agree that every discussion which is syphoned away from this bboard and into private email devalues the OpenACS community.
I hesitate to mention .LRN again, as it's status as a for-client project means to me that this community has precisely zero say in it's future, but perhaps Ben is wondering what I am asking for if not for control.
Consider the state the project would now be in if at it's inception a bboard was created, either here or at OF's site, and possibly made read only to the public, and the OF developers had design and development discussions there rather than send each other email or use an internal bboard. Would .LRN now have less strong leadership or any less direction? Would it be any less of an open source project? No (in fact at this point in time .LRN is not an open source project because no code, no documentation has been released. Hence "unavailable").
A slightly cleaner way to do it would've been to solicit interest from others first, inviting people with a serious interest in contributing actively to the upgrade to join the effort. They might've gotten a couple of other folks involved at an early who would've been willing to roll up their sleeves and get down to it.
So what you're suggesting Don is that before work begins the community (made up of professional developers with other time commitments, hobyists etc., let's not foget) should be solicited for active, willing to roll up their sleeves and get down to it contributors, i.e. people who will commit to dedicating some significant ammount of time. These developers will then depart and between themselves via email or some other private means collaborate on the project for a week, month, year... Everything that is generated during the discussions -- the bugs encountered, the solutions assesed and rejected, the knowledge gained about some corner of the toolkit -- are all discarded, lost forever. During this period of private collaboration, anyone unable to meet the original commitment deadline is denied the oppertunity for involvement. When the project is complete the active developers, sleeves rolled, will present to the Community, and it will provide the thumbs up, thumbs down?
Sarcasm acknowledged, you seem to be saying that collaboration of the form I'm talking about i.e. letting people openly see what is going on, has some overhead which increases with the number of people involved, and that our Open Community System is not up to the task of facilitating that collaboration. Should we conclude from the number of participants in this thread that they are the only ones who are interested? Should we be having this discussion via email because the burden of over 100 people looking over our shoulders is holding us back?
My one and only suggestion is that if you feel as I do that design and development discussion is the valuable information which binds this community together, and that posting it to the bboard rather than keeping it private, where you are able to, is not too much of a burden, then you do so. To suggest otherwise leaves me genuinely quite puzzled.
