Forum OpenACS Q&A: Code freeze in favor of documentation for 4.7

And as a gesture, if you want to suggest a particular piece/package/doc I will happily complete it and contribute that way.
(don't give me anything way technical, Id rather do something for newer/less experienced users)

I could take that on if its useful, unless there's a higher priority

But as for your last point, perhaps we should consider checking the documents as part of the acceptance test process.. i.e. documentation bugs for want of a better term

We can't really stop all code going into 4.7, for instance Collaboraid has a contract with the University of Heidelberg to work on globalization and they're already in the midst of it.

But knowing the Collaboriad folk and from what I've seen of the University of Heidelberg both understand the importance of documentation.  I'd be very surprised if we couldn't get some help from Collaboraid during a "documentation blitz month".

Would an approach of this sort work?  We've talked about doing something similar with bugs...

PACKAGE DEVELOPER'S GUIDE

- We need a package developer's guide and tutorial for how to build "good" OpenACS packages.

It should cover things like

- how to use ad_form,

- how to design your data model

- when and how to use content repository,

- how to internationalize it,

- how to write proper tests,

- how to write your requirements and design doucments and the end-user documentation,

- coding style,

- API design,

- other standard services like workflow, notifications, attachments, etc.: When to use them and how,

- UI guidelines,

- etc.

This is critical, because it's prescriptive, so it'll improve the quality of all new packages being built, and at the same time make it easier for new people to get started on building applications for us. And it must get updated to contain the latest knowledge about best-practices. Therefore we need a maintainer of this document, who can get support from others, of course.

END-USER DOCUMENTATION

This is important. I think it would be a really valuable investment to come up with a simple framework for this, so online help can be invoked in a standard way throughout the toolkit. Perhaps something as simple as adding a "Help" link to the default-master template, and having a directory at package/www/help store the help text files. They should also have a consistent look and style, since they'll become part of the UI.

REQUIREMENTS AND DESIGN DOCUMENTS

These are important for documenting assumptions and design decisions. But they must be kept short and to the point, and let's not try to be too strict on the templates for this, since this seems to encourage people to put lots of irrelevant details in there. It can also be really hard to write your assumptions clearly, since they're frequently assumptions you've made without realizing. I like Joel's approach to specifications: http://www.joelonsoftware.com/articles/fog0000000036.html

DEVELOPER DOCUMENTATION

Service packages or subsystems like CR, notifications, workflow, form builder/ad_form, etc., need a document explaining how to use them. These also need to be kept short and to the point, and shouldn't deal with the details of the APIs.

Other developer documentation, like API documentation and such, should be kept in the code.

-

Hm, this list got long after all. I'm not too happy with saying "if you can't write documentation, you can't contribute". I think there is going to be an eternal problem, because it is intrinsic to the people and the process.

Perhaps we can change the attitude. I think there's a good chance that if we improve the documentation to a point where it's actually useful, then the more people start using it, the more people want to contribute to it. But we should refrain from setting expectations that are so high that they're not going to be met, anyway.

Hm. These were loose thoughts. I hope they were useful to some.

/Lars

• Resource - who has time to do it? A document properly covering the items you discuss, complete with well-commented examples and toolkit references, is not a trivial piece of work. That's not a bad outline for about half of an OpenACS O'Reilly book. It could only be done with someone who has intimate knowledge of how the toolkit works. There may be as many as a dozen people in our community who have that knowledge but I doubt if there's more. If our community were more robust financially, it would be a good project to fund, pay someone for a solid month or so to put together a really good document. Perhaps I'm overestimating the effort required. Am I? What do you think? Any ideas as to how we can get something like this written short of paying for it?
• If we implement some or all of our labor-saving ideas in 4.7, the contents of such a document would be obsolete before it was completed unless the writing of it were put off until these new tools are written. This is a minor thing, IMO.

Roberto ... when I suggest a "document blitz month" you can be assured that it would come complete with arm-twisting, begging, and personal cajoling from me. And of course yourself and I assume other leaders in the community. I would also, of course, commit to working on documentation myself.

If we just lock down the tree and implement a moratorium on code changes we may just find that no code gets written and, damn!, neither does any documentation.

I agree entirely with you in spirit, though, we need to do something to tackle the documentation problem, and getting a bunch of us involved working on it at once is a good idea.

The second part of my point was to try to limit documentation requirements in some places. For instance, there's really no need to include data model or APIs in the documentation. There should be comments in the SQL file, and ad_proc documentation of the Tcl API for sure. But separate documentation should only document the big picture/idea type things, such as defining terms and how they related, e.g. "a portal consists of one or more portal elements, which are instances of portal datasources", etc. Let the data model file tell you the details. Code is documentation, too, as I believe is part of the XP approach to documentation.

So what I'm trying to say is also that we should prioritize our documentation effort, and focus on the most important one, the package developer's guide being an important candidate. If we ask for too much, I'm afraid that we'll get less and lower quality documentation. I think we all agree on that, but obviously the hard part is drawing that line.

/Lars

The package developers guide is something that is useful. I think the "Things to remember doc" that I started should have been this way. Jon's doc is broken down into user roles, my doc is broken down to file extension. The things to remember kinda lost its steam like most doc efforts. I hope people can help me with it since some items (in dark red) are items I need help. I will be interested and will contribute to the package developers guide, but not all section. In particular "when and how to use CR".

Jeff,

I think you have stepped up on this issue. Here is a link to the things 2 remember doc in case you will find some pieces useful.

http://www.infiniteinfo.com/openacs/things2remember.html

Either I continue to maintain that doc or we dump it and make a better one. If we continue to maintain my doc and make a new package developer guide it would be nice to get some help in making the 2 docs consistent. Don't want to put something on the doc that is contrary to the package developers guide.

Roberto,

I think your concern is correct. I don't think it would be easy to do it in the existing packages. Perhaps new packages that comes in must have a higher standard in terms of docs. Also maybe if we can acknowledge the developers for putting in good docs. If we can give more recognition to docs, since most developers hack in the code to get recognized. Maybe we can get people doing the docs. We can start with yourself and Vinod, most older member know that you are the doc master and Vinod. But that info is not visible enough for new developers. Why not have a section or place that is more visible to major doc contributors like you and Vinod? Maybe new developers may want that recognition and start doing docs too. I guess the docs has always been a hard nut to crack even outside of OpenACS.

Jun, your document is useful and yes, it would be nice to have everything be consistent.  One thing I would say is I would like
to split out just the negative things and make it a "Common Errors"
document and take all the afirmative suggestions and incorporate them
into the developer doc.

Ben, collaboration would be welcome.  I have an outline in mind
but had not thought too deeply about specific examples (the
above on news was my first stab at it).  I will write out an
outline and you can have a look and see where we go from there
(and anyone else who wants to be involved is welcome as well).
I do recognize you are very busy documenting the APIs you have
already developed and writing proposals so I don't want to make
too large a claim on your time :)

I think there is an awful lot that can go on in parallel and people
should also think about taking a look at the api documentation since
that can (and should) be a much better guide to best practices
if we just take the time to document what we think people should
really be using.  The embeded documentation for ad_form,
ad_page_contract, and export_vars are good examples of what we can
achieve on specific important interfaces.  Currently about 1/3 of
the functions lack any documentation and about 1/3 have neither
-public or -private designations (which means we can't say "look
at only -public procs").  I would say there are also about 50
procs which should probably be deprecated (although not so many in
the core now).

Sadly I am completely focused on developer documentation and
hopefully someone will step forward on the user side.  I also wonder
if we should have a document oriented towards designers and
people who don't need to dig in the code but do need to understand
some aspects of the system (templating mostly but also how to
theme subsites, etc).  Skin halfway covers this ground
but is out of date and I think easily overlooked.

/Lars

For that purpose a demo package has its place, IMHO, because it does not confuse the newcomer with tons of functionalities. Also it is a very good starting point for problemset-like assignments, e.g. "extend the notes package by one text field and write an upgrade script".

As soon as one understands the basic concepts it is certainly better to study a real package though. So I don't think a demo-only package for the CR would be necessary.

I'll sit down later today and ad_formize notes if nobody has done so yet.

I respectfully request that subsite support be included in the News sample application and documentation (if it isn't already).

Subsite functionality is very important to me, so in my own self-interest I hereby volunteer to help code, document, or proof-read the sample application (especially with regard to subsite support) as the leaders of this project see fit.

While I am no expert on subsites, I did manage to muddle through the creation of a custom package (for internal use) that supports subsites and seems to function properly.  However, I based my work on the pre-CR Notes example.  Helping out with this project would allow me to give something back to the community while simultaneously upgrading my skillset to include the CR and ad-forms.

I had to make some design compromises based on what I perceived to be limitations of the core subsite code at the time (4.5 beta), so this would also give me a chance to make sure that my understanding is in-line with the current "best practices" of the community.

I have to finish some homework for school, so tomorrow I'll post a summary of what has been discussed so far, with a set of tasks to be accomplished based on what has been discussed here and my understanding of documentation status.

I'd also like to work on the developer docs.

Please don't forget the upgrade script for the perms.  I hope it would not be too hard.  We need to move the current custom perms already existing in a running system to the default ones.

I will probably look again at news when I get back next week.

To tell you the truth, I hardly look at the docs on my 4.5 installation because they are oftentimes incomplete or simply incorrect.  I usually just jump into the code with my good friend 'grep'.

As for 'grep' as a documentation tool, I also rely heavily on it.. but that is definately no way to inspire new users ;)

And of course to 'learn-from-code' does require at least 'some' good introductory docs.

To be honest, in true XP fashion, I'm not convinced of the real need for tech docs and design docs (my opinion).

90% of what people need docs for is when they're new, unfamiliar. They don't want to know how tables are joined, or how request processors function, they want to know how to build a web app, whats important to know, what they can do without knowing, and what the 'process' of using the system is...

'it feels like butter thats been spread over too much bread'

There's no getting around it. The resources aren't there to cope with everyhting to the same level. My argument therefore is that we focus on docs that help the people who need it most. Newer users, non-techs, techies with less experience.

The reasoning here is that I think everyone would accept that once you know OpenACS well you can make progress rapidly and do many usefult things....

But.. and I guess this is virtually a design flaw, its learning curve is way to steep initially. The initial mountain to climb is too steep for many...

I recall when we ran a bootcamp for aD (with Eve, Dave and so on attending) that they themselves barely had a grasp of it all!

So, we either flatten the curve, or we provide good tools (docs) top help people up it...

I'm kind of saying, 'Don't write docs for me, I;ve got the hang of it now... write them for people who need them'..

Reretably I have been busy with several things and only now got a chance to browse at this thread.

As some of you know (althought it doesn't show here) I am writing a course that will be used here at Univ. of Sydney. It is still quite immature but it is getting there.

The course is not designed for end users, but it will contain a number of demos for some of the applications.

The course is designed for undergrad students in e-commerce, they have courses in business and courses in software engineering. This one in particular will be called something like "Web application frameworks", and of course the tutorial/labs use OACS.

I expect that someone who finishes the course successfully will be able to use OACS as a webmaster, and will have an idea of the underlying issues, just enough to modify a package.

I will update the page were the content is in the next week or so, and would appreciate feedback.

cheers