Forum OpenACS Development: New Plan for improving Edit-This-Page
I have written up a new plan with goals for improving edit-this-page. It is subject to change. Please comment in this thread.
Evolving version lives at Edit This Page 2.0 OpenACS Package Development Plan.
I have included the original version in this posting.
Last updated 2003-07-02 20:14 EDT
This is a preliminary plan to update the OpenACS package edit-this-page to offer a richer content management application while moving it towards using more built in content-repository features. Along the way it will also result in work on adding such features as dynamic form generation based on object_types.
Currently edit-this-page fills a need for a simple way to add pages in a hierachal structure to an OpenACS site. It is difficult to extend ETP in a way that conforms to accepted OpenACS conventions. It recreates much of the built in functionality of the content-repository. In taking advantage of the current services OpenACS offers, this project should be able to identify area of OpenACS that need improvement, especially in the area of content management.
The plan is broken into several steps. Currently I have four phases. Each phase should be completable in a short period of time, generally less than one month. The goal is to drive development of a CMS solution for OpenACS. I am basing this work on the edit-this-page package because it is already in use in several sites, and I hope to offer an upgrade path for the existing users. In addition I am going to look into the work done by Jun Yamog on his BCDS and BCMS packages to see where code and concepts can be shared. Both Jun and myself believe that a CMS framework can be developed that allows various CMS type applications to be built on top of by adding user interface elements. By enabling rapid advance of development we should be able to reach the critical point in Pind's Rule of Five and realize what parts a content management system built on OpenACS really needs.Phase 1
- Convert all UI forms to use ad_form/form builder
- Allow editing of all default attibutes on one form. This will make it easy to create most content items. Worry about dynamic form geneartaion for additional attributes later.
- Make sure all content_type attributes are registered in acs_attributes for built in and custom content types (define requirement in the documentation, I can't see how to register the attributes automatically)
- Use content::get_content to get content instead of rolling our own and/or use CR views to query for content.
- Allow file-upload of HTML content to get around browser form limitations
- Allow file-upload of images, word docs etc. Store in filesystem. Use cr_item mapping table to associate files with content. (In future version add support for a media storage facility, for now, just "attach" uploads to the item. Possibly use a simplified attachments package for now)
- Test using cr_item_child mapping table to define sort order for items in folders instead of the acs_object tree sortkey
- Improve susbsite support, add general comments support via parameter, add notifications support on folders and pages.
- Use CR content_type/content_item/context to assign templates. This will allow assignment of templates by application/content-type, etp package instance (folder), or idividually assign a template to one page (content_item)
- At this point we should be able to allow multiple content types per folder. Right now ETP support 3 types for each folder anyway, the application type, symlink, and extlink. So its just a matter of offering the UI to add a content type. The existence of the UI can be parameterized.
- Store templates in the CR for versioning, publish to the filesystem for rendering, or store to custom CR storage location. (possibly defer to later phase depending on progress). This works along with assigning templates using built in CR functions.
- Allow creation or extension of new content types through web interface. Requires dynamic form generation.(Jeff Davis is working on this)
- Support static publishing of finished HTML pages or partially rendered dynamic adps. (this is already in CR), at this point the content type template will be merged with the content and pblished as an ADP which may contain a master reference as well includes. We won't have any other tcl setup I don't think. Any variables filled from content attributes should already be rendered at this point.
- Support custom ADP tags to get used by content editors and template editors to provide easier access to includable templates.
- Add default workflow, with tcl api support for customized workflows with possibility of a web UI.
- Add support for the new categories package.
Besides these items, at some point Oracle support will be added back in.
The end result is going to be so different that making an upgrade path to a totally new package would not be any harder, so that leaves (in my mind) the only benefit being that there is something that actually works sooner in the cycle.
I think given that the CR framework is quite strong, and will drive the backend design of any CMS system, building it from scratch will be easier and yield a better result.
I also think that the types and auto-generation issues are not going to be on a CMS shopper's wish list, mostly because they won't understand what to use them for. A better early hit will be the ability to auto-build navigation from a cr folder tree, easy ways (plural) to edit the cr tree and items. he only cr items i think we really need are:
* styles (adp or adp/tcl)
* attachments (arbitrary blobs attached to other items - ideally auto-generating dimensions for images)
* single page adp chunks (or adp/tcl chunk pairs)
* sub-tree adp chunkc (or adp/tcl chunk pairs)
* internal links
* external links
all except blobs should be database bound.
The one i called sub-tree is what midgard calls "dynamic pages", an example is a page that sits at say /foo and is rendered for /foo/bar /foo/bar/sheep ad nauseum, and gets the sub urls (bar, bar/sheep respectively) as an argc/argv pair so it can render accordingly.
I know the latter is not what we expect in ACS, and if we wanted to do that we would register an aolserver hook, but for a cms content programmer, it actually turns out to be amazingly flexible and is what cms programmers expect.
That sort of system doesn't have a lot in common with etp, and I feel that taking etp along for the ride is just unnecessary bagagge.
Lots of item in phase 1 and 2 are already done in BCMS. We are finishing up our current project. Which should bring in an updated wizard. The next project will make use of CR hence further improvement of BCMS.
Maybe we can meet over IRC around 2nd week of July. And discuss a lot of phase 1 and 2. We can also divide the work on that meeting. As I see it if BCMS can fullfill all the phase 1 and 2 functional needs. ETP 2 can be doing the UI and upgrade scripts from ETP 1 and making use of CR. Again BCMS has no data model, it uses CR solely. Maybe 2 plsql, but that is it.
As I see it the cr_item_child is one thing that needs to be worked on for phase 1. Maybe you can teach me about this? Pretty much the other items just remains to be used and refined.
Also who would like to participate in the development of the CMS? So we can organize and initially meet over IRC. How about it?
Templates: Can we make sure that the we have a common template support per node / subsite / site structure? I'm just concerned that if we use a certain templates e.g. for forms in survey the forms in ETP will look different. Or are we talking about different templates here (aka, template for the structure of the content vs. template for the display of the content).
Why do you need dynamic form generation (and how does it differ from something like survey form generation)? Does it provide the ability to create widgets on the fly? And, if we add new content types, wouldn't we have to think about how we are able to display new types (e.g inline media stream)?
For the MLM we are allowing custom queries and double checking that all variables used in the template for mail generation are returned by the custom query. Would something like this make sense for Uploading of ADP pages (associate an ADP page with a (custom) query).
I agree with Jun's idea for a planning session. But before that, I think we should get as much community feedback as possible through the forums. We should also review other CMS platforms out there and "borrow" their best ideas. Perhaps even synthesize all this into a more detailed spec/proposal that people can review.
I think that after this exercise, we'll know whether we'll need to build something from scratch or base it on the existing package. Either way, Mark's point certainly deserves consideration.
Finally, should we move this discussion over to the CMS forum? If not, then what's the point of that forum's existence? Also, I believe there has already been some valuable discussion over there on the qualities of an ideal CMS.
I've recently been making some changes in regard to content templates. I made them while trying to get file storage to work with virtual URLs, and the code is currently being reviewed by Jeff and Don. Hopefully I'll be able to commit some changes pretty soon.
I've been hacking on content::init and content::get_content and also made a single call to content_template.new() automatically create a revision of the template, make it live (or not), etc ... On the PG side, I made a new version of content_template__new() and a convenience wrapper to do the same as on Oracle. But no Tcl wrappers, though.
Hope this helps.
Maybe that is a good idea. Maybe I should start with fresh code, to develop my ideas, keeping in mind a migration path from ETP. That will lead to a cleaner design I suspect.
One of the goals I have is to _not_ require custom tcl code for every page. I definitely want to make it easier to build site. I don't know how close we can get, but I want to start with that as a goal anyway.
That is the reason for dynamic forms, for content entry, not for user level forms, at least at this point. Also that is the reason to fully use acs_attributes. We should be able to generate the data sources based on the metadata in acs_attributes. This way one index.vuh file will be sufficient and the adp fragments are all that needs to be edited.
The groundwork for this already exists in CMS, but the UI is tied to closely to the underlying mechanics. The goal is to move the services out to a tcl library so the user interface is easier to customize.
I definitely agree that widgets for dynamic navigation etc will be very useful. In general I want to make most features available to the CMS user through includes.
I would like to understand more about "sub-trees" that you mention. I think it might be realted to an idea I was saving until it could be implemented. That is assigning categories. Then being able to address categories as a virutal directory. So files under the "foo" folder in the "bar" category could be addressed like this http://example.com/foor/bar which would show an index of all items in the bar category.
I am not sure I am following you :)
The templates I am referrign to would be for content display. I haven't thought about how to associate content with multiple subsites, but I suspect the built in CR template mapping function can be used. It has a context which can be any abitratry string. One way to use that would be to specify a different content template for content in different subsites.
The forms would be strictly for content editing. It might not even be too important if we have a fairly comprehensive set of default content types.
Great, I am glad to see improvements to the content repository.
Unfortunately, the forums package doesn't offer a user interface to move threads. It would be a handy addition. I had forgotton about the CMS forum, although I had participated in those threads.
I think the important thing to do is to get some working code we can base further discussion on.
I think we might want to move some of these into the CR itself.
Maybe, but not too sure. Some have suggested this. But I think we can do this later on? Or do the other senior developers here think that we should out right put the tcl api into CR package?
I am thinking of just creating a service package if it does prove to be useful. Migrate it to CR.
This is a feature supported by the CMS package all along, and is supported by other real-world CMS systems.
The alternative, such as was done by Planet (because the original contractor didn't bother to study the existing code much), is to define content types for a content-driven site by hand and write the forms and display templates by hand.
I've been hesitant to look into doing anything soon in regard to importing code into our main code base because
1. I've wanted people to explore what's already available in the CMS/CR rather than blindly build more wheels where it is not necessary to do so. And to understand where we do need more wheels (better API in other words)
2. Several people have been working on bits and pieces of the problem for personal or client projects; I want to see an organized effort.
3. I'm a believer in design (but not to the point where every detail needs to be worked out.) I'd like people to take the time to really think this stuff out before jumping in and implementing stuff that's going to end up in our code base. This is an area of *major* importance and a clean, robust API design is crucial IMO. If Jun's stuff or Goldstein's stuff or Ola's stuff etc etc is just glommed together into some sort of mega-union of features we won't achieve the kind of clean consistency that makes an API shine.
4. I've really been hoping enough people would get together on this to form a critical mass and to make this stuff work with all objects, not just CR objects, if at all possible (though it is obviously possible that the CR API may include more features, for instance a better API built around the insert views for revisions, an approach that doesn't work for regular objects)
We will try to see if we can get more people aboard. I guess one of the problem I see is that people puts their concern in the CMS area. But not much is done. Right now Dave is looking into improving CMS through ETP2. We seem to have similar ideas. I have done a good chunk of things he wants, so he may likely reuse it. He will likely tell me which parts suck and give suggestions to change it.
Maybe we can announce on the Q&A forum when we will meet on IRC. So we can gather more support in improving the CMS.
I am not really too keen in having BCMS tcl go into CR. Since my primary concern is to help me dealing with CMS related projects. Its there on contrib to so maybe other people can use it other than me. I find it useful so maybe other people can find it useful. If it does prove to be useful lets just decide what do with it when the time come.
Instead of disucssing implementation details of a CMS, I want to pick a few key features, and implement them. This will be a good test of what the CR does now, should do, and a test of what BCMS adds in building a CMS product.