Forum OpenACS Development: Proposal for hooking forms to metadata

I've put together a quick prototype for interfacing the form templating package and the acs metadata. It also includes code for making most of the metadata acs objects. For a first look see

Demo and Docs

Tom Jackson's query writer also covers some of the same ground. It's worth having a look at it as well.

The problem with the stuff driven from the acs_attributes alone is that the attributes don't really tell you enough about what widget should be used or populate select widgets etc. cms has that additional data for driving widgets (cm_attribute_widgets, cm_attribute_widget_params are the two tables, content::new_item_form is the form creation code) and it lets you do quite fancy things but the datamodel presumes that there will only ever be one form for editing a given content item and also the code forces you to edit one object at a time. For real world problems you end up with several interrelated objects you want to be able to edit simlulaneously (for example, an article you might want to be able to edit the list of associated photos, the author data, title, blurb etc. and it's probably the case that that is at least 3 content_items (probably even more than that).

Anyway, I think driving the UI from metadata is a great idea and am a big fan of doing it that way but I also think we need a better datamodel and better supporting code so that our saving programmer time won't come at the expense of creating horrifyingly bad UI experiences.

I think one place where we fall down now is in not having nicer higher order input widgets (like a standard address widget or phone entry widgets) and doing multifield validation (oh and not signing enough variables but thats not a UI thing).

There's no way that automagical generation of forms etc will solve all problems, that's for sure, Jeff's right about the potential for UI hell.

This is one reason I'm still interested in extending ad_form to be able to generate blocks of elements from content (or object) attributes - this would give one a way to blend custom form needs with automatically generate elements from attributes.

I think there's room for both approaches - simplifying the coding of more complex cases while providing essentially automated handling of simpler cases.

I think Tom's work also includes code to generate package/proc stuff (new?) and that's also duplication of effort of code that already exists in acs-subsite.

Maybe we should start up a forum for discussion of how to best coordinate efforts and reduce the redundancy of effort we seem to be seeing today?

I really hate to see so many people working in parallel on the same essential problem!

Yes, this is exactly my intent, except I doubt we'll end up with anything that makes *everyone* happy.  Also, I don't mean to imply that folks shouldn't experiment with different approaches.

What I'm asking for, I guess, is ongoing discussion and some effort at communication.  As this thread makes obvious, I (and others) can't necessarily even judge what is and what is not duplicate effort!

And I know all of us are working hard on projects with deadlines, and that servicing our clients comes before coordinating with the community due to the need to put food on the table.

The CR/CMS API has been used to build one complex package, the CMS itself.  From a functionality POV it's very usable, it's only from a UI POV that it's not.  And Jeff's using it (and improving the API) to write another.

Your query writing package sounds interesting, especially in its new form (the first version seemed really clunky, which I don't mind saying because you're saying the same thing and rewriting it in part because of this.)  If it turns out to be really good, Jeff may decide to put less effort into improving the CR/CMS API, for instance.  Maybe we could stick him with rewriting the admin UI that everyone hates so much instead!

I've been out of town but here are my comments on the comments.

• I put Jaguar on my Mac and I've been messing with Objective-C so I've got objects are objects on the brain.
• I had seen the acs-subsite code (odd place for it). The code I wrote serves two purposes. Make object_types etc objects and generate forms from object_types. I was insprired in part by add_form_elements.
• I'm going to mess with query-writer this week. I noticed it only runs on Postgres and I run Oracle.
• I really like the form templating stuff because it will do almost all the work for free but you can still customize form layout if you like. In fact you can template the same code differently by instance, something that's difficult with the rest of the templating system.

I've spent that last month looking at the java/tomcat/struts/turbine. Some parts are interesting but the thing that struck me was how much code you have to write to do anything. My wife ( a graphic designer ) can write tcl pages to process forms, send email and insert into a database and she does not think it's hard. I think java etc is hard, too much complexity to solve the 'scalablity problem'. It solves it by ensuring you'll never have one.

That said I think there is much room for improvement in openacs. The metadata model is a good start but not complete. This causes poeple to not use it or extend it in package specific ways. There are two ways to implement form validation. ad_page_contract and the forms package. This causes a UI problem. There are things I like about both and perhaps ad_page_contract could be extented to work better with the forms package.

One thing I did like about struts is the MVC model. It seems like this could be implemented in the request proceesor but that would be a whole different discusstion.

Where to go from here? I think the first steps would be to make the object_types etc objects and extend the attributes to support form templates. I think this would help the permission system and make the form system more usable.

By making object types objects you can give groups of people access to groups of objects. For example you could make an abstract type called developer_forum have supertype of forum, give the developer group access to developer_forum and write queries like

select object_id 
from acs_objects 
where object_type = 'developer_forum'

and get a list of forum objects the developer group can access. You can also control what attributes users have access to as well as what enums they could choose. This could be done by granting permissions instead of writting code.

Added data to the attributes so forms can be generated from the meta data would allow packages to be more dynamic and configurable. It would also allow the development of a RAD package. The are pieces of that now but I would be nice to add to the core because even if a package was not completed by RAD it could be started that way. That would help insure that sql runs on all supported databases and the sql object interfaces and metadata are created in a consistant manor.

I think all of this can be done with a second set of tables so there will no impact on existing apps. Should we start a discussion about what should be in those tables?

By default the templating system pairs by name so index.tcl is paired with index.adp. You can override this but I've never had much luck with that. You can place files in the filesystem with paths like that those created with site map and override index.tcl and index.adp, but it's both or none. It would be nice if the request processor would allow you to override just index.adp so you could change the look without changing the code.

The form package is slightly better because you can specify a form template in the form tag. You still have to write code but it just seams cleaner to me.

Now that I say this I guess fixing the first would also allow forms to have different templates since the form template is in the adp file.

We implemented customized .adp templating for scripts for Greenpeace, but not in a general way.  The only barrier to doing something general is deciding on just how we want it to work.  For GP we searched first in a custom location then used the existing paired .adp file (index.adp for index.tcl for instance) as a fallback, that way subsites only need to customize those templates they actually want to change.

But in the general case I'm not sure how it should work.

Regarding object types as objects ... Barry, could you e-mail me a tarball of your changes and (hopefully) update scripts to take a look at?  As I said above I do agree that this would be useful, and yes, permissions are why.  We were looking at this in the design of a "general ratings" package (that hasn't been implemented as the volunteer got a real job, a never-ending one because the company's using Java.)  The idea was that rating dimensions were to be ACS object types, and if these types were objects you could control who could rate an object using a particular dimension (rather than just "user can rate all aspects of a particular object")  The volunteer had a lot of experiences with knowledge management systems and had found that level of granularity important in the past.

Anyway ... I'd love to look at your object-types-as-objects changes.  4.7 is the perfect place to slide in fundamental changes of this sort.

form templating vs. ad_page_contract.  When I wrote ad_form (have you played with it?) I considered trying to do a tight integration with ad_page_contract but decided against it for the time being.  There are several reasons for that.  Among other things pages handling multiple forms aren't uncommon, so you want more than one ad_form call for the separate forms. And individual forms frequently need to be built piecemeal, this the "ad_form -extend" construct to add to an existing form.  On the other hand ... ad_page_contract sets a contract for the page and should be a one-call device.  Also errors are returned in an entirely different way by the form builder (ad_form) and ad_page_contract.  And ad_form handles the flow of processing self-submit forms for you in a way that's straightforward but doesn't blend in well with the ad_page_contract paradigm (IMO)

These aren't simple design issues to solve, so I decided to keep ad_form and ad_page_contract separate.  At the moment I think it would be a lot easier to document how the two interact.