Forum OpenACS Development: Rich internet apps on OpenACS

Collapse
Posted by Jade Rubick on

Rapid, Rich Internet Application Development on OpenACS

First of all, these are some things that influenced my thinking on rich internet applications (RIAs): Now let's look at rapid development. I've looked into RIAs platforms, and it looks to me like the most viable platforms are either doing it in Javascript, or using a framework like Macromedia Flex or Laszlo. Laszlo, being open-source, is my favorite, and I don't think we can expect anyone here in the OpenACS community to plop down $12K to use Macromedia Flex.

I recommend spending some time looking at Laszlo. The basic way it works is it has a simple XML-based syntax for building web applications. This XML is then compiled into Flash. Currently, Flash is the only output format, but they plan to be able to compile apps to Java and .NET in the future as well.

Laszlo currently requires a Java servlet server, running their code, to compile and serve their apps, but they have plans for server-less applications. The beta for this should be out in a month or so.

Laszlo can consume XML formatted results from OpenACS, and commit data via GET and POST statements.

Dave and Alfred and I have been doing a lot of discussion on IRC about an idea (of Alfred's, I believe), to simplify development by making .xql files smarter. The general idea is to remove the need for the .tcl file in simple cases. I think with smart templates, we could even get rid of having to define the adp file unless you wanted to customize things.

Currently, we have three files for each URL (I'm ignoring that they can be mapped to multiple URLs at this point). ADP TCL XQL. Sometimes, if it is on Oracle and Postgres, it can have three XQL files as well. Alfred's idea is to add more smarts to the xql file. Here's a query from project-manager:

  <fullquery name="dependency_query">
    <querytext>
        SELECT
        t.title as task_title,
        to_char(t.end_date,'MM/DD/YYYY') as end_date,
        t.percent_complete,
        i.live_revision,
        d.parent_task_id,
        d.dependency_type
        FROM
        pm_tasks_revisionsx t, cr_items i, pm_task_dependency d
        WHERE
        d.task_id        = :task_id and
        d.parent_task_id = t.item_id and 
        t.revision_id    = i.live_revision and
        t.item_id        = i.item_id
        [template::list::orderby_clause -name dependency -orderby]
    </querytext>
  </fullquery>
Why do we have to pull this into the .tcl file and then into the .adp file? Why not just go directly from .xql to .adp? We could add a couple of tags, such as how many rows are returned, sample data the graphic designers can work with when nothing is there, and default permissions checks.

The principle here is to make the simplest case trivially easy, and made customization just as possible as it is now. You can still do the db_foreach in your .tcl file, and use it in your .adp file as usual. But by default, you don't have to.

Extending this even further, we add in the idea that if you present a special variable to the page, you are returned an XML document, with the values filled in from the .xql file. This lets us build Laszlo front-ends that interact much more dynamically. If you send it another special variable, it will output all the data sources, in a format that a graphic designer will understand. They'll see what the variables are, and possibly even an example using their variables. This will make it much easier for a graphic designer to design the .adp page (which overcomes one of our problems right now).

Tangental topic: If we had a well-written default template, it could stuff information from the database into a Laszlo table without even writing a line of code. Your workflow would be like this:

  • Write up the SQL query to get the data you want from the database. Put it in your .xql file.
  • Put up an .adp file that looks like this:
      <master>
      <property name="title">Show dependencies<property>
      <property name="use_laszlo_p">t<property>
    
The templating system, seeing that use_laszlo_p is set, would pull the data from the .xql file (via an XQL get), and stuff it into a Laszlo style table or widget.

end of tangental topic

I know that Dave and Alfred have some other ideas related to this, so I'm going to post this and let them fill in any blanks they see.

Collapse
Posted by Talli Somekh on
Considering that Macromedia is getting into the elearning space, I think that Jade's post should be very relevant for the .LRN folks.

talli

Collapse
Posted by Alfred Werner on
I am NOT recommending that we redesign our system to be a laszlo backend :) I do find some of their concepts to be 'simplifying assumptions' that would make our lives easier. If we decide to generate XML output from our data with a specific tag, that should be easy.

One thing I really like, that laszlo touches on, but we can do as well, is the availability of front-end tags for displaying data results in a variety of ways.

They achieve this by always returning data in the same form they use XML and Xpath syntax, we can use TCL lists of lists  or arrays of arrays or whatever we think works best in our environment.

A simple example of what I would like to see is a series of 'smart tags' that take a dataset and display it in the manner suitable to that tag.

Lets say an XQL query returns some rows with a variety of data, one field is the 'content-type' field. In our scenario now, we would say:
<multiple name="contents">
<h4>@contents.content_type@</h4>
<group column="content_type">
@contents.description@
</group>
</multiple>

To make that work, there are 3 different openacs roles involved.
First, there is a DBA who designs a dataspec, and writes out the queries associated with the tables.
Second, there is a programmer who writes a db_multirow to access that the queries. The db_multirow needs to be in the .tcl file associated with the .adp file.
Third, there is a graphic designer who wants to do a page layout, incorporating the data from the query.

I would like to eliminate step 2 (in most cases) by extending the tag syntax to include a pointer to the XQL query directly, with the assumption that we will always return data in a know format so the acs-templating library can handle arbitrary datasets.

So -
<fullquery name='myqueryname'>
<querytext>
select etc, etc
</querytext>
</fullquery>

<multiple name="contents" datasource='myqueryname>
<h4>@contents.content_type@</h4>
<group column="content_type">
@contents.description@
</group>
</multiple>

If I now decide I don't like the multiple/group display, I just change the widget. Let's say there is a date field in the results. I change my widget to say

<calendar name="contents" datasource='myqueryname defaultview="day">
<group column="posting_date">
<a href=@contents.description@>@contents.title</a>
</group>
</calendar>

Of course the exact syntax of each widget gets worked out, but that's the idea.

Add in pagination, grid, list etc... and spend some time on sites that you find by googling 'UI design patterns'

I'm intentionally ignoring forms at the moment. I think the ability to create less brittle displays of information is a key step forward.

The XQL should be like a file handle - it should be self describing, but not imply what context it gets used in.

The role of an XQL is to describe what in other contexts is called a datasource.

Like any idea, there are obvious branching points, I think the first and most obvious would be to extend the datasource attribute to include fetching xml-rpc,soap, etc data for local display. If we continue to use tcl lists of lists, then we would wrap that call with a conversion to our internal formatting. I think most of the work for this has been done, it's a matter of assembling the right pieces.

There are several additional things that would make our XQL files more useful. I'll stick to my top 2 for now:
Add a field for permissions required to access the query.
Add syntax to provide 'default/sample' data so a designer can work in an offline/not connected to the database local mode.

Collapse
Posted by Nick Carroll on
Wow that Laszlo platform is pretty cool.  The serverless version of Laszlo could possibly mean that all OpenACS has to do is serve datasets to the Laszlo client via XML/HTTP or Web services.
Collapse
Posted by Don Baccus on
Rather than bury presentation information in .XQL files, which means it might need to be replicated in both PG and Oracle files, which means in time the two version will drift and/or one will break, our maintenance resources being what they are ... and seeing as duplication is something to avoid anyway ... And seeing as having some HTML buried in .xql, some (gasp, still, though we work to get rid of it) in .tcl, and much in .adp files ... Why not investigate modifying some of the more useful tags to take an optional datasource? For instance, your "multiple" tag would fit wonderfully into an .adp file ... When we eventually finish automatic generation of forms and perhaps output lists by making the maintenance of datamodel metadata much easier than it is today and move some of the CR ideas over to mainstream ACS objects ... one can envision that perhaps "formtemplate" and "listtemplate" template tags could just take a datasource directly with some parameters if one wants to autogenerate simple forms/display pages. Still, when all is said and done, at the moment I don't think our generation of tcl/xql/adp is the main issue in regard to rapid development. It is the tedious building of the datamodel and supporting procs that slows me down the most. We're working on fixing that (ever so slowly). Even today, proper providing of metadata allows one to automatically generate new objects with a simple tcl proc call that supplies the datatype and form name ...
Collapse
Posted by Alfred Werner on
Don - the original proposal /discussion <b>EXACTLY</b> what you are saying. The only thing discussed in adding to the xql files were two things - a permissions attribute (so the coder knows what the intentions are - ultimately that would be enforced by the page) and 'demo data' - sort of the lorem ipsum of database queries, so a UI person could work on pages in 'artist mode' which would not execute actual queries, but only return sample data. The original suggestion was to add a datasrc flag to the adp tag libarary (and to extend the tag library) and go DIRECTLY to the query without intermediate tcl code, since the tags are really tcl code anyway. example (without looking at real xql syntax or permssions naming): <xql><query name="onethread">select topic, poster, posting_date from forum</query> <permissions>public</permissions> <data><topic>Re: Re: Rich internet apps on OpenACS</topic><poster>Alfred Werner</poster><posting_date>12/10/04 12:16 AM</posting_date> </xql> You can then use a tag in your adp page : <multiple datasrc="onethread" paginate=yes perpage=25> @stuff@ </multiple> Since there is a date field, you might decide to use a calendar widget instead: <calendar datasrc="onethread" defaultview="month"> @stuff@ </calendar> That's the general idea ... Main focus will be to identify a core set of tags and their semantics, and to eliminate the middle tcl layer unless necesary for customizations.
Collapse
Posted by Alfred Werner on
doh - bad formatting choice!

I'll post a cleaner example at some point.

Main idea -

xql has the query and a name.

in the pages:

<mutltiple datasrc="forums" paginate="yes" perpage=25>
@stuff@
</multiple>

you switch to a calendar view:
<calendar datasrc="forums" datefield="posting_date" label="topic">
@stuff@
</calendar>

In most cases, you shouldn't have to put a tcl layer between the adp and xql to use this ...

Collapse
Posted by Mark Aufflick on
As one who now spends the vast majority of time on programming tasks not including OpenACS (and therefore someone who's views may be out of step with the current OACS community) I agree with Don that the worst time waster in an OpenACS project (especially starting a new one) is the datamodel + procs creation process. You *could* automate some of that, but if you have to rely on code generation I think that reflects an underlying issue (as in the bloated Java web frameworks).

That's really what people love about Rails. Sure Ruby is much nicer to write than TCL, but when you start an OpenACS package you are writing sql, pl/pgsql and then tcl.

If you could define a subclass of a content repositary object and that did all the work for you, then you'd be cooking.

Working at the code level for subclassing (rather than the rdbms level) also makes it easier to upgrade to things like lazlo later since added functionality in the base class will bubble up.

Collapse
Posted by Malte Sussdorff on
This tedious work (of the helper procs and datamodel generation code) was the reason why we created the package-builder at one time. It is far from perfect and probably the other approach taken with UML is considerably better, but maybe someone is willing to invest some time into improving either one to make code generation considerably simpler. If you have all the code for your add/edit/display/list of "objects" ready at hand, then you could (quickly) modify it to suit your needs.
Collapse
Posted by Flash6 Designer on
Hi,

has any checked this??

I found it interesting.
http://www.fruition.in/openmacro.html

a light weight framework to script in xml and generate flash websites.

Collapse
Posted by Matthew Dodwell on
Is the OAK UML tool being maintained? I used it and liked it for code generation, but the save files become easily corrupted, losing all work, so I've got back to hand-crafting scripts at present.

Matthew