Forum OpenACS Development: Re: The future of OpenACS

Why not providing/ exposing functionality and data over different protocol channels (SOAP and XML-RPC), without the need of multiplying the coding effort (e.g.,rewriting the serving bits of code, interface descriptions)? In fact, multi-protocol support is something rather new in framework design, most established frameworks are rather short-handed in this respect. Note, by multi-protocol i do not simply mean providing an xml-rpc and soap package and having the individual developer deal with exposing their code to them. There should be a certain protocol transparency from the developer's perspective. I don't want to give examples from literature, but just a hands-on example: Ruby on Rails comes with a somewhat multi-protocol support (at a rather low level, for my taste) in its Action Web Service module. By switching around some parameters, you either go for an XML-RPC or SOAP call/provision, through a common interface.

Everything should be automatic! This is like ASP.NET, just add a few decorations to you code and it becomes a web service...or not. Actually if you want to have complex types or validate the input you are totally screwed, now you need to write your own code for doing this. Apache AXIS is worse. Although it _is_ true that once the bits get to the protocol level the developer doesn't need to know anything, but by that time who cares? There are no switches to throw anywhere in any system, and you still lack the most important ingredient: a published description of the service and how to access it. Maybe somebody will recognize the importance of following a standard so that external clients can actually use your interface.

Now, what are the issues with OpenACS? The main issue is that most OpenACS applications depend upon page views. By that I mean that pages serve as a procedure. This makes it nearly impossible to reuse this code by a different protocol. For instance, my query-writer package requires all form submissions to go through a special page, very similar to rpc, minus the xml. Until I write a procedure, or set of procedures which covers this functionality, no amount of developer support will help me.

Another issue specific to Tcl is important to recognize: Tcl does not maintain type information, ad_proc actually complicates the situation, and we know nothing about certain things like return types, use of uplevel/upvar, etc. Many programming languages know all this information and so it is 'easy' to serialize a data structure to XML. This is why there are so many rpc packages in every language. But rpc has many problems which were never solved because they are much harder. First problem is that server and client really need to use the same language for full interoperability. Otherwise, either one or the other needs to learn to handle conversions. Second is that it is unlikely that anyone anymore really wants to expose an internal procedure to the network. The best method for handling this is to write a wrapper procedure to restrict use or perform any additional checks that usually take place prior to execution. One of these 'checks' is validation. This is where the wheels fall off of rpc and it is why nobody, not .NET or AXIS performs validation out of the box! And by validation, in this situation I only mean simple validation like is 4 an int? Derived types and complex type validation...this isn't even supported if you wished to do it. Just make up you own method of figuring out if the input is valid, test, debug and hope.

So if the big boys are not offering this as a feature of their products, with billions spent on development, maybe it isn't so easy. Configuring a procedure as a remote application, or whatever it is called, will always require a certain amount of information. How to collect, store and expose this information so it can be reused by new protocols, maybe at the same time...that would be helping the developer.

Hmmm maybe look at tWSDL. tWSDL impliments a single protocol: SOAP with document/literal. What is probably not appreciated is that the protocol is just a few pages of code, and is chosen based upon the client request. If someone were to write their own 'binding' and protocol handler they could reuse the entire tWSDL code base. The service would be fully described by the WSDL, input messages would be validated prior to use. If XML was not used as the representation, then the protocol handler would use some other method besides tDOM to deserialize and some other method to serialize the result. Writing the code is the least time consuming part of this. Deciding what your protocol is and how it works will take the most time.

One thing has become clear over the last few years: developers are starting to recognize that interoperability is the key. Interoperability is achieved by restricting the number of options and requiring public descriptions of services which don't require out of channel communications.

A note about code generation from WSDL. TWiST does this, not just stubs creation like AJAX. But it is only for the server. I have begun work on a TWiST Client. The client can run along with the server, but it will very likely not require AOLserver, that is it will be pure Tcl. But I'm developing it against Google's AdSense service. Everything is https, so I'm using ns_https to pull in the WSDL to read. The service developed on AXIS and uses a feature not in tWSDL/TWiST: SOAP headers. Anyway, the goal is a client which is created from reading the WSDL with API creation for each operation. Creating a document and sending it should work just like in the TWiST server. The return document will be available, but there will likely be a series of generic procedures for turning the document (as a Tcl representation) into Tcl structures, lists, arrays, etc. Of course there will be optional or automatic validation on the return document.

Working with a procedure defined via ad_proc actually presents very similar issues as an application page. ad_proc serves many purposes, but the way it works makes it harder to expose as a web service. This is why TWiST works the way it does: configuration is done via <ws>proc, which provides a wrapper for the internal API. Inside this wrapper, you can specify any amount of code. Maybe you could source a tcl page and do something with the resultant state. Whatever goes on inside the wrapper, you just need to return a list which represents the output message.

One point about typing with TWiST: more important is the structural typing, such as the names and order of the complexType elements, how many of each of the elements, can it be missing, is there a default, if missing, does this indicate it is 'nil', etc. Otherwise, your return document will just be a smashed together mess, but internally TWiST does not care about types, just structure. Essentially the type information is a description of the template: what is the document going to look like. Nobody would ever suggest we just run a tcl script and return a list of all the variables, the results need to be formatted, and this is what the type specification does: defines the non-data parts of the document. The wrapper code doesn't need to know anything about XML or any of this configuration, it just needs to return a list which will be formatted into a document.

When the TWiST client is done it will only handle document/literal, whatever is available now is in the wsclient package of tWSDL. I think everything is in a web page, and it requires the ssl module to be installed, at least to use ns_https.