Forum OpenACS Development: Re: SOA? - Generic XML-RPC Bridge to Database API

Collapse
Posted by Frank Bergmann on
Hi Roc,

I briefly checked out your code and it looks great, obvious! Unfortunately, it is quite OpenACS-centric. Or do you have an example of an external application (.Net, Perl, ...) accessing the service?

Also: I don't see how we would impose fine-grain security policies with your system. We need to be able to restrict read/write access to certain object types. For example, an external Microsoft timesheet application should be able to read "Projects" and create and modify "Timesheet" objects, but should not be able to read financial documents nor human resources stuff.

So I've got my doubts whether your package allows us to reach our goal: Provide system integrators with the easiest possible option (shortest code in Perl RPC::XML) to connect their existing systems with ours.

Or do you see an option to integrate our needs into your package?

Cheers,
Frank

The package is oacs centric in the sense that it runs queries through soap and expects a return in the same form that the db_ API will return it internally, therefore you can use stuff like <multiple>> and list-builder in your site even running queries through soap.

Although we don't have non-oacs examples for the server API (the one that will be running the queries, and returning them in a db_ way), I'm pretty sure you can build it any soap server-language environment.

Fine-grain security policies was not the goal of this package, first due our project goals, that was to run queries in this way, within an internal-secure-restricted network. Since this package does what it is meant to do, you might be able to build a top of it the security policies, in the same way that db_ API does not inherently check them but other procs does.

Collapse
Posted by Jade Rubick on
Hi Frank:

I think our package may fit your needs -- it's not OpenACS specific, and it allows you to only expose the procs you want.

I like Roc's API, however. Tying the client side to the db_ api is a smart idea.

Collapse
Posted by Wolfgang Winkler on
Hi Jade!

Is there already a test version of your SOAP/WSDL Client-Server? I need a decent SOAP implementation in the next weeks and my options so far are TclSOAP or, preferably tsoap or java. But I'd like to see your package as well.

Collapse
Posted by Jade Rubick on
Hi Wolfgang:

I'll ask Tom to respond.

Collapse
Posted by Tom Jackson on
Wolfgang,

Jade asked me to report the status of our project, tentatively called tWSDL.

The first thing to keep in mind is that this project is completely independent of OACS, maybe more important: there are no additional dependencies beyond using AOLserver and tDOM. Parts of it run with tclsh, most runs with libnsd loaded, the client side will likely be completely independent of AOLserver (maybe not independent of libnsd).

WSDL is a step beyond XML-RPC and SOAP. WSDL is more like a data model for a service. An important new feature is that WSDL defines message types. By defining types, WSDL enables type checking prior to invoking a procedure. The same information can be used to figure out how to construct messages, where to send them, and how to validate the result.

Unfortunately Tcl doesn't specify types, so there is a little more work required if you want very good type checking. On the other hand, Tcl's very simple (and few) data structures makes it easy to model a procedure call and result in XML Schema.

Speaking in OACS language: WSDL is like a page contract, where you can equate a page to an WSDL operation.

WSDL could potentially cover a huge number of implimentations. However, tWSDL is first restricted by the WS-I Basic Profile. This means that the basic protocol is SOAP/HTTP using document/literal, plus a few hundred other restrictions. (Literal is required, tWSDL may end up supporting rpc/literal). These choices greatly reduce the complexity of XML processing. Third parties can add their own protocols, by adding a routine which processes a small slice of the request. Note that collections of operations, called 'portTypes', are completely independent of the protocol, so everything below portTypes: operations, messages and types never change if you add a protocol.

So what do we want to be able to do with tWSDL?

Server side:

- It should be possible for the developer who knows zero about WSDL/XML/SOAP to easily create a service based upon a Tcl procedure.
- It should also be relatively easy to create a service which enforces type checking on the input and output documents.
- More advanced developers should be able to construct services which can process and produce XML documents at the complexity of, for instance, an invoice or sales transaction.

Client side:

- A tWSDL client should be able to read a WSDL file and create a client Tcl API.
- Client code can then use the new client API to invoke the service.
- In the simple case, a single call to a client API will return a Tcl result.
- In more complex cases, client API's will construct an XML document, then invoke.
- Client will have full ability to validate outgoing and incoming documents and report faults.

So where is tWSDL right now?

The server side is progressing very well. We can create a service based upon any Tcl procedure which produces a simple result (list).

Extending this to more complex results will be relatively easy. Although there are automatic procedures for handling the usual case, these can be replaced with custom code. Eventually there will be helper procedures to make it easier to write custom code.

It is important to point out that custom code doesn't need to deal with any of the other parts of tWSDL. Essentially the custom code which invokes the procedure is handed a Tcl representation of the XML document (by reference). This document could be of any complexity. The custom code completely handles this processing until it returns a reference to the result XML. In the current API, this 'custom' code is auto-generated.

The separation of the invoking code from the service ensures that the developer is not limited by 'my' preconceived ideas: any XML document can be processed and/or returned.

Although simple things are simple, the simplist thing: no brainer WSDL, is still a few weeks away. This is nothing more than a wrapper with a few conventions on how things should be done. At the moment we have two services, one imaginary (no real backend), one real. These two services offer enough diversity to build the no brainer service.

Since tWSDL will be used with ACS code, I will develop a switch for ad_proc, which Jade mentioned above. The switch will employ the conventions (hmmm, maybe we can configure those), to build a service at runtime. In any case the switch will do nothing more than cause a single proc to run which will use introspection to create a no brainer service. It should be just as easy to call the proc outside of ad_proc (at the correct time).

Another important part of tWSDL is fault/error handling. Currently client errors are handled: incorrect target namespace and invalid document (type errors). Server errors are not handled yet, but will most likely return a stacktrace. You still get a 500 response, which is all most clients need to know that it is a server error. Error checking/handling is done according to the WS-I Basic Profile.

Code Release:

I'm open to suggestions here. I don't mind releasing this package now, but anyone using this needs to remember that major changes could result. However, I have already done a handfull of major changes and the code changes to the application code have been easy and fast to apply. Application code relies mostly on metadata, so there is very little risk that a big change in implimentation would change the amount and type of metadata required. Most changes so far have been to the form in which the metadata is presented to the API.

OACS has a unique and valuable API, there is also a huge investment in page flows which cannot be replicated by a web service. A web service can potentially be useful in OACS for existing and potential packages, but not every package. The only thing that is really universal across all OACS installations is the Tcl API. tWSDL offers the potential to expose any Tcl API as a web service. In addition, it should be noted that OACS also uses conventions related to types and data structures. It would not be difficult to exploit this metadata to build high quality web services with very little additional effort by the developer.

There are still many questions to be resolved for any web service package including security and how a web service should fit into the request processor. The fact that web services use a different processing model than a pure HTTP service makes this a difficult question.

Collapse
Posted by Wolfgang Winkler on
Tom,

Wow, this was an extensive answer. The first part could be the about page of a website 😊.

I think it is a very good idea to keep your package independent of OACS. So one could setup a simple aolserver installation (e.g. for pulling information from webservices) without the need for a full blown OACS installation.

I see a strong need for TCL/AOLSERVER/OACS as more and more companies are providing their services via webservices.

We will start two independent projects in the next weeks, where in one case we implement the server (client is C# .NET), and in the other case we are the client (server is JAVA).

If it is possible, I'd like to try tWSDL. I think it would be a good testrun for your project and the API, as both projects will feature a rich set of functions and services.

Collapse
Posted by Tom Jackson on
It is more likely to work well with services defined by .NET, as (unbelievably) MS generates very high quality WSDL files. tWSDL will generate a WSDL file to match .NET, but this work is not complete yet. I still need to add the 'types' element. The client will read the WSDL file to generate the API, but it needs to be compliant with what the tWSDL server would generate. At the moment, I'm using an XSL transform on the WSDL file. Someone could also create an XSLT to generate the server side application in Tcl, all that would be left is to write the Tcl procedure, and maybe any special type validators.

I'll try to find a place to put the current documentation on line, and figure out how to get a current snapshot of the code. We are using svn instead of cvs.

Collapse
Posted by Wolfgang Winkler on
Are there any news concerning tWSDL? Is there a downloadable version somewhere?
Collapse
Posted by Jade Rubick on
Hi Wolfgang:

The server portion is in a working state, and we're doing work to integrate it into our own implementation of ad_proc (mostly done).

We'd like to do a similar example integration with OpenACS when we release it, so that a lot of the complexity is hidden from the developer. We'd prefer that when people do the first look at it, their experience is "wow, that's easy" rather than seeing too much complexity (which is inherit in WSDL).

But if you're willing to deal with the underlying API, we can release an early, pre-release version.

Otherwise, I expect we'll have something ready within the next few weeks.

Collapse
Posted by Wolfgang Winkler on
A prerelease version would be very handy. I've no problem with the complexity and maybe could provide some real world testing experiences.
Collapse
Posted by Wolfgang Winkler on
Either I've no clue how google code works, or there are no files in the repository.
Collapse
Posted by Tom Jackson on

Wolfgang,

You can now browse the current code at: http://twsdl.googlecode.com/svn/trunk/.

Or do a svn checkout:

svn checkout http://twsdl.googlecode.com/svn/trunk/ twsdl