Forum OpenACS Development: XOTcl remoting services
There have recently been a couple of enticing announcements about
SOAP-based remoting solutions available to the OpenACS community,
namely TWiST by Tom Jackson and SOAP-DB by Rocael.
As many of you know from Stefan's presentation in Boston, we are doing
as well a substantial development at our institute in Vienna. We do
not want to fall into a "mine is better" discussion, but emphasize the
architectural approach we are following and sketch here the provided
services of mostly Stefan's SOAP and ORB implementation. Therefore, we
try to give a broader picture of the developed (and developing) XOTcl
based remoting services.
These XOTcl based remoting services are not meant to reproduce the
"classic OpenACS scenario" of proposing "just-another" package to the
community, as pointed out correctly by Don, but the intention is to
develop a more generic and framework-integrated solution (not
necessarily ours), capable to work with several
marshaling/demarshaling packages (not necessarily ours) and naming
Towards a remoting architecture for OpenACS
Stefan is developing a generic remoting framework for OpenACS / .LRN
as part of his thesis for more than a year now. This project started
with a thorough review of the existing "half a dozen" OpenACS based
remoting packages (as Andrew Piskorski points out in
particular soap-gateway, the tsoap port and the XML-RPC package. With
that input and the observation, that there is an an obvious
correspondence between WSDL and Service Contracts, it became clear
that we have in OpenACS a good chance to come up with a highly
generic solution, which could make the framework attractive
for highly flexible middleware applications. As we are academics,
our profession is to investigate and develop new approaches with
the potential for publication.
Stefan's work currently contains two parts:
- XOTcl-request-broker: a CORBA-style request broker (aka xorb).
the request broker as a plug-in infrastructure for various
remoting protocols (SOAP, XML-RPC, ...)
- xotcl-soap: is the currently most dominantly used service
- support for Tcl procs and XOTcl classes and objects
- server and client-side implementation of SOAP 1.1
- architectural support for maintaining state
between invocations (one might call a specific
instance of an object (e.g. a certain shopping basket)
and modify it)
- publishing interface (including dynamic WSDL 1.1 generation,
based on service contracts)
- SOAP 1.1 support (SOAP marshaller / demarshaller on top of tdom)
- Modular / extensible request handling: xosoap allows for registering
interceptors / handlers that add functionality (logging,
authentication/ authorization, caching, custom) for various scopes
(per-service, explicit sequencing, ...)
- more thorough test cases (currently just about 100 tests)
- Complete compliance tests with SOAPBuilder Interoperability Lab
- as always after larger changes, code cleanup
- Enhance and test publishing mechanism
- Legacy test: replace ALL acs::sc::invoke uses by its xorb equivalent
- Polishing consumer-side stub creation
- Rewrite support for XSD complex data types based on XOTcl slots
- Provide for WS-I compliance of WSDL generation
- Updating documentation and providing more examples
What is a request broker (xorb) all about?
Procedure call abstraction is a well-established concept in OpenACS
through service contracts. Request brokerage is available in many
different middleware flavors, e.g. CORBA, .NET Remoting, Web Services (in their
SOAP variant), DCOM, Java RMI etc. The developed request broker is
basically designed as a generic object-oriented broker for abstract
procedure calls, i.e. local and remote ones.
Request brokerage is a generic form of request redirection, which is
fairly common in OO frameworks. XOTcl itself provides several novel
means for providing chains of redirectors (e.g. through mixins). One
research goal is to rethink these redirection approaches in terms of
remoting and middleware. A part of this discussion came up in
Note that functions/methods can be in general called explicitly,
or via URLs (the request processor) or via remoting services.
Publishing services by annotations:
OpenACS supports declaring ad_procs as public or private. Similar to
this, XOTcl supports the same flags. The declarations introduced via
the remoting infrastructure through mixins allows to go one step
further and to publish methods for remoting purposes through
annotations. This allows great reuse of code and supports all OO
features such as sub-classing (specialization) etc., such one can
provide a customized remoting services, where everything is the same
as in a more generic one except a few differences.
Role of Service Contracts
Service contracts - as they are available in OpenACS today - are a
very powerful construct somewhat similar to service announcements in
middleware and software-bus architectures, but they are not developer
friendly (as they break locality) and they are not compatible with the
needs of todays remoting SOAs. The request broker addresses these
issues and provides an easier to use and more local interface to
service contracts based on XOTcl objects. These service definitions
can be transformed automatically into WSDL and can be more easily
The overall idea we are trying to follow is to enrich existing
concepts rather than to replace them. Object orientation in terms of
XOTcl is not meant to provide a new language to replace existing TCL
code, but rather to deliver advantages of OO abstractions to the
OpenACS developers. It is fully compatible to existing TCL code. The
ideas of extending service contracts for remoting applications is
analogous. Essentially, one can replace occurrences of acs::sc::invoke
uses by its xorb counterparts.
The current features of Stefan's packages include:
The framework is successfully deployed in the context of European
Research Projects (www.icamp.eu, www.prolixproject.org) and is
currently evaluated for EU4ALL by Emma.
Although the mentioned features are fully functioning, changes in the
naming of the concepts, or in the composition of the components are
not unlikely. For example, in the icamp project, we are developing a
lors mediator interfacing with a SQI based repositories for federated
search of learning resources. Queries are distributed via SOAP to
different servers, the results are collected and transformed to RSS
and delivered to the asking client. In order to work in larger
quantities, we had to use asynchronous queries (asking simultanueos n
repositories, not blocking connection threads), which lead to strange
problems in aolserver with vwait. Now we have rewritten the basic
communication infrastructure, and things work smoothly. However, the
handling of asynchronous queries and the new basic infrastructure lead
to some interface changes as well. So, we want to gain more experience
in larger applications before we release the package to the general
What remains to be done
We should be able to provide a public prerelase withing the next weeks (early Mach)
and organize a tutorial at the Vienna OpenACS conference, if there is interest.
There are a couple of issues that will result in this code being a bit marginalized unless they're resolved.
1. I recognize the need for Stefan to be able to continue to view this code as a research project. However, OpenACS and .LRN are production systems and folks need to be able to depend on code stability. So at some point you're going to have to be able to say "here is our published API, it will remain stable at least in regard to backwards compatibility with previous releases".
2. Oracle support. Everyone knows that I've suggested we drop this twice now. Both times, the community has shouted me down. At the moment, at least, anything that's incorporated into OpenACS core or the mainstream .LRN stuff needs to support Oracle. I don't expect U Vienna to do this but at some point either a) the community becomes convinced to no longer support Oracle or b) Vienna partners with people using Oracle for .LRN/OpenACS in order to get a port working (and this is really impossible until the code stablizes).
I'm not trying to be negative with either of these points, just trying to state the reality within which we operate if we want our code to be adopted into the framework.
Btw, I don't consider your attitude negative at all, _honi soit qui mal y pense_.
First, I want to make sure that there is no misunderstanding: We certainly do call for a more generic and viable solution in terms of call abstraction & remoting for OpenACS. However, this is not necessarily to be found in our work alone! All I want to stress at this point (before we can play our card = upcoming prerelease) is that any decision (either by the single developer or the OCT) should not be limited to a rough "SOAP-goes-Core".
ad-1 :: published API & code stability
As for the API, this is -- as prominently mentioned by Gustaf Neumann -- our top priority for the prerelease. As long as you refer by "code stability" to a stable interface, we certainly have to meet this fundamental requirement. However, the interface is already matured (though still not stable) through incorporating the experiences from the projects mentioned above and beyond. Moreover, the interface will certainly be familiar to many of you as we tried to stick with established notational forms, for instance, when declaring a contract the OOish interface resembles the good old arraylist.
ad-2 :: oracle support
I cannot comment on the fundamental issue here, nor the overall opinion in the community. Currently, the entire request broker operates on the established collection of db calls offered by acs-service-contract with only minor extensions. However, there is currently not a fundamental rewrite at this level (which might though be a good thing to do at some point) and therefore the Oracle issue should not be a barrier in this respect.
concerning 2: most parts of xotcl-core don't depend on the database at all, mostly the oo layer for the content-repository does. Stefan's code does not depend on this. Only the xotcl request broker uses the database to interface with the service contracts.
Technically, we could factor out the database dependencies for xotcl-core and make it different packages, which will complicate bootstrapping and increase the number of small packages. Alternatively, we could define a separate project (e.g. ooacs) which shares large parts of the code base with oacs, but which does not have the oracle constraints.
I believe that the software has achieved the specified goals for the server. The example API docs are a little old, but the example services work and are the best example of how to use the API. Also TWiST is an example of how to create a higher level API for a specific purpose, such as wrapping the database API.
tWSDL has a number of layered abstractions which are not fully explained in the documentation (yet). These abstractions will not likely change as they reflect the standards.
One intention of on our report above was to contribute to a more structured discussion on the state of (a)
abstract dispatch mechanisms in OpenACS and (b) of remoting support for OpenACS.
As a first step, we might come up with a state-of-the-art report (xowiki-based), requirements gathering based upon the results of the existing solutions, review of available code, ... we can then distil this into a consumable comparison scheme.
Again, this would also be an interesting task for a thematic workshop during scheduled Vienna meeting, such a comparison being the joint work artefact.
note, that we are not taking about making it part of acs-core ...I'm anticipating a bit ... web services are becoming more and more important, and at some point it's going to be hard for us *not* to provide one central, approved solution possibly supported as an acs-core package.
Thanks for pointing out that Stefan's work does very little db querying on its own. Sounds like Oracle support won't be a big deal if the decision is made to do so at some point.