Forum OpenACS Q&A: Re: Which version of xotcl-soap (xosoap) should I use?

Collapse
Posted by Stefan Sobernig on
Hi Justis!

Thanks for giving xorb/xosoap a try. I will try to help you through getting it up and running:

Let me make some general comments at the very beginning:

  1. All the errors reported by you are due to using OUTDATED examples. This is partly my fault as I (to admit it frankly) to remove some of them from the tarballs. I will take care of this as soon as possible. Thanks for pointing me to this issue .... However, all examples cited in the main documentation document are kept up-to-date (see below)!
  2. The key reference for the on-going development, how-tos etc. is http://alice.wu-wien.ac.at:8000/xorb-doc/. As core part, it contains primers on using xosoap (bothing consuming and providing services) which are also reflected in ready-to-take examples in the package distribution. The examples you refer to (Poem, my Boston and Vienna conference slides are inherently outdated, as stated prominently at the beginning of the xorb-doc document!). NOTE, however, that our main server at VUE (alice.wu-wien.ac.at:8000, media.wu-wien.ac.at) has been down for three days now (probably hacked). But as I am currently spending a holiday leave far away from Vienna, I can hardly do anything about it personally. We hope to get it up and running again soon. Anyway, there are some work-arounds available (see below).
  3. The first stable and, therefore, public release is 0.4+. Therefore, we do not recommend to use 0.3 anymore (for various reasons).
  4. The current and most stable version is 0.4.2 of xorb and xosoap which corresponds to the trunk version. It also requires more recent versions of xotcl (at least 1.5.4) and xotcl-core (0.73+). I will try to guide you through setting these up below.

So, I have checked out xotcl-soap and xotcl-request-broker from their SVN repository and attempted to install them on a box where we were experimenting with XoWiki. It was a no-go with xotcl-soap 0.4, because our xotcl-core was too old and the download server at media.wu-wien.ac.at seems to be offline.

I am wondering, do you refer to getting xotcl-core or xotcl as such? xotcl is actually hosted at media.wu-wien.ac.at which is down, xotcl-core however is best retrieved from the openacs cvs.

I suggest the following steps (to work-around the current availability issues):

  1. In case you need a recent installation of xotcl (at least 1.5.4, preferably 1.5.6), I provide the source tarball at for the time of media.wu-wien.ac.at being down: stefan.thinkersfoot.net/xotcl-1.5.6.tar.gz
  2. Get a recent xotcl-core:
    cvs -d:pserver:mailto:anonymous@cvs.openacs.org:/cvsroot co -r HEAD xotcl-core
  3. Get the trunk versions of xorb (0.4.2):
    svn co http://svn.thinkersfoot.net/xotcl-request-broker/trunk xotcl-request-broker
  4. Get the xosoap trunk (0.4.2)
    svn co http://svn.thinkersfoot.net/xosoap/trunk xotcl-soap
  5. You will now find a local copy of the xorb-doc document (currently not available) at xotcl-request-broker/doc/manual/manual.pdf or xotcl-request-broker/doc/index.html (xhtml derivate). ALTERNATIVELY, point your browser to http://teldev.wu-wien.ac.at:8000/request-broker/doc/ which is one of our staging machines which runs a recent version of xorb/xosoap.
  6. Follow the examples in the quickstart section (or look at http://teldev.wu-wien.ac.at:8000/request-broker/doc/#x1-170003.2)

This should get you going with 0.4.2. If otherwise, please, report back and I will commit the time necessary to guide you through. I will be back in Vienna tomorrow afternoon, and will try to get our main machines (alice, media) up and running again.

Thanks for your patience!

//stefan

Collapse
Posted by Justis Peters on
Stefan,

Thanks so much for the reply. You were right that my roadblock was about gettting XoTCL itself, not xotcl-core. I already had xotcl-core 0.8 from CVS and it didn't work. I was guessing it was because I still had XoTCL 1.5.1 and needed 1.5.6. So, the temporary download you provided is very much appreciated. The alternate locations for the documentation is also appreciated!

I haven't gotten a whole lot further, though. I ran into the following error when compiling XoTCL 1.5.6 and I'm not sure whether it's a problem in XoTCL's makefile or in the options we used in compiling AOLServer 4.5. Here's what I get:

/tmp/build/xotcl-1.5.6 # make && make install-aol rm -f libxotcl1.5.6.so
gcc -shared -pipe -g -Wall -Wno-implicit-int -fPIC -Wl,--export-dynamic -o libxotcl1.5.6.so xotcl.o xotclError.o xotclMetaData.o xotclObjectData.o xotclProfile.o xotclTrace.o xotclUtil.o xotclShadow.o xotclCompile.o aolstub.o xotclStubInit.o -L/usr/local/lib -ltclstub8.4
/usr/lib64/gcc/x86_64-suse-linux/4.1.0/../../../../x86_64-suse-linux/bin/ld: xotcl.o: relocation R_X86_64_PC32 against `LookupVarFromTable85' can not be used when making a shared object; recompile with -fPIC
/usr/lib64/gcc/x86_64-suse-linux/4.1.0/../../../../x86_64-suse-linux/bin/ld: final link failed: Bad value
collect2: ld returned 1 exit status
make: *** [libxotcl1.5.6.so] Error 1

Any suggestions?

Kind regards,
Justis

Collapse
Posted by Justis Peters on

Stefan wrote:


"This should get you going with 0.4.2. If otherwise, please, report back and I will commit the time necessary to guide you through. I will be back in Vienna tomorrow afternoon, and will try to get our main machines (alice, media) up and running again."

Reply:


P.S. Thanks for the help while on vacation. If your site did get hacked, you have my sympathy. I hope you have a safe trip back to Vienna.

Kind regards,
Justis

Collapse
Posted by Stefan Sobernig on
Justis,

Please, post the followings details:

  • Which tcl version are you using?
  • What are the configure options you passed to the xotcl and aolserver 4.5 builds, respectively?

Collapse
Posted by Justis Peters on
Stefan,

Our TCL is 8.4.13.

Here are the options I am passing to xotcl-1.5.6/configure:

CC=gcc; export CC; ./configure --enable-threads --enable-symbols --prefix=/usr/local/aolserver45 --exec-prefix=/usr/local/aolserver45 --with-tcl=/usr/local/lib/

Here are the options that I think were used on aolserver45. I didn't build it, but this line was in the command history along with "cd aolserver45" and "make install", so I'm going to guess that it's correct:

./configure --prefix=/usr/local/aolserver45 --with-tcl=/usr/local/lib/

If it helps, I tried rebuilding xotcl-1.5.1 with exactly the same options I am using for xotcl-1.5.6 and it works just fine. Hopefully, that helps us narrow in on the issue.

Kind regards,
Justis

Collapse
Posted by Stefan Sobernig on
Justis,

Thanks for providing the wsdl descriptions to Tom and me. Though I am puzzled why you don't have any public url pointing to them since they are auto-generated by Axis?!

Anyway, as for the compilation issues: I fear that you experience an issue specific to x86_64 which is related to a documented bug in binutils. See http://aspn.activestate.com/ASPN/Mail/Message/tcl-core/3573780 and make sure that you have a sufficiently recent binutils distribution. However, I do not have access to a comparable build environment so I can not get my hands dirty ...

Hope this gives you a pointer ...

Collapse
Posted by Stefan Sobernig on
For the sake of completeness: Our main machines are up and running again. This includes:
Collapse
Posted by Justis Peters on
Stefan,

You're right that these are auto-generated by Axis, but our deployments are all behind a corporate firewall. In addition to the attachments that I sent to you and to Tom, Tom has posted copies on his website:
http://junom.com/document/openacs/business-wsdl/

Thanks for the tip on our compilation errors. What you suggested is what I already feared. For now, I am going to retreat to testing within a 32-bit VM instead of in the 64-bit VM that my client provided. I'll work to get past the 64-bit compilation hurdles once we've got a proof-of-concept working with xotcl-soap. Right now, I just need to keep us moving forward to a decision on how best to begin consuming web services from within our existing OpenACS apps. Before dropping the topic, though, let me note these two points: 1) we have some sysadmins who know our environment very well and might solve this for me in mere minutes; 2) if this turns out being something related to XoTCL rather than our environment, I may be able to get permission to share a copy of the Xen VM where the error occurred.

Thanks for the enthusiastic help. It has been appreciated.

Kind regards,
Justis

Collapse
Posted by Stefan Sobernig on
Justis,

Right now, I just need to keep us moving forward to a decision on how best to begin consuming web services from within our existing OpenACS apps.

I know. You already received a rough primer from Tom on approaching your scenarios, mine on xosoap will follow tomorrow (Dec 30, 2007, GMT+1). I am quite busy these days, so am just now arriving at this point on my todo list.
I will post my suggestions to this thread ...

cheers
//s

Collapse
Posted by Justis Peters on
Stefan,

I hope you didn't take my comment as suggesting you should move faster. I'm greatly appreciative of the help you're providing and I'm definitely interested in xotcl-soap, but I don't want my excitement to turn into a burden on your todo list.

Myself, I'm juggling 4 other projects in addition to this one, and they all need to show forward progress in the near future. Thus, I am working over the holidays. I'm traveling with family, but ask "where's the internet connection" as soon as we arrive :)

I've got to head back to one of the other projects for a bit. If there's anything I can provide before tomorrow that would help you provide better recommendations, please let me know. I'll be glad to do so.

Kind regards,
Justis

Collapse
Posted by Stefan Sobernig on
Justis,


I hope you didn't take my comment as suggesting you should move faster.

No, no ... I didn't, I just wanted to express my awareness of your shortage in time ...

Finally, I reviewed the interface descriptions you provided to me by email (and which Tom is kindly hosting). A single one (jasperserver, http://junom.com/document/openacs/business-wsdl/jasperserver-repository.wsdl.xml) adopts the RPC/Encoded style, the other three, however, realise the document/literal style.

As for the RPC/Encoded style, realising a consumer is straight forward. Please, find and review a working example at http://svn.thinkersfoot.net/xosoap/trunk/www/demo/demo-ivc-jasperserver-consumer.tcl.
The above proxy was tested against an auto-generated mock service based on the wsdl provided.

As for the document/literal ones, the main problem is that document/literal support for the CONSUMER/CLIENT part has not been accomplished. This is, on the one hand, due to the fact that this feature has been ranked low in priority (at least so far), on the other hand, I am only now investigating a major rewrite towards a more general xml data binding infrastructure as discussed in http://openacs.org/forums/message-view?message_id=1392828. The latter is key to a sound support of document/literal style. Though I could think of some rough work-arounds for realising a simplistic document/literal service (jdbc one), the advanced features required by the others (TaskManager, unit_on_found) such as choice compounds etc. will only be available in the next major release of xosoap.

From my perspective, you now have three options:

  • Depending on your time schedule, you might wait for an early preview of the xosoap rewrite that comes with the necessary features. I will be working on this in the first week of January, and there is a certain chance that I might provide support for the features your scenarios require. However, I can't make this a firm promise, the entire rewrite will only be finished for the OpenACS conference in Guatemela City in the mid of February. Also Tom might be working on tWSDL/TWiST consumer part (!?) in the meantime ...
  • Provide RPC style wrappers in an alien environment, such as Axis, that hide the document/literal issues from xosoap consumers. I, personally, consider this a considerable extra amount of work and source of complexity. But is might be feasible ...
  • For the time being, assemble and use the raw xml messages required by the services (using tdom). It is quite easy to extract sample or template raw messages from the interface descriptions. Tools such as http://soapui.org make this an ease ... Of course, this is only a temporary solution but still a valid recommendation.
Again, I will be working on the xosoap rewrite over the next week. Let me know whether you can afford waiting for xosoap to mature in these directions, so I can put some effort in getting your scenarios supported ...

cheers
//s

Collapse
Posted by Tom Jackson on

The main issue, in my mind, is that you have several examples which will never be handled by a generic client without some hand programming. Any complexType which adds attributes is going to require hand coding. There is simply no generic interpretation of how to handle values that are in both attributes and as element content. Just think of a database row: you have one 'array like' list of columns. If you have a complexType with attributes, where do they go (automatically?) Because of this, tWSDL/TWiST store data as element content and don't automatically use attributes for any content. This doesn't mean attributes can't be used (except if you have mixedContent). It also does not mean that tWSDL/TWiST can't store and manipulate XML with attributes, in fact, this is a requirement. An example from

 <xsd:complexType name="MessagesType">
  <xsd:sequence>
   <xsd:element name="Message" type="MessageType" minOccurs="0" maxOccurs="unbounded"/>
   </xsd:sequence>
  <xsd:attribute name="version" type="xsd:string"/>
</xsd:complexType>

An example element would be:

 <MessagesType version="1.0.2">
  <MessageType>...
  </MessageType>
 </MessagesType>

So the version attribute cannot be combined with the child elements automatically, but why? First, the attributes and elements are in different namespaces (not xmlns namespaces, but more like how Tcl commands, variables, etc. can have the same name without conflict. Therefore it is possible to have a child element of MessageType named 'version'. (Also on the schema type level, simpleTypes and complexTypes can have the same name without conflict.)

Anyway, although you can't do automatic mapping to an API in this case, the data is still available for hand coding. This is really a minor issue overall. What is more critical to automatic client programming is how reasonable the WSDL description is, and if there are any custom extensions added on to the basic service description. Custom extensions always require developer involvement. Even the meaning of any extension has to be understood by a developer before any code can be written. If the point of a WSDL is to allow for automatic client configuration, then extensions are a great way to remove these benefits. Any SOAP headers fall into the same category, although SOAP headers may be easier to handle.

The TaskManager WSDL imports several tag libraries for special features. This indicates that both the client and server use special programming logic to interact. This is way beyond data handling and representation. In general there is no reason to use special tags unless you don't want generic clients to use the service. All data and types dealing with specialized service configuration could be just as easily delivered via SOAP messages like all other data.

Of course if there is any chaining of operations, this also leads to some need to provide client side programming logic. This is exactly like providing a pageflow on a website. Getting or submitting data via a web service can be the smallest part of the overall client design (this is the typical case).

Collapse
Posted by Stefan Sobernig on
Tom,


The main issue, in my mind, is that you have several examples which will never be handled by a generic client without some hand programming.
Any complexType which adds attributes is going to require hand coding. There is simply no generic interpretation of how to handle values that are in both attributes and as element content.

I wouldn't sign the above statement. While it is common sense that xml/xml-s idiosyncrasies are critical when using xml as representation/ streaming format, it is a question of design whether to you want developers of your framework to express these more or less subtle differences when creating their mappings.

The attribute vs. element issue is one of the most trivial cases IF considered isolated from other xml/xml-s idiosynchrasies. It gets more complex once you consider the nillability constraint, just to name a prominent one. The approach taken by most frameworks (Axis, .NET, ...) is to generate per-type marshallers (note that "type" refers to "class type" with classes being representations of element types) from xml-s descriptions embedded in wsdls. They, then, take care of streaming ("serialising") certain members of an object into element attributes. It's a rather declarative approach.

In the end, it boilds down to the question for which overall purpose you want to conceptualise and devise your framework, which "remoting" lifecycles you want to support!
In conventional (OO)-RPC middleware, you deal with rather atomic, operation-level interactions (of objects) which may allow a design that simply rejects element attributes as representation flavour. This is fully valid and acceptable! In this life-cycle, developer devise procedures or objects first, representations come second. Once you enter other scenarios, where "document/ content models" are devised first, and language representations (procs or objects) are secondary, you end up with the need to support more of the xml/xml-s idionsynchrasies. This key distinction is also prominently mentioned at the beginning of the article posted above with the authors referring to schema-first vs. object-first approaches.

Collapse
Posted by Tom Jackson on
I'm talking about WSDL, which is a standard for describing a web service. Anyone who wishes to pick up a web service client have have it 'understand' the web service automatically should know that there are limits on when this can be done. The term for this understanding is interoperability. And it means that there are no ambiguous features in the description.

However, even WSDL is a pretty loose standard. I've pointed out before that an easier standard to follow is the WS-I Basic Profile:

http://www.ws-i.org/Profiles/BasicProfile-1.0-2004-04-16.html

Maybe you think I'm talking about how you translate an existing interface into a web service? Not sure, but the issue is a client issue, not a server issue. WSDL is used to abstract the interface away from the programming language used on the server. Otherwise, consuming an Axis service would require an Axis client, ruling out the use of OpenACS, C# or any other client. One of the main problems for interoperability with web services is their continued tie to the RPC style. It is easy to implement because it is based upon mapping programming types to a message format. Assuming client and server are based upon the same programming language, this is easy, but it isn't interoperable outside these rigid and narrow limits.

In the present case we have an Axis based service. The service is defined using a WSDL. My assumption was that the client should automatically read and understand the definition so that it can construct client API to invoke the operations. When the data is transfered as element content only, it is easier to create client API. In this case the parent could be considered a class or an object, and the child elements are slots, or whatever the correct word would be for the language. If you add attributes, there is an ambiguous quality. Without human analysis, you can't determine if the element attribute is logically part of the element data, or a comment on the data itself. For instance, the element content could be encoded, so an attribute could be targeted at the serializer/deserializer code and not the application.

This is, again, the easy case: data element have attributes. The hard case is a WSDL which has extensions. Who could ever provide a client which understood private extensions automatically, without hand coding? Nobody, outside the narrow meaning of the basic WSDL, very little can or should have a fixed meaning.

Collapse
Posted by Tom Jackson on
As you mention the difficulty of handling nil values. Here is a link to the decision matrix for tWSDL:

http://junom.com/document/openacs/business-wsdl/nil.txt

There are a large number (32) of possible combinations mapping to a handful of results.

Collapse
Posted by Stefan Sobernig on
Tom,


If you add attributes, there is an ambiguous quality. Without human analysis, you can't determine if the element attribute is logically part of the element data, or a comment on the data itself. For instance, the element content could be encoded, so an attribute could be targeted at the serializer/deserializer code and not the application.

Again, this is what I referred to as "content-/ document-model-first" versus "language-construct-first" (or, schema-first versus object-first). It depends on your scenario and the set of requirements embodied in coded conventions. Justis' WSDLs are auto-generated by Axis which implies an object-first simplification. For the resulting Java objects, it does not make any difference whether an object member (i.e. property or slot in XOTcl terms) is transmitted as element or element attribute. That is, in object-first scenarios, there is hardly any quality difference between element and attribute content because the underlying object systems does not make these distinctions.

Your statement is certainly valid but only for schema-first scenarios, where we cannot stream-line schema information in the meta-object level of an object system. But this is certainly NOT an issue with Justis' WSDLs ...


Without human analysis, you can't determine if the element attribute is logically part of the element data, or a comment on the data itself. For instance, the element content could be encoded, so an attribute could be targeted at the serializer/deserializer code and not the application.

Again, you refer to something which does not concern Justis' WSDLs at all. In Document/Literal style, document content is not "encoded" as it would be with SOAP Encoding or whatsoever. It is a proper document that can be thrown against any XML schema.

Collapse
Posted by Malte Sussdorff on
Stefan, how far did you manage to get with document/literal and is there a way to help speed up the process? I have to connect to a framework at http://atena.ios.krakow.pl/efws/ef.asmx?WSDL and though I am at the moment mainly concerned about the production part (and not the consumption one), I need consumption working as well (which is why I wanted to go with xosoap instead of TwiST).

Also, how would you implement the "getProject" glueing in OpenACS if I have a TCL procedure that returns the project_id when called.

Collapse
Posted by Stefan Sobernig on
Malte,

Also, how would you implement the "getProject" glueing in OpenACS if I have a TCL procedure that returns the project_id when called.

I tried to sketch a ready-to-take sample for you below ... did I get it right when assuming that you want to expose an existing tcl proc through a soap channel?



# / / / / / / / / / / / / / / / / / / / / / / / / / /
# $Id$

# -) prerequisites

namespace eval :some_name_space {
proc getProject {input} {
# retrieve project_id based on input ...
set project_id 1
return $project_id
}
}

namespace eval ::cognovis {

::xo::db::require package xotcl-request-broker
namespace import ::xorb::*

# 1-) provide an interface description aka 'service contract'
ServiceContract EchoProject -defines {
Abstract getProject \
-arguments {
input:xsString
} -returns returnValue:xsInteger \
-description {
A sample echo operation that returns
an arbitrary integer value.
}
} -ad_doc {
This is the sample interface description as requested
by Malte ...
}

# 1a-) deploy your interface description
EchoProject deploy

# 2-) Provide 'servant' code and register it with the invoker:
# 'service implementation'
ServiceImplementation EchoProjectImpl \
-implements EchoProject \
-using {
Delegate getProject \
-for ::some_name_space::getProject \
{Delegates calls to the servant proc indicated ...}
}

# 3a-) deploy your service implementation
EchoProjectImpl deploy
}

As for the second issue, i.e. document/literal support for the consumer side:


... how far did you manage to get with document/literal and is there a way to help speed up the process?

I am closing down and started rewriting xosoap allover, I have a somewhat firm deadline for the Jan 24, so, hopefully, till Sunday, there will be something operative (though not completed). Currently I can't think of anything that I can easily delegate to you, but I will think a bit and come back to you ... thanks for the kind offer!

Collapse
Posted by Malte Sussdorff on
Hi Stefan, thanks a lot and thanks for the answer and the information about the deadline. Jan 24th doesnt look so bad from my point of view :-).

Now, as for exposing the above statement, where can I access it? I was looking through the documentation and the FAQ for a hint under which URL I would be able to connect to it (and be it only for round trip self test), but I must have overread / overlooked this.

I found /request-broker/admin and see the service is there (horray). And I found /xosoap/services which says "0 services". But that's pretty much as far as I got so far, I somewhat hoped to find the URL where I could say "url?s=WSDL" (which I found in the FAQ).

Collapse
Posted by Stefan Sobernig on
For the sake of completeness (and the records). I spotted a critical typo in the code sample above that had evaded my eyes:

# 2-) Provide 'servant' code and register it with the invoker:

# 'service implementation'

ServiceImplementation EchoProjectImpl \

-implements EchoProject \

-using {

Delegate getProject \

-for ::some_name_space::getProject \

{Delegates calls to the servant proc indicated ...}

}

needs to be

# 2-) Provide 'servant' code and register it with the invoker:

# 'service implementation'

ServiceImplementation EchoProjectImpl \

-implements EchoProject \

-using {

Delegate getProject \

-for ::some_name_space::getProject \

-ad_doc { Delegates calls to the servant proc indicated ...}

}

Note the missing thingy is the "-ad_doc" flag ...