Forum OpenACS Development: Re: Tcl Web Services Toolkit: TWiST

Posted by Tom Jackson on

I've completed the overhaul of the XML package, so I have another release/tag. The overhaul didn't change the TWiST API, although there were small adjustments to the tWSDL API. The reason for this work was that as tWSDL was developed, the XML procedures were put in toward the end, once I knew that the internal representation was working okay. This delay resulted in three ways to create XML nodes. Now there is only one method. This also allowed me to handle more valid documents from clients.

Another change is that XML node structure is checked/validated prior to the datatype of the children. This should speed up rejection of invalid documents.

The new svn release is here:

svn checkout twsdl

I just copied this over my install at and it seems to work as expected.

One very simple client is wsdlpull. It is a C++ command line tool and API. It is available here (with links to the source):

I've tested it on the example web service. It produces the expected results and it also rejects invalid inputs prior to sending them to the server. TWiST has a built in tool for testing which does allow invalid inputs so that you can test the response documents.

Posted by Claudio Pasolini on
Hi Tom,

after downloading the release-0.6 tag and playing a little with TWiST and the basic tWSDL API I have some doubts as how to proceed further.

First of all checking the environment sourcing init.tcl with nstclsh I get the following error:

[24/Feb/2007:11:59:47][4792.3083896496][-main-] Error: TWS:no server
while executing
"ns_info $option"
("foreach" body line 2)
invoked from within
"foreach option {address hostname name server servers config} {
log Notice "---->\[ns_info $option\] = '[ns_info $option]'"
(in namespace eval "::wsdl::server" script line 3)
invoked from within
"namespace eval ::wsdl::server {
variable hostHeaderNames
foreach option {address hostname name server servers config} {
log Notice "---->\[ns..."
(file "/usr/local/aolserver_45_r0/servers/server1/modules/tcl/twsdl/packages/wsdl/tcl/wsdl-server-init.tcl" line 7)
invoked from within
"::source /usr/local/aolserver_45_r0/servers/server1/modules/tcl/twsdl/packages/wsdl/tcl/wsdl-server-init.tcl"
("uplevel" body line 1)
invoked from within
"uplevel ::source "$file""
(procedure "::tws::sourceFile" line 4)
invoked from within
"::tws::sourceFile [file normalize [file join [file dirname [info script]] "wsdl-server-init.tcl"]]"
(file "./packages/wsdl/tcl/wsdl-init.tcl" line 11)
invoked from within
"::source ./packages/wsdl/tcl/wsdl-init.tcl"
("uplevel" body line 1)
invoked from within
"uplevel ::source "$file""
(procedure "::tws::sourceFile" line 4)
invoked from within
"::tws::sourceFile [::tws::util::package::directory [file join $packageName tcl ${packageName}-init.tcl]]"
(procedure "init" line 4)
invoked from within
"init $package" PACKAGE wsdl
[24/Feb/2007:11:59:47][4792.3083896496][-main-] Notice: TWS:Package xml: Initializing...
Note that this error don't show up during aolserver startup, which however continues to exhibit the previosly posted error.

Regarding the basic tWSDL API I finally built a working sample service which accepts a user_id and returns the name and the email of the user.

So doing I learned how to build complex types, how to define an operation and associate to it a tcl proc and now I'd like to comlicate the matters a little, but I don't know how to populate my complex types within my tcl proc.

Given the following definitions:

eval [::wsdl::elements::modelGroup::sequence::new $usersns oneUser {
{name userdecoder::name}
{email userdecoder::email}}]

eval [::wsdl::elements::modelGroup::sequence::new $usersns Users {
{oneUser userdecoder::oneUser}}]

how do I put the values pulled from the db into the Users structure?

A few other questions.

Is it possible to view the generated WSDL, like in TWiST?

When you define a ::wsdl::operations::new each parameter of the tcl proc is followed by the string 'Value': what is its meaning? Is it mandatory?


Posted by Claudio Pasolini on
I'd like to rectify my previous post regarding the basic tWSDL API.
  • I found how to show the WSDL definitions using xml::document::print ::wsdb::definitions::${serverName}]
  • I ingenuously assumed that once defined a complex type I could use it as a component of a higher level complex type, but it is not so: when I try
    eval [::wsdl::elements::modelGroup::sequence::new $usersns Users {
    {oneUser userdecoder::oneUser}}]
    I get the error
    can't read "::wsdb::types::twist2ns::oneUser::validate": no such variable
    and so my question is: how do I define higher level complex types?
Posted by Tom Jackson on
The tWSDL API modelGroup::sequence::new is currently restricted to the case where all child elements are simpleTypes. This is a common use case. It is also restricted to a each child occurring zero or one time.

I know the API call seems simple enough, but it hides a lot of code generation, including the type validation routines and the invoke routine. These routines would look different than the simple case. This seems to suck pretty bad until you consider the number of Tcl API which take or return multi-dimensional data.

It turns out that type validation is the easiest to extend to multi-dimensional data, but overall, the problem is not too difficult and was planned for in advance.

This is the meaning of the {ElementA Value ElementB Value} configuration (called a conversionList). The modelGroup::sequence API assumes actual values for each child element. This configuration is used in the invoke API (which is auto-generated) by the API ::xml::childElementsAsListWithConversions

Currently this simple API only handles the Value type, although if you look at it, other types are commented about.

Let me just say that developing the type system and the API is very delicate, it has to reflect the method of type development used by XML-Schema, otherwise we cannot publish the interface in a way that a generic client can use. Beyond the simple interface, the developer must become more aware of how things work throughout the entire system and in XML-Schema, WSDL, etc.

I'll post another note about the invoke procedure interface, which is a key abstraction. Any developer using tWSDL for complex services must understand this interface.

Posted by Tom Jackson on

Here is an example auto-generated invoke procedure:

proc ::wsdb::operations::openacs::CheckEmailOperation::Invoke {
} {
variable conversionList
::xml::childElementsAsListWithConversions $inputXMLNS $conversionList
return [::wsdb::elements::openacs::CheckEmailResponse::new $outputXMLNS [::openacs::CheckEmail $Email]]

This tiny proc is the interface between the XML world and the Tcl world. This procedure could be completely hand written as long as it takes the same input (where to find the Tcl rep of the XML document and where to put the output document) and returns the same output (reference to the return document, a child of outputXMLNS).

In addition, the operation's namespace variable 'invoke' must be set to the fully qualified proc name.

From the point of view of the server, the size and complexity of the document isn't important. But obviously the underlying Tcl API which can handle complex data is beyond the scope of the interface: it is hidden inside the invoke procedure. This is the purpose of this interface, otherwise it would take a few seconds of thought to discover something you couldn't handle, and hacking would begin. The invoke procedure is independent of other parts of the tWSDL API. You can completely test this procedure without the need to run the SOAP server, therefore it is also independent of the binding or protocol. If someone adds a new binding, the invoke procedures remain valid.

One use case which I am working on is a complexType which contains a sequence of children all of the same type, such as a sequence of database rows. It works as expected, but I don't yet have a higher level API to write the code.

If you run the modelGroup::sequence::new API with 'puts' or 'log Notice' instead of 'eval', you can see what code is actually generated.

Posted by Tom Jackson on

Both errors you have reported are due to the fact that a server isn't defined. When you use nstclsh, as a command line tool, you will not have certain functions and data structures: ns_conn, etc. This is normal, it is not a bug. To use tWSDL to deliver a service, you must currently use a virtual server.

Virtual servers are enabled by moving the nssock* drivers to the global modules ns_section:

ns_section ns/servers
ns_param jnm {Junom Server}
ns_param twsdl {tWSDL}

ns_section ns/modules
ns_param nssock1 $bindir/

ns_section ns/module/nssock1
ns_param port 8001
ns_param hostname
ns_param address
# Important:
ns_param defaultserver jnm

# Virtual Server Host headers:
ns_section ns/module/nssock1/servers
ns_param jnm maria:8001
ns_param jnm junom:8001
ns_param jnm

There are probably other changes to enable virtual servers, but maybe someone has done this for OpenACS?

The other option is to replace the script in wsdl-server-init.tcl.

Posted by Claudio Pasolini on
Just another qwick question.
I installed the wsdlpull client, but I'm getting this error, no matter what service I'm calling:
Error processing
An Exception occurred at 0:0
Wsdl Parser Exception : could not be opened
Apparently the compilation went good: could you kindly post the command you are using to invoke your stockQuoter service?
Posted by Tom Jackson on
Claudio, I cannot speak for these other services, but you can grab my example WSDL file (one for openacs):

From my experience the wsdlpull client more correctly constructs SOAP messages. It doesn't know how to verify restrictions based upon patterns (regexp), but it does verify enumerations.

On the command to invoke stockQuoter, I guess you mean what SOAP message to send? In the stockquoter/www directory, there should be a post.tcl file, which probably needs to be edited so it sends to the correct address. Otherwise what you are asking for is a client, and wsdlpull looks like the easiest to use.

Otherwise, move the stockquoter procs to another file under pageroot and use the <ws>* API.

Posted by vivek krishna on
If you installed from the rpm then this could be a problem
Apparently there is an issue with the rpms.Try compiling from source or better still checking out from CVS.I was able to invoke the webservice without any problems

wsdl getRate usa India

Hope this helps

If you still face a problem post it on the wsdlpull mailing list