Forum OpenACS Development: Re: Tcl Web Services Toolkit: TWiST

Posted by Tom Jackson on
Seriously Malte,

You have no idea what you are talking about. Your assumption is that because an external tool is unable to figure it out that it is broken. This is similar to your mistaken fix for wsdlpull. Yes wsdlpull is broken in certain respects. But instead of trying to understand the 'bug', you simply changed the datatype so that wsdlpull no longer complained about it.

So the headsup: Malte is making unsubstantiated claims, he has no idea what he is talking about, and he provides zero information about what he is doing.

I ask again that these useless comments be moved to another thread, clearly labeled as misinformation.

Better yet, Malte: just move the discussion to your website, put up your modified code, change the name and have fun.

Btw, If you had provided more information prior to this, I might have been able to help. I actually did find a bug while tracking down the wsdlpull issue. One line of code was added, but wsdlpull still doesn't work. However, it will likely fix the Eclipse issue, but who knows since you have provided no information, just a headsup. So Malte, this is what I call hacking. You are hacking away and calling it fixing. Instead of reporting the behavior you see, you take credit for fixing something without understanding the issue.

Also, note that your headsup helps nobody. Which is why I have asked that your comments be removed to another thread. In the future, if you really want to improve the software, why not report information that would allow me to track down the issue? And if you report a fix, why not include it so that someone else can use it? If it is unimportant to you that everyone benefit from your fix, then why even mention it in the first place?

I think we need another thread:

Why TWiST Sucks, and What I'm Going to do to Fix It.

Posted by Malte Sussdorff on
<wsdl:definitions xmlns:projectm="urn:tcl:projectm" targetNamespace="urn:tcl:projectm" xmlns:tns="urn:tcl:projectm" xmlns:wsdl="" xmlns:wsoap="" xmlns:whttp="" xmlns:xsd="">

In the above definition you can see there exists an "xmlns:projectm". This is not automatically added by TWiST to the definition (::wsdl::definitions::new), but I manually ammended the wsdl::definitions::new procedure and added it there using:

lappend attributes "xmlns:projectm" "$targetNamespace"

right after

lappend attributes "xmlns:tns" "$targetNamespace"

My assumption would be that TWiST is adding the Namespace which I use (projectm), especially if used in the type name for the elements in a sequence, automatically in the definition. That is not the case when using the stock examples provided by Tom and cause Eclipse to break. After I inserted the code which resulted in the insert of xmlns:projectm="..." in the WSDL definition, Eclipse worked and the partners where able to access the webservices provided.

Posted by Tom Jackson on
Yes, you are correct about one thing: projectm shows up as a prefix in the xml schema. However, your fix does not remove this prefix. I noticed this issue when looking into the wsdlpull issue. However, as you noted yourself, my example site doesn't work with wsdlpull either. But it also doesn't have the extra prefix. (This is a slightly older version than the trunk code):

So what is the bug? TWiST creates a mapping of the xml namespace to the prefix of your choice. However, it fails to trim off the leading ::, it passes ::projectm to tWSDL when creating the map. That is probably the bug. However, tWSDL could just trim this, because the leading : are illegal. So in the future, if another user API is developed, if the same mistake is made, the mapping will still work.

Now, during writing the WSDL file, the targetNamespace is your xml namespace. When you include types which are in the targetNamespace, the prefix is replaced with 'tns'. If an incorrect map is used, the replacement doesn't take place.

In the future, multiple namespaces will be supported, and this will be handled by creating another schema in the wsdl file and importing it into the targetNamespace. But, the schema for the targetNamespace will always use either the tns prefix or the built in xml-schema namespace.

The wsdlpull issue is a problem with wsdlpull, not TWiST. The problem will go away with the next update. Hopefully very soon. wsdlpull does not understand renaming of simpleTypes, at least within the same namespace. The problem will go away because the renaming isn't necessary anymore. I plan, eventually, on moving to a directed graph construction to plan the order of writing out xml schemas, but an intermediate solution is on the way.

Your current fix with the lappend is not actually evil, just in the incorrect place. I think you can achieve the same effect by setting:


to the targetNamespace. This will only affect the projectm wsdl.

You can also set the targetNamespace instead of accepting the default 'urn:tcl:yourns':

<ws>namespace init ::stock
<ws>namespace schema ::stock ""

But a simple fix is here:

in wsdl/tcl/wsdl-schema-procs.tcl

in proc ::wsdl::schema::new, add a trim of schemaAlias:

if {!$initialized} {
# trim schemaAlias for buggy user APIs
set schemaAlias [string trim $schemaAlias :]

Posted by Malte Sussdorff on
I like the idea of having the prefix default to tns: if the namespace is actually the target namespace and your fix just does that. I assume (and here I have to wait for the partners tomorrow, as we are in Europe and it is late already) that this is fixing the issue Eclipse has been having as well, as they had been able to parse correctly your code at Thanks for that simple fix.

As for naming the schema using "<ws>namespace schema ::projectm "" ", I remembered it not adding an additional xmlns::projectm="..." to the definitions, just setting the targetNamespace differently. (Look at the source code of which is using your example). Sadly this still is an issue Eclipse is choking at (but do not ask me why...). So, if you could add the first call (adding the Namespace as an xmlns: definition) when doing the second (defining a custom schema), I assume it would work. Though I think for most implementations the fix you provided should be just fine.

One thing in your posting caught my attention.

"wsdlpull does not understand renaming of simpleTypes, at least within the same namespace. The problem will go away because the renaming isn't necessary anymore."

I was wondering about that. If I use <ws>proc ::projectm:GetTask {task_id:integer}, you are creating the custom "simpleType" with the name "project_id" and the base="xsd:integer". When I (during testing) made sure that not the custom type but the original xsd:integer was used as the type in the xsd:element, it worked. Obviously that is not a  solution if you want to have further restrictions like enumeration or patterns. But for strictly the simple types I don't think you need to create a custom simpleType, but again, I haven't read the specs thoroughly enough to say that is allowed by the standard. I can only speak for wsdlpull behaviour in this particular instance.

Posted by Tom Jackson on
"As for naming the schema using "<ws>namespace schema ::projectm "" ", I remembered it not adding an additional xmlns::projectm="..." to the definitions,..."

You are right, it doesn't fix the bug, it replaces the default targetNamespace with your choice. Note that a lot of defaults exist as variables in <ws>namespace.

The problem with wsdlpull is that there is another layer of derivation for the types which it isn't handling correctly by wsdlpull. I believe the schema is correct. One way to test is to remove the intermediate type and point directly to the global simpleType. I'm guessing, but I think that wsdlpull is interpreting the definition as empty:

<xsd:simpleType name="DayNumber">
<xsd:restriction base="tns:dayNumber"/>

Posted by vivek krishna on

As the developer of wsdlpull,somebody brought my attention to
this bug. as far as i understand its a bug in simple types.One such bug is caused by the creating another type DayNumber which is just a wrapper for "daynumber" note the case. Due to this the check for valid integers in the enumeration fails for the 3rd operation in the because wsdlpull does not check validity recursively up the type tree yet. I am workin on fixing this but an immediate fix would be to remove the (redundant ,IMHO)wrapper
type "DayNumber". Feel free to contact me if you do have issues in wsdlpull

vivek Krishna

Posted by Tom Jackson on

Thanks for keeping track of this discussion. Personally I like wsdlpull, which is why I provided links to your software. I immediately recognized limitations for the built in client 'wsdl'. These limitations are to be expected in this field, especially with a simple example use of the API. The point is that the WSDL is correct, but individual clients may not take into account every conceivable representation of data types. This is not necessarily a bug for the client. In my opinion, XML Schema provides so many ways to represent the same structures, that it is unreasonable to expect independent client and server software to 'understand' each other.

I agree with Vivek that there is an unnecessary, although legal, layer (at the moment) in the type definitions of TWiST. This will be removed as I have stated before. But this fix will not make wsdlpull interoperate fully with TWiST. There will still be limitations with types based upon patterns and complexTypes beyond what a simple array could represent. My understanding is that the 'wsdl' client of wsdlpull is merely an example use of the underlying API. I like it, and it helps me check for certain errors. But it is unreasonable to expect that any client or server will ever support much of the XML Schema standard for type definition. There are a handful of companies investing billions into this problem with little success. XML Schema is something of a disaster in flexibility. In response, I have placed well defined limits on how types are represented in tWSDL.

As Vivek has mentioned, a tool in the cure is to represent types as a tree going back to the built in types. I wish to use a tree so that the XML Schema is as simple as possible, but this might also help the clients. At least clients can apply a rational and consistent process in reading the WSDL and developing the type validation code. However, as an intermediate step, or maybe a final step I have decided to flatten the type hierarchy. Every complexType is at a global level, allowing it to be reused as a child of other complexTypes. Elements with a simpleType content will be protected inside their parent elements, but their simpleType will be exposed as a global type in the case that the type is a derived type. But as I said, even this simplification will not guarantee that the wsdlpull client will interoperate in every case.

Posted by Malte Sussdorff on
Tom, as you asked for it twice already and I am not sure how many others are actually following this discussion,here is some information:

You always have the option to write to the OCT at oct "" if you find comments made offensive and would like them to be removed from the forum, or moved or labelled. I actually pondered about asking for that for some of your comments, but maybe this is just your communication style.

Other than that I am delighted to read that my comments do have an impact on TWiST, after all it seems you did find an outstanding issue. And despite dimissing me as stupid you actually take the time to look into things :-). As you seem to prefer me not to give any feedback on TWiST, I will duely respect your wishes. After all the feedback was supposed to help you to improve the software, not to help you insult me.

Posted by Tom Jackson on
You are right that my style of communication isn't always nice, in fact, I'm pretty sure that most everyone here already knows that. I blow things way out of proportion, I make incorrect assumptions, etc.

On the other hand, I'm very receptive to anything indicating a bug. I think the main issue here is fairly simple: TWiST and tWSDL are for machine communication. This is vastly different than human communication. A guiding principle is to make things as simple as possible for clients (software), without regard to humans. But by simple, I don't mean accommodating. The best starting point for that is the WS-I Basic Profile. However, tWSDL has limitations above and beyond those outlined in that document.

OpenACS packages tend to be standardized for another purpose: pleasing humans. I chose to please the machines first, because even though I can argue with humans, machines refuse to even respond if they are not addressed correctly. I also have to please myself. I would rather TWiST do one thing very well, without errors, than to take on another layer of development too soon. I'm also starting to think that any OpenACS package should start over at the same level as TWiST, not above it. Then the OpenACS API could amount to a graphical interface to develop and deploy and monitor web services. Essentially you would have one page per equivalent TWiST API and use the guts of this API as a script on a Tcl page, with minor changes.