Forum OpenACS Development: Remoting (Formerly: The future of OpenACS)

Now, what are the issues with OpenACS? The main issue is that most OpenACS applications depend upon page views. By that I mean that pages serve as a procedure. This makes it nearly impossible to reuse this code by a different protocol.

Another issue specific to Tcl is important to recognize: Tcl does not maintain type information, ad_proc actually complicates the situation, and we know nothing about certain things like return types, use of uplevel/upvar, etc. Many programming languages know all this information and so it is 'easy' to serialize a data structure to XML. This is why there are so many rpc packages in every language.

Second is that it is unlikely that anyone anymore really wants to expose an internal procedure to the network. The best method for handling this is to write a wrapper procedure to restrict use or perform any additional checks that usually take place prior to execution.

One of these 'checks' is validation. This is where the wheels fall off of rpc and it is why nobody, not .NET or AXIS performs validation out of the box! And by validation, in this situation I only mean simple validation like is 4 an int? Derived types and complex type validation...this isn't even supported if you wished to do it.

Just make up your own method of figuring out if the input is valid, test, debug and hope.

(Sorry, Tom, but I think discussions should be kept open to the interested public, so from time to time we should either describe details discussed or refrain from going into a certain level of detail) For those not so familar with these terms (and still interested and following): WSDL basically describes four aspects: 1-) A remote interface (operations, parameters etc.), 2-) "types" for these parameters (parameters are actually grouped in input and output messages), 3-) rules how to create the XML messages used to exchange operation calls between two ends, and 4-) . The term "Document/Literal" refers to this rule set, and describes two aspects:

• "Document" vs. "RPC": Document means, the XML message to expect from either side is "well-formed and valid XML document" you can not only throw in XML parsers but also validate against schema descriptions (RelaxNG, XML Schema). The opposite style is RPC, and in fact, the only difference (looking at XML messages in Document and RPC styles) is that in RPC, the name of the remote operation is part of the message, as an XML element. In Document style, it is not!!!! So what is the key difference now, well, how should the receiving end of this message now which local operation to invoke on??? So what the key difference here is that dispatch resolution needs to be handled differently (taking http header information, etc.)
• "Literal" vs. "Encoded": Literal translates to "literal meaning" and simply says take the XML message that comes in as what it is, a completed and self-contained XML document. Encoded, on the contrary, means that it is not generic XML, but a dialect of XML that one should expect. For instance, associatives arrays can be written differently (serialised differently).

One thing has become clear over the last few years: developers are starting to recognize that interoperability is the key. Interoperability is achieved by restricting the number of options and requiring public descriptions of services which don't require out of channel communications.

Why don't we compare apples to apples. I am talking about real software. TWiST, .NET, AXIS, as they exist today. The purpose of this software is to solve a problem, not philosophy. The tiny problem I am interested in solving is exposing useful operations over a network and making it as easy as possible for remote users or programs to figure out how to actually invoke the operations.

At any rate, TWiST exists and works. Does the software you are speaking of exist? Could you give examples? The comments I have made about ASP.NET and AXIS are well documented. The reason I know is that after I wrote TWiST, I started hunting around for how these other solutions did the same thing as TWiST. Well, very poorly and not out of the box. However, I could be wrong, and I would really and truly be interested in parallel examples, of even the simplist web service. But the requirements are that the developer:

1. Doesn't have to write their own WSDL file or move it into place by hand.
2. Can abstractly describe simpleTypes not based upon a given programming language.
3. Can describe how to restrict simpleTypes from built in abstract types.
4. Can describe how to combine simpleTypes into complexTypes
5. Doesn't have to write validation code or type creation code as a separate step from description.
6. Can name their input and output parameters as they wish without rewriting their internal code.
7. Don't have to edit source code files to make configuration changes to the service.

Okay, that is enough, Actually if you find examples of how to do ANY of these with either .NET or AXIS, please pass them along. I am stuck on writing a howto to find such examples.

There is one fact which always stands out for me on this topic, I'll restate it and hope that someone can set me straight, because if it is not a true fact which always must be kept in mind, I need to rethink my whole way of writing code. Here is the fact: data cannot be created out of thin air. Maybe this is one of those laws of thermodynamics.

The situation is this: some code exists in memory or in a file. This code cannot 'remote' itself without some help. If this fact is not fully appreciated, those who fail to appreciate it will wait in vain for this 'remoting' to happen all by itself. If things were otherwise, we would have programming languages which consisted of one word, because everything else has already been figured out.

Skip the one word. There is proposed a great remoting program. You install it and suddenly it does exactly what you want without any input. It speaks whatever language or protocol you can think of. Not only does it speak, but anyone or any program which contacts it understands everything. Communication is perfect.

Until I find this program, or it finds me, I'll try to focus on figuring out what information is needed to move from an internal code to a useful service. What language will I use to do this? WSDL. Will I follow any advice from others, or just consider my creative genius enough? No, I'll follow the WS-I Basic Profile.

I'll use a WSDL file to hold information, data, created by a person in order to communicate what I wish to be a service. By following a certain format, the WSDL format, I reduce the amount of information I need to create. The form of the WSDL file is fixed by certain standards. The form of the messages is fixed, the description of the types and operations follows a fixed pattern. All I have to do is provide a few pieces of real data. Maybe this is called work, maybe thinking, but it doesn't come out of thin air. This is what folks get paid for. If we could remote a procedure by adding a switch, then remoting would be a one bit operation. One bit of data. We can argue all the way around this for years, but the information has to come from somewhere. All TWiST does is provide a format for collecting this information in one place so that tWSDL can do all the boring code writing which doesn't contain any more content than the TWiST file. This is what has to be seen: a few lines of a configuration file is expanded into a working web service. Developers don't need to spend hours or days writing code which essentially has no more data content than the description of the service.

I'd wish you wouldn't have played this card, as most of the issues listed and mentioned can easily be discussed without referring to YOUR and MY piece of work. If I remember correctly, the idea of the thread (as it was started by Malte) was to collect ideas and requirements for remoting solutions for OpenACS, and not whether YOURS or MINE are better suited.

Ummm, you started this thread.

I have never looked at your software, don't know where to get it or how it works. I use my software as an example for several reasons, but I'm not promoting it. Anyway, the main reason for discussing software which actually exists is because it is obvious how things are done, while talking about stuff which could work or might be better is a waste of time. Until something is boiled down to code, everything sounds possible, that is the nature of programming languages: anything can be done, in any language.

Another reason to talk about existing code is that code has behavior. If you want to discuss a future requirement, it is good to know how other people have actually solved the problem, what is involved. That is why I was looking for examples from other major players in 'remoting'. I talked about, and provided links to the ones I know about. Personally I'm not interested in re-inventing the wheel. When someone tries to code to a standard, they are giving up that part of the process. I have said I tried to follow the recommendations in WS-I Basic Profile. If you don't wish to follow a standard, it becomes very difficult to compare it to anything else. It does make it easier for you to claim that your software works and does what you want.

I don't quite understand why you are so interested in examples from compiler language environments. Scripting languages are much more powerful in this respect!

I listed the examples I could find, I asked for better examples. I'm still waiting.

This I don't quite understand, provided that I understood your guiding scenario. You want to have a language-bound code block turned into a service + service description, but you want simple types (xs based types) not defined in the very language the code block was written? this is conflicting and contradictory to your scenario.

Ahhh! Yes! This is exactly the advantage of Tcl over most other languages in this area. First consider the following: Tcl is best as a glue language. In OpenACS it is a glue language. OpenACS doesn't store information as a Tcl structure, it takes in some string and passes on a string to another application. Usually stuff goes into a database, which does have type requirements. Most procedures don't care what they take in, so you claim that the typing of this input or output doesn't make sense. Actually what doesn't make sense is caring about constructing types in Tcl just to pass them on to another application. But if you can test the type information, you can pass back information to the client in a useful form. Once the procedure executes, you pass from client error to server error, which is much less useful. This is why we have filters in ad_page_contract, not because the Tcl layer necessarily cares, but it is easier to return error information to the client.

Besides simple typing and derivation, more important is structural typing. Structural typing is very important. Every langauge is slightly different and the point of WSDL using XML-Schema to describe how to construct types is that it is theoretically easy to build software which could validate a type (simple or complex). Does XML-Schema suck: big time. But one of the biggest sucking parts is that it tries too hard to accommodate optimized validation routines and fuzzy types.

Anyway, I'm not soo sure what is contradictory: WSDL defines an interface, at some point there has to be code which translates from an XML/SOAP document to Tcl. This is required.

The question is rather what is the realm of validation:

The question is what helps the client, targeted error messages which relate to the data input, or a stacktrace from the server? Of course, validation also protects the server from attack. I'm not too impressed with MS releasing code which is all too happy to stick any data into any variable. My general rule is to validate as soon as possible, why wait? Oh, and it is a requirement with SOAP/WS-I that client errors should lead to no change in state on the server. The best way to ensure that is by validation before execution.

This is a matter of framework design and, again, the employment of design patterns: Provide signature adapters with the adapters being exposed as service.

Yes again! <ws>proc is an adapter! Actually it is a description of an adapter. The adapter is written by tWSDL based upon the input to <ws>proc. Very complex adapters have to be written by hand, although I haven't thought of a realistic example which would require this.

This is a basic requirement and the solution in most frameworks is referred to as "deployment descriptors". They come in various flavours, in AXIS they are XML-based descriptors, in ASP.NET they come as annotation tags, ...

Yes, this is how they do it. All the examples I have found are for very simple services. I know they can do more, obviously, but how easy it is? More importantly, how do you learn how to do it, where are the complex examples? I'm interested in example code. Another thing that is still missing is using annotations _and_ using complexTypes or derived types and not having to edit a bunch of files.

I would like to promote something, two things: efficiency and simplicity. If you can boil out the information content which distinguishes one remoting service from another, you can write code which uses this tiny amount of information to write the actual code. One example of what information is required for a service is a WSDL file. It isn't the only example, but it is a good one. It doesn't contain all the information needed since the adapter information is missing. A more complex solution will require more information, more software to handle the complexity, etc.

Anyway, I'm not interested in your remoting whatever. TWiST does what it does and doesn't need to be integrated with some ambiguous uber-code. Why not just release the code you have right now?

Basic ASP.NET description:
http://www.stardeveloper.com/articles/display.html?article=2001071401&page=5

Adding validation (note what isn't shown, how to validate):
http://msdn.microsoft.com/msdnmag/issues/03/07/XMLSchemaValidation/default.aspx?loc=&fig=true#fig10

AXIS:
http://ws.apache.org/axis/java/user-guide.html

IBM on JAX-RPC/JAX-WS (configuration vs annotation):
http://www.ibm.com/developerworks/library/j-cwt08025.html

ASP.NET with WS-I (Is there more info somewhere?):
http://msdn2.microsoft.com/en-us/library/ms230196%28VS.90%29.aspx

IBM: compare JAX-RPC to JAX-WS (hello world example):
http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc3/index.html?S_TACT=105AGX04&S_CMP=HP