Forum OpenACS Development: Re: Remoting (Formerly: The future of OpenACS)

Collapse
Posted by Tom Jackson on
Stefan,

Why don't we compare apples to apples. I am talking about real software. TWiST, .NET, AXIS, as they exist today. The purpose of this software is to solve a problem, not philosophy. The tiny problem I am interested in solving is exposing useful operations over a network and making it as easy as possible for remote users or programs to figure out how to actually invoke the operations.

At any rate, TWiST exists and works. Does the software you are speaking of exist? Could you give examples? The comments I have made about ASP.NET and AXIS are well documented. The reason I know is that after I wrote TWiST, I started hunting around for how these other solutions did the same thing as TWiST. Well, very poorly and not out of the box. However, I could be wrong, and I would really and truly be interested in parallel examples, of even the simplist web service. But the requirements are that the developer:

1. Doesn't have to write their own WSDL file or move it into place by hand.
2. Can abstractly describe simpleTypes not based upon a given programming language.
3. Can describe how to restrict simpleTypes from built in abstract types.
4. Can describe how to combine simpleTypes into complexTypes
5. Doesn't have to write validation code or type creation code as a separate step from description.
6. Can name their input and output parameters as they wish without rewriting their internal code.
7. Don't have to edit source code files to make configuration changes to the service.

Okay, that is enough, Actually if you find examples of how to do ANY of these with either .NET or AXIS, please pass them along. I am stuck on writing a howto to find such examples.

There is one fact which always stands out for me on this topic, I'll restate it and hope that someone can set me straight, because if it is not a true fact which always must be kept in mind, I need to rethink my whole way of writing code. Here is the fact: data cannot be created out of thin air. Maybe this is one of those laws of thermodynamics.

The situation is this: some code exists in memory or in a file. This code cannot 'remote' itself without some help. If this fact is not fully appreciated, those who fail to appreciate it will wait in vain for this 'remoting' to happen all by itself. If things were otherwise, we would have programming languages which consisted of one word, because everything else has already been figured out.

Skip the one word. There is proposed a great remoting program. You install it and suddenly it does exactly what you want without any input. It speaks whatever language or protocol you can think of. Not only does it speak, but anyone or any program which contacts it understands everything. Communication is perfect.

Until I find this program, or it finds me, I'll try to focus on figuring out what information is needed to move from an internal code to a useful service. What language will I use to do this? WSDL. Will I follow any advice from others, or just consider my creative genius enough? No, I'll follow the WS-I Basic Profile.

I'll use a WSDL file to hold information, data, created by a person in order to communicate what I wish to be a service. By following a certain format, the WSDL format, I reduce the amount of information I need to create. The form of the WSDL file is fixed by certain standards. The form of the messages is fixed, the description of the types and operations follows a fixed pattern. All I have to do is provide a few pieces of real data. Maybe this is called work, maybe thinking, but it doesn't come out of thin air. This is what folks get paid for. If we could remote a procedure by adding a switch, then remoting would be a one bit operation. One bit of data. We can argue all the way around this for years, but the information has to come from somewhere. All TWiST does is provide a format for collecting this information in one place so that tWSDL can do all the boring code writing which doesn't contain any more content than the TWiST file. This is what has to be seen: a few lines of a configuration file is expanded into a working web service. Developers don't need to spend hours or days writing code which essentially has no more data content than the description of the service.

Collapse
Posted by Stefan Sobernig on
Sorry for not continuing the discussion any earlier, I was out of reach of any internet access for the last two days.
I am talking about real software [...] At any rate, TWiST exists and works. Does the software you are speaking of exist? Could you give examples?
I'd wish you wouldn't have played this card, as most of the issues listed and mentioned can easily be discussed without referring to YOUR and MY piece of work. If I remember correctly, the idea of the thread (as it was started by Malte) was to collect ideas and requirements for remoting solutions for OpenACS, and not whether YOURS or MINE are better suited. This is a valid level of discussion, but a different. Anyway, yes there is such a MY, and it is already in use by a smaller community. I did not release it to a wider public simply because documentation was so awfully lacking that i did not dare to do so. This has changed, I will be happy to put you on my list of addressees of the upcoming announcement. In the meantime, you might refer to the slide set of a talk given at the last OACS conference. At least, this should convince you that is not pure philosophy, although philosophy is an important ingredient. As for your requirements list below, I need to state that initially, you just referred to "out-of-the-box validation not being supported" by existing framework, what you are looking for by far more than that. Anyway, let me try to restate your quest, just to make sure i get it right: I have a tcl code block (a proc) and want to expose it as a SOAP-enabled service the easiest way possible. the infrastructure should take care of gathering service-declaring information from the proc declaration. how to achieve this in a tcl environment without annotating/extra-declaration efforts by the developer.
1. Doesn't have to write their own WSDL file or move it into place by hand.
Ok, auto-generation of wsdl files out of existing code. there are quite a few examples available. However, the resources you posted below indicate them already. So, I am wondering why you are still desperately looking for more sources. The main difference between AXIS an ASP.NET (in all implementations, MS and mono's) is that AXIS uses static stub creation (incl. java > wsdl) while ASP.NET does this dynamically with some annotation sugar (it is reflection-heaviy in implementation).
  • AXIS: there is a wsdl compiler available, called Java2WSDL. this achieves what you seem to be looking for, however, under heavy constraints of (a) compiler languages as such and (b) java's limitations. there need to be an Interface construct, java primitive types are mapped to xsd built-in types, java types are mapped to xs complex types. Achieving this on the fly (as you intend) is hard in compiler languages, as reflection is a costly thing in these environments.
  • ASP.NET uses annotators and internally reflection to register web methods etc. with the dispatcher framework. again, .NET primitive types become xsd simple types, typed arguments or return values become complex xs types, etc. If you are interested in the details, a look in the source code is necessary which might be hard. But you might refer to mono's code base which is open to the public domain and comes with an asp.net implementation. there are also some archived discussions on mono's devel list.
I don't quite understand why you are so interested in examples from compiler language environments. Scripting languages are much more powerful in this respect!
2. Can abstractly describe simpleTypes not based upon a given programming language.
This I don't quite understand, provided that I understood your guiding scenario. You want to have a language-bound code block turned into a service + service description, but you want simple types (xs based types) not defined in the very language the code block was written? this is conflicting and contradictory to your scenario. If I want to deploy my tcl proc as service, I don't want to specify a constraint derived type "myInteger" based on the base type xsd:interger used when the service description is built etc. Was it you saying "interoperability is result of the restriction of options"? This is not separation of concerns as I understand it, but I might be wrong in my reading of your scenario. Btw., describing data types in abstract manner usually is referred to as abstract data types (ADTs) but I think you are purely referring to what in type creation in XML Schema world here.
3. Can describe how to restrict simpleTypes from built in abstract types.
The same goes here. A Tcl developer does not want to bother with these issues which are XML Schema specific.
4. Can describe how to combine simpleTypes into complexTypes
This is usually done (in the OO world) by mapping object types (class schemes and their primitive-typed attributes) to complex types. In procedural world, this requires some add-on abstraction.
5. Doesn't have to write validation code or type creation code as a separate step from description.
The question is rather what is the realm of validation: The streamed data represention (XML element against element description) or the language-level representation (Tcl string that is validated to be conforming to a notational form, character set and value space). In the streamed scope, use schema validation. In the language level, you need to provide your own validator. C'est la vie, but as you are centred on WSDL/XML Schema, you need to think of option (1).
6. Can name their input and output parameters as they wish without rewriting their internal code.
Again, separation of concern is not clear to me. I read the above requirement as follows: My exposed proc has a different signature (internal code, as you call it) as I want it to be exposed. This is a matter of framework design and, again, the employment of design patterns: Provide signature adapters with the adapters being exposed as service.
7. Don't have to edit source code files to make configuration changes to the service.
This is a basic requirement and the solution in most frameworks is referred to as "deployment descriptors". They come in various flavours, in AXIS they are XML-based descriptors, in ASP.NET they come as annotation tags, ...

I will take up you questions/ requirements and include them (however, somehow clarified in my reading) in my documentation so the interested reader can then figure out the design decisions for MY remoting fetish.

Collapse
Posted by Tom Jackson on
I'd wish you wouldn't have played this card, as most of the issues listed and mentioned can easily be discussed without referring to YOUR and MY piece of work. If I remember correctly, the idea of the thread (as it was started by Malte) was to collect ideas and requirements for remoting solutions for OpenACS, and not whether YOURS or MINE are better suited.

Ummm, you started this thread.

I have never looked at your software, don't know where to get it or how it works. I use my software as an example for several reasons, but I'm not promoting it. Anyway, the main reason for discussing software which actually exists is because it is obvious how things are done, while talking about stuff which could work or might be better is a waste of time. Until something is boiled down to code, everything sounds possible, that is the nature of programming languages: anything can be done, in any language.

Another reason to talk about existing code is that code has behavior. If you want to discuss a future requirement, it is good to know how other people have actually solved the problem, what is involved. That is why I was looking for examples from other major players in 'remoting'. I talked about, and provided links to the ones I know about. Personally I'm not interested in re-inventing the wheel. When someone tries to code to a standard, they are giving up that part of the process. I have said I tried to follow the recommendations in WS-I Basic Profile. If you don't wish to follow a standard, it becomes very difficult to compare it to anything else. It does make it easier for you to claim that your software works and does what you want.

I don't quite understand why you are so interested in examples from compiler language environments. Scripting languages are much more powerful in this respect!

I listed the examples I could find, I asked for better examples. I'm still waiting.

This I don't quite understand, provided that I understood your guiding scenario. You want to have a language-bound code block turned into a service + service description, but you want simple types (xs based types) not defined in the very language the code block was written? this is conflicting and contradictory to your scenario.

Ahhh! Yes! This is exactly the advantage of Tcl over most other languages in this area. First consider the following: Tcl is best as a glue language. In OpenACS it is a glue language. OpenACS doesn't store information as a Tcl structure, it takes in some string and passes on a string to another application. Usually stuff goes into a database, which does have type requirements. Most procedures don't care what they take in, so you claim that the typing of this input or output doesn't make sense. Actually what doesn't make sense is caring about constructing types in Tcl just to pass them on to another application. But if you can test the type information, you can pass back information to the client in a useful form. Once the procedure executes, you pass from client error to server error, which is much less useful. This is why we have filters in ad_page_contract, not because the Tcl layer necessarily cares, but it is easier to return error information to the client.

Besides simple typing and derivation, more important is structural typing. Structural typing is very important. Every langauge is slightly different and the point of WSDL using XML-Schema to describe how to construct types is that it is theoretically easy to build software which could validate a type (simple or complex). Does XML-Schema suck: big time. But one of the biggest sucking parts is that it tries too hard to accommodate optimized validation routines and fuzzy types.

Anyway, I'm not soo sure what is contradictory: WSDL defines an interface, at some point there has to be code which translates from an XML/SOAP document to Tcl. This is required.

The question is rather what is the realm of validation:

The question is what helps the client, targeted error messages which relate to the data input, or a stacktrace from the server? Of course, validation also protects the server from attack. I'm not too impressed with MS releasing code which is all too happy to stick any data into any variable. My general rule is to validate as soon as possible, why wait? Oh, and it is a requirement with SOAP/WS-I that client errors should lead to no change in state on the server. The best way to ensure that is by validation before execution.

This is a matter of framework design and, again, the employment of design patterns: Provide signature adapters with the adapters being exposed as service.

Yes again! <ws>proc is an adapter! Actually it is a description of an adapter. The adapter is written by tWSDL based upon the input to <ws>proc. Very complex adapters have to be written by hand, although I haven't thought of a realistic example which would require this.

This is a basic requirement and the solution in most frameworks is referred to as "deployment descriptors". They come in various flavours, in AXIS they are XML-based descriptors, in ASP.NET they come as annotation tags, ...

Yes, this is how they do it. All the examples I have found are for very simple services. I know they can do more, obviously, but how easy it is? More importantly, how do you learn how to do it, where are the complex examples? I'm interested in example code. Another thing that is still missing is using annotations _and_ using complexTypes or derived types and not having to edit a bunch of files.

I would like to promote something, two things: efficiency and simplicity. If you can boil out the information content which distinguishes one remoting service from another, you can write code which uses this tiny amount of information to write the actual code. One example of what information is required for a service is a WSDL file. It isn't the only example, but it is a good one. It doesn't contain all the information needed since the adapter information is missing. A more complex solution will require more information, more software to handle the complexity, etc.

Collapse
Posted by Stefan Sobernig on
Ummm, you started this thread.
Nope, if I remember correctly, it was Malte starting the thread on remoting, by collecting some rough scenarios. I continued by summarizing what had been discussed during the OpenACS Spring Conference, still focussing on a general road mapping. It was you going into design + implementation details (of TWiST/tWSDL) which I somehow found inappropriate in a Future-of-OpenACS thread and therefore forked the original. However, my idea was still a general road mapping. C'est tout.
I listed the examples I could find, I asked for better examples. I'm still waiting.
Mmmh, if you are looking for "better examples" to the AXIS/.NET/... examples posted by you, simple googling won't be enough. I had good experiences with browsing the code repositories (especially the test suites contained in there) and getting my hands dirty with re-implementing and extending test cases for my purposes. If this is no option for you, mailing to the developer lists may help (that's why I indicated an example thread from the mono-devel list previously). At least, either of these options is better than posting a request to this forum.
What can be done? Have the most experienced people with regards to webservices (Stefan and Tom) along with interested OCT members come up with a generic strategy how to support webservices in OpenACS.
Taking up Malte's original statement, I would like to call a spade a spade. Tom, would you be willing to contribute (a) to a phase of general + detailed requirements gathering/ road mapping and, subsequently, (b) realising implementation efforts, even though they go beyond the scope of TWiST/tWSDL and might even replace these two as self-contained solutions?
Collapse
Posted by Tom Jackson on
I really wish I could figure out what I did that was inappropriate. Discussing code? This isn't a tea party.

Anyway, I'm not interested in your remoting whatever. TWiST does what it does and doesn't need to be integrated with some ambiguous uber-code. Why not just release the code you have right now?

Collapse
Posted by Stefan Sobernig on
Tom,
Anyway, I'm not interested in your remoting whatever.

I got this strong impression, indeed! However, you constantly become set on MY, YOURS, THEIRS etc. I was never referring to MY fetish necessarily being THE solution for OpenACS, I just wanted to know whether there is esprit for co-ordinated action.

TWiST does what it does and doesn't need to be integrated with some ambiguous uber-code.

At least in this respect, I got an honest answer. Thanks for making this clear to me.

Why not just release the code you have right now?
Why? The only thing that prevents me from doing so is the fact that I am currently on holidays and that I am not fully available to support (early) adopters. I will be back to office around the end of next week, so expect the release around that weekend.

I, therefore, close this enlightening tea party from my side ...