Forum OpenACS Development: The future of OpenACS
How did this report come about? Well, some of my OACS sites were actually ACS sites and have been upgraded since 1999, before OACS was even a reality. I therefore have not just history, but a vested interest in OACS.
Users have always complained, but generally it was ignored. The systems worked, so don't rock the boat. Now, though, things have changed. If your system looks like crap, is still stuck in the year 2000 for technology, or is just plain hard to use, end users will not use your system. Most likely, they will talk to their friends and request the system be changed to use more friendly alternatives like using mash-ups.
As much as I hate PHP and MySQL, they have the killer apps. Wordpress is one of them, so are the several forum apps. Lay people see these apps and want them. They are not interested in the geek speak about the benefits of embedded TCL and database support. They want a solution now, sadly OACS doesn't provide one.
Even in its role as a toolkit, it fails in many regards. To quote Jim Davidson (one of the architects of AOLserver) in a recent forum posting to the AOLserver list (slightly edited):
Anyway, I've spent some time with LAMP recently -- quite clever all that PHP/MySql stuff.
To compare, AOLserver:
-- Still has a scalable architecture and good underlying code quality
-- Is tightly woven with Tcl which appears less and less popular each year (I could be wrong)
-- Is lacking many extensions or has cruddy alternatives (e.g., ns_http instead of curl)
-- Has horrible, incomplete, and often misleading documentation
-- Isn't so easy to install and configure
while LAMP:
-- Has great documentation with threaded discussions
-- Relies on PHP which, fair or unfair, appears quite popular
-- Has possibly too many overlapping extensions
-- Is hard to install and configure unless you're using that xampp stuff
The infrastructure is old and showing its weaknesses. Whatever technical head start AOLserver had is not very important anymore. People are not looking for community sites anymore, but rather ways to facilitate social networking.
Many of the offline conversations I have with other OACS users and ex-users revolve around the fact that OACS is now a third class citizen, with almost all development going to dotLRN. To make matters worse the toolkit is so far behind technologically many have started using other alternatives.
I also understand many of the people actively writing code work for companies who work almost exclusively with dotLRN. That is fine, but when dotLRN dependent code gets into OACS, it becomes a community problem.
<h2>What can be done?</h2>
- I already have written on xowiki (see http://jongriffin.com/articles/oacs-needs).
- More of the core needs to be re-factored using xoTcl.
- A logical theming/templating system needs to be created. Most LAMP applications use smarty templates. OACS has a better template system that could maybe be created as a separate from OACS stand alone module.
- The css is a mess, too many independent files that if you change, get written over with upgrades. LAMP uses a /templates directory, OACS should follow this. No packages should have css or resources in them.
- Templates should be editable from the browser, this includes css. There should be no need for shell access for a designer to change something.
- Split out any modules that don’t require database access into standard AOLserver modules.This will allow other developers to use the codebase of OACS without the overhead of the entire toolkit (i.e. I only want a blog)
- Think about moving to naviserver. This project adds many extensions that could be useful for OACS.
The above is just some of what needs to be done. Am I volunteering, well no. I don’t really have time.
I am mentioning this as someone who likes and uses OACS, so don't take it the wrong way. Most people just try it and dump it and never say anything. Also, ex-users just see the whole aolserver/tcl problem not worth fighting.
LAMP exists, what would changes to OACS add to LAMP? If merely a me-too bullet point, it seems pointless, LAMP exists! Use it if it satisfies your customer's needs.
Refactor to xoTcl? For one I can say that I can never figure out what is going on in an xoTcl script. If the original code works, why would it need refactoring to xoTcl, why not refactor to Java or LAMP?
One thing OACS requires is a database. It is impossible to refactor OACS into modules which would work without a database. Also, remember that a lot of code is defined with ad_proc, etc., so moving code out of OACS is nearly impossible.
It is a short job to allow editing of templates via a web browser. The problem is more with supporting this as a core function, and then writing packages which use the feature.
One interesting point is that LAMP in most generic form simply means using the best tools at each layer, and keep code in the most applicable layer. Our layers are Tcl, AOLserver, AOLserver Modules, OACS-Core, OACS Packages.
So LAMP not so much a layered system as a code grouping and coordination system. The OACS system can push code groupings into different layers using general or specific API. This is more flexible that LAMP, but usually what you get with LAMP is an application which has to handle the whole process as a single bag of code. The OACS system allows for lots of reuse.
The fact that more specific tools are not used in OACS is really a matter of developer taste, which I have no interest in changing. I can say that I write packages which handle specific tasks so I can get lots of reuse and lots of separation of function. I've also written a package which allows lots of configuration/customization by design. Each content type can have a separate master, type, page and item template (or reuse an existing or default template). Each item can also have a different template. Templates are also content types, so they can have parameters and customizations: added and updated in the same way as other content.
All templates are editable via a web interface. My 14 year old edited her blog template by herself. The same installation has other applications, like an inventory/product list with ecommerce and photo galleries for each product. So it can be done, without any changes to OACS or interference from dotLRN. Just add data, configuration and templates, no Tcl coding, no database coding.
As I said above, this isn't about bashing OACS so much as sharing my and others observations.
I really don't expect anything to change other than the number of people using OACS and AOLserver to continue to drop. It has always been a niche player so the reality is it doesn't really matter how many people are using it.
As much as I love TCL, the reality is that most products in the commercial world are developed in PHP. I don't want to learn PHP but I may be forced to.
As bad as PHP is, having to use MySQL is what I dread the most. The few sites I have that use it (qmailtoaster and squirrelmail) make me ill. Postgres is 1000% better than MySQL will ever be just due to the fact that it was designed correctly. Now if they could only make the object relational part work with foreign keys and etc.
If you really know Tcl and OACS, you must know that writing an app will be faster with these tools than with PHP, but maybe not fast for a marketable product. These things don't write themselves. Remodeling your kitchen doesn't improve your cooking skills.
Am I volunteering, well no. I don’t really have time.That's the only part of your post that will stick in my long-term memory, sorry.
Well, a couple of other things ...
I very much dislike the notion that all templates should go in a global directory and that css and resources should all be global.
Follow that model to its logical conclusion and EVERYTHING should follow a flat model, no modularization at all. It's so 1945.
Reason: When we upgraded in the past we *always* blew it up because we forgot something that changed in a template or in a CSS file. This is not happening anymore and we just do a comparison if we need to update the our custom CSS / template with a version from OpenACS. With the latest upgrade and overhaul we decided to exchange the templates (custom) with the ones from OpenACS and modify the OpenACS ones to include the visible changes (visible for the user that is).
=> Yes, we keep everything in one place but no, we do not do this in OpenACS itself which is fine.
If there is interest, someone can probably quickly write up a "create customer package" script, which will create a package and copy all templates and CSS files into the package. Additionally it would change the parameters to use the new location for the templates. This might go as far as change the location for the /lib/ code from acs-subsite (e.g. login, user-new, user-info, email-confirm).
Furthermore I think we could add a function there that let's us edit the templates and CSS files from the browser. And if we do this, we could make each file an object in the content repository and save the published version on the harddisk.
I am willing to help anyone write this code. I am also willing to provide code snippets and explanations how to do this in XoTCL, if someone wants to learn this. Sadly I wont have the time to write it all by myself (and, more to the poinnt, involve myself in testing it out).
There is no need to refactor everything in XoTCL. I start doing this if I need to work considerably on content items by using Generic::CrItem functions from xotcl-core, making my life easier, but there is no need in my opinion to do this for existing code. As Don mentioned, it also makes working on core much harder for developers that have no knowledge of XoTCL. And finding the places where suddenly variables pop up from or where a function is defined is harder when you have inheritance.
Gustaf, what would be helpful here would be an API Browser for classes which lets you search for something like "xowiki::FormPage new" and display you the new function wherever it is defined with the link to the file it is defined on (in this case I think it is ::Generic::CrItem new, but I might be wrong). Or, if this exists, and explanation for an XoTCL dummy like myself where to find it 😊.
I will write up a customization document this week on how we do customizations, based also on the feedback from the community in the customizations thread. If this is going to become a best practice is something for you to decide then.
Also, one should note that most of your needs come from the use of OpenACS as a CMS, which is where XoWIKI is getting at . Just to put them into context.
So keep on using OpenACS, keep on bringing the feedback and don't feel bad about selective memories of other people. Just don't hope we can do much about it unless you find someone committed.
Tom, is your package available somewhere to look at and would it be a worthwhile addition to OpenACS as a whole. It seems to me that it is highly useful for custom build content types, yet I do not understand how this would work on the existing types like users. Yet I see a huge benefit if this work could make it into a generic dynamic attribute solution, as it would fill the missing gap. Do you store the templates in the database or do you write them to disk as well?
On Gustaf's behalf, xotcl-core, since its very beginning, comes with a two flavours of "api browsers", a native xotcl or object-centric one, and a seamless OACS api browser integration. After having installed xotcl-core, point your browser to /xotcl/ and enjoy! Alternatively, look for objects and/or methods in the good old api browser, and you won't be disappointed.
The above works nicely when a function is documented with ad_instproc, sadly it miserably fails for anything else.
Furthermore, when I type in e.g. "get_object_type", I see the generic class procedure (which is documented, so things are fine). Sadly, unless I know that a FormPage is a CrItem, I would not understand what xowiki::FormPage get_object_type does. So, I would love to see, if I type in "xowiki::FormPage get_object_type" that the API Browser finds me the Generic::CrItem get_object_type procedure and displays this (probably along with a hint that this is the version being used by xowiki::FormPage).
Does this make my problems a little bit clearer with the currently provided documentation? I do have the slight feeling, regardless what can be done with the API Browser and the XoTCL browser we still need make a lot more procedures using ad_instproc and Classes using the -ad_doc parameter for us to really "digg" XoTCL in OpenACS.
And I agree with Gustaf that on the XoTCL Browser having "Show all methods including inherited ones" will probably lead to pages of procedures / methods being displayed, but at least it helps us reading the code.
Not having developed in Java or other OO languages, how do the IDEs for them solve the problem of inherited Methods and the documentation for them. Any Pointers?
Tom, is your package available somewhere to look at and would it be a worthwhile addition to OpenACS as a whole. It seems to me that it is highly useful for custom build content types, yet I do not understand how this would work on the existing types like users. Yet I see a huge benefit if this work could make it into a generic dynamic attribute solution, as it would fill the missing gap. Do you store the templates in the database or do you write them to disk as well?
You can browse the package at http://rmadilo.com/files/cams/
This is not the same thing as building custom types, but look at the data model. Each content type begins with a title and content. You can add attributes to the type. When you add content, you first enter the title/content and then you can add the attributes. For instance, your content type could have an image attribute, or call it whatever you want MyImage, whatever. When you add an attribute, you define the content type and the form widget. If it is a file widget, you can upload a file, or more than one file for the attribute. The package name CAMS stands for Content Application Management System. The idea was to allow developers to create certain types of applications focused on content: how to get it into the system and how to display it.
Templates are used for the master, content type and content item. In addition, the content of each item can be treated as a template. Special tags can be included in the template to format/insert the attributes associated with a content item. So if you upload an image, you can use the same data to insert a link to the image, or display it inline, anywhere in the content.
New templates are stored in the database, they are content items. But they are also 'compiled' and stored as files for use. The 'compiling' just applies any attributes, so that the same base template could be reused.
Some applications are here: http://whogeenie.com/cams/
Furthermore I think we could add a function there that let's us edit the templates and CSS files from the browser.Note that this is a totally different issue than "where package-specific resource files should go", and I didn't disagree with it.
So if it's good to put all templates in /templates, and all resources in /www/resources, then clearly all tcl scripts should to into /tcl and all library scripts should to into /lib and all package info files into /info, right?
If you don't please so, please create a coherent argument as to why customized (or untouched) tcl scripts should be treated differently than customized (or untouched) resource files.
Tom:
If the original code works, why would it need refactoring to xoTcl, why not refactor to Java or LAMP?
Why not refactoring in Java or LAMP? Wasn't there a project trying to refactor ACS in Java, which failed badly? XoTcl is a Tcl extension which integrates therefore nicely with pre-existing code, which is not the case for LAMP alternatives.
There are a couple of reasons for refactoring "working" code in xotcl
- Make applications reusable through subclassing
- Improve flexibility and maintainability
- Improve orthogonality (why is argument passing and checking in ad-procs, page contracts, widgets and for adp includes different?)
- Reduce the need for the notorious upvar/uplevel necessity by using objects.
- Last but not least: improve speed (invocation overhead of an XoTcl method is significantly less than the invocation overhead of an ad_proc) and reduce memory consumption (for every ad-proc two tcl procs are added to the blueprint; that is not the case for xotcl based ad-procs and ad-instprocs)
Furthermore, through higher reusability, xotcl code is typically much shorter than tcl-only code. For example xotcl-core and xowiki are together significantly smaller than e.g. assessment, just by counting the number of lines:
assessment: 49.000 lines
xotcl-core: 6.900 lines
xowiki: 29.000 lines (4 of the 5 largest files are javascript libraries used by xowiki)
I doubt very much that a change in the OACS core or changing to xoTcl would somehow help write the eventual killer app
The only openacs application currently available with some killer-app potential is xowiki, which can be easily configured as a collaborative wiki, a blog, an announcement system, an faq module, a collaborative networking tool, a slide presenter, a podcaster, etc.
Wasn't there a project trying to refactor ACS in Java, which failed badly?No, not really. It was overdesigned and overly complex attempt at a redesign of ACS 4 (which has some overdesigned and overly complex elements already :)
For instance, you didn't write HTML-based template files ala ACS 4. You wrote XML which got fed through XSLT to create HTML which then got streamed to the user, and whatever the abstract attractiveness of this approach might be, the particular implementation was obtruse and difficult to use by experienced programmers. Designers? Forget it.
That's just one example.
A straightforward rewrite might've been interested, in the sense at least of giving some empirical evidence that a Java-based framework's always going to be a PITA to work with.
One might say it had a similar relationship to ACS 4 as ACS 4 did to ACS 3, with a language change thrown in for fun.
My comment about xotcl simply reflects a belief that AOLServer+tcl+xotcl will be no more popular than AOLserver+tcl.
Well, there's still a pro-Mysql bias in the web world, too. Mysql's gotten better, at least, though the idea of giving up PG would make me sick :)
Why not refactoring in Java or LAMP?
If you delete the first part of the sentence, you can make any conclusion you want. The first part was "If the original code works, why would it need refactoring to xoTcl". Working code doesn't need to be refactored to make that working code nicer to look at. Refactoring must be directed at reuse by higher level APIs. Obviously there could be internal refactoring which has no external impact: who cares about this?
Unfortunately I sense that refactoring into xoTcl will have a vast impact on other code. xoTcl is not an extension of Tcl. It is a new language. If it were an extension, then the authors would not complain about uplevel and upvar. These are not evil commands. This single complaint informs us that the authors see a deficit in Tcl. Believe me, it doesn't exist. The only deficit is in the understanding of how to use Tcl. Why is it that ignorance makes people think they are smart? This whole discussion could be reformulated as: is Tcl Java? Thank whoever, the answer is no. Am I, again, overreacting? Then explain how Tk extended Tcl without complaint.
Anyway, there is a huge difference in saying xoTcl is not good and saying that it doesn't offer much benefit, if any, to an existing code base written in Tcl. I perceive them as distinct programming models. They are presented as distinct, so there doesn't seem to be much argument over this fact. Why is this new language being introduced, and why is it being accepted? Tcl does not have the deficiencies which concern a few here. Tcl is easily adapted to any object model, I would argue that Tcl is very object oriented, maybe exclusively object oriented. Tcl acts on objects. Even commands are objects, so the Tcl model is objects acting upon other objects. Maybe the difference with Tcl is that the same object can be both the actor and that which is acted upon. Some people are offended by this. Screw 'em.
That said, I just want to address the multiple CSS issue. Its not an issue at all.
Having specific CSS in a specific place makes sense. Although it does make the page load slowly.
In this case, what we really need is a tool that understands CSS. Lee started work on that (template::head) which makes OpenACS aware of the CSS/JS etc. Once OpenACS is aware of this, it can dynamcially create and cache consolidated CSS/JS files that load quickly, but still allow for modularity.
Gustaf, XoTcl code might be faster but so far its incredibly difficult for most OpenACS developers to read. Processors are cheap and programmers are expensive. As I said before, having consisten code is a goal, but so far there hasn't been any effort to produce any sort of documentation on how to USE XoTcl wisely within the OpenACS framework. Unfortunately, until that happens you'll end up doing all the work yourself.
I would concur with this. I've been using XOWiki for the last couple of weeks as it looks like a perfect fit for an upcoming project but I find it hard work to figure out whats going on. I'm not new to OO (don't get me started on J2EE!) and I've read the tutorial on XOTcl http://media.wu-wien.ac.at/doc/tutorial.html and worked through the code but I'm still struggling to figure out how some of it works.
I appreciate the time Gustaf has spent on this and I can see the advantages that he talks about. I think it could be very productive but its not going to be taken up for other packages and projects simply because we can't work out how to do it.
Perhaps what we need is an "XOTcl in OpenACS" tutorial along the lines of the XOTcl Tutorial.
- Certanly, How OACS manages CSS is a problem. I'm not sure what it's the best solution, but our approach now, at least, is modular and, for recurrent users (because of the cache) works quite well. But yes, first time, I had to load a lot of files.
- It's impossible to install OACS in 20 hours (complexity and well trained people), so my students were always happy with LAMP, but ..., it's not fare compare a typical LAMP app with OACS. And for me, Aolserver and Postgres is easy. TCL, OR AOLServer API it's not cool (neither Emacs), but works very well (It solve my problems quickly)
But, what is the future? I miss some things, some bugs that remain a lot of time, and a lot of distributed effort. But I think it's a common problem of a complex and mature project in open source world (like OACS).
But. sure. we have to learn of other approaches
Rewriting in XoTCL does not make any sense to me either. Why change a running system to make it run slightly better yet radically different with even less documentation for the developer to go around than there is for OpenACS per se.
This said, I love XoTCL and what it did for my application development. And more of it will popup in OpenACS CVS in the medium time for new functions. But I would not go and rewrite e.g. Forums in XoTCL unless there is a compelling reason to do so (read: customer requirement).
What would in my opinion be killer apps for OpenACS are things which haven't been discussed here:
WEBSERVICES
TWiST works great and I am sure Stefans code works beautifully as well. Yet we need to bring them into OpenACS properly and devise methods where an administrator can quickly say: "I would like this information available as a webservice". Ideas on that:
-- Make a form a webservice. If we use ad_form, provide a switch that, if querying a URL with a certain parameter gives you a webservice for the form.
-- Enhance listbuilder to return WSDL formatted XML in addition to CSV
I am aware that I will get lot's of bashing about this for not understanding the concepts of Webservices, but what it boils down to is:
"I need the information on this table available in my financial data system"
"We need to interface our user creation with yours to keep it in sync"
"We have a Yahoo Widget which wants to add a new task"
If OpenACS as a toolkit provides interfaces OUT OF THE BOX, without the developer having to do anything about it, then we talk killer app. Oh, and yes, we would need to have security on the webservices and lots and lots of other things to take into account, above is just a suggestion.
What can be done? Have the most experienced people with regards to webservices (Stefan and Tom) along with interested OCT members come up with a generic strategy how to support webservices in OpenACS. Get "customers" like Quest and ]po[ and E4 involved to state what they need and see if the model suits them. Then collaborate together to come up with the funding for a generic solution to be put into OpenACS. But as there a customers with needs I have the slight suspicion there will be money as well (or at least development help).
EXTEND FIELDS
I think the reason that we have 5 (or more) ways of dealing with dynamic extension of data fields shows me that there is huge demand, great intellect and capacity for building it and ZERO coordination to make this a killer feature for OpenACS. But this is what customers want.
They want to be able in their application to extend the fields a user has to fill out when signing up, WITHOUT hiring a developer to make that happen.
They want to store their custom datafields for projects and show them in a project overview.
Why is it at the moment impossible to do this with OpenACS?
If the most common needs for a user of OpenACS can only be achieved by hiring a developer who is knowledgeable in OpenACS, chances for him using the toolkit are slim, because he can use a LAMP application and hire a smart PHP kid from Ukraine for $20 per hour to make the changes for him.
What can be done about it? Collaborate on https://openacs.org/xowiki/dyntypefields by writing up what we want to have and pointing to bits and pieces of code that already provides this. Then ask who can work on achieving this and then have those who can work on it vote on which existing solution to base the new one (or to write a different one from scratch).
DOCUMENTATION
Our documentation sucks and the one for the packages is even worse (XoWIKI becoming an exception to the norm). But without documentation on how to achieve certain things in OpenACS we will not be able to attract new users. And if our goal "only" is to aquire new developers who love the toolkit, than at least the developer documentation needs to be up to date. We (Anett and I) have worked over the last months to clean up https://openacs.org/test-doc and enhance it bit by bit with new information (mainly get an update for TCL for Dummies there), yet this showed us how MUCH we are out of date and how difficult this job to update the information is going to be. And to make matters worse, it probably needs the best people we have (in terms of development experience) to write this documentation to attract new developers, which is a catch 22 as they are usually in high demand (due to the fact that the documentation sucks so badly that noone else can quickly learn this).
What can be done about it? Among the companies in need for new developers come up with a "fund" for the outcome to have a developer tutorial along with updated Problem Sets and training / teaching instructions. They need this anyway if they want to grow, so they might just as well shoulder the burden and not do the same effort each.
Then try to get some EU or other funding to partly cover the writing of documentation for packages used in the funded project and have them written. There are excellent writers for documentation out there and I am fairly sure you can point them to our application and as long as you are open for questions, they can write up (and screencast) what they have been doing on the application, without having used it for a couple of years. And yes, this obviously depends on the level of complexity of the application.
ATTRACT NEW DEVELOPERS
We have a lot of universities using .LRN. They have in house staff that are knowledgeable with the solution. They have students who might need to learn about developing these kind of solutions, so why not teach them? As Gustaf mentioned, a lot of volunteer effort's benefits come from the acknowledgment of your work. So why not invite OCT members to be guest lecturing a course at your institution? This would already get into the direction to providing the tutorial as discussed above and if the lecture is taped and edited it can then be used as a Web based Training. I am not an expert in this, but with all the effort going into LORSM and IMS LD, I assume we have enough knowledge in the community to achieve this.
WEBSERVICESThere is certainly need for a co-ordinated initiative along these lines, but I do fear that web services alone are not the killer app OpenACS is looking for. At the last OpenACS conference, we all sat together and shared our requirement sets which told kind of a fairy tale of what is needed. And these do not strip down to web services as such. I will try to sum up some key issues discussed back then in April further down, in appreciation of your list of issues and to foster further discussion.
I am aware that I will get lot's of bashing about this for not understanding the concepts of WebservicesAt least not from my side ;) The scenarios you sketched in some one-liners are certainly valid, however, they might not directly be related to web services, they could be solved (better, more elegantly, ...) by alternative approaches to the same idea of remoting or don't even require middleware as such. But, I think, this is the wrong place to go into further details ... I'd like to continue your requirements gathering, by listing and commenting some requirements we came up with back in April. Let's rephrase "web services" to something more general, such as "remoting". hope, you agree.
interaction styles
While this a quite generic term, it subsums all that has been discusse under will buzzwords such as RPC, Document or Message style, asynchronous remoting, eventing (more recently). These discussion usually boil down to flame wars on terminology: What and why should we support either of these? Or, are they the very same? Back at our get-together in April, we collected 4-5 scenarios, mainly by you, Malte, Frank, Nima and Neophytos, if I remember correctly. In view of interaction styles, I got the impression that there is an articulated need for support of "data exchange & integration scenarios" rather than conventional component integration (or integration of remote functionality). This would shift the requirement set in this category. It simply is an area of important strategic decision making.single vs. multi-protocol support
Why not providing/ exposing functionality and data over different protocol channels (SOAP and XML-RPC), without the need of multiplying the coding effort (e.g.,rewriting the serving bits of code, interface descriptions)? In fact, multi-protocol support is something rather new in framework design, most established frameworks are rather short-handed in this respect. Note, by multi-protocol i do not simply mean providing an xml-rpc and soap package and having the individual developer deal with exposing their code to them. There should be a certain protocol transparency from the developer's perspective. I don't want to give examples from literature, but just a hands-on example: Ruby on Rails comes with a somewhat multi-protocol support (at a rather low level, for my taste) in its Action Web Service module. By switching around some parameters, you either go for an XML-RPC or SOAP call/provision, through a common interface.developer support
This is an area to gain grounds (at least with respect to other web development environments).- "scaffolding environments": This is the place where your "ad_form" scenario goes, but this might be easily amended by considering such a form-support for xowiki.
- as simple as is, a nice (xslt-based) annotating wsdl/wadl reader for developers (this has been pioneered before, can't find the link now)
- code generation from wsdl/wadl (or better dynamic stub creation)
- testing environments, i.e. integration with an automated test environment to allow for testbed runs of remoting actions.
- some (web-based) logging and monitoring functionality, something even lacking in most commercial platforms.
inter-framework compatibility
The nasty thing about remoting (especially following the illusion created by standards) is that it only pretends to make applications interoperate. the list of incompatabilities is never ending, did anyone out there suceeded to interact with .NET remoting from whatever environment in the intial and first run? This is certainly beyond our influence (partly), but it might be a good selling point to come up with a solution addressing these. And a couple of these issues can smoothly be resolved or just properly documented. There are a couple of directions at hand.security & remoting
Most would summarize this requirement by calling for support of WS-Security (and related stuff) as propagated by OASIS and WS-I industry groups. However, most problems (apart from implementing this rather complex security schemes) occur at lower levels (what about ssl/tsl?) or on-top of WS-Security. Just to name one: How to design and realise configurable access to functionality and data offered over remoting channels? Some would now say, okay let's go for WS-Policy, but this is another major task. Rather straight-forward but powerful "invocation access policies" (note, by access I do not mean "access control") date back to the time of early CORBA. A basic scheme can be implemented quite straight forward, similar to xowiki's policies (xorb actually features such policies). To summarize my main messages:- "Web services" is not enough. This is both a frustrating and encouraging statement. Frustrating, because it means considerable efforts to provide some basic support. Encouraging, because we do not need (and should not) reproduce an existing scheme, instead look for a niche and go for it.
- I highlighted some possible niches above, note, remoting support for web environments means something different than writing yet another AXIS, gSOAP, .NET Remoting or the like. It needs different focus (developer support, ...)
- It can't be done as one man's show ;) and needs some backing (funding, dedicated dev time, ...). But this goes without saying, actually.
Why not providing/ exposing functionality and data over different protocol channels (SOAP and XML-RPC), without the need of multiplying the coding effort (e.g.,rewriting the serving bits of code, interface descriptions)? In fact, multi-protocol support is something rather new in framework design, most established frameworks are rather short-handed in this respect. Note, by multi-protocol i do not simply mean providing an xml-rpc and soap package and having the individual developer deal with exposing their code to them. There should be a certain protocol transparency from the developer's perspective. I don't want to give examples from literature, but just a hands-on example: Ruby on Rails comes with a somewhat multi-protocol support (at a rather low level, for my taste) in its Action Web Service module. By switching around some parameters, you either go for an XML-RPC or SOAP call/provision, through a common interface.
Everything should be automatic! This is like ASP.NET, just add a few decorations to you code and it becomes a web service...or not. Actually if you want to have complex types or validate the input you are totally screwed, now you need to write your own code for doing this. Apache AXIS is worse. Although it _is_ true that once the bits get to the protocol level the developer doesn't need to know anything, but by that time who cares? There are no switches to throw anywhere in any system, and you still lack the most important ingredient: a published description of the service and how to access it. Maybe somebody will recognize the importance of following a standard so that external clients can actually use your interface.
Now, what are the issues with OpenACS? The main issue is that most OpenACS applications depend upon page views. By that I mean that pages serve as a procedure. This makes it nearly impossible to reuse this code by a different protocol. For instance, my query-writer package requires all form submissions to go through a special page, very similar to rpc, minus the xml. Until I write a procedure, or set of procedures which covers this functionality, no amount of developer support will help me.
Another issue specific to Tcl is important to recognize: Tcl does not maintain type information, ad_proc actually complicates the situation, and we know nothing about certain things like return types, use of uplevel/upvar, etc. Many programming languages know all this information and so it is 'easy' to serialize a data structure to XML. This is why there are so many rpc packages in every language. But rpc has many problems which were never solved because they are much harder. First problem is that server and client really need to use the same language for full interoperability. Otherwise, either one or the other needs to learn to handle conversions. Second is that it is unlikely that anyone anymore really wants to expose an internal procedure to the network. The best method for handling this is to write a wrapper procedure to restrict use or perform any additional checks that usually take place prior to execution. One of these 'checks' is validation. This is where the wheels fall off of rpc and it is why nobody, not .NET or AXIS performs validation out of the box! And by validation, in this situation I only mean simple validation like is 4 an int? Derived types and complex type validation...this isn't even supported if you wished to do it. Just make up you own method of figuring out if the input is valid, test, debug and hope.
So if the big boys are not offering this as a feature of their products, with billions spent on development, maybe it isn't so easy. Configuring a procedure as a remote application, or whatever it is called, will always require a certain amount of information. How to collect, store and expose this information so it can be reused by new protocols, maybe at the same time...that would be helping the developer.
Hmmm maybe look at tWSDL. tWSDL impliments a single protocol: SOAP with document/literal. What is probably not appreciated is that the protocol is just a few pages of code, and is chosen based upon the client request. If someone were to write their own 'binding' and protocol handler they could reuse the entire tWSDL code base. The service would be fully described by the WSDL, input messages would be validated prior to use. If XML was not used as the representation, then the protocol handler would use some other method besides tDOM to deserialize and some other method to serialize the result. Writing the code is the least time consuming part of this. Deciding what your protocol is and how it works will take the most time.
One thing has become clear over the last few years: developers are starting to recognize that interoperability is the key. Interoperability is achieved by restricting the number of options and requiring public descriptions of services which don't require out of channel communications.
A note about code generation from WSDL. TWiST does this, not just stubs creation like AJAX. But it is only for the server. I have begun work on a TWiST Client. The client can run along with the server, but it will very likely not require AOLserver, that is it will be pure Tcl. But I'm developing it against Google's AdSense service. Everything is https, so I'm using ns_https to pull in the WSDL to read. The service developed on AXIS and uses a feature not in tWSDL/TWiST: SOAP headers. Anyway, the goal is a client which is created from reading the WSDL with API creation for each operation. Creating a document and sending it should work just like in the TWiST server. The return document will be available, but there will likely be a series of generic procedures for turning the document (as a Tcl representation) into Tcl structures, lists, arrays, etc. Of course there will be optional or automatic validation on the return document.
Working with a procedure defined via ad_proc actually presents very similar issues as an application page. ad_proc serves many purposes, but the way it works makes it harder to expose as a web service. This is why TWiST works the way it does: configuration is done via <ws>proc, which provides a wrapper for the internal API. Inside this wrapper, you can specify any amount of code. Maybe you could source a tcl page and do something with the resultant state. Whatever goes on inside the wrapper, you just need to return a list which represents the output message.
One point about typing with TWiST: more important is the structural typing, such as the names and order of the complexType elements, how many of each of the elements, can it be missing, is there a default, if missing, does this indicate it is 'nil', etc. Otherwise, your return document will just be a smashed together mess, but internally TWiST does not care about types, just structure. Essentially the type information is a description of the template: what is the document going to look like. Nobody would ever suggest we just run a tcl script and return a list of all the variables, the results need to be formatted, and this is what the type specification does: defines the non-data parts of the document. The wrapper code doesn't need to know anything about XML or any of this configuration, it just needs to return a list which will be formatted into a document.
When the TWiST client is done it will only handle document/literal, whatever is available now is in the wsclient package of tWSDL. I think everything is in a web page, and it requires the ssl module to be installed, at least to use ns_https.
One can always as xotcl, where and how a certain method is looked up. In the particular case,
Gustaf, what would be helpful here would be an API Browser for classes which lets you search for something like "xowiki::FormPage new" and display you the new function wherever it is defined with the link to the file it is defined on (in this case I think it is ::Generic::CrItem new, but I might be wrong). Or, if this exists, and explanation for an XoTCL dummy like myself where to find it 😊.
::xowiki::FormPage procsearch new tells, that new a method of ::xotcl::Class implemented in C.Your request is certainly valid. I have put an updated object/class browser into cvs head, which show the class hierarchy for classes together with the provided methods. It flags as well built-in methods with a<sup>C</sup>
Tom
That is not very constructive. Read my posting again, i address arguments for refactoring in general as well.
If you delete the first part of the sentence, you can make any conclusion you want.
Why is XOTcl no extension? A strange argument. I have not complained about upvar/uplevel in general (Tcl has so far no other way of doing a call by reference), but about its widely use e.g. in the templating code where variables are set and read on various call stack frames. Have you every tried to figure out, why e.g. an ad-form does not find variables in an adp_eval or adp_include? My impression was: better not touch this code. This is bad from a development point of view. What i am arguing that in many such cases there are better ways to encapsulate state or behavior, especially when efficient objects are available.
xoTcl is not an extension of Tcl. It is a new language. If it were an extension, then the authors would not complain about uplevel and upvar.
i am not understanding your paragraph with the "single complaint", the deficit of Tcl, the ignorance, and "people think they are smart". Maybe, a forum is not the best place to discuss this.
Well, by which definition? There were a vast number of articles written in the 80s and 90s about classification schemes for object orientation and object based systems (e.g. Peter Wegner). Tcl does not fit these definitions (i am in no way arguing that tcl is a bad language). Certainly, object orientation can be achieved without object oriented languages, object oriented support can be built in practically all programm languages on the framework or application layer. What's wrong for providing language support for object oriented aspects to make it more consistent?
I would argue that Tcl is very object oriented, maybe exclusively object oriented
Also openacs extends the language Tcl by ad_procs, introducing non positional arguments. Tcl is an extension language, this makes it powerful.
Steve
Learning XoTcl or oo by studying a full-featured system like xowiki is as complex as learning Tcl from OpenACS. The xowiki documentation and FAQ are growing, but it is always hard to provide documentation for the right target group. Some people want an OO introduction, others are interested in the interaction with aolserver, others are need enduser documentation, or "Inside XoWiki". Feel free to ask question in the forums or send emails to me, i am always trying to be helpful. Gradually, the documentation improves, but it is slowed down by the innovation process within xotcl-core and xowiki.
I have been using XOWiki for the last couple of weeks as it looks like a perfect fit for an upcoming project but I find it hard work to figure out whats going on.
Malte
Rewriting openacs code exactly like it is just for the sake of rewriting it does not make much sense for me either. Nobody argues for this. I have rewritten some code of openacs a couple of functions in the past. This task is a lot of work, mentally disturbing (some "objects" have a complex, grown behavior), the outcoming code is not a text-book example how object oriented code should be written. I am not arguing that someone should rewrite the code, but i am pointing to the benefits of the parts i have done in the past.
Rewriting in XoTCL does not make any sense to me either.
Haven't you asked for an XoWiki widget for forms? Wouldn't it be nice to have tags, categories, web 2.0 gadgets, social networking ... in forums as well? No, i am not volunteering in rewriting forums, but i want to point out that orthogonality and object oriented reuse makes sense for many openacs packages from an end user point of view as well.
But I would not go and rewrite e.g. Forums in XoTCL unless there is a compelling reason to do so (read: customer requirement).
