Forum OpenACS Q&A: Will Dr. OpenACS survive? Or why I stopped worrying and learned to love the .LRN consortium?

Will OpenACS survive? Is the .LRN Consortium necessary to its survival?

Let me put forward some controversial propositions for your consideration.

1. In a trivial sense OpenACS will survive whether or not .LRN exists or succeeds. In the trivial sense, OpenACS will survive as a project of interest to a small group of enthusiasts and hobbyists and the occasional one-off organizational adopter of the software. That's where we currently are with OpenACS and .LRN. I don't mean to imply that the project is trivial for the enthusiasts and hobbyists. Far from it. Nor do I mean to belittle anyone or the value of the project. We would not have chosen it as the foundation for .LRN unless we strongly believed in the technical merits of the framework and the brilliance of the community. However, OpenACS remains trivial in terms of its standing in the landscape of open source projects (test: if we were to do a recognition poll at O'Reilly Open Source Conference, my bet is that less then five percent will have heard of OpenACS and perhaps only one in a hundred will have some inkling of what project specifics.). It's also trivial (in the sense of not worthy of serious consideration) from the perspective of enterprise adopters.

2. I predict that over the next eighteen months there will be a brutal industry consolidation of open source projects, dictated strictly on economics. Those left standing will fall into one of two categories, trivial or professional. Only a handful of open source projects will fall into the "professional" category. What will be some of the distinguish features of "professional" open source projects? It will need to play in the enterprise. For that issues such as Apache vs AOL Server are less important than quality, risk, transparency, and business model. Quality: people are now astute enough to know that not all open source projects are the same. How do we know that the software we are using or intend to use has a high degree of quality and can withstand installation in a mission critical enterprise? The notion of ".LRN-certified" components is one of the ways in which the consortium intends to address the quality issue. Risk: Organizations are increasingly concerned about legal liability and exposure whjile using open source software. The SCO case began it but we are also now under threat by the sorry state of patent laws, particularly in the U.S. GPL by itself is not enough for risk mitigation. The consortium is stepping up and beginning to address this topic. While we cannot provide indemnification, the consortium will be taking steps to diminish legal risks. Transparency: Organizations will not adopt OpenACS and .LRN in mission critical environments unless they know with confidence that the software release process is predictable and well managed. This ranges from everything to an accurate and up-to-date bugtracker to clear statements and status of active projects. Business Model: the project will not be viable---in the sense of gaining adoption and interest in the enterprise market---unless vendors and companies are able to realize profit and a business model around the software.

What does this mean for OpenACS and .LRN? For OpenACS it does not mean changing anything except for taking more interest and also participating fully in the .LRN consortium. The OpenACS technical community is healthy and well governed. I would also like to see more involvement though among the community members in the .LRN project. The challenge is greater for the .LRN community. .LRN users and institutions need to contribute and participate more in both the OpenACS and .LRN communities. We have to educate .LRN users that they need to give back more to the community: that's the spirit of open source but also required if we are going to survive.

Some opening thoughts and I welcome your comments and flames. :)

I predict that over the next eighteen months there will be a brutal industry consolidation of open source projects

Why? What is your evidence for this? And what is special about the next 18 months, as opposed to the 18 months after that, or the previous 18 months?

The issues of quality, risk, transparency, and business model exist now, regardless of Al's premise. Discussion on the chronology or merits of any predictions are secondary --whether they are based on "peak oil" calculations or some kind of religion inspired end of the world scenario.

The project processes need more attention to eek out the best OpenACS and dotLRN has to offer.

This is useful dialogue. Let me say that my intention was not to criticize OpenACS. On the contrary, I believe that OpenACS is better governed than .LRN and continues to make great strides in terms of governance and transparency. Having said that, the .LRN consortium is finally getting its act together. But also recognize that organizing a "user community" comprised of academics has its challenges.

I believe that Jeff's formulation is the best way of looking at this: "Maybe if by brutal you mean that there is tremendous opportunity for growth in open source which some projects will miss out on because they don't make the right collective decisions about how to manage themselves then maybe I would agree."

That's what I would like to focus the discussion on. How should we manage ourselves so that we fully take advantage of the opportunities for growth? The "we" is OpenACS and .LRN. There is potential for great synergy between OpenACS and .LRN. Let's havea a no holds barred and dialogue on what we need to do shore up the relationship so that we have a powerful and unbeatable partnership as we move forward.

I want to offer elaboration on some of my points. My starting premise is that the battle of open source vs proprietary is over. It's now accepted within firms that open source software has a place: it's not whether (to use open source), but how and when.

But with this maturity and acceptance of open source, a new set of considerations have entered into the picture. These considerations were just nore relevant and worthy of consideration for open source projects as late as even a year ago. The maturity of "open source" means that we have entered the "market" completely and inexorably with all that it entails.

Now, let me come back to my distinction between trivial vs professional open source projects. I admit that it was a poor choice of words. The distinction I had in mind is the difference between a project that meets hobbyist's needs vs a a project that aspires to meet market demand. That's not to belittle hobbyists because there might be open source projects that are extremely innovative and push the envelope with new technology or approach. At the other end of the spectrum -- and it is a continuum I believe--might be other projects that not only beautifully fit a market demand but are able to *sustain* their competitive advantage.

One of my principal points is that the rules of the adoption has changed and will continue to change as open source becomes more and more mainstream. The threshold is much higher for adoption. The criteria that we (e.g. MIT Sloan) used four years ago for adopting open source is not the same criteria we would use today. For example, intellectual property and liability is a big issue for firms as they consider open source. If we aspire to play in the "market", then there needs to be a real story on how we are handling this issue. And that story can't be just, "hey we are GPL".

Moreover, the same types of pressures that firms face in trying to maintain competitive advantage also now confront open source projects. Let me quote Michael Porter (Harvard Business School): "Being 'all things to all people' is a recipe for strategic mediocrity and below average performance, because it often means that a firm has no competitive advantage at all."

Now, I am assuming that OpenACS aspires to meet a market demand. I know .LRN certainly does. There are essentially two ways in which firms maintain competitive advantage: low cost or differentiation. For OpenACs and .LRN it's probably not low cost. So, what is our differentiation and what is our strategy for maintaining that differentiation? What are we unique in that buyers value?

I don't want to imply by any means that we (".LRN") have the answers. I wish to suggest that these are some of the questions we should be asking and trying to answer jointly.

Let me come back to the "brutal competition" bit. The good news is that markets are looking to and are now open (no pun intended) to open source products filling the need. The bad news is that we are in a market, which means that we are faced with competition. And, therefore, survival.

Will OpenACS survive as an activity for hobbyists. If that's what the community aspires to, then it's a good bet that it will continue to survive as an obscure and relatively unknown project. If, however, OpenACS aspires to fill a market need, then the jury is still out. But if we find ways to strengthen the partnership between OpenACS and .LRN, while finding our true differentiation, then I am confident that we will have a very combination to play in the market.

Andrew: "Why? What is your evidence for this? And what is special about the next 18 months, as opposed to the 18 months after that, or the previous 18 months?"

Yes. 18 months is arbitrary. But it's not 18 years.

Well, as far as differentiation goes, that question is probably best asked of the OpenACS community members who have investigated alternative tools in-depth, in order to learn what the competitors do well and poorly, and thus perhaps shed light on how to further improve and differentiate OpenACS.

We have at least one active contributor (Jun? I forget.) with extensive Red Hat CCM experience, and probably a whole bunch more (e.g.) who have between them either evaluated or used for real one or two dozen of other systems as well. And hasn't Lars been doing lots of work with Ruby on Rails lately? What's the story there?

Perhaps competitive market analysis simply means getting all those OpenACS folks together in one room for a day. Preferably with beer, and if they have enough interest, preferably with preparatory work by each participant beforehand as well.

Please allow me to add a thought to the discussion here:

OpenACS has its roots in a university and grew due to market demands. Now the market shifted from the technology perspective to LAMP, J2EE but OpenACS cannot simply do the same. Now from what I see an active and lively community demonstrate this ability by growth either by the number of active developers in total or by the number of freshmen it attracts to fill the shoes of those who left. Which product of OpenACS can serve best to fullfill this task?

I believe it is .LRN. No other product have we at present that can reach so many young students and even professors who could all become devoted active community members and improve this toolkit. So basically OpenACS has now the great chance to be brought back to where it came from: the university.

I think there are some issues with making the primary focus for all OpenACS activities be dotLRN. For one thing it's very big and very complicated hence much harder to develop for than simply writing packages on acs-core, it's quite rigid in the site structure which makes it unsuited to things outside the course management/education market.

I personally think the way forward is to take it out of the university and the reason I think that is that the innovative things I think draw people in are not coming from dotLRN. I think things like Lors, the assessment package, etc are important to the success of dotlrn in the "Enterprise Level University Service" market but your run of the mill student just isn't going to care. They are not going to set up a website to run their own little university or deliver learning objects but they might set up a website to run their blog and maybe a photo album and other things, but it might get to the point where OpenACS is really a high-octane alternative to gallery and moveable type at which point I think we would be more attractive to people getting started.

Nima, my point is if you want real growth in the developer base you need to cater to the people who want blogs, photo-albums, and other personal things. That pool of developers is many many orders of magnatude larger than the pool of people who want lors or assessment.

Jeff and Nima's lines of thought are not necessarily orthogonal.

One of Jeff's point, I believe, is that the principal driver of innovation in open source is developers (students or otherwise) wanting to "scratch their own itch". Cool stuff is more likely to flow from a developer wanting to do something different with blogs or photos than programming from pre-defined specifications for a new LMS feature, no matter how important and valuable that might be for the relevant institution.

As an open source technical community OpenACS needs to continue to be attractive and inviting to developers so that they feel that they can do "cool" things. Lars left the community partially I suspect for this very reason. In his opinion, Ruby is a cooler and more flexible development environment.

The knock on OpenACS is that it's not mainstream. I think that's less important than the fact that OpenACS has the perception of being rather obscure ("medieval scholastics" is how one developer described it) and not terribly inviting to newbies. We compound the perception of obscurity by lack of visibility, not participating in the larger open source community. Perhaps this stems from an ultra macho attitude that only "real men" do OpenACS and its inaccessibility and obscurity is part of its strength. Let me enter as evidence the fact that for the fourth year in a row there will not be a presentation or even a presence at O'Reilly conference. If you are not at O'Reilly, you don't exist. It's as simple as that.

So, this is a challenge for OpenACS independently of .LRN. What is about the framework that should be attractive for developers? What can be done or needs to be done to make it more attractive? What can be done to make the cost of entry lower?

If the OpenACS web site had a sign "Developers Welcome, Enter, Come Here!", the question that needs to be answered is "Why". Why, compared to other frameworks? Why is it worth my while to join the OpenACS community? What can I do with it that I can't do with other frameworks? And how do I go about doing it?

I went to a talk by Rasmus Lerdof several years ago at O'Reilly. He was stunned by the success of PHP. Surprisingly, he attributed most of the success to the documentation associated with PHP. His paramount goal was to make it as easy as possible for newbies to pick up the technology and quickly join the community.

I agree with Talli, it is a waste of time marketing to developers, it's needed a "serious and consistent" marketing effort for .LRN, that should take place in the short time. By serious, isn't just a few random hours of someone trying to do something about it, in the other side we don't need an entire army, just a continuos and dedicated effort of someone who really understand what's going on with the .LRN market segment. And by consistent, is that we need to have a great image, and not just good persons, for instance the dotlrn.org website is very poor for marketing itself.

Can we have .LRN marketing itself continuously and loudly in its segment?

My post was certainly provocative and may very well be becuase I'm an asshole, but it's not because I'm wrong.

And I'm not asking for an apology from anyone at .LRN. But I am saying publicly what I've been saying privately for two years now and what others are also saying.

The investment that .LRN institutions have made have not been trivial, that's certainly true. But the effectiveness of the organizing of the .LRN project has been abysmal.

Some people have done well in the community and it may just be that the project can only sustain those. But if .LRN's problems are entirely related to resources and marketing then what good is it pontificating about what OpenACS can do about it?

talli

Yes QA/Release management is crucial in any project, and .LRN is aware of that. Of course, Joel is the right person, but while we don't have a paid person to handle that, .LRN will relay its product quality on random volunteering efforts or will never expand its functionality base.

BTW, the new version of the graphical installer is scheduled for the next week, and will be maintained for the next 2 years by the University of Reading, UK.

After months of working with ACS 4.x I recently picked up OpenACS at `stable' version 5.1.4. As some might know I've been an active member of the OpenACS community for several years now and as such am quite familiar with the internals of OpenACS.

Even installation of OpenACS core required patches to run on PostgreSQL 7.3, which by OpenACS's policy should be a supported version.

After installation, tsearch2_driver required a patch to work with PG 7.3. Lars-blogger fails when setting up blogs for several users. You might say that these are small problems, but I found them very discouraging. Compounded with administration navigation that appears to be designed without rhyme or reason was enough of an incentive to look for a different web development framework.

Not that these specific bugs/gripes are so bad that I feel compelled to look around. No, it is the fact that after all these years OpenACS still produces questionable releases, has outdated documentation, lacks comprehensive support for SOAP, relies on a legacy ecommerce solution with a fair share of warts (no offence Torben) and has trouble supporting even 2 databases.

Things I'll miss most about OpenACS? AOLserver, tDOM and the templating system.

I haven't forsaken OpenACS, but OpenACS is losing its appeal. And in that respect I'm with Talli's comments on .LRN. As a developer it also makes economic sense to look at more mainstream tools/languages, especially when those tools solve the same problems just as well.

/Bart

I want to apologize to the community for losing my temper and my showing disrespect towards Talli. I was learning from the discussion until things degenerated into name calling and personal attacks.

I remain committed to the community, but I no longer plan to post in public except if it is to provide information. I am open to suggestions, comments, and criticism at any time in private.

In the spirit of being constructive and also being true to my thoughts, here are some (hopefully not too rambling) thoughts and ideas:

Al's point about consolidation strikes a chord with me. It is a hunch for sure (intuition sounds better), but it feels true - and many of us will have seen the pattern in other industries.

What Talli and Bart have had to say strikes a chord with me also (very musical today...).

Taking into account that I have been using ACS (on and off) since beta versions of Tcl ACS 4, I know ACS quite well. I'm not scared by upvar or form widgets and I positively love writing pl/pgsql and tuning queries. I love the package manager and see more benefits than disadvantages in the monolithic structure of the system.

OpenACS as an overall system however, is old. The lack of true Object Orientation in it's design makes (non-destructive) customisation hard and core development slow. The lack of rich data structures (or objects) makes Tcl unsatisfactory (although OO Tcl extenstions may solve this weakness).

The flip sides are the body of code and the community. With the exception of good OO modelling, all areas of my professional personality have grown due to interaction with members of the OACS community (and the community itself). Possibly more than due to any other IT professional experience.

The body of code can also not be ignored. It is simply to good to throw away.

Now here comes the constructive bit! If we can let ourselves admit that there are key flaws in the toolkit, then perhaps we can get to a point where we then realise that we can deal with those weaknesses without throwing it all away.

The question should become "what are our strengths - what is it that we are so reluctant to throw away". Well there is certainly the core object model with it's permissioning and relationships ... there's the package and subsite management ...

Once we see what the strengths are, we can focus on how to provide them in a way that people will want to use. Perhaps we could write wrapper APIs for them in another language (like, say, Perl which is supported by an AOLserver module, or maybe Ruby). Design a module template and developer quickstart kit for people familiar with that language - this would allow them to quickly get a server up, and start creating objects and using the api in their package using all the tools of that language.

This is just one idea, obviously, but if we don't do something radical like this, then (my hunch is that) the OpenACS project will lose a lot of it's headline developers and with them a major reason for the quality of the community.

I truly believe that developers are the key to core growth of a project like ours. It doesn't matter how much a manager might like something - if their in-house developers reject it or they cannot find enough developers to work on it, it becomes a non-option.

On the other hand, if developers begin to love it, it will work it's way into production by osmosis - then the value and productivity will be seen by management. That principle is why Perl is still among the most common languages used by investment banks. It's unsexy and very old-school - but you just can't argue with the productivity it offers when you see perl developers working alongside java developers.

A problem is certainly that we are not a company. There is no central provider of funding who is interested in making sure that OpenACS doesn't become marginal. Implementing an idea like this or one more radical would take some time. It's a real pity that the time and money spent on Java ACS wasn't spent more wisely, but I'm sure there's a thread about that already... ;)

PS: Which kind of doctor is Dr. OpenACS??
PPS: Thanks to Al and others for the courage and time taken in starting and contributing to this useful discussion

I’m a doctor and learned much of what I know about web programming through openACS (So maybe I’m one Dr. openACS :) ), but without .LRN I’m sure I would have lost interest by now. Why, because .LRN held the promise of the future and I hope it still does.

It is not my primary business so I don’t know how other communities work. But as I have stated before, it is my opinion that we should look into the community first. Although, there doesn’t seem to be any support for that, I still felt I had to say it one more time.

Suppose you would buy software from a company that would sell you this toolkit for $20,- , but without support (even a forum), and given the registered amount of bugs in the toolkit, would you buy if you had a goal other than programming you own custom system?

Bugs currently drive even talented programmers away, eventhough they are familiar with openACS.
What if you are the leader of a company with 8000 part-time employees what would you do if things didn’t work out? Hire a new set of full-time developers and not care why the 8000 aren’t productive (anymore)? Would it be useful to have a roadmap or to empty the bugtracker ? I’m not sure, but I feel management is needed more to keep developers than to gain new developers. Investing to keep customers is usually cheaper than to gain customers.

Interesting. I thought this includes new packages as well. Which packages are included? Only the core packages?

Just a little comment - I regretfully agree with Nima that windows support would be a big boost.

It could also bring less talented developers and decrease the quality of the forums. Controversial but true - a larger user base would most likely mean that the average skill level would decrease. The average OpenACS developer is definately above the industry average in terms of skill level.

And it may not be so hard now that postgres 8 natively supports windows.

A sad fact is that building a windows specific binary installer would be far simpler than a linux one.

It seems like a no-brainer for me, but our community has limited windows developer skills.

Heck - it can't be that hard. Make the installer, get an articl or two in major developer magazines or even better - get on the cover CD. Small business intranet in a box? OpenACS has it all.

Add some API enhancement like my previous post and I think we would be very suprised at the outcome.

Exactly my point - most OACS developers probably don't have a windows box - I don't!

I have printed and framed Mark's posting, especially the comment:

"I truly believe that developers are the key to core growth of a project like ours."

That is the core strength of the community and we need to find every means to leverage it.

Nima:

OpenACS has a huge amount of documentation. All APIs are documented at https://openacs.org/api-doc/ and the same URL within any installation.

Consistent APIs are an ongoing challange, so we are working on it.

"As a newbie I find it extremely hard to understand how I can define objects in OpenACS." This I find very hard to believe. Besides the huge amount of existing example code there is the tutorial: https://openacs.org/doc/openacs-5-1/tutorial-database.html and the lower-level, but extremely detailed https://openacs.org/doc/openacs-5-1/object-system-design.html

Please if you are looking for answers to a question, and can't find it in the documentation, please ask on the forums. If it is hard to find something, please let us know. Anyone who can help out with documentation is welcome to.

While on the subject of documentation one place I think we need to streamline the existing documentation is on the install. It has been said that the install documentation shows how to build a production ready environment. That should be a seperate advanced document. A goal should be to have a one page install document that gets a minimum system running. With many links to more advanced topics.

I am going to respond to Bart's concerns. I know he is not the only one who has these concerns. My greatest problem with these type of complaints (and not Bart expressing them in particular) is that they only come out in a greater discussion of marketing, or why OpenACS doesn't have more developers. Why aren't these issues brought up individually in the forums, or as bug reports?

"Even installation of OpenACS core required patches to run on PostgreSQL 7.3, which by OpenACS's policy should be a supported version."

Interesting. Are these problems documented somewhere so that the community can act on them?

"After installation, tsearch2_driver required a patch to work with PG 7.3."
A patch to what? To tsearch2, postgresql, or the tsearch2-driver package. Its a new package, so it might have problems. Was a bug report posted?

"Lars-blogger fails when setting up blogs for several users."
Is this in the bugtracker?

"Compounded with administration navigation that appears to be designed without rhyme or reason was enough of an incentive to look for a different web development framework."

Was there a discussion about admin iterface problems? As far as I know, most people thought the interface was improved, but certainly had room for more improvement. Did you propose and ways the interface could be improved?

"...it is the fact that after all these years OpenACS still produces questionable releases, has outdated documentation, lacks comprehensive support for SOAP" Was there a volunteer to work on this?

"... relies on a legacy ecommerce solution with a fair share of warts (no offence Torben)" Was there a volunteer to write a new one? It is my understanding that people are using ecommerce in real web sites, so the perceived problems are not so great to stop anyone using it. Certainly it has not improved at the same rate as Amazon.com.

I encourage anyone to ask questions, post problems, and make suggestions! If necessary write a proposal, design a mockup for a new interface, write code. We have a pretty much open CVS repository now. Anyone who wants commit has been granted it. We know how to reverse commits if there is a problem. If anyone needs help in writing new documentation and getting it intergrated with the docbook version is offered assistance.

OpenACS

I am not a doctor yet 😊 but i have some thought about OpenACS i want to share with all of you that have been taking this great conversation in this forum keeping the flames public available. We can learn with this kind of discussion too if we could deal with it in a positive manner.

Why I came to OpenACS and why I love it and why I want this community survive? It is just because i perceive the value of this community-driven software development as a template for education. Not just for my personal education but also for other who want to involve with opensource.

As a template for education OpenACS survival is almost entirely dependent on the participation of the community that supports their efforts, regardless of their specific association with the project. Other reasons like we don't do Q&A, market analysis, or use focus groups seems to me less important for OpenACS survival than create group processes into this community to Build Good Software and Education

Rob Reynolds wrote once a interesting article for Xplana Learning where he suggested there are the elements that go into the making of good software and education. They are here:

* Abundant dialog -- the conversations occur and proliferate at every level, not just top-down. There are active, grass roots discussions that are just as important as the formal conversations taking place;

* Quality listening -- since information sharing and product evolution are the goals, listening is the most important element in every stage. Every detail can be important and people stop talking hen they do not feel they have been heard;

* True positive feedback -- positive feedback in the sense of a positive feedback loop -- information that actually causes a change in or modification to the system;

* Openness to real change -- positive feedback is a design feature that means the process/system is open to change, in fact, that it is expecting change to occur;

* Open-ended inquiry -- a driving force in quality developments is the asking of questions that have no set or pre-determined answer.

Maybe these elements could help us to create a group process that is dependent on asking questions, allowing new information to alter previous plans, and listening to the input of others. We have tools for enhance this kind of group process (public available TIPS, forums, IRC logs, maybe we could explore Survey). We have community-friendly for this (knowledgeable, transparent). If we see just only this thread we have great positive feedback 😊

.LRN

I really think .LRN is a little different from OpenACS. The most important differece i think is that .LRN is the unique expressions not only of OpenACS community or .LRN Consortium, but also and much more of their Users. .LRN competitors like Moodle, Atutor, ILIAS and others are good because there is a collaborative User community surrounding each of them in which people listen to the ideas of others and are willing to change. Where are .LRN users? Where are them using .LRN (at university, at k-12, at reseach centers etc)? What type of .LRN user are them (Educational administrator, Systems administrator, Teacher etc)? When will be the next .LRN conference? Is there something like a DDP (dotLRN Documentation Project)? Tips and Tricks Using .LRN for Teaching for Researching for Universitary Portal for Courseware etc? Reviews on IT news, EduInfo etc?

I think .LRN Consortium has great advances on it. Quality survey from Rafael covered most of the "where". Sloan and Bruce Spears covered some aspects on teaching using .LRN. Al and Cesar Brea has in touch with conferences. E-lane has working on methodologies and producing contents for e-learning. Dave Bauer and others has focusing on discussions about Good practices in elearning- either here in forums and his personal page at thedesignexperience. Could we glue all of these advances and show them for people who want to try .LRN?

I think install and TCL are both non-issues to some degree. Look at Rails -- even less people know Ruby than TCL (is my guess; certainly neither is mainstream) but people learn it because Rails is just that cool.

AOLServer OTOH is relatively big. It means you have to be a fairly solid sysadmin to get OpenACS running, and you need a more expensive virtual-root setup instead of a shared apache one. Hobbyists won't bother.

Al - I would love to take up your offer ... if I wasn't getting married in three weeks ;)

Just a tip with running postgres 7.x under cygwin - anyone I know who has tried to run a decent size openacs database on it has found serious stability issues.

Perhaps that is better with 7.4.2, but I wouldn't reccomend anyone runs production postgres databases on windows without postgres 8.

Andrew,

Talli and I have been talking offline and he said the same thing. Some blood letting from time to time is healthy. The "Talli bomb" set off an energetic discussion.

I will enter back into the arena.

It's not that I have bruised feeliings or that I am sensitive guy. I have a pretty thick skin. I was concerned about the impression it would make on people who tend to be silent i.e. "I am not going to speak my mind because I will get blasted or be subject to personal attacks."

But I realize now that needs to be part of the education process in open source communities. Any debate where the stakes are high and people are passionate will lead to some fisticuffs and blood. But at the end of the day we should all be able to go out and get a beer. That's the way it goes.

Incidentally, Frank's OpenACS and Project/Open installer for Windows sounds like exactly the right approach to me, for Windows - just not for Linux.

A clone of that same installer for Linux might be useful, and might be better than nothing, but seems clearly inferior to leveraging the fabulous stock support provided by all the modern Linux package management systems and tools (rpm, deb, apt-get, and yum).

In this thread and in others I hear three recurring themes, which apply to OpenACS and .LRN. We need:

1. orderly and predictable releases;
2. an improved installer;
3. quality code base;

This is not exactly "differentiation" but it would go a long way if we were able quickly bring ourselves to the point where those who are trying to "sell/market" OpenACS/.LRN can legitimately and credibly claim:

"We have a product that you can install quickly, upgrade easily, and, by the way, is enterprise-class out-of-the-box."

By enterprise-class I mean a product that a) is usable in a mission critical production setting because of a high degree of code quality; b) scales from a small site (e.g. small NGO) to a large educational institution (e.g. Valencia with 20,000 - 40,000 users); c) integrates into the enterprise (e.g. external authentication).

What f$*(%#! product out there, with a feature set comparable to OpenACS/.LRN (commercial or open source), can make this claim?

I believe that we are 90% on the way, and we just need that extra 10% of effort to get there.

I will offer some thoughts soon on how we might be able to get there. But I think this should be the area of focus for the next 3 months and we should all work together and go after it ruthlessly.

In response to Jonathon re: installation. The first time I downloaded and set up a dev Rails setup, I was up and running in under 10 minutes. That probably includes download time as well!

In fact the one thing about Rails that gets you interested it is it's instant satisfaction. Along with it's easy install, the 'scaffolding' concept that uses database introspection to auto-build template pages means that you can have a little test site running much faster than even PHP.

A bit later, the "everything's an object" philosohpy of Ruby gets you hooked enough that the dumb (in my opinion) single db table hierarchy setup doesn't seem so bad.

The Rails community is also looking seriously at that inheritance issue, and there certainly seem to be developers of good calibre involved.

And while less people know Ruby than TCL, Ruby does not have the limitations of TCL and is a language that computer scientists can get excited about.

Note that Rails is not a competitor to OpenACS if you want to use some OACS packages. But once the Rails community settles on a common authentication system, it will become a very serious competitior for people who use OpenACS mostly for the framework/toolkit.

John,

I suspect you would find there are far fewer bugs introduced relative to the number fixed if you upgraded. Based on tickets we see for dotlrn that many of the open tickets end up being closed in the new release.

That said, the upgrade process from an older version (particularly if customized locally) is not painless.

It's an area we have worked harder at and I think over time are getting better at (upgradability and reproducible delivery). You are right that it's a pretty intractible problem for a 2-tier architecture although given the persistence layers I have seen that abstract that, it's the problem I would rather have.

Is there a good write-up on the sakai permissioning? I don't think in principle there is anything that prevents us from doing the same.

I want to thank John for his comments (they are always "spot on), but also tease them out a bit in a series of posts.

During one of our conversations several years ago over coffee in Cambridge (US), I recall John saying:

"Hey, Al, you guys really need to focus on an easy installer and make .LRN a configuration option for OpenACS."

John reminds me of a physics professor that I had in college. His statements even at face value were valuable but the real insight was something deeper: take this thought, develop it, and you will be rewarded. Let me take the John installer thought a bit further, even if John ultimately might disagree with where I am taking it.

The starting point is that there should be an identical installer for OpenACS and .LRN. It can be implemented using switches or whatever, The deeper point here is that we should not be view OpenACS and .LRN as separate products and processes. As John indicates, "The impression that .LRN is another layer on top of OpenACS with different developers having different agendas does not help."

If OpenACS and .LRN are equivalent in terms of code base, how are they different? First, .LRN would always be a subset of OpenACS and not an "additional layer" on top of OpenACS. Secondly, ".LRN certified" components would need to pass an additional set of QA to receive the designation. Third, in addition to a higher level of quality code of individual components, the ".LRN-certification" would also provide assurance of integration among the certified components. Fourth, .LRN-certified components would also be tested for scalability. Finally, there would be a commitment from the consortium members and the community to always keep the focus on quality for the already certified components. We would also always provide an easy upgrade path for the latest release.

In short, .LRN would be that integrated subset of OpenACS which is enterprise ready.

I believe that it benefits neither OpenACS nor .LRN to think of .LRN as software for the educational community. I come back to another point that John keeps reminding us of, namely that a course should be viewed as just a "special case" of a community. Now, this is the reason why we (MIT Sloan) went down this path. We were not interested in building a course management system to compete with Blackboard of WebCT. What we wanted and needed was a platform that would support knowledge communities of all kinds, whether it's a student organization, a research community, or a project community.

Again, here is a one line sales/marketing pitch for OpenACS /.LRN:

"We have a product that you can install quickly, upgrade easily, and is enterprise ready out-of-the-box."

The pitch to be an educational instutions (K-12 to ., a non-profit, a corporation, any organization that has a need for collaboration and online communities.

"And, by the way, the software is backed by the .LRN Consortium, which consists of these organizations...."

The list of organizations should be deeper than educational institutions and ideally include everyone who is using the certified components.

In short, we should begin to view ourselves as a single community that produces the highest quality software to support online communities of every sort.

Here is sakai versus openacs from the standpoint of grants.

I would think since we have heirarchical permissions, permissions inheritence, nested groups and relational segments I doubt there is anything the Sakai crud implementation offers that we don't already do (or could do reasonably easily).

Having looked at the realms bit, it's also clear that we are not the only ones saddled with a cryptic "advanced admin" interface.

I don't think I want to be in an openacs vs sakai debate either (as I don't know much of anything about sakai or really about any other e-learning platforms). I am more interested in seeing something that works well for permissioning as it's one of those really hard things to get right.

The permissioning you saw there for openacs was local to an event calendar under a particular subsite and not sitewide -- although with inherited permissions you can permission on the subsite directly rather than per application (which is something sakai seems to lack -- driving things more from roles and direct permissions as far as I can tell).

One thing that's also not clear in the sakai example is whether something like file storage which has a folder structure can permission a folder and have that permission inherited by it's children.

What I really want to find is an example of role based permissions that people really like but I have not seen one yet.

Thanks for that info Nick - I for one have never checked those communities out.

It again reminds us that the community and code base are our core assets I guess.

Of course if you want to compare $ investments into the code base, OpenACS factors in ArsDigita ACS 4 which in turn comtains part of the ACS 3 legacy - I know that we're talking well over USD 10 million here... Someone may have actual figures on that.

It's a very good point you raise too about the barriers to entry. Every time I consider Java for a web project I am amazed at the complexity of what are considered standard toolkits. I pity a university student trying to wrap their head around Jakarta Struts when they have barely learnt to think in Objects!

Nick

You are right and wrong, but mostly right. Sakai has not yet commited to a full open source development philosophy (hence they have redefined the term 'community source' for their own purposes) and the project is midway. The intended new architecture is only now getting seriious attention after a period of merging features from the 4 schools into a CHEF-plus environment.

I agree fully that OpenACS can compete well. My remarks were intended to convey the areas in which I felt OpenACS/dotLRN could improve with reference to a concrete example.

I'll bow out with a reminder that many Universities do research (and other things) as well as teach and that a solution for all of the communities on a University will always be attractive. OpenACS/dotLRN has that potential, but I don't hear it discussed much in the forums.

John, the university of Darmstadt in Germany mentioned that they went for .LRN because they wanted to have an Open Toolkit that allows their Professors to do easy research and especially research in E-Learning. I'm unsure about the status of this effort, but it strengthens your point and one of the more interesting features .LRN has to offer is support for research communities and research projects (using e.g. the project manager).

Thanks a lot for your comments and your insights into Sakai and getting this discussion about Sakai started in the first place. At least I learned quite a bit out of it (and can advise universities better).

Interesting conversation.

Allow me to add my own comment.

I am the CIO at a medium sized company in Canada, 400 to 600 employees. We are definately in the minor league here with respect some of the institutions using OACS and .LRN

Currently we run Lotus Notes as our groupware solution but we are evaluating ditching that beast. Basically Notes/Domino is really a wonderfull idea, run by a top heavy, closed organization, hugely expensive, and a system that once you get heavily invested you are tied to it.

Luckily we are not heavily invested in it, so we have choices.

We have looked at everything under the sun here in terms of an 'Enterprise Class Application Server' and various technologies.

I understand where you guys are coming from, but you really are missing the boat in your discussions here. Some thoughts from a non educational organizations perspective:

- Anything on .NET is a no go for us. MS has shown it self to be a proven and consistently immoral and untrustworthy company, and we won't be part of anything from withing their sphere of influence. Anyone that deploys anything on .NET is putting on their own shackles of slavery - MONO alternatively is not production ready.

- Java based solutions ... too many options to discuss, however I don't see anything from that toolset that really does what OACS does.

- LAMP ... this is fine for task specific stuff. If you want CMS, Mambo Server or Typo3 are amazing, if you want project management you have tons of options, if you want frameworks the options are amazing. For Python alone there are something like 8 frameworks available and Python isn't even used as a web language per se!!!

I love the LAMP stuff, but the community seems large and active only because there are a ton of tiny and a few big projects that completely reinvent everything every time. If you are going to start anyting on LAMP, at worst you will rewrite an OACS type of framework your self, or at best use someone elses that is either too small to have any mind share, too young to have been tested, or if its something that is successfull like Typo3 or Mambo server, well they reinvented their own frameworks and I dare anyone to merge anything from those two projects into the other. For that matter, I dare any one to get any two separate LAMP project to work together.

For us, I could deploy a few LAMP applications, but in the end, we will be maintaining each one separately, patching each one separately, coding each one separately and administering each one separately. They won't talk to each other, they won't have the same permission system ... etc.

LAMP is a no go for anything other than quick and dirty, but to be fair OACS has the quick and dirty down better than anything in LAMP.

- Standalone frameworks/apps like GNUE.org. Okay I personally love this project, I think they have the right idea and I think they have some potential for delivering strong solutions. The project is old and is fairly stable, yet at the same time it's still is lacking in certain features that people look for in things like that. Many other apps/frameworks in this category are in similar positions. This is a wait and see category still before I make up my mind about it.

- OACS ... I think everyone needs to go and read up on some of the history of AOL server again, and why its such a terrific base to have a framework on.

AOL Server - probably the most powerfull and scalable web server available out there ( maybe outpowered by Zeus ).

TCL - if you can dig into python, you should be able to understand TCL. Learning TCL is a non issue, from what I can see it's very simple, if you can learn Ruby or Python you can dig TCL.

OACS - one single integrated framework, on one single language ( I know there is a Python binding out there, but there really is no need for it TCL seems fine. Perhaps some expert programmers can fill me in on TCL'S shortcommings but so far I haven't been able to google anything more than WAAAH, NO ONE USES IT, I'M NOT L33T) on one of the worlds fastest and most scalable webservers.

All tested, all production proven, a single permissions system, all applications are a subset of OACS and are organized and function similarly ... WOW!

Do you guys even realize what you have here? Holy shit!

I'm just a PHB, and it took a bit, installing the .LRN package in debian is one command, got it up and running, figured out that .LRN is just a SINGLE APPLICATION built on OACS, uninstalled .LRN from the framework, installed a good number of applications my self via web gui, and setup permissions and a few sub sites all with little or no help from IRC.

I created a NEW APPLICATION with the click of a button, following the introductory application tutorial, and restarting the DOTLRN service in Debian and VOILA! I have my own application ( okay it's just the tutorial example files but they work ) running.

Hello? This is amazing!

Look, what this project lacks is sex appeal. Here are some things that might help and probably have been mentioned already:

- sex appeal. Look https://www.sitepoint.com/mambo-server-cms/ - it's lickable and the OACS site needs the same treatment.

- the .LRN debian package and the .LRN instance needs to stopo presenting it self as a separate instance of OACS because it is not. It should only be, and is, an application available from the OACS repository. Nothing more, nothing less, just a beautiful, well thought out nicely programmed.

- LORS needs some debugging and integration into .LRN. Once you add the ability to deliver SCORM class content via .LRN you have a Moodle and Sakai killer. .LRN already exists and works, Moodle is simply an instance of LORS and as such lacks the scope of .LRN, so when you merge the scope of .LRN with the ability to deliver online course, you can just "apt-get install dotlrn" and have any potential organization running in minutes with a basic server for testing blow away everyone at the presentation. This is huge, I can't underscore this enough.

- The overall user interface needs a rethink. I've spoken to quite a few people on the IRC channel, everyone knows this but no one has time. Fine, but this is minor and easily fixed.

- The applications in the repositories needs to be further debugged, further push towards bug fixing made, although everything is mostly good and mostly works. For us, .LRN, Project Management and a few other packages work great and will be Piloted in the next month or two if not sooner.

- OACS needs to make some noise. Never mind the negative crap, there is nothing out there like OACS. You can't deploy any framework faster, you can't get a framework with the production proven scalability of OACS and AOL server, you simply can't develop applications as fast as you can on OACS. I just don't think anyone knows about OACS. Holy christ, if i had this back at the Web Development firm before the dotbomb took place we would of killed all our competition!

We could of delivered websites almost instantly by simply reusing various applications, and perhaps adding a few more and concentrating on just creating the look and feel while charging for development time as well.

Okay that's it for now. Sorry for the rambling post, but it's 6am.

What I'm getting at is that OACS is so far ahead of any other framework out there that it's funny! Any shortcomings already mentioned in this thread I don't see as significant at all. Just tell people about OACS and they will come, which brings up the last rather interesting point, would you like to know how I found about OACS and AOL server?

I had no clue it existed, I was evaluating opengroupware.org and via some misunderstanding of their documentation ( they as well reinvented their own application server yet again! ) i googled up AOL server, by which I found OACS, by which i discovered that it has a unified security model, by which I discovered .LRN, LORS, and a ton of other applications, all that I installed with a few simple commands.

Just tell them you exist godamnit!

Nick,

I'd love to hear more about your eportfolio project. Keep us updated please. I think the community features along with a flexible eportfolio that an add any sort of object into the portfolio would be very powerful.

It might tie into some of the work on communities of practice thats on HEAD as well.

Hi Guys,

I'll be keeping everyone posted about the portfolio package that I will be developing. I will start a new thread when I get things rolling. I have some tight deadlines, so hopefully I'll have a prototype up and running by April. I need a production ready application done by June. As we'll be getting students to use it in the second semester that starts in July.

I hope I can get the support of the community behind this project, as I know eportfolios are the latest must-haves in education and in HR. So there will be a need for this type of application.

Cheers,
Nick.

PS. I agree with the others in that OpenACS needs sex appeal. For starters, someone with expertise in designing icons should turn "Alex" into one of those MacOS X candy style icons.

Nicks right - it's time for Alex to go ...

I love Alex (and Karen). Even Apple booted Clarus the dog-cow (moof!) from the page layout dialog.

Getting more professional and sexy won't kill the familiar feel of our community, and it might just help protect it.

Try typing "https://openacs.org/wiki/";

Nick, thanks for the detailed post on the Sakai initiative, it confirms many of my suspicions. I would be very surprised to see them adopt a true open source model of development (regardless of license specifics) because it seems clear to me that quite a few people are staking at least parts of their careers on the success of this project ... and they're going to want CONTROL.

I wonder what would happen if we post a link to a summary of your comments to Slashdot? We had tons of fun when we publicized Ben Adida's "Why Not MySQL?" article on Slashdot a few years ago.

Richard ... our community is well aware of the fact that we're are own worst enemy, because we don't effectively promote our software or ourselves. That's not the only reason few folks know of us, there are people out there who turn their noses up at any non-Java solution, or any non-LAMP solution, etc. But we're so tiny that even if you subtract zealots from our population of potential users and customers, we only have to be successful in attracting new hackers to double or triple the size of the active development community ...

Now there have been some promotional activities, such as Reuben Lerner's articles in The Linux Journal. But not enough, not nearly enough ...

We have not been pushing the wiki since Dave want's to change the wiki formatting language from wikit to something else like markdown and I have been thinking vaguely about a mediawiki formatting plugin (mediawiki == wikipedia so there is a large corpus of test documents :) ).

We did not want to be saddled with a big conversion job.

Thanks for your interest Robert.

First let me state that while I know others who share my views, I also know that there are others who do not and all positions have merit!

You asked what size developments is OACS/TCL not suitable for? Well I would say that that is not quite the right question, because OACS is already much bigger than what TCL is traditionally suitable for. The fact that it is possible is due to a number of factors which I see as :

1. the quality of the original ArsDigita designers and developers;
2. pushing the object model onto the DB layer (which by itself turns out to be actually quite a good idea anyway);
3. the richness of the AOLServer API

Point 3 is the key to why we don't suffer quite as badly as others from TCLs major weakness IMO - lack of rich data structures. Think hashes in Perl or collections in Objective C and there is no corresponding TCL concept. Everything is a list of strings. And that leads to another weakness in the scoping / substitution / execution model (or lack of). Anything quoted in curly brackets {} is (normally) untouched. So for instance when you are building a complex set of parameters (eg. an ad_form spec) with nested {} lists, you can't (easily) insert variables to be substituted. In some cases that is such a pain that our framework kludges substitution for you - but by the time it happens you are in a different scope, and so the stack level needs to be played with to get the right values to substitute. But that turns out to ber not quite always what you want (see Jade's ad_form notes for examples).

Sorry if the latter part of that paragraph is too technical, but it's a bit late for me to find a clearer way of describing it.

As I said earlier though, it should be possible for us to support two or so languages calling our core API (although that has it's own perils).

Can others maybe refine or add to this? In the end, you can do anything with anything if you put your mind to it, but in my experience of large/complex projects, dealing with TCL starts to diminish the speed benefits that the OACS base gives you.

And you are right - this problem happens everywhere. The designers of NaviServer (now AOLServer) chose TCL because they were short on time, TCL was designed for embedding inside other systems and was threadsafe. None of other alternatives were, at the time, both of those. I'm sure if Ruby was around then or Perl was less monolithic they would have chosen otherwise.

In small projects, something like Java loses hands down to TCL (or any other agile-ish language). In larger projects TCL alone would lose, but in terms of OACS versus a Java framework, it would still be hard for a Java one to beat OACS - TCL aside. Perl or Ruby would be much closer if not better.

Where the crunch comes though, is maintenance. You can't (IMO) make a very large system cheap to maintain without a really clear, simple object method reflected in both the db and code layers - and without inheritance you can't make sure that logic is never duplicated. Additionally, without objects, rich structures or a real scoping model, complex TCL code will never be able to be fully independant at the abstraction layers - there is too much interdependancy of evaluation contexts.

The interesting thing that comes from that is that a lot of OACS work is effectively maintenance - re-tooling existing code to do something slightly different.

Anyway - these issues are, i'm sure, well discussed elsewhere. With funding it would be amazing to write a new API layer on top to allow an object language to interface with the existing toolkit, but I am not sure how that would happen without a sponsor. Anyone friends with some member of a Royal family? That's how R&D was funded in the old days ;)

TCL is an excellent fit for the vast majority of web pages. It would have been cool to see PyWX take off for that last 5% or so, though.

Thanks for those great replies!

I see what you guys are getting at.

If OO can handle the last 5%, what about the OO extensions to tcl? Could they be integrated/extend aol server and fulfill that last bit and resolve some of the above commentary? How about the Python extension for aolserver?

"In your defence software example, I would put money on the fact that the Ada developers were much more highly skilled than the (more commodity) C++ developers (although that is a complete guess)"

Yes, it's a complete guess - and wrong. Wrong guesses of this kind expose the religious bigotry that underlies many discussion about language and software engineering. C afficionadoes have dismissed the *many* efforts to quantify improvements in code quality when proper tools are used since, oh, the late 1970s. Speaking from personal experience (and as someone who's been a C programmer since 1974).

And, of course, the same argument from authority using a guess driven by belief can be used against any claim for the superiority of the OOPs paradigm. And vice-versa, too.

Which is why such arguments are worthless.

"But to get to the point of using the framework to solve problems that generally arise we encounter a learning curve of the framework. Although Openacs provides a certain degree of abstraction IMHO its not enough."

OpenACS is a far too much a grab bag of things. In some areas our abstractions aren't orthoganal therefore less useful than they should be (a bunch of things that only apply to the content repository, rather than all objects, for historical reasons, being the primary example).

This has more to do with a messy implementation and design history than the choice of implementation language. If a modern OO language had been used, we'd still have a system with a content repository built by aD folks on the west coast who made no effort to work with their peers at aD on the east cost, glued to the side of the ACS kernel developed by those east coast aD'ers.

That was due to poor software project (or software company) management ... not programming paradigm. Or, more fairly perhaps, it was simply a reflection of the fact that aD had a bunch of high-visibility clients demanding that complex projects get done in a short period of time.

We can't shed our history, though. That's why I fundamentally find these discussions not very interesting.

We'd make different choices if we were to start over ... but unless we're willing to start over, we're stuck with the history.

One example has to do with RDBMS support. If we were starting the project today, we'd ignore Oracle, I'm sure of it. Many of us would jettison Oracle today if it weren't for legacy users. It would greatly decrease our maintenance efforts and would simplify new development if we only had to develop for Postgres.

Yet, when we made the decision to support PG and Oracle equally when we took over the code base from aD, it was undoubtably the right decision. Malte had an early success story, being able to support a client's ACS 4.x packages on an early OpenACS core seamlessly, others had similar experiences. It gave our project legitimacy and led to a steady trickle of ex-aD'ers becoming part of our project.

But viewed from today's perspective ... man, it sucks having to support two RDBMS's. Imagine a toolkit without a single .XQL file!

History is like that.

Roc, Thanks you for supplying this reference. I found it very useful.

http://home.pacbell.net/ouster/scripting.html

I think it can be used in our marketing toolbox.

Jun,

I agree, I think the openacs site needs a new face lift. If you can get your designer friends to create a new site design for us, then that would be great!

Don, I can't deny that recursive descent parsers are another good model - I am too embedded in the Perl community to ignore that! Closures and parsers together are a powerful combination. Of course TCL doesn't support closures either...

Should we perhaps start summarizing the ideas brought up by this thread, and classify their difficulty and importance. For example, sexing up the layout (possibly with the skinning stuff) in the toolkit and the website would be high impact for relatively low effort - of course it requires designers. I believe the intention was to have the Collaboraid .LRN design put into .LRN 2.1 or 2.2 - maybe we should chase that dow because it's certainly an improvement and would look quite good on the Openacs.org site.

I have (very) limited time right now (two deadlines + a wedding ;) but I'm willing to make a first pass at it (summarising) or make a second pass over someone elses first pass.

Well if noone else does, I'll look at it when I'm back from my honeymoon (5 weeks from now).

Ever since John's post I want a "lickable" Look n feel :)

I guess I would be dissatisfied with Ruby/Python as well as frameworks based on them at some point of time 😉 as I am with TCL in Openacs land but in general I have been very impressed with what Openacs can do for me. At the end of the day I see it as cool toolkit which enables me to solve problems quite effectively.

My experiences with other frameworks made me really appreciate the openacs community , the focus and quality of the contributors.

This particular thing got me started to ask my programmer friends to take a look at openacs and its possibilities. The intial reaction was the "TCLing" feeling and the difficulty they faced firing up an instance without going through a lot of technical hazzles. My exploration is in the direction of making openacs interesting enough for increasing adoption among developers. Most need simple and quick way of demonstrating the development of an application like how Rails provides ( without having to go through the learning curve). I believe this is a very potent approach.

But I guess we are getting there partly with the installer.

Don, if PostgreSQL 8.0 had been released 4+ years ago I probably wouldn't be using Oracle at all, either. And, probably, if 8 or 9 years ago Illustra (PostgreSQL's uncle?) had worked half as well as PostgreSQL does now, Philip would never have even tried Oracle, and ACS and OpenACS never would have used it all!

But, fundamentally, why is supporting multiple databases so costly in OpenACS? And must it necessarily be so? (I suspect the answer is "mostly yes", but I am not at all sure.)

ACS 4.0 added the first hooks for multi-db support, but at the same time also made multi-db support harder by moving so much functionality into PL/SQL code. That seems rather ironic.

Lots of other projects claim to support every database under the sun, and even that doing so is easy. My impression however, is that most of those do so via dead-simple data models, and generally making trivial demands on the RDBMS. And thus presumably many of those applications pay an extremely heavy price for their "multi-RDBMS support", whether their authors realize that or not. (The cost is merely hidden under other accounts, like "can't scale" and "general suckiness".)

Considering alternate realities can be educational, sometimes. So, what I'm wondering about, is or was there some middle ground? Some way that, if you were hypothetically starting from scratch, you could get 80 to 90% of the RDBMS power goodness of OpenACS, while reducing the cost of multi-db support down to, say, 10% of what it is now?

Andrew, I wouldn't bury so much logic in PL/[pg]SQL for starters. In reality, we can auto-generate packages the include "new" and "delete" functions and as the dynamic types package grows (and gets moved into OpenACS proper as it should, as it becomes integrated better and better) I'm ready to propose we do so and that packages include nothing else.

Why keep "new" and "del"? Because some old code does more than a simple "create supertypeobject/insert params into my type specific table" operation (bad idea, IMO, but that's life). So for such types we need to call "new" so we might as well auto-generate "new" functions for new types.

Anyway ... over time our idea is to move more and more to high-level Tcl APIs for the declaration and manipulation of types.

One thing we won't do, and which will always make query support more "difficult" for us than for frameworks that bury logic in the application layer (generally the object layer) rather than use complex queries, is, well, get rid of those complex queries.

Because as you point out ... using the RDBMS properly as we do aids scalability. The simplistic approach of (say) building a container object containing a list of object info returned by a simple query, that is then filtered by some other set of application object is by its nature, I suspect, less efficient than writing a complex query that returns exactly what you want in the first place.

Tcl is excellent, not sucky. Or rather, IMNSHO, the only way anyone can legimately claim that Tcl sucks is by taking the stance - the supportable stance, in my opinion - that all programming languages suck, to greater or lesser degrees. And on that measure, Tcl sucks much less than most. Overall, I consider Tcl a major asset to OpenACS, not a hindrance.

I think this may largely be because Tcl does a lot of stuff well, and nothing really wrong. There are various useful things which it does not even attempt to do at all. (Perhaps closures, continuations, and macros as in Scheme. Certainly vector orientation, matrix math, and sohpisticated statistical libraries, as found in S and its open-source dialect and R. One could go on and on listing things like static type inference, etc.)

To use one example I am intimately familiar with, although S is a very useful language overall, it feels replete with small but obnoxious legacy design and implementation flaws. So even though I use S for things I would never want to use Tcl for, I would argue that Tcl is the better designed language overall. Apples and oranges there, but if you have to compare them, Tcl looks like the cleaner fruit to me.

A language which locks you into broken misdesigned features (as seems very common) is much worse than simply staying out of the way and giving you tools to extend the language as you need (Tcl does; most non-Lisp languages do not). Tcl seems to strike a good balance there.

I naively hope to live long enough to see programming languages which are an order of magnitude or more better than what we have now. (And demonstrated as such scientifically; not just via hand-wavy philosophizing like I'm doing here.) Maybe from those Haskell-using academics. Maybe atypical practitioners like Jean-Claude Wippler will come through with something. Or both. And Tcl, Python, Scheme, and Ruby will all seem more backward than Fortran 77 and COBOL seem today. I don't see that happening terribly soon, though.

Anyway, probably none of this is news to anyone here, so enough of my rambling. I just wanted to emphasize that technically anyway (ignoring marketing), Tcl is a large net asset.

Jonathon is probably wise in wanting to avoid a 'TCL sucks/TCL's ok' thread. Ulimately though, we need to decide on a way forward and hearing peoples gripes makes it possible to estimate the biggest issues and needs.

XOTcl is not such a bad idea, but the key is deciding on an approach that best deals with our limitiations and un-attractiveness to developers and implementing it. If it's XOTcl, then we need to implement ACS object creation and manipulation as XOTcl objects, and make a new Object based wrapper on top of form builder.

Looking at some XOTcl docs, it does give us a lot of what we need:

• Real class-based abstraction
• Mixins (aka. interfaces in Java)
• Object passing - which effectively deals with the lack of references and rich data structures in Tcl

If there were replacements for the core toolkit like forms, permissions & subsite, that accepted object passing, most developers would be saved from upvar without adding another language (although XOTcl could rightfully be called an additional language).

It's still unsexy however, and has no sizeable community. Which leads me to a question: by what criteria should we choose the way forward?

I think the sexiness requirement should be limited to the look & feel, while any developer environment choices should be guided by the following:

• Practicability - what benefits it would provide
• Pragmatism - how realistic it would be to implement in OpenACS
• Profile - how big and prominent is the community surrounding the language/technology

IMO Object Orientation wins all of those critera against the alternatives. Whether XOTcl, Perl, Ruby or another language fits those criteria better than any other is a more complex question.

If nothing else, this thread makes clear that many are not happy with the status quo.

Malte you are right that it is not really what should be in this thread, Maybe we should start a few parallel threads based on this discussion.

I would like to stress that all of my discussions surrounding the language are because this thread has identified that TCL and it's shortcomings are one of the barriers to getting wider developer involvement.

OO or another language does not address any of the other issues, such as visibility. But taking visibility, if we are to market ourselves to developers (or their managers) then we must solve the developer issues first to avoid a poor first impression.

On the subject of languages, your idea to use ns_java was exactly my idea that I mentioned earlier about ns_perl. I don't particularly like the Java way of doing things, but with luck it may be possible to enable ns_java and ns_perl at the same time (and my dislike of Java doesn't count as a choice criteria ;)

The catch 22 is that any other supported language would need to be documented and supported nearly as well as the TCL Api. Again with luck, api calls should be simple translations allowing the api-doc to show calling examples from all supported languages.

OTOH, if the people most excited about the toolkit are the ones without enough experience to recognize its shortcomings, shouldn't that be an indication that something is deeply wrong?

Wow.

This community is amazing.

I've learned a ton from this thread. Thank you for replies to my questions, those are enormously helpfull.

From my naiive point of view XOTcl sounds/reads like the cleanest way to add the OO capabilites to the framework. And I agree, the sexiness part really should focus on the look and feel aspects of the project. One of the things that really attracts me to OACS is that there aren't bindings for every language under the sun - while I am a huge fan of Python, after learning and reading about OACS/AOLServer, it is clear that Tcl strikes the right balance for the category that it's playing in (more or less). When I install OACS, I know I don't haveto worry about compiling modules, debugging configurations, tracking down esoteric bugs just because I want to run say a calendar application from the repository (as an example). One language, one server, all done pretty much right.

I think the greater consideration is thinking in terms of application "scale". I would think that comments about java callbacks would add the greatest ability to scale beyond the frameworks current scope. Things like Perl, Python, etc, simply do not seem to have a scope much greater than what Tcl/XOTcl would provide anyway (outside of maybe the huge number of libraries, yet still, no one in their right mind writes stuff in Perl and Python where Java is called for). It would seem to me that Perl and Python support would scale OACS laterally moreso than vertically, whereas Java would allow the platform to scale both laterally and vertically simultaneously. The only question that arises from the java proposition is, just how much scaling do you need - if you want something that extends OACS so far beyond what it was designed for, perhaps one would of taken that into account in the first place and started with Java in the first place?

Hmmm ... this topic is very quickly getting me in over my head technically. I guess just count my vote in toward XOTcl as a solution that appeals to me on aesthetic merrits in relationship to the big picture

As a side note, it would be neat if we had the ability on the site to break discussions like these into a workflow and start hammering out some details. Say something as simple as Volunteers, Proposal/Specification, Vote/Feedback (inclusive of forum), Project Subsite, Organization of Task ( why isn't Project-Open.com on here? ), Progress and Bug Tracking, Publish. Perhaps there is one that I'm not aware of?

*sigh* so much to do, so little time.

Some tiny notes to anyone considering look and feel proposals for OACS ( I am planning on jumping in as soon as I get the hang of creating simple applications here ):

- this needs to be an official project with a project manager, perhaps listed under the projects directory, perhaps somewhere else as a special project (can this site be a project it self perhaps in the same way .LRN is?)
- start small, no big changes.
- the layout should stay mostly the same for the time being.
- incremental layout changes are better than huge changes as they let everyone 'fine tune' the site rather than invest huge amounts of energy into a site upgrade and then haveto live with it for a year or two.
- the site should remain 100% wide
- start by changing the colours, css, and fine tuning some of the visual cues such as colours, sight lines, links, etc. first.
- an alpha/beta replica should be available for public viewing/comments
- all workfiles should be available in some repository (preferably using Gimp, yes it is good enough to replace PhotoShop), such as CVS/SVN so that anyone can pick up the work where it left off, and incrementally push the look and feel forward one small step at a time.
- keep the logo for now. a logo change is huge. people may not like alex, but then again, when Tux was proposed as a mascot for Linux everyone thought that was stupid too. If a logo change is needed, I think that should be handled properly and not rushed.

This will keep the project from being overwhelming, and allow for incremental upgrades.

Jonathon, Jeff & Harish all have good points. I would also like to point out that the whole fixation with OO starts with the fact that Tcl has no rich data structures. Whether OO is or is not the best or even a good way to program, programming without rich data structures is simply abominable.

Yers the funtionality of our toolkit rocks. But show any sesoned (non-TCL) developer the quirks of ad_form, and then show him the source and they will run away screaming.

No disrespect to ad_form intended - I love it so much I implemented it in Perl! But it just takes too much wizzardry to be called structured programming.

So as I suggested a few threads ago, maybe we should agree to call this thread closed, and appoint a few people to summarise the thread into a list of good & bad points about the OpenACS tookit & community. Then from that list we can agree on areas that are important to improve, and what strengths we should be marketing.

At that point, the actual approach to deal with each weakness becomes a seperate issue that can be dealt with by different groups.

Who agrees that the useful debate is just about done (we are, after all, getting a little cyclical in recent posts)?

ACS is the open source web application framework we extracted from Photo.net. When we built Photo.net we didn't know we were building the ACS at the same time, but that's exactly how it happened. Photo.net came first; the ACS was born from Photo.net. Photo.net was the divine chicken, the ACS was the egg.

Sorry, I don't see the point to my post. Nor yours ...

Sorry - I should have put in some text of my own.

I had no point - I just thought it was an interesting article, and contained discussions not irrelevant to this thread.

Well, hey, fame's great, fleeting or otherwise. I well remember the standing I had after co-authoring the first commercial Pascal compiler, which just happened to generate code efficient enough to make it usable for the creation of system software for the PDP-11. Our company was well-known, too. We did a little public back-patting, as well.

Yet, today, hardly anyone's even heard of my old company. In 10 years, hardly anyone will be aware of Ars Digita (and if they are it will probably be because the OpenACS project continues to live).

Maybe Ruby on Rails will stick. Best of luck to them. But the fact that they happen to be the fad du jour is in no way a predictor of the odds of their project or company having any stamina, of being more than a flash in the pan.

Remember when Java was going to kill OpenACS, like three years ago?

Caroline Meeks made a good point today - she loses bids to LAMP far more often than anything else ...

Don's PDP-11 Pascal compiler Well Don, Google does know, (perhaps surprising given the date), if only just barely:

"The story of TECO on the PDP-8 is convoluted. Russ Hamm implemented TECO under his OS8 (without a slash) system, and then gave a listing to Don Baccus at the Oregon Museum of Science and Industry (OMSI) who, along with Barry Smith ported it to PS/8. This was the beginning of what became Oregon Software, later famous for OMSI Pascal."

I'll take a happy average programmer over a disgruntled, frustrated master programmer any day.

Given that the best software engineers are documented to be one to two orders of magnitude more productive than your average programmer ... I'd say this statement is one of the most stupid ones I've seen in a long time.

Having worked with both happy average programmers and disgruntled, frustrated master programmers I think I would prefer the former too.

Also the numbers for team productivity I have seen say a 90th percentile team is about 4 times as productive as a 15th percentile team, and the 10x number was from the worst programmer to the best (and derived from a really small dataset) -- the number for average to best is lower...

Hi,

I've got stucked reading those debates about the language to choose for openACS. I think those debates are worthless. Lets see why.

Lets go back to the begining (long ago) and remember:

TCL is not your friend
======================
Do NOT try to do fancy stuff in Tcl and if you do, if it doesn't work quickly, you might want to DROP IT. Don't be stubborn with Tcl. It is not worth it, especially if you are just beginning to learn how to work with it. The language is fairly simple, and most of the power of OpenACS comes from the API, not the programming language."

The database is your friend
===========================
Both Oracle and Postgres are supported by OpenACS, and they both are very good databases.

Yes, SQL and Oracle (now PostgreSQL too) can be challenging to get to know. It can be frustrating to realize that you can do whatever you want to do in 3-10 lines of code, but that it'll take you a while to figure out what those few lines need to be. But once you do, you'll be able to do lots of the stuff you want to do using the database. Given how powerful an RDBMS like Oracle or PostgreSQL is, you can, and should, do as much as you can in the database. Be creative in coming up with queries that can get as much stuff out at once as you can, so you don't need to do multiple queries (use nested queries, joins, etc.) The database is fast and powerful compared to Tcl (or ANY OTHER LANGUAGE between the database and the Web server). Use that to your advantage.

==========

The funniest thing is that all of this information belongs to our own pages: https://openacs.org/education/psets/advice

How can we go forward if we do not even know our own history and documentation?.

Félix.

Jonathan, while your argument sounds fine in principle, it is so general that I'm not sure whether it really means anything or not in practice. What specific stuff was being forced into the database and thus killing scalability? And what would you recommend instead?

(And don't just say files for file storage, we all know that stuffing them into the RDBMS as LOBs is silly and always was silly. File vs. LOB is not a useful example of anything.)

I can think of some examples, btw, but they are fairly esoteric. E.g., I worked on a project which shoved real-time options pricing data (from an electronic feed; on only a small number of options however) into Oracle almost solely so that Oracle could fire triggers based on that data.

That actually worked adequately in that case, but handling a wider variety of options may have strained it severely, and larger numbers of (possibly user-defined...) triggers might well have collapsed it entirely. (The production system never saw such loads, unfortunately, and I don't remember what our scalability testing said about data feed and trigger-related loads, or after all these years, whether we even checked that at all.)

Note however, that the "stuff real-time options prices into Oracle and fire triggers" design decision had nothing at all to do with some sort of belief in an "everything in the database" principle. It was simply that Oracle's triggers were the only tool we knew of at the time for doing "active database" type stuff - and indeed, are unfortunately almost the only common such tool.

Oracle's Advanced Queuing software might (or might not) have been better than simply inserting prices into tables. And Jerry Asher later pointed out the massive superiority of the Rete algorithm for such things over Oracle's (naieve) trigger implementation. Stonebraker's Streambase stuff is no doubt also relevent.

So yes, if I was designing a similar major application all over again, I would definitely try to get the "hard active" data-driven aspects out of the (poorly implemented) RDBMS native triggers into some more efficient and more powerful system. (Perhaps I would even write a PostgreSQL module to do so.) Plus at least two other major architectural differences which I haven't mentioned here at all.

But somehow I doubt that's what you meant in your "everything in the database" comments...

ok so this weekend I'll have built all the binary rpms for the dependant software for multiple distros:

SLP 9.2
SLES 8,9
FC 2,3
RHEL 3,4
CentOS 3,4

Who do I contact/bribe to get the officall openacs repositories(apt4rpm/yum) set up?

Should this be on one of the actuall openacs.org machines?
If not, who is in charge of the DNS for pointing the appropiate host at the appropiate IP?

That brings me to the next question...
What should the repository host be called?
repo.openacs.org?
packages.openacs.org?
somethingelse.openacs.org?

Derek and Bart, these are great contributions! You've got my ear. How can I help you? I've started a new thread for this.

https://openacs.org/forums/message-view?message_id=278642

Never mind the negative crap? Are you kidding? Most of that negative "crap" is true, so why should we not mind it?
It's not like the bugs/problems are going away any time soon.

IMHO, it would be much easier for OACS developers to start over than to keep on fixing the never ending flow of bugs/problems. OACS is way too complicated to keep on developing and keep your sanity at the same time.

Hint to anybody new to OACS - not a single book exist for OACS and/or AOL server. There is a reason for that. If you are not a programmer/developer yourself, think twice before taking on OACS.

Just my 2 cents

http://philip.greenspun.com/panda/

Well Vic, in which case you have all of our blessings to go troll another open source project.

Meanwhile the rest of us will merrily go on using this wonderful, enterprise class framework to build applications, corporate intranets, services and businesses.

The lesson here to anyone new is that you have to pick the bright tool for the right job. If you don't have the skills to do so, hire someone who does have the skills to help you make the right choice. No single project can cover all bases. Sometimes OACS is the answer sometimes it isn't. For us it has saved us time and money (and sanity) no other framework can offer and we wouldn't work in any other framework. For you the calculation may be different.

Otherwise all you will end up is a troll on a forum of a project you don't understand looking like a fool.

Vic May,

How about three documents (User, Administrator, and Developer) starting from one url? https://openacs.org/xowiki/openacs-handbook Ignore the categories on the left side-bar if you want to get more of a document tree perspective.

cheers,

Torben

Vic,

It's true that those pages try to keep 1 topic per page, and are organized somewhat organically to fit various reader's needs.

It seems like you are wanting documentation in a traditional format. What about this? https://openacs.org/test-doc/

cheers,
Torben