Forum OpenACS Q&A: Will Dr. OpenACS survive? Or why I stopped worrying and learned to love the .LRN consortium?

Will OpenACS survive? Is the .LRN Consortium necessary to its survival?

Let me put forward some controversial propositions for your consideration.

1. In a trivial sense OpenACS will survive whether or not .LRN exists or succeeds. In the trivial sense, OpenACS will survive as a project of interest to a small group of enthusiasts and hobbyists and the occasional one-off organizational adopter of the software. That's where we currently are with OpenACS and .LRN. I don't mean to imply that the project is trivial for the enthusiasts and hobbyists. Far from it. Nor do I mean to belittle anyone or the value of the project. We would not have chosen it as the foundation for .LRN unless we strongly believed in the technical merits of the framework and the brilliance of the community. However, OpenACS remains trivial in terms of its standing in the landscape of open source projects (test: if we were to do a recognition poll at O'Reilly Open Source Conference, my bet is that less then five percent will have heard of OpenACS and perhaps only one in a hundred will have some inkling of what project specifics.). It's also trivial (in the sense of not worthy of serious consideration) from the perspective of enterprise adopters.

2. I predict that over the next eighteen months there will be a brutal industry consolidation of open source projects, dictated strictly on economics. Those left standing will fall into one of two categories, trivial or professional. Only a handful of open source projects will fall into the "professional" category. What will be some of the distinguish features of "professional" open source projects? It will need to play in the enterprise. For that issues such as Apache vs AOL Server are less important than quality, risk, transparency, and business model. Quality: people are now astute enough to know that not all open source projects are the same. How do we know that the software we are using or intend to use has a high degree of quality and can withstand installation in a mission critical enterprise? The notion of ".LRN-certified" components is one of the ways in which the consortium intends to address the quality issue. Risk: Organizations are increasingly concerned about legal liability and exposure whjile using open source software. The SCO case began it but we are also now under threat by the sorry state of patent laws, particularly in the U.S. GPL by itself is not enough for risk mitigation. The consortium is stepping up and beginning to address this topic. While we cannot provide indemnification, the consortium will be taking steps to diminish legal risks. Transparency: Organizations will not adopt OpenACS and .LRN in mission critical environments unless they know with confidence that the software release process is predictable and well managed. This ranges from everything to an accurate and up-to-date bugtracker to clear statements and status of active projects. Business Model: the project will not be viable---in the sense of gaining adoption and interest in the enterprise market---unless vendors and companies are able to realize profit and a business model around the software.

What does this mean for OpenACS and .LRN? For OpenACS it does not mean changing anything except for taking more interest and also participating fully in the .LRN consortium. The OpenACS technical community is healthy and well governed. I would also like to see more involvement though among the community members in the .LRN project. The challenge is greater for the .LRN community. .LRN users and institutions need to contribute and participate more in both the OpenACS and .LRN communities. We have to educate .LRN users that they need to give back more to the community: that's the spirit of open source but also required if we are going to survive.

Some opening thoughts and I welcome your comments and flames. :)

You highlight some key points: quality, risk, transparency, and business model

Quality: ..How do we know that the software we are using or intend to use has a high degree of quality and can withstand installation in a mission critical enterprise?

The development process needs to be explicitly described, including stating values and principles used in technical decision making, and why those are used --including how results are verified. The documentation is working towards that, but is only valuable if it gets used. The feedback loop needs to be optimized. Perhaps at the end of each page there should be a link to create a context specific comment in the bugtracker instead of general comments?

Risk: Organizations are increasingly concerned about legal liability and exposure while using open source software. ..the consortium will be taking steps to diminish legal risks.

Is there some sort of explicit statement that code contributors can agree to, to help minimize the appearance of risk?

Transparency: Organizations will not adopt OpenACS and .LRN in mission critical environments unless they know with confidence that the software release process is predictable and well managed. This ranges from everything to an accurate and up-to-date bugtracker to clear statements and status of active projects.

I believe there should be a list of questions or checks that should be used to verify project phases, such as when issuing new versions etc. I'm sure that Joel et al are addressing these now. These just need to be reviewable by others. Again, documentation can help with this.

Business Model: the project will not be viable---in the sense of gaining adoption and interest in the enterprise market---unless vendors and companies are able to realize profit and a business model around the software.

The collective interests of business are represented in the concerns that are voiced in the forums. Getting those into the process is part of quality initiatives and improving transparency of the projects. However, if OpenACS is to be represented by a nonprofit, then I believe the business model of OpenACS still needs to be addressed in the context of risk. There needs to be some kind of for-profit representation of OpenACS to minimize risk and integrate with the corporate culture.

I doubt an enterprise level forprofit company will ever seriously consider choosing a nonprofit's product over a profit's one in a bidding process, because nonprofits are not permitted to compete against forprofits in their main markets. There are legal and economic risks that the enterprise avoids by choosing a product from a forprofit.

I predict that over the next eighteen months there will be a brutal industry consolidation of open source projects
Why? What is your evidence for this? And what is special about the next 18 months, as opposed to the 18 months after that, or the previous 18 months?
I am with Andrew in that I don't really see what would drive a "brutal industry consolidation" (not to mention I don't see any mechanism for making it brutal given how most developers are associated with projects to begin with). Maybe if by brutal you mean that there is tremendous opportunity for growth in open source which some projects will miss out on because they don't make the right collective decisions about how to manage themselves then maybe I would agree.

I would be interested in hearing what form you think OpenACS involvement in .LRN would take. It's not entirely clear to me what "participating fully in the .LRN consortium" would mean for me as an OpenACS participant and as someone selling consulting and hosting services for OpenACS.

The issues of quality, risk, transparency, and business model exist now, regardless of Al's premise. Discussion on the chronology or merits of any predictions are secondary --whether they are based on "peak oil" calculations or some kind of religion inspired end of the world scenario.

The project processes need more attention to eek out the best OpenACS and dotLRN has to offer.

I agree with Jeff and Andrew above regarding "brutal consolidation". I can see the practical fallout of the SCO lawsuit - which has the fingerprints of MS help all over it - being a slowdown of adoption of Open Source software by the timid. But the consolidation argument doesn't necessarily follow from this at all.

"Organizations will not adopt OpenACS and .LRN in mission critical environments unless they know with confidence that the software release process is predictable and well managed. This ranges from everything to an accurate and up-to-date bugtracker to clear statements and status of active projects."

While I know you've been frustrated about our release process etc, the fact is we're slowly getting better at it (largely due to Joel). On the other hand ...

"I would also like to see more involvement though among the community members in the .LRN project."

The same argument in regard to slipping schedules, at times apparent poor coordination, and general organizational ineptness that can very validly be applied to the OpenACS community and project at various times in its history (including to some extent today) can also be applied to .LRN governance and marketing.

It is true there's less obvious enthusiasm for .LRN in the OpenACS community today than there was 18 months ago, I think. It's also true that the formation of the .LRN foundation/consortium was what, a year late?

If the OpenACS community does a better job of pushing out timely releases, fixing bugs, etc the technical future of the .LRN project will be much brighter.

If the .LRN consortium, now that it exists, does a better job of meeting promises made in a timely fashion I think you'll find enthusiasm within the OpenACS community for the project and participation in the consortium as advisors, gadflies or whatever will rise again to the levels we saw 18-24 months ago.

"The challenge is greater for the .LRN community. .LRN users and institutions need to contribute and participate more in both the OpenACS and .LRN communities. We have to educate .LRN users that they need to give back more to the community: that's the spirit of open source but also required if we are going to survive."

No disagreement here, but you can't force communities to grow, you can only provide an environment favorable to their growth, IMO. If there's anything we can do to be more welcoming I'm sure you'll find a lot of people willing to help out.

This is useful dialogue. Let me say that my intention was not to criticize OpenACS. On the contrary, I believe that OpenACS is better governed than .LRN and continues to make great strides in terms of governance and transparency. Having said that, the .LRN consortium is finally getting its act together. But also recognize that organizing a "user community" comprised of academics has its challenges.

I believe that Jeff's formulation is the best way of looking at this: "Maybe if by brutal you mean that there is tremendous opportunity for growth in open source which some projects will miss out on because they don't make the right collective decisions about how to manage themselves then maybe I would agree."

That's what I would like to focus the discussion on. How should we manage ourselves so that we fully take advantage of the opportunities for growth? The "we" is OpenACS and .LRN. There is potential for great synergy between OpenACS and .LRN. Let's havea a no holds barred and dialogue on what we need to do shore up the relationship so that we have a powerful and unbeatable partnership as we move forward.

What I would think is cool is that all the packages and technology (the code in both OpenACS and .LRN) is thought of as "OpenACS".

Some of the issue is historical - the code for the communities ending up in what we call ".LRN". Obviously, that is something that is of generic use for OpenACS. That and some installation issues, there because a "difference" in "running OpenACS" and "running .LRN". Then one thing led to another and the separate gets more and more engrained.

Ignoring these historical things, it really all is the SAME code and it's ALL OpenACS. .LRN is a given configuration and set of packages, it's a consortium, it's a group of users willing to spend time, energy and money to make OpenACS of good use for education.

I've been out pounding the streets for sales in the past couple of weeks and this works quite well:

  1. Describing OpenACS and it's benefits
  2. Explaining that .LRN is a configuration of OpenACS for education and a consortium
Keep it simple.
I want to offer elaboration on some of my points. My starting premise is that the battle of open source vs proprietary is over. It's now accepted within firms that open source software has a place: it's not whether (to use open source), but how and when.

But with this maturity and acceptance of open source, a new set of considerations have entered into the picture. These considerations were just nore relevant and worthy of consideration for open source projects as late as even a year ago. The maturity of "open source" means that we have entered the "market" completely and inexorably with all that it entails.

Now, let me come back to my distinction between trivial vs professional open source projects. I admit that it was a poor choice of words. The distinction I had in mind is the difference between a project that meets hobbyist's needs vs a a project that aspires to meet market demand. That's not to belittle hobbyists because there might be open source projects that are extremely innovative and push the envelope with new technology or approach. At the other end of the spectrum -- and it is a continuum I believe--might be other projects that not only beautifully fit a market demand but are able to *sustain* their competitive advantage.

One of my principal points is that the rules of the adoption has changed and will continue to change as open source becomes more and more mainstream. The threshold is much higher for adoption. The criteria that we (e.g. MIT Sloan) used four years ago for adopting open source is not the same criteria we would use today. For example, intellectual property and liability is a big issue for firms as they consider open source. If we aspire to play in the "market", then there needs to be a real story on how we are handling this issue. And that story can't be just, "hey we are GPL".

Moreover, the same types of pressures that firms face in trying to maintain competitive advantage also now confront open source projects. Let me quote Michael Porter (Harvard Business School): "Being 'all things to all people' is a recipe for strategic mediocrity and below average performance, because it often means that a firm has no competitive advantage at all."

Now, I am assuming that OpenACS aspires to meet a market demand. I know .LRN certainly does. There are essentially two ways in which firms maintain competitive advantage: low cost or differentiation. For OpenACs and .LRN it's probably not low cost. So, what is our differentiation and what is our strategy for maintaining that differentiation? What are we unique in that buyers value?

I don't want to imply by any means that we (".LRN") have the answers. I wish to suggest that these are some of the questions we should be asking and trying to answer jointly.

Let me come back to the "brutal competition" bit. The good news is that markets are looking to and are now open (no pun intended) to open source products filling the need. The bad news is that we are in a market, which means that we are faced with competition. And, therefore, survival.

Will OpenACS survive as an activity for hobbyists. If that's what the community aspires to, then it's a good bet that it will continue to survive as an obscure and relatively unknown project. If, however, OpenACS aspires to fill a market need, then the jury is still out. But if we find ways to strengthen the partnership between OpenACS and .LRN, while finding our true differentiation, then I am confident that we will have a very combination to play in the market.

Don: "I can see the practical fallout of the SCO lawsuit - which has the fingerprints of MS help all over it - being a slowdown of adoption of Open Source software by the timid. But the consolidation argument doesn't necessarily follow from this at all."

My argument about consolidation has nothing to do with the SCO lawsuit or FUD. I am claiming, on the contrary, that it follows from maturity and success of open source, namely that it's now an accepted market phenomenon. Markets don't tolerate an infinite array of products in a particular space. Very quickly a handful of incumbents emerge.

Andrew: "Why? What is your evidence for this? And what is special about the next 18 months, as opposed to the 18 months after that, or the previous 18 months?"

Yes. 18 months is arbitrary. But it's not 18 years.

Here's some irony. There is product placement from a competitor of dotLRN's in this apropos article about team morale: Morale of the Story[1].

1. http://www.fastcompany.com/magazine/92/open_playbook.html

Well, as far as differentiation goes, that question is probably best asked of the OpenACS community members who have investigated alternative tools in-depth, in order to learn what the competitors do well and poorly, and thus perhaps shed light on how to further improve and differentiate OpenACS.

We have at least one active contributor (Jun? I forget.) with extensive Red Hat CCM experience, and probably a whole bunch more (e.g.) who have between them either evaluated or used for real one or two dozen of other systems as well. And hasn't Lars been doing lots of work with Ruby on Rails lately? What's the story there?

Perhaps competitive market analysis simply means getting all those OpenACS folks together in one room for a day. Preferably with beer, and if they have enough interest, preferably with preparatory work by each participant beforehand as well.

I had an enlightening conversation this afternoon with a member of the OCT. He/She reminded me tactfully that for this community actions speak louder than words.

My take away from the conversation was: "The leadership of .LRN needs to demonstrate consistently that it is contributing to the community in positive ways. Establish a track record with that first and then we just might be more receptive to your ideas when you start pontificating."

Please allow me to add a thought to the discussion here:

OpenACS has its roots in a university and grew due to market demands. Now the market shifted from the technology perspective to LAMP, J2EE but OpenACS cannot simply do the same. Now from what I see an active and lively community demonstrate this ability by growth either by the number of active developers in total or by the number of freshmen it attracts to fill the shoes of those who left. Which product of OpenACS can serve best to fullfill this task?

I believe it is .LRN. No other product have we at present that can reach so many young students and even professors who could all become devoted active community members and improve this toolkit. So basically OpenACS has now the great chance to be brought back to where it came from: the university.

Sadly, Nima, not many computer science departments are using .LRN at the moment, in contrast to the time when OpenACS began it's roots. And those using .LRN are not teaching courses on it (within the curriculum). Therefore the major drive we had when Philip was teaching 6916 is gone and needs to be revived for your idea to blossom. But i totally agree with it, though I'm not enthusiastic about the chances for success, but hopefully others can correct me.
I think there are some issues with making the primary focus for all OpenACS activities be dotLRN. For one thing it's very big and very complicated hence much harder to develop for than simply writing packages on acs-core, it's quite rigid in the site structure which makes it unsuited to things outside the course management/education market.

I personally think the way forward is to take it out of the university and the reason I think that is that the innovative things I think draw people in are not coming from dotLRN. I think things like Lors, the assessment package, etc are important to the success of dotlrn in the "Enterprise Level University Service" market but your run of the mill student just isn't going to care. They are not going to set up a website to run their own little university or deliver learning objects but they might set up a website to run their blog and maybe a photo album and other things, but it might get to the point where OpenACS is really a high-octane alternative to gallery and moveable type at which point I think we would be more attractive to people getting started.

Malte - I agree that there is no reason for enthusiasm but I don't see any other potential ground for new developers than from universities either.

Jeff- I disagree. You say

"I think things like Lors, the assessment package, etc are important to the success of dotlrn in the "Enterprise Level University Service" market but your run of the mill student just isn't going to care."

but right here is the proof that assessment, lors and many more package were only possible because universities are getting involved. Neither assessment nor lors depend on dotLRN.

Also I don't target normal students. Universities that go beyond simply installing the system and feel committed are my aim. And these packages were developed to a great extend by folks from universities who have entered the community now.

It's not about dotLRN in the first place! I have enough experience here in Mannheim to say that OpenACS is what dotLRN is all about. Students learn a great deal about web based development developing with OpenACS. A package is an OpenACS package in the first place and later on integrated with dotLRN. But dotLRN is at the end of the day the platform where all these activities at the university level are brought together and show fruits.

Nima, my point is if you want real growth in the developer base you need to cater to the people who want blogs, photo-albums, and other personal things. That pool of developers is many many orders of magnatude larger than the pool of people who want lors or assessment.
This is an enlightening discussion and am learning a great deal. Will comment on recent posts soon.

Good firms manage the transition from one generation of products to another, concurrently adjusting the organization and processes.

In a way, this discussion is about the next generation of OpenACS and .LRN as products. We don't have to reach for solutions just yet. It's helpful to understand the landscape and where people think we are and where we should be going.

Jeff and Nima's lines of thought are not necessarily orthogonal.

One of Jeff's point, I believe, is that the principal driver of innovation in open source is developers (students or otherwise) wanting to "scratch their own itch". Cool stuff is more likely to flow from a developer wanting to do something different with blogs or photos than programming from pre-defined specifications for a new LMS feature, no matter how important and valuable that might be for the relevant institution.

As an open source technical community OpenACS needs to continue to be attractive and inviting to developers so that they feel that they can do "cool" things. Lars left the community partially I suspect for this very reason. In his opinion, Ruby is a cooler and more flexible development environment.

The knock on OpenACS is that it's not mainstream. I think that's less important than the fact that OpenACS has the perception of being rather obscure ("medieval scholastics" is how one developer described it) and not terribly inviting to newbies. We compound the perception of obscurity by lack of visibility, not participating in the larger open source community. Perhaps this stems from an ultra macho attitude that only "real men" do OpenACS and its inaccessibility and obscurity is part of its strength. Let me enter as evidence the fact that for the fourth year in a row there will not be a presentation or even a presence at O'Reilly conference. If you are not at O'Reilly, you don't exist. It's as simple as that.

So, this is a challenge for OpenACS independently of .LRN. What is about the framework that should be attractive for developers? What can be done or needs to be done to make it more attractive? What can be done to make the cost of entry lower?

If the OpenACS web site had a sign "Developers Welcome, Enter, Come Here!", the question that needs to be answered is "Why". Why, compared to other frameworks? Why is it worth my while to join the OpenACS community? What can I do with it that I can't do with other frameworks? And how do I go about doing it?

I went to a talk by Rasmus Lerdof several years ago at O'Reilly. He was stunned by the success of PHP. Surprisingly, he attributed most of the success to the documentation associated with PHP. His paramount goal was to make it as easy as possible for newbies to pick up the technology and quickly join the community.

I'm not sure I buy these ad hoc, arbitrary marketing insights about Open Source. "Itch scratching", "consolidation" etc are just buzzwords. I'm sure that emacs and GCC would be surprised about not existing because their not at Oreilly. They don't show up because they realize that Oreilly is pretty much a clique that needs an excuse to spend $1500 a year.

The truth of why people are leaving the community is not because technical inflexibility and obscurity - that's been true since the beginning of the project. We've always been dealing with the fact we're not OO, we don't use a P-lang, we don't use Apache and we don't use MySQL. Nothing new with that.

The reason that people are leaving the community is because the promise that it held in the form of .LRN has been totally unfulfilled.

There are many ways to show how .LRN has botched the lead it had on Sakai but we're all familiar with that. The question is whether anything can be regained either in attractive new developers (which I think is unlikely) or attracting new customers.

Focusing on attracting new developers is a waste of time. We have a technology that no one is terribly excited about putting on their resume (tcl and AOLserver). It doesn't play well in heterogeneous environments. And the learning curve is such that very little previous knowledge can be applied other than general database and programming experience.

These things can be improved in order to attract new developers, but it requires that things like the RAD facilities are improved, the object systems, merged, etc. That's apparently not going to happen anytime soon given the resources we currently have. Expecting that we can attract new resources that will take up these hard problems is wishful thinking.

Attracting new customers, though, may be possible and that is only through marketing. Marketing, though, is the major competence that is lacking in the OpenACS and .LRN communities. While OpenACS doesn't need marketing competence - we are a developer community, if we had it, great, if we don't we'll survive - .LRN's incompetence in this area is both discouraging and mind boggling.

It certainly doesn't need to be that way. .LRN has been around for 4 or 5 years or so and has a fairly impressive roster of users. But it's leadership has been disorganized in gathering momentum and shown a lack of creativity in managing to do the most important thing to make the project effective - raise money.

That is fundamentally the important issue here. If .LRN doesn't raise at least $1M in short time, it is doomed. That money should go to marketing itself better and getting out of the obscurity in which it languishes. Once that happens, .LRN should hire developers to document the system and and make RAD more accessible.

But the chances of that happening are increasingly slim. So why should anyone want to work with a group that can't pull itself together after 5 years of existence?

talli

I agree with Talli, it is a waste of time marketing to developers, it's needed a "serious and consistent" marketing effort for .LRN, that should take place in the short time. By serious, isn't just a few random hours of someone trying to do something about it, in the other side we don't need an entire army, just a continuos and dedicated effort of someone who really understand what's going on with the .LRN market segment. And by consistent, is that we need to have a great image, and not just good persons, for instance the dotlrn.org website is very poor for marketing itself.

Can we have .LRN marketing itself continuously and loudly in its segment?

I love Talli like a brother so I didn't mind saying that his comments are not only off the mark but offensive. I don't mind accepting criticism but have no patience with people who are only looking to cast blame.

I did some calculations last night. .LRN institutions have invested at least $1.5 million of hard dollars in OpenACS. If you count soft dollars, which is much harder to measure, that investment amounts to at least another $1 million. That's not a trivial investment.

Going into open source at the time we did was a risky decision. Give us some credit for that. The leadership of .LRN is also not being paid by its institutions to participate in the community, which means that we are also volunteering our time and this ends up being our second and third jobs.

We are not perfect. We are not omnipotent. Have we have contributed to OpenACS. Yes. Could we do more? Yes. Could we do better? Yes. But we have nothing to apologize for, least to Talli.

So, Talli: Go ____ yourself.

My post was certainly provocative and may very well be becuase I'm an asshole, but it's not because I'm wrong.

And I'm not asking for an apology from anyone at .LRN. But I am saying publicly what I've been saying privately for two years now and what others are also saying.

The investment that .LRN institutions have made have not been trivial, that's certainly true. But the effectiveness of the organizing of the .LRN project has been abysmal.

Some people have done well in the community and it may just be that the project can only sustain those. But if .LRN's problems are entirely related to resources and marketing then what good is it pontificating about what OpenACS can do about it?

talli

How about some data:
   Month | Users |    Lines
 2003-01 |    12 |    90841
 2003-02 |    10 |    24452
 2003-03 |    11 |    85862
 2003-04 |    10 |    14755
 2003-05 |    15 |    25375
 2003-06 |    18 |    20226
 2003-07 |    14 |    23313
 2003-08 |    15 |    65943
 2003-09 |    19 |    53634
 2003-10 |    21 |    43687
 2003-11 |    23 |    78636
 2003-12 |    19 |   165153
 2004-01 |    14 |    40731
 2004-02 |    18 |    41652
 2004-03 |    25 |    48118
 2004-04 |    19 |    12471
 2004-05 |    23 |    26610
 2004-06 |    22 |    34546
 2004-07 |    24 |    21309
 2004-08 |    20 |    10683
 2004-09 |    18 |    14363
 2004-10 |    23 |   105241
 2004-11 |    21 |    18463
 2004-12 |    25 |    24756
 2005-01 |    23 |    22949
 2005-02 |    21 |    19417
This is the number of unique committers to openacs.org and total lines changed by month. There isn't really anything it that data that says there is some crisis with retaining developers. Otoh, it does not show much growth either.

I don't think "marketing to developers" is that important although I do think making openacs install smoothly and have a shallow lead in to get people started working would make a difference in developer adoption.

I would like to see dotLRN spending it's money differently but I think the investment has been critical. The code coming out of e-Lane and other dotLRN project has been useful and the work on release/QA from Sloan has helped as well.

What I would like to see is more direct investment by dotLRN in QA/Release management. One problem is that Joel has gotten busy and he is *really good* at all the things most of the rest of us are bad at. I have said it before -- finding money to pay Joel to do QA/Release management would have tremendous ROI.

Another issue is that e-Lane is producing a lot of code but I don't see anything like the level of involvement in general quality improvements coming from them. I think in part that's a level of experience issue but it's something that the dotLRN and e-Lane members should consider.

I know it's hard to allocate Al's "hard dollars" to things other than projects with deliverables but to address the "Quality" issue the consortium is going to have to find a way to do just that.

Yes QA/Release management is crucial in any project, and .LRN is aware of that. Of course, Joel is the right person, but while we don't have a paid person to handle that, .LRN will relay its product quality on random volunteering efforts or will never expand its functionality base.

BTW, the new version of the graphical installer is scheduled for the next week, and will be maintained for the next 2 years by the University of Reading, UK.

Roc,

It's not a binary choice of either qa or expanded functionality. As long as dotLRN relies on "random
volunteering efforts" dotLRN quality will remain as
random.

I also think that given a choice between higher quality
and more functionality at this point higher quality
is the more relevant.

After months of working with ACS 4.x I recently picked up OpenACS at `stable' version 5.1.4. As some might know I've been an active member of the OpenACS community for several years now and as such am quite familiar with the internals of OpenACS.

Even installation of OpenACS core required patches to run on PostgreSQL 7.3, which by OpenACS's policy should be a supported version.

After installation, tsearch2_driver required a patch to work with PG 7.3. Lars-blogger fails when setting up blogs for several users. You might say that these are small problems, but I found them very discouraging. Compounded with administration navigation that appears to be designed without rhyme or reason was enough of an incentive to look for a different web development framework.

Not that these specific bugs/gripes are so bad that I feel compelled to look around. No, it is the fact that after all these years OpenACS still produces questionable releases, has outdated documentation, lacks comprehensive support for SOAP, relies on a legacy ecommerce solution with a fair share of warts (no offence Torben) and has trouble supporting even 2 databases.

Things I'll miss most about OpenACS? AOLserver, tDOM and the templating system.

I haven't forsaken OpenACS, but OpenACS is losing its appeal. And in that respect I'm with Talli's comments on .LRN. As a developer it also makes economic sense to look at more mainstream tools/languages, especially when those tools solve the same problems just as well.

/Bart

Jeff, I agree, so maybe we can start a paypal-donate to have someone to perform at least partially the job in a periodic way.
I want to apologize to the community for losing my temper and my showing disrespect towards Talli. I was learning from the discussion until things degenerated into name calling and personal attacks.

I remain committed to the community, but I no longer plan to post in public except if it is to provide information. I am open to suggestions, comments, and criticism at any time in private.

For those of you who are unsure of a constructive suggestion, here is an example from Jeff:

"What I would like to see is more direct investment by dotLRN in QA/Release management. One problem is that Joel has gotten busy and he is *really good* at all the things most of the rest of us are bad at. I have said it before -- finding money to pay Joel to do QA/Release management would have tremendous ROI."

If you have similar suggestions, please let me know offline. I appreciate your feedback.

In the spirit of being constructive and also being true to my thoughts, here are some (hopefully not too rambling) thoughts and ideas:

Al's point about consolidation strikes a chord with me. It is a hunch for sure (intuition sounds better), but it feels true - and many of us will have seen the pattern in other industries.

What Talli and Bart have had to say strikes a chord with me also (very musical today...).

Taking into account that I have been using ACS (on and off) since beta versions of Tcl ACS 4, I know ACS quite well. I'm not scared by upvar or form widgets and I positively love writing pl/pgsql and tuning queries. I love the package manager and see more benefits than disadvantages in the monolithic structure of the system.

OpenACS as an overall system however, is old. The lack of true Object Orientation in it's design makes (non-destructive) customisation hard and core development slow. The lack of rich data structures (or objects) makes Tcl unsatisfactory (although OO Tcl extenstions may solve this weakness).

The flip sides are the body of code and the community. With the exception of good OO modelling, all areas of my professional personality have grown due to interaction with members of the OACS community (and the community itself). Possibly more than due to any other IT professional experience.

The body of code can also not be ignored. It is simply to good to throw away.

Now here comes the constructive bit! If we can let ourselves admit that there are key flaws in the toolkit, then perhaps we can get to a point where we then realise that we can deal with those weaknesses without throwing it all away.

The question should become "what are our strengths - what is it that we are so reluctant to throw away". Well there is certainly the core object model with it's permissioning and relationships ... there's the package and subsite management ...

Once we see what the strengths are, we can focus on how to provide them in a way that people will want to use. Perhaps we could write wrapper APIs for them in another language (like, say, Perl which is supported by an AOLserver module, or maybe Ruby). Design a module template and developer quickstart kit for people familiar with that language - this would allow them to quickly get a server up, and start creating objects and using the api in their package using all the tools of that language.

This is just one idea, obviously, but if we don't do something radical like this, then (my hunch is that) the OpenACS project will lose a lot of it's headline developers and with them a major reason for the quality of the community.

I truly believe that developers are the key to core growth of a project like ours. It doesn't matter how much a manager might like something - if their in-house developers reject it or they cannot find enough developers to work on it, it becomes a non-option.

On the other hand, if developers begin to love it, it will work it's way into production by osmosis - then the value and productivity will be seen by management. That principle is why Perl is still among the most common languages used by investment banks. It's unsexy and very old-school - but you just can't argue with the productivity it offers when you see perl developers working alongside java developers.

A problem is certainly that we are not a company. There is no central provider of funding who is interested in making sure that OpenACS doesn't become marginal. Implementing an idea like this or one more radical would take some time. It's a real pity that the time and money spent on Java ACS wasn't spent more wisely, but I'm sure there's a thread about that already... ;)

PS: Which kind of doctor is Dr. OpenACS??
PPS: Thanks to Al and others for the courage and time taken in starting and contributing to this useful discussion

Jeff - Thanks for the data on users and lines. Put a linear or even logarithmic curve on that and you will see that even though the number of committers grew the number of lines are falling [1]. It would be interesting to see who these new committers are and where they are coming from. My guess is from universities!

What I learn from the data is, that OpenACS is becoming less attractive to developers with each day. In the past a bunch of active developers committed alot. Now many of those are leaving or less active and are accompanied by many who commit less.

I believe that "marketing developers" is very important. If the toolkit is attractive then developers are attracted to implement projects they are offered with this toolkit. If it is not they will choose another toolkit. So either the toolkit is sexy or OpenACS will decline. Marketing developers is from my understanding to offer them a great toolkit that is superior enough to compete with mainstream technologies. So what we need to offer is what has been said before:

- Quality and release management. OpenACS is almost great on that. I hope that .LRN will improve as well. We have problems with regular releases, communication of the release status, upgrade scripts.

- Installation. Windows is very important. I know some folks prefer linux over windows. But windows has a huge market. Noone really questions Oracle versus Postgres because Oracle has its market. We should find a way to support windows and linux without any patches and long installlation procedures.

- Documentation. A developer is attracted by a good documentation that shows him how easily he can implement a package and where to find the API. We need a well documented and consistent API. I know the gurus now say "documentation is for the weak". This doesn't help. It's like going to the Dr. (OpenACS) and ask for medication and get the response "darwin's law baby...be strong or die". No! We need a documentation that allows a user to write his own package in 30 min. As a newbie I find it extremely hard to understand how I can define objects in OpenACS. Maybe OAK CASE [2] or the upcoming package builder will close that gap. The templating system is great but not completely documented I guess.

Anyway. It's interesting that this discussion is going on now for at least 2 years. Which is a sign to me that business opportunities for developers with OpenACS and .LRN is decreasing. :(

Ideas?

[1] http://madura.bwl.uni-mannheim.de:8080/data.jpg
[2] http://www.weg.ee.usyd.edu.au/people/laetitia/

I’m a doctor and learned much of what I know about web programming through openACS (So maybe I’m one Dr. openACS :) ), but without .LRN I’m sure I would have lost interest by now. Why, because .LRN held the promise of the future and I hope it still does.

It is not my primary business so I don’t know how other communities work. But as I have stated before, it is my opinion that we should look into the community first. Although, there doesn’t seem to be any support for that, I still felt I had to say it one more time.

Suppose you would buy software from a company that would sell you this toolkit for $20,- , but without support (even a forum), and given the registered amount of bugs in the toolkit, would you buy if you had a goal other than programming you own custom system?

Bugs currently drive even talented programmers away, eventhough they are familiar with openACS.
What if you are the leader of a company with 8000 part-time employees what would you do if things didn’t work out? Hire a new set of full-time developers and not care why the 8000 aren’t productive (anymore)? Would it be useful to have a roadmap or to empty the bugtracker ? I’m not sure, but I feel management is needed more to keep developers than to gain new developers. Investing to keep customers is usually cheaper than to gain customers.

Nima, I think that is the wrong inference to draw from that data, I think the number of lines changing declining over time has more to do with the toolkit maturing than it does with declining interest. I would hope that it would continue to decline while the number of participants increases. I don't think it should be construed negatively at all, I think it's a sign of health. Also, those counts are lines changed so exclude the large imports from the e-Lane work.
Interesting. I thought this includes new packages as well. Which packages are included? Only the core packages?
All packages are included but the line count from cvs add
is not (cvs does not show lines changed for an add).
Just a little comment - I regretfully agree with Nima that windows support would be a big boost.

It could also bring less talented developers and decrease the quality of the forums. Controversial but true - a larger user base would most likely mean that the average skill level would decrease. The average OpenACS developer is definately above the industry average in terms of skill level.

And it may not be so hard now that postgres 8 natively supports windows.

A sad fact is that building a windows specific binary installer would be far simpler than a linux one.

It seems like a no-brainer for me, but our community has limited windows developer skills.

Heck - it can't be that hard. Make the installer, get an articl or two in major developer magazines or even better - get on the cover CD. Small business intranet in a box? OpenACS has it all.

Add some API enhancement like my previous post and I think we would be very suprised at the outcome.

I think the big roadblock for windows was always PG support
and now that that is in much better shape I think it
would be a great idea to get OpenACS working there.

If someone buys me a dual opteron box running windows
I would be happy to spend some time making it work :)

Exactly my point - most OACS developers probably don't have a windows box - I don't!
"If someone buys me a dual opteron box running windows
I would be happy to spend some time making it work :)"

Done. Go for it.

I have printed and framed Mark's posting, especially the comment:

"I truly believe that developers are the key to core growth of a project like ours."

That is the core strength of the community and we need to find every means to leverage it.

Mark, same offer if you go after the binary installer.
Nima:

OpenACS has a huge amount of documentation. All APIs are documented at https://openacs.org/api-doc/ and the same URL within any installation.

Consistent APIs are an ongoing challange, so we are working on it.

"As a newbie I find it extremely hard to understand how I can define objects in OpenACS." This I find very hard to believe. Besides the huge amount of existing example code there is the tutorial: https://openacs.org/doc/openacs-5-1/tutorial-database.html and the lower-level, but extremely detailed https://openacs.org/doc/openacs-5-1/object-system-design.html

Please if you are looking for answers to a question, and can't find it in the documentation, please ask on the forums. If it is hard to find something, please let us know. Anyone who can help out with documentation is welcome to.

While on the subject of documentation one place I think we need to streamline the existing documentation is on the install. It has been said that the install documentation shows how to build a production ready environment. That should be a seperate advanced document. A goal should be to have a one page install document that gets a minimum system running. With many links to more advanced topics.

Hey! I have a windows box. But I don't have an ibook G4. Would your offer also include that? If yes, count on me, I will create the windows installer!
I am going to respond to Bart's concerns. I know he is not the only one who has these concerns. My greatest problem with these type of complaints (and not Bart expressing them in particular) is that they only come out in a greater discussion of marketing, or why OpenACS doesn't have more developers. Why aren't these issues brought up individually in the forums, or as bug reports?

"Even installation of OpenACS core required patches to run on PostgreSQL 7.3, which by OpenACS's policy should be a supported version."

Interesting. Are these problems documented somewhere so that the community can act on them?

"After installation, tsearch2_driver required a patch to work with PG 7.3."
A patch to what? To tsearch2, postgresql, or the tsearch2-driver package. Its a new package, so it might have problems. Was a bug report posted?

"Lars-blogger fails when setting up blogs for several users."
Is this in the bugtracker?

"Compounded with administration navigation that appears to be designed without rhyme or reason was enough of an incentive to look for a different web development framework."

Was there a discussion about admin iterface problems? As far as I know, most people thought the interface was improved, but certainly had room for more improvement. Did you propose and ways the interface could be improved?

"...it is the fact that after all these years OpenACS still produces questionable releases, has outdated documentation, lacks comprehensive support for SOAP" Was there a volunteer to work on this?

"... relies on a legacy ecommerce solution with a fair share of warts (no offence Torben)" Was there a volunteer to write a new one? It is my understanding that people are using ecommerce in real web sites, so the perceived problems are not so great to stop anyone using it. Certainly it has not improved at the same rate as Amazon.com.

I encourage anyone to ask questions, post problems, and make suggestions! If necessary write a proposal, design a mockup for a new interface, write code. We have a pretty much open CVS repository now. Anyone who wants commit has been granted it. We know how to reverse commits if there is a problem. If anyone needs help in writing new documentation and getting it intergrated with the docbook version is offered assistance.

I am a web developer who has experience with PHP, PostgreSQL, and Apache. I have been reading the OpenACS forums for the last year and a half.  Below is my assessment of the good and bad of OpenACS.

Bad:

1. OpenACS is hard to install.  I tried to install openacs about 10 times and have been successful only 2 or 3 times.  Not only does openacs use non-mainstream technology, but it's hard to install. This is analogous to even when you are able to get a customer in the store (rare), you can't make the sale.

I tried installing everything from source and even tried installing the debian dotLRN package.  Installing from source has so many steps- it's too complicated.  One thing wrong and openacs doesn't work.  With the debian dotlrn package, eventually I got to a point where I needed SSL to work with AOLserver.  I tried to generate a certificate.  I couldn't get it to work.

2.  OpenACS is dependent on AOLserver.  Why can't openacs use another web server like apache?  I don't understand the openacs\aolserver dependency, but I guess it's the ability of aolserver to cache tcl scripts.  Why can't openacs use a memory cache that's not tied to a web server (e.g. memcached.)

3.  TCL is not a popular language anymore.  It's not worth it for me to learn tcl, just to develop for aolserver.  So even if openacs was easy to install and worked with apache, I still wouldn't develop for openacs.  I would install openacs and use it, but I  wouldn't develop for it.

Good

1.  The database schema is very clean and solves important web application problems in a generic way - base objects, users, permissions, and tree queries.  I have looked at the PostgreSQL sql files extensively.  I like the consistency of how the tables, columns, indexes, and stored procedures are named.  I recommend to anyone who wants to learn plpgsql to look at the openacs source code.

2.  I like that there are many packages available and they are all structured in the same way- work with the acs-kernel package and have a consistent directory structure.  This is the man reason to use a CMS.  The PHP world has some nice applications, but each solves generic problems over and over in a different way.  Most PHP CMS software that does exist has undecipherable code.

3.  I like that openacs accepts and maintains packages from individuals.  In contrast, the zope world has tons of packages, but each is maintain by an individual, various greatly in quality, and eventually breaks with new installations.

4. I like the openacs community- friendly, knowledgeable, and transparent.  I like the public available TIPS, forums, and IRC logs.  I read all of them to learn about generic web application and database issues.

George Essig

OpenACS

I am not a doctor yet 😊 but i have some thought about OpenACS i want to share with all of you that have been taking this great conversation in this forum keeping the flames public available. We can learn with this kind of discussion too if we could deal with it in a positive manner.

Why I came to OpenACS and why I love it and why I want this community survive? It is just because i perceive the value of this community-driven software development as a template for education. Not just for my personal education but also for other who want to involve with opensource.

As a template for education OpenACS survival is almost entirely dependent on the participation of the community that supports their efforts, regardless of their specific association with the project. Other reasons like we don't do Q&A, market analysis, or use focus groups seems to me less important for OpenACS survival than create group processes into this community to Build Good Software and Education

Rob Reynolds wrote once a interesting article for Xplana Learning where he suggested there are the elements that go into the making of good software and education. They are here:

* Abundant dialog -- the conversations occur and proliferate at every level, not just top-down. There are active, grass roots discussions that are just as important as the formal conversations taking place;

* Quality listening -- since information sharing and product evolution are the goals, listening is the most important element in every stage. Every detail can be important and people stop talking hen they do not feel they have been heard;

* True positive feedback -- positive feedback in the sense of a positive feedback loop -- information that actually causes a change in or modification to the system;

* Openness to real change -- positive feedback is a design feature that means the process/system is open to change, in fact, that it is expecting change to occur;

* Open-ended inquiry -- a driving force in quality developments is the asking of questions that have no set or pre-determined answer.

Maybe these elements could help us to create a group process that is dependent on asking questions, allowing new information to alter previous plans, and listening to the input of others. We have tools for enhance this kind of group process (public available TIPS, forums, IRC logs, maybe we could explore Survey). We have community-friendly for this (knowledgeable, transparent). If we see just only this thread we have great positive feedback 😊

.LRN

I really think .LRN is a little different from OpenACS. The most important differece i think is that .LRN is the unique expressions not only of OpenACS community or .LRN Consortium, but also and much more of their Users. .LRN competitors like Moodle, Atutor, ILIAS and others are good because there is a collaborative User community surrounding each of them in which people listen to the ideas of others and are willing to change. Where are .LRN users? Where are them using .LRN (at university, at k-12, at reseach centers etc)? What type of .LRN user are them (Educational administrator, Systems administrator, Teacher etc)? When will be the next .LRN conference? Is there something like a DDP (dotLRN Documentation Project)? Tips and Tricks Using .LRN for Teaching for Researching for Universitary Portal for Courseware etc? Reviews on IT news, EduInfo etc?

I think .LRN Consortium has great advances on it. Quality survey from Rafael covered most of the "where". Sloan and Bruce Spears covered some aspects on teaching using .LRN. Al and Cesar Brea has in touch with conferences. E-lane has working on methodologies and producing contents for e-learning. Dave Bauer and others has focusing on discussions about Good practices in elearning- either here in forums and his personal page at thedesignexperience. Could we glue all of these advances and show them for people who want to try .LRN?

Great feedback and suggestions. Please keep it coming.
I think install and TCL are both non-issues to some degree. Look at Rails -- even less people know Ruby than TCL (is my guess; certainly neither is mainstream) but people learn it because Rails is just that cool.

AOLServer OTOH is relatively big. It means you have to be a fairly solid sysadmin to get OpenACS running, and you need a more expensive virtual-root setup instead of a shared apache one. Hobbyists won't bother.

Dave asked `Why aren't these issues brought up individually in the forums, or as bug reports?'.

Well, because you and I were working on some of the bugs I encountered together, communicating on the #OpenACS IRC channel. The OpenACS core fix required was the `folder__new' proc we discussed. See https://openacs.org/irc/log/2005-02-18 for a transcript. I later discovered the bug in the tsearch2_driver for which I've emailed you a patch directly as you are the maintainer of that package. Maybe you overlooked it in your inbox?

I did not log the lars-blogger bug as I was fed up with constant stream of bugs encountered so far.

Dave also wondered if `I proposed ways the interface could be improved?'. No I didn't. Personally, I can't find rhyme or reason in the admin interface and consider it a step back from the previous interface. Specifically the labyrinth of pages for maintaining packages (both installed or new). See https://openacs.org/irc/log/2005-02-19 for my uncensored expressions of frustration with OpenACS.

Regarding ecommerce, remember that I cleaned up the code significantly about 3 years ago and since have assisted Torben on occasion. You can't tell me that I don't know the state of ecommerce. Yes, commercial sites are running ecommerce but that doesn't mean that ecommerce's code is of good quality. If all you have is a hammer....

My point is that I'm disappointed with OpenACS's progress. There have been some excellent contributions, yours included Dave, but overall I feel that the OpenACS framework suffers from lack of organization in the OpenACS community. Which translates -for me at least- that OpenACS isn't ready to be used as Praesagus's Intranet site. A site that should assist Praesagus in their day to day work not detract from it. As it stands installing the (then) latest release of OpenACS was giving me more headaches that I cared for.

Harsh? Maybe.

/Bart

Al - I would love to take up your offer ... if I wasn't getting married in three weeks ;)
Hi,

- Installation. Windows is very important. I know
some folks prefer linux over windows. But windows
has a huge market. Noone really questions Oracle
versus Postgres because Oracle has its market.
We should find a way to support windows and linux
without any patches and long installlation procedures.

I've just posted an updated version of "Project/Open" that includes a Windows installer based on the free Inno Setup (https://openacs.org/forums/message-view?message%5fid=273854). Just press "Next" a few times and you are running OpenACS on a Windows 2000 machine within ~12 minutes. No XP nor 2003 yet, but coming soon.

The P/O installer is based on the free Inno Setup, as oposed to the installer from Vlassis Rizopoulos which is based on the proprietary InstallAnywhere. Both installers include CygWin, together with PostgreSQL 7.4.3 on CygWin. This way, you can distribute everything in a single binary. No need for an extra PostgreSQL 8.0 installation, except that the CygWin version is a bit slower and doesn't support searching (yet). But that are no issues for trying out the software.

Also, our P/O installer still contains an entire DotLrn installation, which came from Vlassis. I haven't tested the DotLrn installation recently, but it should still still ok. I guess it's only a few hours of work to include the latest DotLrn version.

We would be very, very happy to share the installer (and part of the testing work...) with the community. Inno Setup supports preprocessor directives, which could allow us to build installers for both DotLrn and Project/Open from the same source file.

The installer source is included in the ProjectOpen-3.0.beta3.Wininstaller.exe file and will install if you select the "Installer Source Code" option during install.

If you're interested, please contact me (at the other discussion, please) and I'll setup a chat or a Skype online conference for a hands-on session with the installer.

Bests,
Frank

Just a tip with running postgres 7.x under cygwin - anyone I know who has tried to run a decent size openacs database on it has found serious stability issues.

Perhaps that is better with 7.4.2, but I wouldn't reccomend anyone runs production postgres databases on windows without postgres 8.

Al, why this sudden switch to "offline" and "no longer plan to post in public"? So you and Talli had a little public tiff in this thread, so what? A bit of flamage never killed anyone. I would rather see folks like you and Talli offer your thoughts and opinions with no punches pulled. I like more communication and transparency, even if they sometimes cause a few bruised feelings - they're worth that side effect.
Andrew,

Talli and I have been talking offline and he said the same thing. Some blood letting from time to time is healthy. The "Talli bomb" set off an energetic discussion.

I will enter back into the arena.

It's not that I have bruised feeliings or that I am sensitive guy. I have a pretty thick skin. I was concerned about the impression it would make on people who tend to be silent i.e. "I am not going to speak my mind because I will get blasted or be subject to personal attacks."

But I realize now that needs to be part of the education process in open source communities. Any debate where the stakes are high and people are passionate will lead to some fisticuffs and blood. But at the end of the day we should all be able to go out and get a beer. That's the way it goes.

Collapse
Posted by Andrew Piskorski on
Why the heck would you want a Linux binary installer that works like a Windows binary installer? Yuck! Linux has something much better - packages! For Linux, what OpenACS needs isn't some silly pointy-clicky GUI binary installer, but rather:

  1. Source rpm and deb packages for all OpenACS dependencies, all OpenACS core packages, and each non-core OpenACS package.
  2. Compiled binary packages of the above for some of the more popular Linux distrubitions.
  3. An OpenACS package repository for use with both apt-get (on both Debian and rpm distros) and yum (for rpm distros), preferably all hosted at one place. (On a packages.openacs.org server, or something like that).
  4. Clear instructions on how to point apt-get or yum at the above repositories and install everything you need for OpenACS. (With the above work done right, this should be very easy, just adding a line or two to one config file, and running one or two commands along the lines of, "apt-get install openacs".)
  5. Optionally, where various Linux package repositories include stock components suitable for use with OpenACS, clear and concise pointers to those as well.

Just yesterday Bart contributed a whole bunch of source rpms which make a good dent in the above list of tasks. Many others have also made similar efforts in the past, but to-date they have been one-off one man efforts which have never pushed through to the full solution above, and which haven't been taken up and fully integrated by the rest of the OpenACS community.

There is no doubt additional related work that would be needed to faciliate the stuff above and/or really leverage it once it's built, but I am convinced that the above is at least roughly the right approach.

Basically 80+% of the difficulty and hassle in installing OpenACS has nothing to do with OpenACS per-se, but is due solely to installing the required software stack. And the repository of Linux packages described above will make all of that software stack difficulty just go away! Then all that is left for the brand new OpenACS user/admin to do is:

  • Creat and initialize a database (for your already installed RDBMS software).
  • Configure an AOLserver instance (using the already installed AOLserver software).
  • Initialize your OpenACS system via the web browser (using the already installed OpenACS files).

Those tasks are pretty easy, are already covered adequately in the current OpenACS docs, will get even easier once they can assume use of all the Linux packages above, and most likely, could also be further automated via the Linux package systems if people end up wanting that.

Collapse
59: Windows installer (response to 54)
Posted by Andrew Piskorski on
Incidentally, Frank's OpenACS and Project/Open installer for Windows sounds like exactly the right approach to me, for Windows - just not for Linux.

A clone of that same installer for Linux might be useful, and might be better than nothing, but seems clearly inferior to leveraging the fabulous stock support provided by all the modern Linux package management systems and tools (rpm, deb, apt-get, and yum).

I would like to thank the OpenACS/dotLRN community for some great software. I would like to reiterate something from a year or two back as a suggestion for the way forward.

We are a research-led University and one of the early appeals of dotLRN was the 'built on OpenACS' as we wanted to support communities as well as courses. We had expected the community would understand and share that goal. For one reason or another, I don't believe it has been realised (I could be wrong, because I am not following as closely as I used to). The impression that dotLRN is another layer on top of OpenACS with different developers having different agendas does not help.

We have not imposed anything on our campus and adoption is based on interest and word of mouth. So far our adoption of communities (backed by a rebadged dotLRN) is about 3 times our adoption of course sites (backed by CourseWork from Stanford). Evidence on 'real use' of VLEs (e.g. only 30% of courses making heavy use of VLE 5 years after introduction) from campuses that have imposed campus-wide platforms leads us to believe we are not unique.

We continue to refer to OpenACS and dotLRN as a benchmark on good design of useful and usable feature sets and I have posted to Sakai forums referring to some of your practices as examples for them to follow.

We are now looking hard at Sakai because it does support both communities and courses out of the box. And it seems likely that many will soon be working to create value-added research tools for the research communities. (see http://www.grids.ac.uk/twiki/bin/view/EResearch/VreProjects for some recently funded 'Virtual Research Environments' in the UK)

I would urge the community to look again at the concept that a class/course is just a special case of a community and that setting up a class is just a configuration option. Then apply the portal to the resulting collection of classes and communities. I think this will make the software more attractive in many ways and allow you to compete with Sakai, which I think is in a more realistic niche (at least for research-led Universities) than competing with Moodle, Blackboard, etc. in the VLE mainstream. A common product should help with alignment of goals and supporting other specialised 'configurations' of a community - e.g. project group would add even more value.

I hope this is of some value and thanks again for some great work!

In this thread and in others I hear three recurring themes, which apply to OpenACS and .LRN. We need:

  1. orderly and predictable releases;
  2. an improved installer;
  3. quality code base;
This is not exactly "differentiation" but it would go a long way if we were able quickly bring ourselves to the point where those who are trying to "sell/market" OpenACS/.LRN can legitimately and credibly claim:

"We have a product that you can install quickly, upgrade easily, and, by the way, is enterprise-class out-of-the-box."

By enterprise-class I mean a product that a) is usable in a mission critical production setting because of a high degree of code quality; b) scales from a small site (e.g. small NGO) to a large educational institution (e.g. Valencia with 20,000 - 40,000 users); c) integrates into the enterprise (e.g. external authentication).

What f$*(%#! product out there, with a feature set comparable to OpenACS/.LRN (commercial or open source), can make this claim?

I believe that we are 90% on the way, and we just need that extra 10% of effort to get there.

I will offer some thoughts soon on how we might be able to get there. But I think this should be the area of focus for the next 3 months and we should all work together and go after it ruthlessly.

John,

I think there has actually been a lot of progress on making dotLRN work as part of an OpenACS deployment rather than as a monolithic portalized front-end. You can create a subsite with forums and membership etc. alongside a dotlrn instance without problems (well, I think anyway -- I do install things this way to test but we don't use dotLRN in our work so I have not built production sites that way).

Is any of what you use dotLRN for publicly visible anywhere?

I am curious if you have written things which would be of value to the community and could contribute back. One thing I think we don't do as well as we could is proactively seek out the things which people are doing to make dotLRN or OpenACS useful in production settings and make sure they make their way back in to the codebase.

In response to Jonathon re: installation. The first time I downloaded and set up a dev Rails setup, I was up and running in under 10 minutes. That probably includes download time as well!

In fact the one thing about Rails that gets you interested it is it's instant satisfaction. Along with it's easy install, the 'scaffolding' concept that uses database introspection to auto-build template pages means that you can have a little test site running much faster than even PHP.

A bit later, the "everything's an object" philosohpy of Ruby gets you hooked enough that the dumb (in my opinion) single db table hierarchy setup doesn't seem so bad.

The Rails community is also looking seriously at that inheritance issue, and there certainly seem to be developers of good calibre involved.

And while less people know Ruby than TCL, Ruby does not have the limitations of TCL and is a language that computer scientists can get excited about.

Note that Rails is not a competitor to OpenACS if you want to use some OACS packages. But once the Rails community settles on a common authentication system, it will become a very serious competitior for people who use OpenACS mostly for the framework/toolkit.

Jeff

This is not publicly visible, but I think we have posted some screenshots previously as part of a collaboraid look at interfaces. We would happily do so again, but our changes are limited:
* professor renamed to admin
* student renamed to member
* 'contributing member' role added for file sharing
plus various bug fixes, mostly in calendar.

Since 'getting dotLRN working' we have not updated, because of a fear of introducing new bugs. This is an area where we are less happy about the community. You are almost *too* smart and I get the impression that new development tends to get priority over QA (with some notable exceptions). We feel this is a drawback to the 2 tier architecture with so much functionality in the database - even a database update could introduce bugs. We have yet to see the planned Hibernate introduction into Sakai and test its scalability and performance, but it seems promising. The other thing we have found much more attractive is the configurability of permissions, with basic CRUD permissions on a role configurable sitewide or in a community by tick box for the relevant administrator.

You have prompted me into some negative thoughts, but I want to emphasise overall admiration for the work and that is selfish reason I want to encourage you. If Sakai fails to deliver, I would like to think OpenACS/dotLRN will be ready to pick up the pieces...

John,

I suspect you would find there are far fewer bugs introduced relative to the number fixed if you upgraded. Based on tickets we see for dotlrn that many of the open tickets end up being closed in the new release.

That said, the upgrade process from an older version (particularly if customized locally) is not painless.

It's an area we have worked harder at and I think over time are getting better at (upgradability and reproducible delivery). You are right that it's a pretty intractible problem for a 2-tier architecture although given the persistence layers I have seen that abstract that, it's the problem I would rather have.

Is there a good write-up on the sakai permissioning? I don't think in principle there is anything that prevents us from doing the same.

Mark,

Coincidentally,

I thought the scaffolding idea made for a neat demo and I built something similar for OpenACS using Lee Denison's dynamic-types package. I have hesitated annoucing this since I haven't written any documentation on it yet. Its on the way, and I plan on showing it around to folks next week around the bug bash.

The code is all on HEAD and oacs-5-1 branches if anyone wants to see it.

I want to thank John for his comments (they are always "spot on), but also tease them out a bit in a series of posts.

During one of our conversations several years ago over coffee in Cambridge (US), I recall John saying:

"Hey, Al, you guys really need to focus on an easy installer and make .LRN a configuration option for OpenACS."

John reminds me of a physics professor that I had in college. His statements even at face value were valuable but the real insight was something deeper: take this thought, develop it, and you will be rewarded. Let me take the John installer thought a bit further, even if John ultimately might disagree with where I am taking it.

The starting point is that there should be an identical installer for OpenACS and .LRN. It can be implemented using switches or whatever, The deeper point here is that we should not be view OpenACS and .LRN as separate products and processes. As John indicates, "The impression that .LRN is another layer on top of OpenACS with different developers having different agendas does not help."

If OpenACS and .LRN are equivalent in terms of code base, how are they different? First, .LRN would always be a subset of OpenACS and not an "additional layer" on top of OpenACS. Secondly, ".LRN certified" components would need to pass an additional set of QA to receive the designation. Third, in addition to a higher level of quality code of individual components, the ".LRN-certification" would also provide assurance of integration among the certified components. Fourth, .LRN-certified components would also be tested for scalability. Finally, there would be a commitment from the consortium members and the community to always keep the focus on quality for the already certified components. We would also always provide an easy upgrade path for the latest release.

In short, .LRN would be that integrated subset of OpenACS which is enterprise ready.

I believe that it benefits neither OpenACS nor .LRN to think of .LRN as software for the educational community. I come back to another point that John keeps reminding us of, namely that a course should be viewed as just a "special case" of a community. Now, this is the reason why we (MIT Sloan) went down this path. We were not interested in building a course management system to compete with Blackboard of WebCT. What we wanted and needed was a platform that would support knowledge communities of all kinds, whether it's a student organization, a research community, or a project community.

Again, here is a one line sales/marketing pitch for OpenACS /.LRN:

"We have a product that you can install quickly, upgrade easily, and is enterprise ready out-of-the-box."

The pitch to be an educational instutions (K-12 to ., a non-profit, a corporation, any organization that has a need for collaboration and online communities.

"And, by the way, the software is backed by the .LRN Consortium, which consists of these organizations...."

The list of organizations should be deeper than educational institutions and ideally include everyone who is using the certified components.

In short, we should begin to view ourselves as a single community that produces the highest quality software to support online communities of every sort.

Jeff

I will try to dig something out, but you can always take a look for yourself at the test site for the impending 1.5 release: http://sakai-stable.mit.edu/ I'm not sure what the plans are for the site, but this morning you could log in with admin and admin to become site admin.

You can also register for a user account and make that an admin account with the above login. There are CRUD permissions at course/project admin level with tick boxes, but then more sophisticated controls in the area called 'realms'.

Here is sakai versus openacs from the standpoint of grants.

I would think since we have heirarchical permissions, permissions inheritence, nested groups and relational segments I doubt there is anything the Sakai crud implementation offers that we don't already do (or could do reasonably easily).

Having looked at the realms bit, it's also clear that we are not the only ones saddled with a cryptic "advanced admin" interface.

OK, I may drop out of this soon, because I don't want to be part of a Sakai vs. OpenACS debate, but I will make one more point referring back to the merging of OpenACS and dotLRN.

Worksite setup starts by asking: are you setting up a course site or a project site? This is the type of integration I would have liked to have seen with OpenACS/dotLRN. Also the illustration for Sakai shows the permissions applied locally within a course or project, not sitewide (which I think is what I am seeing in the OpenACS picture). And finally, I said we hadn't upgraded so I'm a couple of years out of date. Maybe we were wimps not to keep up, but one way or another we picked up the impression that it we might risk introducing bugs if we did.

There is plenty that is wrong with Sakai and many risks to the project. If we had had this from OpenACS/dotLRN 2 years ago, we would have been locked in by now. For established campus-wide systems switching costs are substantial. Our Blackboard users don't want to switch to CourseWork and our CourseWork users don't want to switch to Blackboard.

I don't think I want to be in an openacs vs sakai debate either (as I don't know much of anything about sakai or really about any other e-learning platforms). I am more interested in seeing something that works well for permissioning as it's one of those really hard things to get right.

The permissioning you saw there for openacs was local to an event calendar under a particular subsite and not sitewide -- although with inherited permissions you can permission on the subsite directly rather than per application (which is something sakai seems to lack -- driving things more from roles and direct permissions as far as I can tell).

One thing that's also not clear in the sakai example is whether something like file storage which has a folder structure can permission a folder and have that permission inherited by it's children.

What I really want to find is an example of role based permissions that people really like but I have not seen one yet.

Hi,

I am undertaking a project to develop a portfolio application for the Faculty of Engineering at the University of Sydney. I have considered looking at using Sakai/OSP for my project (and integrate it to our dotTeach program using web services), but after evaluating it for the last couple of weeks I am now in favour of developing an openacs/dotlrn solution.

I thought I might post my experiences with Sakai/OSP, as this thread is kind of heading in that direction.

Developer Community

Sakai has two forums for developers and users respectively. I have participated in the forums, and have found that there is very little activity. Questions are rarely answered, and there is no presence of core sakai developers. These two forums are accessible by the public. However, there are more forums that are only available to $EPP members. I am not a $EPP member as I don't have $10,000USD for such an expense.

So what does $10000 per year buy you in Sakai? More forums, which core developers *probably* participate in. Documentation that explains the sakai framework. Access to tools developed by $EPP members. So much for an "Open Knowledge Initiative".

I have to say that the OSP community is far more helpful than Sakai. However I don't know if this will change due to its integration with the Sakai framework. They may adopt Sakai's policies. In fact they are already considering a site redesign.

The OpenACS community is well established. There is a high volume of activity within forums. Questions get answered. Information is not restricted to members that have money.

There is a democracy within the openacs community. The steering committee are elected from openacs members by openacs members. This is not obvious in Sakai's community. They have a board, but they are made up of senior management from institutions that are $EPP members.

Development progress is heavily dependent on the $EPP members. If the funding dries up, then what will happen to Sakai? Sakai's community has not been tested for "survivability". At least OpenACS has... OpenForce once contributed a lot to the community. They no longer exist, but the community has strived and even flourished. I've noticed that some big names in OpenACS have kind of dropped off the radar recently, but who cares when others can step up and fulfil those roles as community leaders and knowledge experts... questions get answered.

If you can't contribute money to their cause, then they don't want to know you. That was my experience with the Sakai community. I spent some time porting the Sakai datamodel to Postgresql, cos I didn't want to use mysql. It wasn't hard, and I thought others in Sakai's community might find this work of some value. Well I am disappointed to say that I have not heard back from the community about wanting to support postgresql.

Learning Curve

Lets face it, there is a bit of a learning curve that people new to OpenACS have to face to develop something useful. However new developers can get assistance from other openacs developers in the forums... questions get answered. This is not so with sakai. If you don't believe me then go to http://collab.sakaiproject.org/portal create an account and browse the forums. You'll find a lot of postings, but no responses. The chat site isn't much better, and can be summed up with this extract:

" (Feb 22, 2005 10:15 pm) : hello??

(Feb 25, 2005 10:47 am) : Clay! HElp!

(Feb 25, 2005 10:47 am) : have a quetion!

(Feb 25, 2005 10:47 am) : Help John / clay ! I have a question!

(Feb 25, 2005 10:48 am) : I am mailing to the mailto:sakai-dev@collab.sakaiproject.org list (I think!) but I dont get any feedback. I dont know whats happening "

By the way that was not me *begging* for help. :)

I've tried sending email to mailto:sakai-dev@collab.sakaiproject.org as well, but my emails get bounced. I know that the core developers participate in that mailing list, but a non-$EPP member cannot get access to that knowledge base.

Now what do you have to learn in OpenACS to develop something useful? Well there is TCL, Postgresql/SQL and the OpenACS framework.

In Sakai, there is Java, Hibernate, Mysql/sql, Struts, JSF, Spring, Velocity, CHEF framework, and the Sakai framework. There is a hell of a lot more to learn to develop something useful in Sakai.

The learning curve to me is far greater with Sakai, than OpenACS. Moreso because there is so much to learn with Sakai, and very little documentation or community assistence to reduce this learning curve.

Standards

Sakai haven't really implemented any IMS or SCORM specifications. They are working towards making their Tool Portability Profile (TPP) an IMS specification. I don't see what is so great about this, cos OpenACS packages do this quite simply. A package that I develop in OpenACS can be installed and mounted at any institution that uses OpenACS. This is what they are trying to achieve with TPP. Mind you there is no documentation on TPP made available to non-$EPP members, so developing applications that plug into Sakai requires a lot of reverse engineering and source code sifting. Also OpenACS has LORS, so OpenACS is one up in the specifications stake.

Functionality

Sakai: Chat, email archive, discussion (basic forum), resources, web content (aka gate keeper in openacs).

OpenACS: Too many to list.

Evaluation

Now I have to say that I consider myself as a Java developer. All work that I have been paid for have been Java solutions, with very few paid openacs based jobs. So I was quite excited about using Sakai... because it is Java based. In fact that is what I think Sakai is all about... "excitement".

We've all heard that Sakai is a "$6.8 million software development project", and that there are high expectations for great things to come. So what? I've seen million dollar projects crumble and they have all been well documented (google "software runaways"), why would Sakai be any different?

Lets talk about OpenACS... what value in dollar terms can you place on the OpenACS project? I'm fairly certain that if you add up all the contracts that have used openacs as a solution (and contributed to openacs), then you'll find that OpenACS would also be a multi-million dollar project. Moreso when you factor in intangible assets like OpenACS's open knowledge base and the diversity of experience of members of the community.

So my evaluation has lead me to develop my portfolio application in OpenACS. Hopefully there is a need for a portfolio application in OpenACS and dotLRN. I'm sure there is, as there has been talk about it in back channels.

Feel free to post your experiences with Sakai. But in my opinion, unless you are a $EPP member, I would put Sakai on the backburner until they release something amazing.

Cheers,
Nick.

Thanks for that info Nick - I for one have never checked those communities out.

It again reminds us that the community and code base are our core assets I guess.

Of course if you want to compare $ investments into the code base, OpenACS factors in ArsDigita ACS 4 which in turn comtains part of the ACS 3 legacy - I know that we're talking well over USD 10 million here... Someone may have actual figures on that.

It's a very good point you raise too about the barriers to entry. Every time I consider Java for a web project I am amazed at the complexity of what are considered standard toolkits. I pity a university student trying to wrap their head around Jakarta Struts when they have barely learnt to think in Objects!

Well, that cuts both ways... students who have trouble with Struts will almost certainly have trouble with the OpenACS stored procedures as well.
Nick

You are right and wrong, but mostly right. Sakai has not yet commited to a full open source development philosophy (hence they have redefined the term 'community source' for their own purposes) and the project is midway. The intended new architecture is only now getting seriious attention after a period of merging features from the 4 schools into a CHEF-plus environment.

I agree fully that OpenACS can compete well. My remarks were intended to convey the areas in which I felt OpenACS/dotLRN could improve with reference to a concrete example.

I'll bow out with a reminder that many Universities do research (and other things) as well as teach and that a solution for all of the communities on a University will always be attractive. OpenACS/dotLRN has that potential, but I don't hear it discussed much in the forums.

Nick, would you mind if we put your comparison up on the dotlrn.org website under a "competition" area?

Al, Carl, would you mind having this content on the dotlrn.org website and create a "competition" area?

John, the university of Darmstadt in Germany mentioned that they went for .LRN because they wanted to have an Open Toolkit that allows their Professors to do easy research and especially research in E-Learning. I'm unsure about the status of this effort, but it strengthens your point and one of the more interesting features .LRN has to offer is support for research communities and research projects (using e.g. the project manager).

Thanks a lot for your comments and your insights into Sakai and getting this discussion about Sakai started in the first place. At least I learned quite a bit out of it (and can advise universities better).

Collapse
79: Research Communities (response to 1)
Posted by Alfred Essa on
Speaking of research communities, I am pleased to announce that .LRN will be used by two research communities at MIT. Both are led by Eric Von Hippel and Karim Lakhani.

The first site will be called "Distributed Innovation" and centres around Eric Von Hippel's research and new book, Democratizing Innovation. Von Hippel argues that "user-innovation communities" in open source should be applied more widely, for example, in manufacturing.

The second site will be a conversion of the existing Open Source Research Project web site at MIT.

Interesting conversation.

Allow me to add my own comment.

I am the CIO at a medium sized company in Canada, 400 to 600 employees. We are definately in the minor league here with respect some of the institutions using OACS and .LRN

Currently we run Lotus Notes as our groupware solution but we are evaluating ditching that beast. Basically Notes/Domino is really a wonderfull idea, run by a top heavy, closed organization, hugely expensive, and a system that once you get heavily invested you are tied to it.

Luckily we are not heavily invested in it, so we have choices.

We have looked at everything under the sun here in terms of an 'Enterprise Class Application Server' and various technologies.

I understand where you guys are coming from, but you really are missing the boat in your discussions here. Some thoughts from a non educational organizations perspective:

- Anything on .NET is a no go for us. MS has shown it self to be a proven and consistently immoral and untrustworthy company, and we won't be part of anything from withing their sphere of influence. Anyone that deploys anything on .NET is putting on their own shackles of slavery - MONO alternatively is not production ready.

- Java based solutions ... too many options to discuss, however I don't see anything from that toolset that really does what OACS does.

- LAMP ... this is fine for task specific stuff. If you want CMS, Mambo Server or Typo3 are amazing, if you want project management you have tons of options, if you want frameworks the options are amazing. For Python alone there are something like 8 frameworks available and Python isn't even used as a web language per se!!!

I love the LAMP stuff, but the community seems large and active only because there are a ton of tiny and a few big projects that completely reinvent everything every time. If you are going to start anyting on LAMP, at worst you will rewrite an OACS type of framework your self, or at best use someone elses that is either too small to have any mind share, too young to have been tested, or if its something that is successfull like Typo3 or Mambo server, well they reinvented their own frameworks and I dare anyone to merge anything from those two projects into the other. For that matter, I dare any one to get any two separate LAMP project to work together.

For us, I could deploy a few LAMP applications, but in the end, we will be maintaining each one separately, patching each one separately, coding each one separately and administering each one separately. They won't talk to each other, they won't have the same permission system ... etc.

LAMP is a no go for anything other than quick and dirty, but to be fair OACS has the quick and dirty down better than anything in LAMP.

- Standalone frameworks/apps like GNUE.org. Okay I personally love this project, I think they have the right idea and I think they have some potential for delivering strong solutions. The project is old and is fairly stable, yet at the same time it's still is lacking in certain features that people look for in things like that. Many other apps/frameworks in this category are in similar positions. This is a wait and see category still before I make up my mind about it.

- OACS ... I think everyone needs to go and read up on some of the history of AOL server again, and why its such a terrific base to have a framework on.

AOL Server - probably the most powerfull and scalable web server available out there ( maybe outpowered by Zeus ).

TCL - if you can dig into python, you should be able to understand TCL. Learning TCL is a non issue, from what I can see it's very simple, if you can learn Ruby or Python you can dig TCL.

OACS - one single integrated framework, on one single language ( I know there is a Python binding out there, but there really is no need for it TCL seems fine. Perhaps some expert programmers can fill me in on TCL'S shortcommings but so far I haven't been able to google anything more than WAAAH, NO ONE USES IT, I'M NOT L33T) on one of the worlds fastest and most scalable webservers.

All tested, all production proven, a single permissions system, all applications are a subset of OACS and are organized and function similarly ... WOW!

Do you guys even realize what you have here? Holy shit!

I'm just a PHB, and it took a bit, installing the .LRN package in debian is one command, got it up and running, figured out that .LRN is just a SINGLE APPLICATION built on OACS, uninstalled .LRN from the framework, installed a good number of applications my self via web gui, and setup permissions and a few sub sites all with little or no help from IRC.

I created a NEW APPLICATION with the click of a button, following the introductory application tutorial, and restarting the DOTLRN service in Debian and VOILA! I have my own application ( okay it's just the tutorial example files but they work ) running.

Hello? This is amazing!

Look, what this project lacks is sex appeal. Here are some things that might help and probably have been mentioned already:

- sex appeal. Look https://www.sitepoint.com/mambo-server-cms/ - it's lickable and the OACS site needs the same treatment.

- the .LRN debian package and the .LRN instance needs to stopo presenting it self as a separate instance of OACS because it is not. It should only be, and is, an application available from the OACS repository. Nothing more, nothing less, just a beautiful, well thought out nicely programmed.

- LORS needs some debugging and integration into .LRN. Once you add the ability to deliver SCORM class content via .LRN you have a Moodle and Sakai killer. .LRN already exists and works, Moodle is simply an instance of LORS and as such lacks the scope of .LRN, so when you merge the scope of .LRN with the ability to deliver online course, you can just "apt-get install dotlrn" and have any potential organization running in minutes with a basic server for testing blow away everyone at the presentation. This is huge, I can't underscore this enough.

- The overall user interface needs a rethink. I've spoken to quite a few people on the IRC channel, everyone knows this but no one has time. Fine, but this is minor and easily fixed.

- The applications in the repositories needs to be further debugged, further push towards bug fixing made, although everything is mostly good and mostly works. For us, .LRN, Project Management and a few other packages work great and will be Piloted in the next month or two if not sooner.

- OACS needs to make some noise. Never mind the negative crap, there is nothing out there like OACS. You can't deploy any framework faster, you can't get a framework with the production proven scalability of OACS and AOL server, you simply can't develop applications as fast as you can on OACS. I just don't think anyone knows about OACS. Holy christ, if i had this back at the Web Development firm before the dotbomb took place we would of killed all our competition!

We could of delivered websites almost instantly by simply reusing various applications, and perhaps adding a few more and concentrating on just creating the look and feel while charging for development time as well.

Okay that's it for now. Sorry for the rambling post, but it's 6am.

What I'm getting at is that OACS is so far ahead of any other framework out there that it's funny! Any shortcomings already mentioned in this thread I don't see as significant at all. Just tell people about OACS and they will come, which brings up the last rather interesting point, would you like to know how I found about OACS and AOL server?

I had no clue it existed, I was evaluating opengroupware.org and via some misunderstanding of their documentation ( they as well reinvented their own application server yet again! ) i googled up AOL server, by which I found OACS, by which i discovered that it has a unified security model, by which I discovered .LRN, LORS, and a ton of other applications, all that I installed with a few simple commands.

Just tell them you exist godamnit!

Robert, Thank you for your excellent post!
Nick,

I'd love to hear more about your eportfolio project. Keep us updated please. I think the community features along with a flexible eportfolio that an add any sort of object into the portfolio would be very powerful.

It might tie into some of the work on communities of practice thats on HEAD as well.

Nick, ePortfolio is huge especially in the K-12 space. Our early conversations with Apple indicate that they are seriously interested in .LRN, but they need ePortfolio. I will explore to see if there might be funding opportunities to kick start your project.
Hi Guys,

I'll be keeping everyone posted about the portfolio package that I will be developing. I will start a new thread when I get things rolling. I have some tight deadlines, so hopefully I'll have a prototype up and running by April. I need a production ready application done by June. As we'll be getting students to use it in the second semester that starts in July.

I hope I can get the support of the community behind this project, as I know eportfolios are the latest must-haves in education and in HR. So there will be a need for this type of application.

Cheers,
Nick.

PS. I agree with the others in that OpenACS needs sex appeal. For starters, someone with expertise in designing icons should turn "Alex" into one of those MacOS X candy style icons.

Robert, thanks for your excellent post! If nothing else, I learnt about GNUE which looks pretty sweet.

Also let me say that if you can follow the OACS new application tutorial you are no PHB - and I have worked with some real PHBs in my time...

You are 100% right about the sex appeal. The only thing you are a little off the mark in my opinion is that TCL is a limitation. You are right that any half-baked programmer can learn TCL, but we have so many hacks in place to work around TCL's limitations that some of our libraries approach wizzardry - by that i mean they work - it's damn impressive - but you just pray you'll never need to uderstand how it works! Cf. ad_form (whose internals I now understand, but it took a long time...)

With the will and time however, we can solve many of the TCL/API limitations with tools at our disposal (OO TCL, ns_perl, etc.).

Also with the will and time it is not hard to build sex appeal in the site - the framework and community already have sex appeal (as long as we ban developer portraits ;), we just need to strut our stuff a little more with the web look n feel and writing magazine articles etc.

Speaking of UI sex appeal, we really need to get the .LRN redesign that Collaboraid did into cvs :

http://www.collaboraid.biz/developer/blog/archive/2004/05/

The new error/alert bar alone is worth 100 sex-appeal points. You know it's true ;)

Thanks again Robert - we all know what you say is true, but hearing it from the outside brings hope ("A man finds joy in giving an apt reply- and how good is a timely word!" - Proverbs 15:23)

Nicks right - it's time for Alex to go ...

I love Alex (and Karen). Even Apple booted Clarus the dog-cow (moof!) from the page layout dialog.

Getting more professional and sexy won't kill the familiar feel of our community, and it might just help protect it.

Malte,

You can add my comparison to the competition site if you think it would be useful.

Although I have to state that OSP have a good community website. I like the fact that they are using a Wiki system for their documentation. Why don't we have a wiki? I think there was talk about this before. My gripes were more towards Sakai than OSP.

John, the worksite feature in Sakai exists in openacs/dotlrn. The same functionality can be found in /admin/site-map. You can mount any application within the dotlrn domain or at root level. That is essentially what the worksite feature in sakai does.

Cheers,
Nick.

That's not editable for me - is it supposed to be for all logged in users?
Nick, thanks for the detailed post on the Sakai initiative, it confirms many of my suspicions. I would be very surprised to see them adopt a true open source model of development (regardless of license specifics) because it seems clear to me that quite a few people are staking at least parts of their careers on the success of this project ... and they're going to want CONTROL.

I wonder what would happen if we post a link to a summary of your comments to Slashdot? We had tons of fun when we publicized Ben Adida's "Why Not MySQL?" article on Slashdot a few years ago.

Richard ... our community is well aware of the fact that we're are own worst enemy, because we don't effectively promote our software or ourselves. That's not the only reason few folks know of us, there are people out there who turn their noses up at any non-Java solution, or any non-LAMP solution, etc. But we're so tiny that even if you subtract zealots from our population of potential users and customers, we only have to be successful in attracting new hackers to double or triple the size of the active development community ...

Now there have been some promotional activities, such as Reuben Lerner's articles in The Linux Journal. But not enough, not nearly enough ...

Mark, it's a little secret at the moment, I think certain people want to improve the UI or make other improvements before claiming that we really have a wiki package mounted on the site???

"wiki" has been in our CVS tree for some time ...

We have not been pushing the wiki since Dave want's to change the wiki formatting language from wikit to something else like markdown and I have been thinking vaguely about a mediawiki formatting plugin (mediawiki == wikipedia so there is a large corpus of test documents :) ).

We did not want to be saddled with a big conversion job.

Al Essa -- I hope my post is helpfull!

Mark Aufflick -- you're welcome! ... see above :)

I've learned a lot from this thread, keep the brain dump going, this is fascinating.

Mark, regarding your comments on TCL's limitations, I've run into that phenomenon in various degrees in many fields. For example, acquaintances of mine that are also running Lotus Notes/Domino environment which is basically a client, app server and a proprietary scripting language have run into just that ... some rather severe limitations of the platform as they have essentially pushed Domino to do far more than it was ever designed to do. Alternatively the same thing tends to happen in all sorts of areas, apps start out in VB, outgrow it and migrate to C++, etc.... In our discussions sometimes they look back and wish that they had started it all in Java, but then again the initial barriers to entry are much higher, and so goes the circular discussion.

It's one of those things where you start using a tool because it's easy to get into, but over time the scope of your usecase outgrows your toolsets spec ( oh boy, hard to get rid of that dotcom cliche-itis! ), and i can't see any real way to mitigate this as this scope growth tends to happen over such a long period of time that any choice you make is speculative at best even if you have the right decision making and technical skills.

With that being said, would you care to elaborate more on the TCL limitations experienced? Is there any way to outline the issues to essentially a non programmer so that the explanation isn't too abstract? I think this would be something worthy of further exploration as I think this discussion has covered most of the big issues that I think are easily solved with as was mentioned 'merely tripling the OACS developer base' which currently would ammount to repackaging some of the entry points to OACS (easier install, etc.), a post on slashdot and engagement of the community to the active questions so seal the deal so to speak.

It sounds as if you are describing similar issues to what many of my colleagues are running into with Notes/Domino. To turn it around, I think it's fairly obvious what OACS excells at, based on the TCL limitation pionts ( small to medium sized applications ), what categories of development would OACS/TCL not be suitable for? Could OACS successfully develop and ERP set of solution ( project-open.com for example really has an amazing start in this direction already ) on the level of some of the stuff SAP is offering (although they offer so much that in a way they really offer nothing, their last install at hp cost hp 500 million in lost revenues, heh!) ... surely a lot is possible what I'm trying to gauge here is where exactly is the development ceiling here?

Thanks for your interest Robert.

First let me state that while I know others who share my views, I also know that there are others who do not and all positions have merit!

You asked what size developments is OACS/TCL not suitable for? Well I would say that that is not quite the right question, because OACS is already much bigger than what TCL is traditionally suitable for. The fact that it is possible is due to a number of factors which I see as :

  1. the quality of the original ArsDigita designers and developers;
  2. pushing the object model onto the DB layer (which by itself turns out to be actually quite a good idea anyway);
  3. the richness of the AOLServer API

Point 3 is the key to why we don't suffer quite as badly as others from TCLs major weakness IMO - lack of rich data structures. Think hashes in Perl or collections in Objective C and there is no corresponding TCL concept. Everything is a list of strings. And that leads to another weakness in the scoping / substitution / execution model (or lack of). Anything quoted in curly brackets {} is (normally) untouched. So for instance when you are building a complex set of parameters (eg. an ad_form spec) with nested {} lists, you can't (easily) insert variables to be substituted. In some cases that is such a pain that our framework kludges substitution for you - but by the time it happens you are in a different scope, and so the stack level needs to be played with to get the right values to substitute. But that turns out to ber not quite always what you want (see Jade's ad_form notes for examples).

Sorry if the latter part of that paragraph is too technical, but it's a bit late for me to find a clearer way of describing it.

As I said earlier though, it should be possible for us to support two or so languages calling our core API (although that has it's own perils).

Can others maybe refine or add to this? In the end, you can do anything with anything if you put your mind to it, but in my experience of large/complex projects, dealing with TCL starts to diminish the speed benefits that the OACS base gives you.

And you are right - this problem happens everywhere. The designers of NaviServer (now AOLServer) chose TCL because they were short on time, TCL was designed for embedding inside other systems and was threadsafe. None of other alternatives were, at the time, both of those. I'm sure if Ruby was around then or Perl was less monolithic they would have chosen otherwise.

In small projects, something like Java loses hands down to TCL (or any other agile-ish language). In larger projects TCL alone would lose, but in terms of OACS versus a Java framework, it would still be hard for a Java one to beat OACS - TCL aside. Perl or Ruby would be much closer if not better.

Where the crunch comes though, is maintenance. You can't (IMO) make a very large system cheap to maintain without a really clear, simple object method reflected in both the db and code layers - and without inheritance you can't make sure that logic is never duplicated. Additionally, without objects, rich structures or a real scoping model, complex TCL code will never be able to be fully independant at the abstraction layers - there is too much interdependancy of evaluation contexts.

The interesting thing that comes from that is that a lot of OACS work is effectively maintenance - re-tooling existing code to do something slightly different.

Anyway - these issues are, i'm sure, well discussed elsewhere. With funding it would be amazing to write a new API layer on top to allow an object language to interface with the existing toolkit, but I am not sure how that would happen without a sponsor. Anyone friends with some member of a Royal family? That's how R&D was funded in the old days ;)

"As I said earlier though, it should be possible for us to support two or so languages calling our core API (although that has it's own perils)."

A major issue here is that AOLserver works directly on Tcl internal data structures (i.e. TclString and TclObject and the like). Additional languages will suffer from the need to provide for consversion to and from the internal Tcl data structures AOLserver requires in its API. This may or may not be a significant performance barrier for those wanting to investigate such an approach.

This assumes you want the alternative language to be a first-class citizen with the kind of integration with the DB API etc that we have for Tcl.

"Can others maybe refine or add to this? In the end, you can do anything with anything if you put your mind to it, but in my experience of large/complex projects, dealing with TCL starts to diminish the speed benefits that the OACS base gives you."

TCL is not ideal (TM)

"The designers of NaviServer (now AOLServer) chose TCL because they were short on time, TCL was designed for embedding inside other systems and was threadsafe. None of other alternatives were, at the time, both of those. I'm sure if Ruby was around then or Perl was less monolithic they would have chosen otherwise."

Yes, indeed. Tcl also has a very clean implementation that made it easy to work with, I would be surprised if Perl's as well-implemented given its history. Ruby's implementation is probably very clean, though I've never inspected the source.

"Where the crunch comes though, is maintenance. You can't (IMO) make a very large system cheap to maintain without a really clear, simple object method reflected in both the db and code layers"

People have built very large systems in languages like Ada that are very maintainable. I used to consult for the compiler company whose Ada compiler was used for the F-16 avionics systems, and in head-to-head implementation tests bug rates for those writing in Ada were about 1/2 for those writing in either C or C++ (not hard to see why).

OOPs is not a panacea. It's one technique that, properly applied, can help one write clear and maintainable code. Or, not properly applied, unclear and unmaintainable code. Take your pick.

"and without inheritance you can't make sure that logic is never duplicated."

Inheritance is no barrier towards the accidental duplication of logic. Nor is it a precondition for minimizing such logic. People oversell OOPs.

"The interesting thing that comes from that is that a lot of OACS work is effectively maintenance - re-tooling existing code to do something slightly different."

This is true of any large system. The only way it can't be true is if the demands on the system are static and unchanging ... which is pretty much the definition of a lame-duck project which has outlived its usefulness, not something dependent on programming philosophy.

Yes, language differences and programming paradigms do greatly impact maintainability ... but only the EASE of maintenance. Not the NEED for maintainence. If your project's dead, you needn't fix bugs or enhance it ... those needs are externally driven.

If people doing large-scale OOPs web development frameworks like Ruby on Rails, Websphere etc imagine they've gotten rid of (all, most of, much of) the need for maintenance, they're going to have a very unpleasant surprise down the road.

TCL is an excellent fit for the vast majority of web pages. It would have been cool to see PyWX take off for that last 5% or so, though.
Don, thanks for helping clarify some of my thoughts. I am a bit of an OO lover for sure, but it is 100% true that you can do OO just as badly as any other programming technique (I have sure seen that in action...).

I just find that real world data and process structures are neatly mapped by either OO or finite state machines. If you have an environment that has first or second class support for both of these, then your job in designing a good system is well supported.

In your defence software example, I would put money on the fact that the Ada developers were much more highly skilled than the (more commodity) C++ developers (although that is a complete guess). Give the Ada developers smalltalk and see what happens ;)

You are right about perl's internals. The regex engine is so complex that noone has successfully made it deal with mixed utf8 strings (unlike the rest of perl which does),
but that's just for your interest...

My thought about maintenance might be better explained. I was more thinking of when you want to slightly change the behaviour of a package in a client site, but in a way that's not suitable to be committed to cvs. If we had an OO design, you could, say, subclass the forums to allow files to be attached to messages (yes it's a lame example). The subclass would survive minor version upgrades to the core forum package. Currently, we need to hack the forum code (or duplicate a bunch of it) and upgrades then require manual merging.

I'm loving this discussion!

Thanks for those great replies!

I see what you guys are getting at.

If OO can handle the last 5%, what about the OO extensions to tcl? Could they be integrated/extend aol server and fulfill that last bit and resolve some of the above commentary? How about the Python extension for aolserver?

"I was more thinking of when you want to slightly change the behaviour of a package in a client site, but in a way that's not suitable to be committed to cvs. If we had an OO design...."

Isnt the thin object system that Oacs provides enable us to do exactly that. We might run into awkward code if you have to do another level of inheritance , but how often would you need that?.

One the other hand TCL was meant to be used as a glue and to do simple things. I believe the API and the framework that we have in Openacs provides us a way to get around the limitation that we meet with TCL.

But to get to the point of using the framework to solve problems that generally arise we encounter a learning curve of the framework. Although Openacs provides a certain degree of abstraction IMHO its not enough. This is where I think Ruby/Python helps and makes abstractions more obviuous thus leading more code reuse and easier understanding of the framework.

"In your defence software example, I would put money on the fact that the Ada developers were much more highly skilled than the (more commodity) C++ developers (although that is a complete guess)"

Yes, it's a complete guess - and wrong. Wrong guesses of this kind expose the religious bigotry that underlies many discussion about language and software engineering. C afficionadoes have dismissed the *many* efforts to quantify improvements in code quality when proper tools are used since, oh, the late 1970s. Speaking from personal experience (and as someone who's been a C programmer since 1974).

Oh, how much money were you planning to put down, and would you like my bank routing and account number?
And, of course, the same argument from authority using a guess driven by belief can be used against any claim for the superiority of the OOPs paradigm. And vice-versa, too.

Which is why such arguments are worthless.

"I just find that real world data and process structures are neatly mapped by either OO or finite state machines."

And LL grammars are neatly, indeed perfectly, mapped by recursive descent parsers written using the procedural rather than OO paradigm, and are not expressible as FSMs. OO has its place. It's not a panacea and it is by no means the best fit for every programming problem in the world.

"But to get to the point of using the framework to solve problems that generally arise we encounter a learning curve of the framework. Although Openacs provides a certain degree of abstraction IMHO its not enough."

OpenACS is a far too much a grab bag of things. In some areas our abstractions aren't orthoganal therefore less useful than they should be (a bunch of things that only apply to the content repository, rather than all objects, for historical reasons, being the primary example).

This has more to do with a messy implementation and design history than the choice of implementation language. If a modern OO language had been used, we'd still have a system with a content repository built by aD folks on the west coast who made no effort to work with their peers at aD on the east cost, glued to the side of the ACS kernel developed by those east coast aD'ers.

That was due to poor software project (or software company) management ... not programming paradigm. Or, more fairly perhaps, it was simply a reflection of the fact that aD had a bunch of high-visibility clients demanding that complex projects get done in a short period of time.

We can't shed our history, though. That's why I fundamentally find these discussions not very interesting.

We'd make different choices if we were to start over ... but unless we're willing to start over, we're stuck with the history.

One example has to do with RDBMS support. If we were starting the project today, we'd ignore Oracle, I'm sure of it. Many of us would jettison Oracle today if it weren't for legacy users. It would greatly decrease our maintenance efforts and would simplify new development if we only had to develop for Postgres.

Yet, when we made the decision to support PG and Oracle equally when we took over the code base from aD, it was undoubtably the right decision. Malte had an early success story, being able to support a client's ACS 4.x packages on an early OpenACS core seamlessly, others had similar experiences. It gave our project legitimacy and led to a steady trickle of ex-aD'ers becoming part of our project.

But viewed from today's perspective ... man, it sucks having to support two RDBMS's. Imagine a toolkit without a single .XQL file!

History is like that.

Roc, Thanks you for supplying this reference. I found it very useful.

http://home.pacbell.net/ouster/scripting.html

I think it can be used in our marketing toolbox.

Wow such a long thread, I don't follow OACS daily anymore these days. But how I wished I have followed this one and grabbed that dual Opteron, rather than I buying myself an a64 last week.

I think for one OACS is pretty good, although maybe getting too old and less sexy? For me I tried to look at other platforms, although not that actively. It seems I keep on ending up with OACS since the biggest factor not to use OACS is the learning curve. I have gotten over that and for me its a bit backwards to use other platforms. Still you have that thought at the back of your head, that other people are improving other platforms and maybe one day OACS has been left behind. There is also the intagibles of how great the community is, also the fact that I was able to make a few friends here. So I guess in general its a good community and hopefully the good software will improve over time.

I also see the trend that OACS is going away from the quick but dirty ways of ACS3, which is good since things are more structured, but bad since it puts higher learning curve for new developers and growing complexity of the system. I am not saying that OACS is hard, but its getting bigger and deeper. So for a developer that is progressing to understand OACS, there is more things to learn.

In case you guys decide to change the design for OACS site, just email me. I can still find a few of my designer friends to make another facelift of the design. I really think the design is way too old already, was it 2000... 2001? I think its too old.

Anyway just an opinion from the other side of the globe, nice post again here.

Jun,

I agree, I think the openacs site needs a new face lift. If you can get your designer friends to create a new site design for us, then that would be great!

See this redesign study done by short time community member Steve Ivy http://www.thedesignexperience.org/openacs/oacs-design-hack.png
Don, I can't deny that recursive descent parsers are another good model - I am too embedded in the Perl community to ignore that! Closures and parsers together are a powerful combination. Of course TCL doesn't support closures either...

Should we perhaps start summarizing the ideas brought up by this thread, and classify their difficulty and importance. For example, sexing up the layout (possibly with the skinning stuff) in the toolkit and the website would be high impact for relatively low effort - of course it requires designers. I believe the intention was to have the Collaboraid .LRN design put into .LRN 2.1 or 2.2 - maybe we should chase that dow because it's certainly an improvement and would look quite good on the Openacs.org site.

I have (very) limited time right now (two deadlines + a wedding ;) but I'm willing to make a first pass at it (summarising) or make a second pass over someone elses first pass.

The Collaboraid graphic design that was done for Sloan is available, I'm sitting here with DaveB and his checkout of .LRN HEAD appears to have it. Working that into a standard theme usable by the upcoming theme manager package is an obvious thing to do.

As far as the openacs.org site design ... fine w/me. The current design's so much nicer than the previous one but that doesn't mean we should rest on our laurels. If a subgroup of people want to work on new graphics for the openacs.org site proper I suspect the community would be pretty open to it. There's only one way to find out, though ... ask around.

Well if noone else does, I'll look at it when I'm back from my honeymoon (5 weeks from now).

Ever since John's post I want a "lickable" Look n feel :)

while someone is doing the redesign of the site, can we get rid of that "GAY dog"?
I guess I would be dissatisfied with Ruby/Python as well as frameworks based on them at some point of time 😉 as I am with TCL in Openacs land but in general I have been very impressed with what Openacs can do for me. At the end of the day I see it as cool toolkit which enables me to solve problems quite effectively.

My experiences with other frameworks made me really appreciate the openacs community , the focus and quality of the contributors.

This particular thing got me started to ask my programmer friends to take a look at openacs and its possibilities. The intial reaction was the "TCLing" feeling and the difficulty they faced firing up an instance without going through a lot of technical hazzles. My exploration is in the direction of making openacs interesting enough for increasing adoption among developers. Most need simple and quick way of demonstrating the development of an application like how Rails provides ( without having to go through the learning curve). I believe this is a very potent approach.

But I guess we are getting there partly with the installer.

Collapse
Posted by Andrew Piskorski on
Robert, yes, you can use (many of the dozen or so) Tcl OO extensions with AOLserver. XOTcl is probably the most powerful choice, and several folks here have used it with AOLserver and OpenACS - including the XOTcl maintainer, Gustaf Neumann.

So for someone wanting an Object-Oriented language (as opposed to wanting a not-Tcl-language) for use with AOLserver and/or OpenACS, XOTcl is obviously the easier approach. Python or Ruby or whatever would, at best, require additional work integrating the second language interpretor, adding good foreign function call interfaces, etc. etc. Someone might want to do it, but only so that they could program in Python, not just for OO.

As for the "last 5%" of web apps that non-OO tools like Tcl handle poorly, I'm not sure what those are, although I'm somewhat willing to take Jonathan's word for it that they can or do in fact exist. Hm, Jonathan, I see that you were investigating XOTcl back in 2002. Did you try it? How do you like it compared to Python? And how have you seen either one used on that "last 5%"?

Collapse
Posted by Andrew Piskorski on
Don, if PostgreSQL 8.0 had been released 4+ years ago I probably wouldn't be using Oracle at all, either. And, probably, if 8 or 9 years ago Illustra (PostgreSQL's uncle?) had worked half as well as PostgreSQL does now, Philip would never have even tried Oracle, and ACS and OpenACS never would have used it all!

But, fundamentally, why is supporting multiple databases so costly in OpenACS? And must it necessarily be so? (I suspect the answer is "mostly yes", but I am not at all sure.)

ACS 4.0 added the first hooks for multi-db support, but at the same time also made multi-db support harder by moving so much functionality into PL/SQL code. That seems rather ironic.

Lots of other projects claim to support every database under the sun, and even that doing so is easy. My impression however, is that most of those do so via dead-simple data models, and generally making trivial demands on the RDBMS. And thus presumably many of those applications pay an extremely heavy price for their "multi-RDBMS support", whether their authors realize that or not. (The cost is merely hidden under other accounts, like "can't scale" and "general suckiness".)

Considering alternate realities can be educational, sometimes. So, what I'm wondering about, is or was there some middle ground? Some way that, if you were hypothetically starting from scratch, you could get 80 to 90% of the RDBMS power goodness of OpenACS, while reducing the cost of multi-db support down to, say, 10% of what it is now?

Wow. I have no idea what happened with xotcl. My son was born about a month after I made that post; maybe that has something to do with it...

For my purposes I guess I wouldn't need an OO language per se as much as I just need a language that has the concept of an actual reference. Upvar is a poor substitute, but more importantly it still doesn't let you do things like store one hash inside another. It's really quite frustrating when you're building an object cache to have to split everything across 3 or 4 nsv arrays, using "array get|set," when every other language you've used for 10 years wouldn't inflict this on you...

I do enjoy TCL's minimalist style in general, but this particular corner of the language bugs me a lot.

Andrew, I wouldn't bury so much logic in PL/[pg]SQL for starters. In reality, we can auto-generate packages the include "new" and "delete" functions and as the dynamic types package grows (and gets moved into OpenACS proper as it should, as it becomes integrated better and better) I'm ready to propose we do so and that packages include nothing else.

Why keep "new" and "del"? Because some old code does more than a simple "create supertypeobject/insert params into my type specific table" operation (bad idea, IMO, but that's life). So for such types we need to call "new" so we might as well auto-generate "new" functions for new types.

Anyway ... over time our idea is to move more and more to high-level Tcl APIs for the declaration and manipulation of types.

One thing we won't do, and which will always make query support more "difficult" for us than for frameworks that bury logic in the application layer (generally the object layer) rather than use complex queries, is, well, get rid of those complex queries.

Because as you point out ... using the RDBMS properly as we do aids scalability. The simplistic approach of (say) building a container object containing a list of object info returned by a simple query, that is then filtered by some other set of application object is by its nature, I suspect, less efficient than writing a complex query that returns exactly what you want in the first place.

Well, I think we should just keep repeating "Tcl in many ways sucks". That's life. But for this application space ... I think most of us agree that Java sucks more. Maybe Ruby sucks less, but that doesn't put us at the bottom of the barrel when it comes to language suckinesss ...
Collapse
119: Tcl does not suck (response to 118)
Posted by Andrew Piskorski on
Tcl is excellent, not sucky. Or rather, IMNSHO, the only way anyone can legimately claim that Tcl sucks is by taking the stance - the supportable stance, in my opinion - that all programming languages suck, to greater or lesser degrees. And on that measure, Tcl sucks much less than most. Overall, I consider Tcl a major asset to OpenACS, not a hindrance.

I think this may largely be because Tcl does a lot of stuff well, and nothing really wrong. There are various useful things which it does not even attempt to do at all. (Perhaps closures, continuations, and macros as in Scheme. Certainly vector orientation, matrix math, and sohpisticated statistical libraries, as found in S and its open-source dialect and R. One could go on and on listing things like static type inference, etc.)

To use one example I am intimately familiar with, although S is a very useful language overall, it feels replete with small but obnoxious legacy design and implementation flaws. So even though I use S for things I would never want to use Tcl for, I would argue that Tcl is the better designed language overall. Apples and oranges there, but if you have to compare them, Tcl looks like the cleaner fruit to me.

A language which locks you into broken misdesigned features (as seems very common) is much worse than simply staying out of the way and giving you tools to extend the language as you need (Tcl does; most non-Lisp languages do not). Tcl seems to strike a good balance there.

I naively hope to live long enough to see programming languages which are an order of magnitude or more better than what we have now. (And demonstrated as such scientifically; not just via hand-wavy philosophizing like I'm doing here.) Maybe from those Haskell-using academics. Maybe atypical practitioners like Jean-Claude Wippler will come through with something. Or both. And Tcl, Python, Scheme, and Ruby will all seem more backward than Fortran 77 and COBOL seem today. I don't see that happening terribly soon, though.

Anyway, probably none of this is news to anyone here, so enough of my rambling. I just wanted to emphasize that technically anyway (ignoring marketing), Tcl is a large net asset.

Andrew, this isn't 1990... I respect TCL's strengths more than anyone I know outside of the OpenACS community, but sucking "less than most" is not a sufficient condition for excellence anymore.
Jonathon is probably wise in wanting to avoid a 'TCL sucks/TCL's ok' thread. Ulimately though, we need to decide on a way forward and hearing peoples gripes makes it possible to estimate the biggest issues and needs.

XOTcl is not such a bad idea, but the key is deciding on an approach that best deals with our limitiations and un-attractiveness to developers and implementing it. If it's XOTcl, then we need to implement ACS object creation and manipulation as XOTcl objects, and make a new Object based wrapper on top of form builder.

Looking at some XOTcl docs, it does give us a lot of what we need:

  • Real class-based abstraction
  • Mixins (aka. interfaces in Java)
  • Object passing - which effectively deals with the lack of references and rich data structures in Tcl

If there were replacements for the core toolkit like forms, permissions & subsite, that accepted object passing, most developers would be saved from upvar without adding another language (although XOTcl could rightfully be called an additional language).

It's still unsexy however, and has no sizeable community. Which leads me to a question: by what criteria should we choose the way forward?

I think the sexiness requirement should be limited to the look & feel, while any developer environment choices should be guided by the following:

  • Practicability - what benefits it would provide
  • Pragmatism - how realistic it would be to implement in OpenACS
  • Profile - how big and prominent is the community surrounding the language/technology

IMO Object Orientation wins all of those critera against the alternatives. Whether XOTcl, Perl, Ruby or another language fits those criteria better than any other is a more complex question.

If nothing else, this thread makes clear that many are not happy with the status quo.

Collapse
Posted by Malte Sussdorff on
I'm a little bit surprised that the discussion turns to an argument about the benefits of an OO language. If this is seen as a major obstacle to the sexiness of the toolkit, maybe we can find someone to expose ACS Objects and CR Items to Java using ns_java or tclblend or other tools. Same goes for Python and Ruby. After all, we just need to be able to call a different languages procedures. Furthermore, due to the fact that some important parts of the functionality are hidden in the database, we don't have to care about the application layer language.

Therefore I think the easiest way to test if supporting OO will attract more users to OpenACS would be to make sure you expose the TCL API and the ACS Objects to Java and allow the Request Processor to serve pages written in Java along the ones we currently have in TCL. This allows anyone to use the existing functionalities while enabling her to write her own additional functionality in Java. This approach has the benefit of easier connectivity to proprietary third party tools, as they usually tend to come with a Java API.

Sadly I have no clue how much effort this would be and I guess unless someone really needs this for a client project or because of the need to finish a thesis we are not going to see much process.

Collapse
Posted by Mark Aufflick on
Malte you are right that it is not really what should be in this thread, Maybe we should start a few parallel threads based on this discussion.

I would like to stress that all of my discussions surrounding the language are because this thread has identified that TCL and it's shortcomings are one of the barriers to getting wider developer involvement.

OO or another language does not address any of the other issues, such as visibility. But taking visibility, if we are to market ourselves to developers (or their managers) then we must solve the developer issues first to avoid a poor first impression.

On the subject of languages, your idea to use ns_java was exactly my idea that I mentioned earlier about ns_perl. I don't particularly like the Java way of doing things, but with luck it may be possible to enable ns_java and ns_perl at the same time (and my dislike of Java doesn't count as a choice criteria ;)

The catch 22 is that any other supported language would need to be documented and supported nearly as well as the TCL Api. Again with luck, api calls should be simple translations allowing the api-doc to show calling examples from all supported languages.

So we're going to back up our belief, for many backed up with practical experience, that Java is not a particularly useful language for the bulk of web application development by working hard to make sure everything we do plays nice with ns_java, by replacing key concepts in the toolkit with object replacements in order to play nice with Java, etc etc?

I don't think so. I stand behind the things I believe in, and refuse to stand behind the things I don't believe in, and frankly Java doesn't move me in the least. I don't care *what* the application space is.

I'm going on the road for 3-4 weeks to photograph in the Mojave and Sonoran deserts, and the east New Mexico/west Texas shortgrass prairie. Leaving in a couple of weeks. Threads like this, where long-term members of the community wring their hands in angst over every thing that's wrong with our software stack while relative newcomers say, "Gee, guys, your toolkit ROCKS, you just need to get the word out!" are depressing, to say the least.

If people had fixed or resolved a bug for every post they'd made in this thread, we'd have over a hundred fewer open bugs in our bug database ...

OTOH, if the people most excited about the toolkit are the ones without enough experience to recognize its shortcomings, shouldn't that be an indication that something is deeply wrong?
When people talk about OO in the context of openacs I think they mean that things should implement a consistent set of methods so we can treat objects uniformly within the system. We currently have (not always though) new, delete, and name but they are implemented inconsistently or not at all for some object types. When you start talking about things like search indexing or object rendering to html, the interfaces are even less formal. Progress to something like xml and they don't exist at all.

People are saying OO when they really mean provide consistent interface and that certainly is something we all agree is useful, important, possible and probably not particularly hard when it comes right down to it.

Wow.

This community is amazing.

I've learned a ton from this thread. Thank you for replies to my questions, those are enormously helpfull.

From my naiive point of view XOTcl sounds/reads like the cleanest way to add the OO capabilites to the framework. And I agree, the sexiness part really should focus on the look and feel aspects of the project. One of the things that really attracts me to OACS is that there aren't bindings for every language under the sun - while I am a huge fan of Python, after learning and reading about OACS/AOLServer, it is clear that Tcl strikes the right balance for the category that it's playing in (more or less). When I install OACS, I know I don't haveto worry about compiling modules, debugging configurations, tracking down esoteric bugs just because I want to run say a calendar application from the repository (as an example). One language, one server, all done pretty much right.

I think the greater consideration is thinking in terms of application "scale". I would think that comments about java callbacks would add the greatest ability to scale beyond the frameworks current scope. Things like Perl, Python, etc, simply do not seem to have a scope much greater than what Tcl/XOTcl would provide anyway (outside of maybe the huge number of libraries, yet still, no one in their right mind writes stuff in Perl and Python where Java is called for). It would seem to me that Perl and Python support would scale OACS laterally moreso than vertically, whereas Java would allow the platform to scale both laterally and vertically simultaneously. The only question that arises from the java proposition is, just how much scaling do you need - if you want something that extends OACS so far beyond what it was designed for, perhaps one would of taken that into account in the first place and started with Java in the first place?

Hmmm ... this topic is very quickly getting me in over my head technically. I guess just count my vote in toward XOTcl as a solution that appeals to me on aesthetic merrits in relationship to the big picture

As a side note, it would be neat if we had the ability on the site to break discussions like these into a workflow and start hammering out some details. Say something as simple as Volunteers, Proposal/Specification, Vote/Feedback (inclusive of forum), Project Subsite, Organization of Task ( why isn't Project-Open.com on here? ), Progress and Bug Tracking, Publish. Perhaps there is one that I'm not aware of?

*sigh* so much to do, so little time.

Some tiny notes to anyone considering look and feel proposals for OACS ( I am planning on jumping in as soon as I get the hang of creating simple applications here ):

- this needs to be an official project with a project manager, perhaps listed under the projects directory, perhaps somewhere else as a special project (can this site be a project it self perhaps in the same way .LRN is?)
- start small, no big changes.
- the layout should stay mostly the same for the time being.
- incremental layout changes are better than huge changes as they let everyone 'fine tune' the site rather than invest huge amounts of energy into a site upgrade and then haveto live with it for a year or two.
- the site should remain 100% wide
- start by changing the colours, css, and fine tuning some of the visual cues such as colours, sight lines, links, etc. first.
- an alpha/beta replica should be available for public viewing/comments
- all workfiles should be available in some repository (preferably using Gimp, yes it is good enough to replace PhotoShop), such as CVS/SVN so that anyone can pick up the work where it left off, and incrementally push the look and feel forward one small step at a time.
- keep the logo for now. a logo change is huge. people may not like alex, but then again, when Tux was proposed as a mascot for Linux everyone thought that was stupid too. If a logo change is needed, I think that should be handled properly and not rushed.

This will keep the project from being overwhelming, and allow for incremental upgrades.

Openacs is not a magic wand that can solve all the problems for you neither is experience. To identify shortcomings you need foresight and the courage to look within yourself. This I see a lot in the community.

Openacs solves a subset of problems and everyone has his subset of problems to deal with. Some overlap with Openacs some donnot and hence the thread nearing 0.5 MB.

Apart from OO what I donnot see in the community is the recognition of the fact that most developers out there are not the best or technical proficient and amazing but what they need is a tool kit that solves_their_problems without having to qualify into MIT to do that. PHP enables you to do exactly that.
To my understanding one of the reasons that Openacs is unable to attract a larger developer base or a higher degree of contributions is because we donnot think about "them" whereas you would find a plethora of applications out there in PHP world created by the same people.

Imagine the scenario if the same developer is given a powerful framework such as Openacs ,completely opensource , easy to implement or extend simple solutions , a community waiting open arms to support you for any silly problem you have apart from being sexy for development.... Openacs would really ROCK and continue to ROCK n ROLL.

Jonathon, Jeff & Harish all have good points. I would also like to point out that the whole fixation with OO starts with the fact that Tcl has no rich data structures. Whether OO is or is not the best or even a good way to program, programming without rich data structures is simply abominable.

Yers the funtionality of our toolkit rocks. But show any sesoned (non-TCL) developer the quirks of ad_form, and then show him the source and they will run away screaming.

No disrespect to ad_form intended - I love it so much I implemented it in Perl! But it just takes too much wizzardry to be called structured programming.

So as I suggested a few threads ago, maybe we should agree to call this thread closed, and appoint a few people to summarise the thread into a list of good & bad points about the OpenACS tookit & community. Then from that list we can agree on areas that are important to improve, and what strengths we should be marketing.

At that point, the actual approach to deal with each weakness becomes a seperate issue that can be dealt with by different groups.

Who agrees that the useful debate is just about done (we are, after all, getting a little cyclical in recent posts)?

http://www.oreillynet.com/pub/a/network/2005/03/10/basecamp.html

Ruby on Rails is the open source web application framework we extracted from Basecamp. When we built Basecamp we didn't know we were building Rails at the same time, but that's exactly how it happened. Basecamp came first; Rails was born from Basecamp. Basecamp was the divine chicken, Rails was the egg.

I had some natural hesitation about using Ruby at first ("What the #@!* is Ruby?" "Why don't we just use PHP--it served us well before?"), but David Heinemeier Hansson, the first engineer on the Basecamp project, cogently made the case and I bought it. I'm thrilled with the results.

The way I look at it is this: I want developers to be comfortable with their development environment. I'm a designer and a business guy, not a developer. I'm not going to push PHP or Java or whatever just because I've heard of it. I'm going to defer to David on this. And if David chooses Ruby, then Ruby it is. It's all a matter of trust. If you don't trust your developer to choose the right environment, then how can you trust him to build the best application? Trust is critical here. And, further, why would you dare impact your developer's morale by throwing him or her into a language where he can't be as productive or as satisfied? You only get good work from people who enjoy doing the work. I'll take a happy average programmer over a disgruntled, frustrated master programmer any day.

ACS is the open source web application framework we extracted from Photo.net. When we built Photo.net we didn't know we were building the ACS at the same time, but that's exactly how it happened. Photo.net came first; the ACS was born from Photo.net. Photo.net was the divine chicken, the ACS was the egg.

Sorry, I don't see the point to my post. Nor yours ...

I'll take a happy average programmer over a disgruntled, frustrated master programmer any day.

Given that the best software engineers are documented to be one to two orders of magnitude more productive than your average programmer ... I'd say this statement is one of the most stupid ones I've seen in a long time.

OK, successful guy pats himself on back in public. He's getting his 15 minutes of fame. We'll see if it lasts any longer than that.

You may recall the Philip, ArsDigita, and the ACS had easily as much ink, notoriety and fame as these folks. And look! The work was crap! The tools are crap! It's all crap!

But today's darling, well, it's a panacea! The world's problems are solved!

It will be fun to talk about Ruby on Rails in, oh, five years or so. We'll still be here to talk about it, too, no doubt about it.

Sorry - I should have put in some text of my own.

I had no point - I just thought it was an interesting article, and contained discussions not irrelevant to this thread.

And btw Don, your points are totally valid.

I was particularly interested in the section I pasted in because we have discussed, at various points in this thread. a mythical CIO who may or may not make toolkit choices based on his developers opinion.

Here, in the interview, we have an actual president of a prominent and successful product, totally accepting the opinion of his developer over his personal (non-developer) views on PHP.

This in itself is heartening, and further strenghtens the views posted here that providing the right tools for developers is more important than being buzzword compliant.

Well, hey, fame's great, fleeting or otherwise. I well remember the standing I had after co-authoring the first commercial Pascal compiler, which just happened to generate code efficient enough to make it usable for the creation of system software for the PDP-11. Our company was well-known, too. We did a little public back-patting, as well.

Yet, today, hardly anyone's even heard of my old company. In 10 years, hardly anyone will be aware of Ars Digita (and if they are it will probably be because the OpenACS project continues to live).

Maybe Ruby on Rails will stick. Best of luck to them. But the fact that they happen to be the fad du jour is in no way a predictor of the odds of their project or company having any stamina, of being more than a flash in the pan.

Remember when Java was going to kill OpenACS, like three years ago?

Caroline Meeks made a good point today - she loses bids to LAMP far more often than anything else ...

I rarely find anyone here in Australia who ever heard of ArsDigita...

RE: Caroline's point - perhaps it would be illuminating to start a thread on what we lose bids against and what (we believe) were the major factors in losing the bid.

Collapse
Posted by Andrew Piskorski on
Don's PDP-11 Pascal compiler Well Don, Google does know, (perhaps surprising given the date), if only just barely:
"The story of TECO on the PDP-8 is convoluted. Russ Hamm implemented TECO under his OS8 (without a slash) system, and then gave a listing to Don Baccus at the Oregon Museum of Science and Industry (OMSI) who, along with Barry Smith ported it to PS/8. This was the beginning of what became Oregon Software, later famous for OMSI Pascal."
I think a book needs to be published in order to promote OpenACS. This is especially the case if you want students to learn to develop in OpenACS. I've taught a few courses on Java, LAMP and OpenACS, and the biggest complaint from students is the lack of an OpenACS book. I don't know why students are so dependent on books when all the resources can be found online. But that must be a psychological thing, especially when it comes to exam time, when they think that they will only be quizzed on stuff within a finite set of knowledge, such as a book.

There are lots of books on Java, Perl, PHP, etc. We need a book for OpenACS.

Collapse
Posted by Jeff Davis on
I'll take a happy average programmer over a disgruntled, frustrated master programmer any day.

Given that the best software engineers are documented to be one to two orders of magnitude more productive than your average programmer ... I'd say this statement is one of the most stupid ones I've seen in a long time.

Having worked with both happy average programmers and disgruntled, frustrated master programmers I think I would prefer the former too.

Also the numbers for team productivity I have seen say a 90th percentile team is about 4 times as productive as a 15th percentile team, and the 10x number was from the worst programmer to the best (and derived from a really small dataset) -- the number for average to best is lower...

Having worked with both happy average programmers and disgruntled, frustrated master programmers I think I would prefer the former too.

Perhaps more relevant is the apparent fact that the person making that quote was smart enough to realize that without his master programmer, he wasn't going to have a product. I haven't looked at programmer productivity numbers in a couple decades, perhaps average programmers have gotten better. Or perhaps the field's so glutted that nearly everyone's average these days.

Bottom line, though, the original quote consisted of a couple of paragraphs of which one could be written of any number of projects, and the other which simply boils down to "if you have someone good, let them use whatever language they want", which isn't particularly good advice, to be honest.

Hi,

I've got stucked reading those debates about the language to choose for openACS. I think those debates are worthless. Lets see why.

Lets go back to the begining (long ago) and remember:

TCL is not your friend
======================
Do NOT try to do fancy stuff in Tcl and if you do, if it doesn't work quickly, you might want to DROP IT. Don't be stubborn with Tcl. It is not worth it, especially if you are just beginning to learn how to work with it. The language is fairly simple, and most of the power of OpenACS comes from the API, not the programming language."

The database is your friend
===========================
Both Oracle and Postgres are supported by OpenACS, and they both are very good databases.

Yes, SQL and Oracle (now PostgreSQL too) can be challenging to get to know. It can be frustrating to realize that you can do whatever you want to do in 3-10 lines of code, but that it'll take you a while to figure out what those few lines need to be. But once you do, you'll be able to do lots of the stuff you want to do using the database. Given how powerful an RDBMS like Oracle or PostgreSQL is, you can, and should, do as much as you can in the database. Be creative in coming up with queries that can get as much stuff out at once as you can, so you don't need to do multiple queries (use nested queries, joins, etc.) The database is fast and powerful compared to Tcl (or ANY OTHER LANGUAGE between the database and the Web server). Use that to your advantage.

==========

The funniest thing is that all of this information belongs to our own pages: https://openacs.org/education/psets/advice

How can we go forward if we do not even know our own history and documentation?.

Félix.

"Everything in the database" is one of the reasons some aD sites had trouble scaling though, IIANM. Sure, "Oracle|PostgreSQL scales," but for a price. Anything past 4 CPUs gets expensive quickly, hardware-wise.

Some work belongs at the application server level, and forcing that into the database because your app server can't handle it is a kludge at best.

Jonathan, while your argument sounds fine in principle, it is so general that I'm not sure whether it really means anything or not in practice. What specific stuff was being forced into the database and thus killing scalability? And what would you recommend instead?

(And don't just say files for file storage, we all know that stuffing them into the RDBMS as LOBs is silly and always was silly. File vs. LOB is not a useful example of anything.)

I can think of some examples, btw, but they are fairly esoteric. E.g., I worked on a project which shoved real-time options pricing data (from an electronic feed; on only a small number of options however) into Oracle almost solely so that Oracle could fire triggers based on that data.

That actually worked adequately in that case, but handling a wider variety of options may have strained it severely, and larger numbers of (possibly user-defined...) triggers might well have collapsed it entirely. (The production system never saw such loads, unfortunately, and I don't remember what our scalability testing said about data feed and trigger-related loads, or after all these years, whether we even checked that at all.)

Note however, that the "stuff real-time options prices into Oracle and fire triggers" design decision had nothing at all to do with some sort of belief in an "everything in the database" principle. It was simply that Oracle's triggers were the only tool we knew of at the time for doing "active database" type stuff - and indeed, are unfortunately almost the only common such tool.

Oracle's Advanced Queuing software might (or might not) have been better than simply inserting prices into tables. And Jerry Asher later pointed out the massive superiority of the Rete algorithm for such things over Oracle's (naieve) trigger implementation. Stonebraker's Streambase stuff is no doubt also relevent.

So yes, if I was designing a similar major application all over again, I would definitely try to get the "hard active" data-driven aspects out of the (poorly implemented) RDBMS native triggers into some more efficient and more powerful system. (Perhaps I would even write a PostgreSQL module to do so.) Plus at least two other major architectural differences which I haven't mentioned here at all.

But somehow I doubt that's what you meant in your "everything in the database" comments...

Félix, you are mistaken in thinking that we are debating what language to choose for OpenACS. No, we are just discussing random stuff, stuff which might, maybe, end up having something useful to do with how to make OpenACS better. (Or might not.) That's it, really.

However, you are probably wise to point us back to our officially documented advice on such issues. :)

ok so this weekend I'll have built all the binary rpms for the dependant software for multiple distros:

SLP 9.2
SLES 8,9
FC 2,3
RHEL 3,4
CentOS 3,4

Who do I contact/bribe to get the officall openacs repositories(apt4rpm/yum) set up?

Should this be on one of the actuall openacs.org machines?
If not, who is in charge of the DNS for pointing the appropiate host at the appropiate IP?

That brings me to the next question...
What should the repository host be called?
repo.openacs.org?
packages.openacs.org?
somethingelse.openacs.org?

Derek,

did you make the binary RPMs from the source RPMs I made available? If so, did you have to make any changes to them in order to compile them to binary RPMs on the multiple distros you created them for?

Feed back is greatly appreciated.

Derek and Bart, these are great contributions! You've got my ear. How can I help you? I've started a new thread for this.

https://openacs.org/forums/message-view?message_id=278642

Rob, this is great stuff. Can we include it in the Why use OpenACS page?

Is there currently any centralized marketing document for OpenACS?

UI and lickability, I know, is a priority in the community. What, in peoples' opinions, needs to be done to improve look and feel, subsite navigation, and the installation and use of portals?

Cheers.

Never mind the negative crap? Are you kidding? Most of that negative "crap" is true, so why should we not mind it?
It's not like the bugs/problems are going away any time soon.

IMHO, it would be much easier for OACS developers to start over than to keep on fixing the never ending flow of bugs/problems. OACS is way too complicated to keep on developing and keep your sanity at the same time.

Hint to anybody new to OACS - not a single book exist for OACS and/or AOL server. There is a reason for that. If you are not a programmer/developer yourself, think twice before taking on OACS.

Just my 2 cents

Yeah, to complete the picture, other online books are in the "other related tutorials" of the handbook: https://openacs.org/xowiki/docs-dev-tutorial
Anything missed should probably be added there.
Well Vic, in which case you have all of our blessings to go troll another open source project.

Meanwhile the rest of us will merrily go on using this wonderful, enterprise class framework to build applications, corporate intranets, services and businesses.

The lesson here to anyone new is that you have to pick the bright tool for the right job. If you don't have the skills to do so, hire someone who does have the skills to help you make the right choice. No single project can cover all bases. Sometimes OACS is the answer sometimes it isn't. For us it has saved us time and money (and sanity) no other framework can offer and we wouldn't work in any other framework. For you the calculation may be different.

Otherwise all you will end up is a troll on a forum of a project you don't understand looking like a fool.

Just checked the so called Documentation Project again. Seriously, dudes, WITH ALL DUE RESPECT, you have to be kidding. Lots of links don't work and now there is even LESS documentation while there are MORE links pointing to that documentation. How can this be such a mess, while you are creating projects to, supposedly, ORGANIZE things?

Let me give you a hint:

1. We need three documents (just three documents under three urls, not hundred urls)

2. The documents are: Developer, User, Administrator

3. The documents need to have some logical start (LEVEL 0 -- someone who knows nothing about OACS and it's commands) and cover everything, from A to Z. And they need to include documentation for AOL server.

I only found one tiny reference on how to create my first page (this time I did, not before), and even that made me happy.

You may call it trolling but if you look at it soberly and honestly you would have to admit that OACS is falling way behind other projects while it didn't have to. I just came back from Sugarcrm conference. They have done everything in their power to make sure I learn it fast, there is lots of good, clean, working and clear documentation. And they actually listen to their users.

I keep coming back here not for trolling but because I really want this project to succeed.

Regards

Vic May,

How about three documents (User, Administrator, and Developer) starting from one url? https://openacs.org/xowiki/openacs-handbook Ignore the categories on the left side-bar if you want to get more of a document tree perspective.

cheers,

Torben

:)

Let me rephrase that a bit:

Three links with three documents (html and/or pdf), not three links pointing to many other links.

It is pretty obvious that the developers and some users grew up with or grew into OpenACS over many years, probably from it's beginning and all that information seems natural to them. Not so for a person who has never used it before.

Cheers

Vic,

It's true that those pages try to keep 1 topic per page, and are organized somewhat organically to fit various reader's needs.

It seems like you are wanting documentation in a traditional format. What about this? https://openacs.org/test-doc/

cheers,
Torben