Forum OpenACS Q&A: Why do you use OpenACS and .LRN?

Collapse
Posted by Dave Bauer on
I am sure this topic has gone by before, but since someone asked, let's try to get some recent posts on why people are using OpenACS today.

Here is the last article we had about why to chose OpenACS http://www.acornhosting.net/whyoacs

Personally I use OpenACS because it is a good fit for the types of web applications I am building. The features that are especially important are subsites, and the sophisticated groups, roles, and permissions system. In addition I appeciate the way I can build content-driven applications with the content repository, allowing me to use the higher-level apis to handle the database independent storage of content.

Collapse
Posted by Dave Bauer on
I thought of two more huge reasons

1) Templating system, I really appreciate the power and ease of use of the templating system.

2) Database API, this is a very powerful and easy to use feature of OpenACS and a great addition to the AOLserver platform.

Collapse
Posted by Nick Carroll on
I really like the DB API and placing my SQL statements in the XQL files. I've tried managing/maintaining SQL statements in other frameworks, and they just don't compare. Ever tried querying the DB using Java? Just looks messy having an SQL statement passed into Java methods as an argument.
Collapse
Posted by Simon at TCB on
Many reasons. Here are the most prevalent

TCL

This for me is the key usp. TCL is to agile development what Shakespeare is to actors. An extensible, embedded scripting language with hugely powerful features, a dynamic basis and (with the inclusion of XOTcl) a superior object system is a big win for me.

The community

Ok, it may have gone quiet recently and it may not be as large as its could be, but there are some really bright people around. What it lacks in quantity is mitigated by quality, in my experience.

The db interface, i.e. db_1row, db_transaction, db_multirow.

A small element, but a surprisingly neat one. Ok, call me lazy if you like, but the sheer simplicity of interacting with the database is again a major win. Databases are hugely powerful tools and yet somehow never popular with your stereotypical Java developer. I suspect the proliferation of clumsy interface libraries out there has a lot to do with it. I mean, take Hibernate for example, good code it might be.. but why? I love Oracle because of db_*... and I'm not ashamed to say so ;) Surely this must be one of the most beneficial aspects of ACS for new/less experienced users. Its just so simple.

The data model

Its a thing of beauty (a quirky kind of attractive). I thought it the very first time it was released and I still haven't changed my mind. Easy to extend and a good core set of design principles.

Tcl/Adp Pages

I mean, combine this with db_* and... well, how much easier can this stuff be? Try cranking it out using Java/Tomcat... or Apache/Perl... do you want to hit your deadline or not?
I'm always a bit surprised when newbies say its difficult. I begin to wonder if they've installed the same software I have. ;)

Note to self.. early delivery, rapid development, low risk... ACS should be pitching these.

The way I can always seem to solve every problem with it

Websites?! Pah. Never built one (well not much of one). The ACS is really an Integration Engine, and its not surprising, its built with TCL, the mother of all Glue Tools. If there's a gap it can plug it, quickly. With the database underpinning (and potential grid processing with 10g) its the perfect solution the the general business problem of 'and we need some other kind of system to do this..'

I always look forward to working with it

Dunno why, can't rationalise it. Some things have 'it' some don't. ACS is one of those tools (again like TCL) that you build a relationship with. If this could be expressed in marketing/sales terms I'm sure it would be a gold mine. There is far more to be said for enjoying your work than whether its built with this or that. I find it hard to believe anyone could build the same relationship with JBoss.

And perhaps I should end with one reason why I don't like it.

My file watches disappear when I restart the server

It drives me banonkers

Collapse
Posted by Claudio Pasolini on
In the last year I used OpenACS mainly to build an almost complete ERP system.

I used OpenACS in various kinds of applications and I can't think of any of them for wich I wouldn't recommend using it to get the job done.

I find Simon's list of arguments almost identical to that I wanted to write down and so I limit myself to emphasize the wonderful database integration (also thanks to Aolserver) and the templating system.

The one thing that I don't like is the implementation of the service contracts, wich I find too complicated. I think that they are modeled after the WSDL, but personally I find much more easy to deploy a web service via soap-gateway instead of using the service contracts.

Collapse
Posted by Sam Nicholson on
I began this year looking for a system which contained a bug tracker, support forum, with user management and authentication. In addition, I wanted it written in a system that allowed me to use html tagging to call the core system from html, a la PHP. Oh, and it had to be expressly *not* PHP. I've written some extensive applications in PHP, and find that with it I can be very expressive, however the fact that every language bug fix changed the semantics (if not the syntax) of a part of my code finally drove me away from it. It also couldn't be LISP or Perl. For various reasons, I find that I am not up to the task of programming in either.

I went so far about a year ago to search for the (for me) ideal scripting language and to embed it in my favorite webserver. If anyone is interested, I've got ICI embedded in thttpd. Likely one would realize that no matter how hot that combo is (and it *is*), like the mountain climber just outfitted with strong, lightweight gear, one was at the bottom of Everest. I had neither ticket tracker, support forum, nor user management. By February, I was seriously considering returning to PHP, when my websearching landed me at openacs.

I've never been a fan of tcl, but after spending some time I came to the conclusion that tcl gives the power of LISP without the (for me) mind (and finger) numbing notation. Couple that with the AOLserver's decidedly brilliant method of gathering up relevant data in the .tcl and rendering it with the .adp, and this tcl enabled system looked pretty good.

So, here is a system with a reasonable language, a template system (.adp) in which I can be reasonably expressive, coupled with pre-built and community maintained forum, tracker, and user management, and I've just saved a year of coding and debugging.

Unfortunately, we had already selected Postgresql 8 for another use, and felt that we could not support two versions. So we had to wait until OpenACS caught up. I feel that it will be worth the wait. The first time I built our app, back in 2000, it took 6 months, and the majority of that was spent catching up with PHP's language changes, and re-inventing numerous wheels. Though learning tcl and OpenACS has at times been frustrating, I am delighted that everything I prototyped in March still works unchanged today. No PHP code I ever wrote survived a single system update unscathed.

In most other template systems, if one wishes to use the "toolkit" one must adopt the system lock, stock, and barrel. We have critical time series data and filesystem based provisioning data that aren't going to be relational any time soon, if ever. So the icing on the cake for us was that we could use OpenACS' SQL-backed forum, etc, and through the flexibility of .adp, converse with our filesystem stored data as well.

I didn't mean this to be a PHP bash (I've got some very complex PHP-based code still running, albeit behind some serious ICE), but I find that the stability of each of the AOLserver, the TCL, and the OpenACS means that I can spend great effort in getting things done with the confidence that they'll keep running for quite some time. Only Perl and the C language give me anything like that kind of stability, and neither works for the way I write web apps.

So, thanks for making it possible!

Collapse
Posted by Joe Cooper on
I chose OpenACS for http://www.virtualmin.com because I needed a pretty good forums package, a pretty good ecommerce package, and a pretty good bug tracker...and they all needed to work together and I didn't want designing an ecommerce package to become my full-time job. OpenACS looked like all of these things would be extremely easy to implement...it didn't turn out to be easy, but in the end I did get it all working in a reasonable amount of time (certainly less than writing it from scratch, even in the allegedly magical Ruby on Rails) and my users are pretty happy with the results. Some aspects of the deployment turned out nicer than I expected, but most things were disappointing.

I also maintain a Plone site (http://www.scipy.org), and have for several years. The OpenACS learning cliff is a little less deep and deadly than Plone (another contender in this most recent endeavor). Both OACS and Plone, however, have a cliff rather than a learning curve, and I've never managed to scale either one. Being "easier than Plone" is like being smarter than G.W....the bar isn't very high.

I still have daily error emails from OACS 7 months after deploying the site, because I have no clue where to start to fix the problems. They aren't affecting core parts of my site and they don't appear to affect security, so I haven't lost any sleep over it...but it's frustrating. But not as frustrating as Plone. Plone reinvents everything--database, webserver, relational model, object model, Python grammar and object rules, and more. It was just too much. At least in OACS, if I need to do something stupidly dull like adding web content with a tiny bit of live content, I can figure out how to do it from scratch in an hour or two or three or four by reading the database documentation, the TCL documentation, the AOLServer documentation, and the OACS API documentation (actually, the OACS API documentation is a negative entity...one has to read the code to know anything, since the API docs are very nearly always wrong). In Plone, I never even got that far. If I couldn't add it via the ZMI, I just couldn't do it. Both systems are very poorly documented. Both Plone and OACS are inaccurately documented to the point that following the official tutorials does not result in working code. If I could figure out how to do things the right way, I'd write up documentation (I write a lot of documentation, including a published book...I'm not afraid to help out with docs). But I've never had any luck, even the tutorial was a challenge that ended with me as confused as when I started. I simply have to hire someone who already understands the system when I need new code (thanks, Malte!). This is disappointing, as I'm not a programmer but I've written quite a lot of code in my day (ten years of perl, bash, and bits of C/C++ and Python and lisp, so I'm not helpless)--this stuff shouldn't be rocket science. I really thought I'd be contributing useful code to OACS by now, but I'm not much closer than when I started because I can't devote full-time to developing for OACS. With the only other large modular system I've ever worked on seriously (Webmin), I learned perl and the system well enough to write a working Webmin module in a couple of weeks. It took me that long just to fix a few bugs in some of the packages I'm using in OACS, and I've never written a new package from scratch that worked (and it's not for lack of trying).

Lots of cool apps exist for OACS in various states of completion. Some of them (Project Manager) seem to point the way to a much better OACS. So I have high hopes that things are getting better. I was pleasantly surprised by the quality of some packages. Most were disappointing because they simply don't work in current OACS versions, and there appears to be no active maintenance happening for the majority of them. It has become apparent to me that OACS is a framework (like Rails, like Mason, etc.) and the apps are an accidental side-effect. Like all accidents, some turn out alright, but most are just trouble. I wouldn't want to write an ecommerce application from scratch, but I certainly had to write a lot more code in OACS ecommerce than I expected. The impression I got from OpenACS.org was that many of the packages worked, but this is simply not true--and it gets worse with every release. I've heard it referred to as a community system in a box. It isn't. It is a bunch of cool components that kinda-sorta-work if you squint a bit and don't mind writing some code.

Umm...Ok, so I'm being a bit bitchy about stuff that I got for free. The fact is that I wouldn't have been up and running as fast as I was with any other system that I'm aware of and I'm grateful to have it all running. Ecommerce is making sales for us, the forums and FAQ package are helping us keep our customers informed, and the bug-tracker (which is excellent with a few caveats) help us make our software better.

So what's right about OpenACS?

It has a lot of code, and some of it even works. The size of OACS has been given as a negative, and I understand why it would be to a developer that wants to understand everything and build all custom solutions. But for someone like me, I want to write the fewest lines of code possible. Me writing no lines of code is better than me writing a hundred and a hundred is better than five thousand (even if the five thousand solution is better than the hundred line solution). I only had to write a couple hundred lines of code to get virtualmin.com running and providing most of the services we needed. Malte wrote another couple hundred for us in the ec-serial-numbers package, which provided the last mandatory component (software registration via the ecommerce package). My couple hundred lines were hard-fought and painful, and took weeks to write, but I couldn't have written five thousand lines of Ruby in the same period of time (and it probably would have taken 25,000 loc to start from scratch in RoR to get our minimum functionality and half that in some of the other toolkits that have a few of the necessary components pre-written). I don't really want a great framework. I want apps that work together. I've played with RoR, and it was fun. But I'll never build an entire web application from scratch, so it is simply not the right tool for me.

Performance is excellent. I'm sure someone is reading this and scoffing...but my plone site runs nothing complicated, never has more than a half-dozen users, and a ~500 MB database, and the process size is never less than 1000 MB. I run two OACS instances (devel and production) in about the same amount of memory on Virtualmin.com. CPU usage is never a problem, and we have several times the users and usage of SciPy.org. OACS runs just fine. I'm sure a custom solution would be smaller and more efficient (our problem set is not huge), but having a $300 a month server rental bill is cheaper than spending man-months writing code just so it would fit on a $100 a month server. Not to mention the opportunity cost of not making sales while we were waiting for the app to be finished.

OACS didn't reinvent every wheel. PostgreSQL is a database that I've used many times in the past, and it is extremely reliable, fast, efficient, and well-documented. I like PostgreSQL and I trust it. I never trusted the ZODB, and never will. I'm sure the technology of the object database is fascinating, but I don't care one whit about cool technology. I want a site that works. OACS could stand to stop reinventing so many of its own wheels...at least long enough to make a few of the packages work and the documentation a little bit accurate.

I can see the potential of subsites and groups to make it possible for me to do things with our site that would be really good for our customers. But I've never been able to get it to work. Almost none of the packages that I tried are sub-site aware (or even instance aware), so they can't be used in ways that would be useful to me. So, these things are a really cool idea and I thought I would use them heavily, but they are not reality for someone like me. Maybe when my revenues are significantly higher, I can afford to bring someone in to make the improvements we would need to those apps (classified ads and news and press are the big ones that would be useful but simply don't work right in multiple instances--I've tried to figure out how to fix them, and got some advice on the forums, but the advice just didn't penetrate into my opaque view of OACS, so I gave up).

Permissions are another area that sounds great but appear to be impossible to do correctly on this scale. Plone/Zope uses them in pointless and confusing ways, and so does OACS. The bug-tracker is a great example, where OACS permissions don't even seem to apply. I've tried on a half-dozen occasions to add the ability for normal users to comment on issues that they didn't create (ya know, like every other bug tracker in existence). I've spent hours reading the code and the docs by the package author. I've asked in the forums, twice (as have others), with no response. It seems an insurmountable problem--and I can't even understand why it is such a difficult problem. It ought to be one if-then switch, but the logic is tucked away somewhere invisible to me (and others, since it has come up multiple times).

What's been most disappointing?

The original implementors of ACS were wrong about everything they ever did. At least, that must be the case, because everything is being re-written by every developer in wholly different ways (it also appears that every current developer is also wrong in every possible way, since everyone writes their own everything from scratch and doesn't document it). Packages are more broken today than when I started using OACS a year ago. In short, one should now approach OACS as a toolkit just like RoR. One cannot count on any of the packages actually working once installed--even many reasonably new ones are broken in 5.2.0. Apps that were broken in 5.1.x when I started working with OACS are still broken today, and bug reports result in no changes...sometimes even the patches I send take months to be applied or don't get applied at all (Torben is an exception, and has been very helpful, but ecommerce is still utterly unuseable in the state that it comes down from OACS.org and doesn't work at all in 5.2.0). I understand this is a volunteer effort, and I work on several OSS projects myself, but it's time for packages that haven't been touched in a year or more to be marked unusable/unmaintained. If I had known the actual state of packages in OACS, and how much time I would spend figuring out that "this one doesn't work in multiple subsites, so we can't use it yet, and this one doesn't work if you have more than one instance, and this one just gives a traceback and hasn't been touched in CVS in two years, and this one doesn't uninstall without a traceback, and this one only supports Oracle despite apparently having code for PostgreSQL", I might not have chosen to use OACS.

Documentation is copious but wrong. And difficult to read, due to an odd docbook processing quirk that has never been fixed (the doubled up shell output).

APIs are constantly shifting with no documentation of the changes. After upgrading my devel site to 5.2.0 last night, I became aware of a huge swath of deprecated functions in most of the modules I use. I wasn't aware these functions were being removed, and the only mention of it anywhere was in a forum post in February of 2005 that Torben helpfully pointed out to me. Worse, there is no map of what functions replace the deprecated functions. If I can't count on the API, I can't much count on anything, can I? I'm all for removing dead code, improving functionality, etc. But give me something to work with. I have no clue how to replace the deprecated functions as there is no documentation about what they've been replaced with. Maybe I've just gotten involved in OACS at the wrong time. Maybe in a year all of these problems will be resolved and OACS will be a beautifully crafted object oriented festival of goodness that reads like poetry. Or maybe it will be even worse, because everyone wants to re-write everything and no one wants to document the changes and no one wants to maintain packages that aren't proprietary to their company (I don't mind proprietary components to Open Source projects--that's what Virtualmin.com is selling--but take some responsibility for the core, too). I'd help with documentation if I even knew where to start to comprehend what is changing and how it works.

Anyway, I don't care about the language, or the webserver, or whether the code is object oriented, or includes logic in the database, or has a habit of calling me offensive names in the error logs. I want a base set of applications that:

  • Work
  • Work together
  • Can be extended in small ways without huge amounts of effort--I don't care if the implementation is a bit ugly, if it is documented well enough for me to write the necessary ugly code
  • Can be upgraded and maintained without more than a few hours of downtime every 12 months or so

Things that don't matter to me:

  • dotLRN
  • ERP
  • Toolkit coolness
  • Object Orientation (or Functional programming, or XP, agile development, or whatever other buzz-word you like). Either the code is good and it works, or it isn't and it doesn't.

I know that everyone wants a different set of packages to work, and so maybe the OACS that the current developers invision is not the OACS that I'd like to have--it seems that my focus is quite a bit different than most folks here, since a lot of focus is on dotLRN and major overhauls of the core APIs. I'd feel better about taking responsibility for the packages I need to work (ecommerce, ec-serial-numbers, forums, bug-tracker) if the API was a bit more stable than quicksand and changes to the API were documented. I guess I'll just have to see how it all shakes out--I'm pretty well committed to the platform for some time, and I'm also pretty certain that at some point I'll get a grasp on OACS enough to make some larger improvements and write new packages, but my enthusiasm for the platform so far has a negative correlation to the amount of time I spend with it (not because it is bad, but because I cannot grasp it and it gets harder to understand with each new revision because the docs reflect reality less and less).

What I'd like to see happen

Clearly I'd like documentation to improve, and the APIs to stop moving (without documentation).

I'd be almost just as happy if someone would tell me a user-facing package (i.e. not a service used by other packages) that I can look at that is correct, uses the new APIs exclusively, and works. That would give me some examples of how I ought to be doing things. I might even be able to turn time spent with such an example into my own working code. I've asked for such examples in the forums, and no example without caveats was forthcoming. The bug-tracker was given as a good example, but with the caveat that it uses a bunch of functionality that is not yet in the core API but will be in a different form in X months. Besides the fact that the bug-tracker code is wholly opaque to me (as discussed earlier), I'd like an example that only uses the OACS officially sanctioned API. Classified ads was given as another good example, but it doesn't support several now-core aspects of OACS (subsites, for example) and is broken when installed along-side another app (user portraits), because one of them used the content repository in an unsupported way.

Weed out packages that are broken and not maintained. They can stick around in CVS. But don't advertise them. If every user wastes as much time on trying to get broken apps to work as I did, it's a lot of manpower going into very unrewarding pursuits--and it leads to whiny belly-aching like this post (and nobody really needs that!). Fix the packages that are supported. Everything in the repository ought to work after installation without more than a few minor configuration tweaks. If it requires code changes to avoid tracebacks, the package is broken. Unfortunately, almost every package I've installed has required some kind of code change to not traceback under some circumstances in 5.1.5.

Everytime I see someone talking about what they'd like to see changed in OACS, it includes some pet API change they really need because RoR or Plone or something has something similar. I'll be reactionary, and say that I'd rather see the existing APIs get some attention in the form of documentation, examples, and working packages that use the existing APIs. I'm sure everyone has an idea for a killer new form validator or the uberest database-object mapper, but all of those things have been invented by half of the OACS developers in one form or another, and we're no better off for it because it isn't documented and it is used in one or two packages and no one else even knows it exists.

I'll stop my whining for a bit, as in several cases I'm just repeating what others have already said. Don't take it as complaining...Ok, it is complaining...but don't take it as criticism. Ummm...Ok, just take it as wishes from someone who really gets excited about the potential of OACS only to be let down on a daily basis by the reality of the incohesive state of the current codebase and documentation.

Finally, I'll repeat that I'm grateful to have these tools. They work well and it shaved months off of my development time. Thanks to all of the developers for the effort despite my being so ready to issue criticism (and forgive me for being critical before being exceedingly useful to the project--I am almost fanatical about never complaining until I've contributed something significant to development, and in this case I've only filed a few bugs, submitted a few small patches, and funded development of ec-serial-numbers, which I believe Malte has checked into CVS...view my complaints as something for me to atone for when I finally do figure out how to develop for OACS).

Collapse
Posted by Alfred Essa on
Joe, most throughtful, informative and balanced review of OpenACS I have come across in quite awhile. Also, enjoyed the sense of humor. I hope it stimulates further discussion.
Collapse
Posted by Adam Aggeusz Jaworski on
Fantastic post Joe!

I am a big fan of your Webmin, I have build my custom, internally used, small Virtualmin-like admin control panel years ago - about 1990/2000 - using as a base some of Webmin core and modules, and I really appreciate you have posted your review on OpenACS TODAY, since, as you wrote:

"If every user wastes as much time on trying to get broken apps to work as I did, it's a lot of manpower going into very unrewarding pursuits"

You have saved my life - OK, almost 😉

I have found OpenACS just few days ago (and only because I found (via Google) a link to project-open (Frank rocks!) on some Mambo forum archive while looking for really working groupware) and get so excited about it so I started planning whole new project OACS-based, but at the same time I was somehow confused, how is it possible such a GREAT framework AND packages are almost unknown? why developers here are "tired", why there is no buzz around (like mostly hype about RoR) ? why Lars Pind switched to RoR? (I can't believe he switched only because he likes 37signals people), and the like - many questions.

I realized OACS is a BIG beast, but it is not a problem for me, also, this is not a problem for me I know nothing about Tcl, AOLserver, and very little about Postgres, I have lived mainly in a Perl world since 1997, but excited so much about OACS, so now I prefer to learn Tcl, ADP etc. instead of going RoR and writing fancy looking code in minutes - but from scratch everything, reinventing everything again - as the rest of the LAMP world.

I suspected something must be wrong here, but honestly, never ever suspect it is as broken as Joe explained above. WOW! "awesome"! Don't get me wrong, but if OACS weterans here would donate only 50% of time spent on discussing "what and how to improve the OACS homepage" to just make "spring-cleaning" on surface level only, they would get resolved 80% or more their frustration about not-growing community, zero-buzz around there etc. I feel you really have no idea how many great people you have loosed without any notice - just because that plain reasons, explained by Joe in his post.

Joe, thanks again, you saved me months of frustration with OACS. I am still excited, and still believe developers here can change this chaos here to avoid further frustration and dead of project at the end, just think, there is BIG market for OACS, and it doesn't need to be only dotLRN focused, get me for example: I have found OACS "by accident" only thanks to Frank (project-open), and groupware (not only community oriented) tools around OACS are still reinvented in LAMP world, with no success there, and you have here what they are dreaming of!!! Awake!!

To get a picture about how big market waits for OACS based solutions, please read something like:

http://asymptomatic.net/2005/03/16/1398/a-groupware-hole/

Going to my part - as I am not a programmer, just self-educated Perl hacker, I have also some experience with making self-selling websites - mostly in Polish since my English is mainly Perl-based 😉 but with some help from native speakers I can write/compose really self-selling website for OACS, but, wait, I first want to know if community and developers here are open to first make some spring-cleaning as suggested by Joe, since even with best sales-letter ever, there is no way to sell current chaos of bugs, confusing docs, orphaned items etc. here...

Best,
Adam

Collapse
Posted by Ben Koot on
Adam,

These are good points, I have a feeling one of the problems is lack of resource. One of these is time. Right now there are 9425 registered users. However, only very few ever make it to the forum. So effectively we have a sleeping community.

Why don't we start with activating it and thus find the essential resources to clean up. Why not start an internal PR campaign, asking folks to join the party and help with the spring clean-up. Even if only 10 respond we might have motivated some valuable new blood.

I guess based on the feedback recived over the last 2 weeks we have more than enough points to create a priority listing of issues that need to be tackled.

I will try to create a provisional shoppinglist over the next few days.

Just a thought
Ben

Collapse
12: Re: Perfectionism (response to 1)
Posted by Adam Aggeusz Jaworski on
Ben,

excellent idea,

I feel we really should just START, make simple things, rather than great plans which will remain plans only, it is that simple, and to be honest, simple clean-up doesn't require "resources", it is rather matter of DECISION, and as Joe explained, persistent lack of simple clean-up can be a source of real, BIG problems for OACS development and community

I am really new here, so have a problem: who is really, personally, responsible and able to make decisions here? community? governance body? there is no such person, it has to be real person, discussions are nice, but there are many little things where discussion is pointless, only decision and quick action resolves "problem"

as far as I understand situation here, after reading everything about OpenACS 24h a day during last week, the problem is: too much Perfectionism, yes, too much - I know this really well since it refers to me, also

if you are interested, too much Perfectionism leads to having ultra-excellent code, which almost nobody knows about, because you have "forgot", or even doesn't care about these "unimportant" details, while the rest of the world see only those "unimportant" details and thinks in a really simple way: "there can't be anything good behind such chaos on the front" - end of story

please let me know where and how can I (as completely novice) help, it really hurts me seeing such a great engine "in shadow" while thousands of LAMP world people still reinvents the whell - daily!

Best,
Adam

Collapse
Posted by Ben Koot on
Adam,

Thanks for your comments. I have added a new topic to the blog. "I am new to Openacs". I hope this will help newbies and non developers to share ideas as what they expect and would like to solve. See it as a kind of thinktank where issues are solved not just by coding packages.

I have a feeling the package approach has slowely been killing potential breaktrhoughs, as they distract from he real practical usage of individual features, hidden in the packages. I would go as far as to state that all new tricks in web 2.0 are allready available in openacs, but undiscovered. We just need to repackage and simplify them!

Just a simple start. More later.

Ben
(not a developer/coder)

Collapse
Posted by Adam Aggeusz Jaworski on
Ben,

thanks for this "I am new to oacs" blog - I agree it can help unleash some fresh power from newbies, without direct regard to "coders inner world" 😊

Best,
Adam

Collapse
Posted by Matthew Dodwell on
Just to add my 2 penniworth of praise for Joe's post as well. I read it with great interest and agree with it all. Adam and Ben posts too prompt me to join in.

I've been using OACS off and on for about 3 years and love playing with it as Simon was saying in a recent post. As a professional developer, TCL and OACS are tools I look forward to using.

I've produced a few non-profit sites in ACS 3 and have stalled for the last year in upgrading them to 4&5, partly due to the learning cliff, although I've found that using OAKS uml (v0.3) has really helped in producing working skeleton pages.

I know we're a volunteer community and thus resource is variable, but I think we need to use more industry best practices here. For me we need a project manager to run a properly visible project to get a set of solid and maintainable packages. That project manager could be rotated as needed but must be enthuiastic and visible - (and pass the baton on when the need to continue to beat their head against a brick wall is fading!!)

Cheers

Collapse
Posted by Ben Koot on
hey guys,

Great to to see so much enthousiasm, 3 minutes before christmas 😉. I think one thing we need to eliminate is the brick wall.

It will all happen !

Cheers
Ben

Collapse
16: project manager for OACS (response to 1)
Posted by Adam Aggeusz Jaworski on
Matthew,

you have interesting idea, I feel it is worth some consideration and discussion with core/team members, but it seems we are at this point a little off-topic in this thread, and I'm unsure in which forum (or blog) it could be continued

ideas anyone?

Cheers,
Adam

Collapse
Posted by Ben Koot on
Hi guys,

I have opened a new topic https://openacs.org/forums/message-view?message_id=352078

See you there.

Collapse
Posted by Daniël Mantione on
Just some ideas here:

* Have the community write the documentation. Write a tool that allows the community to edit the documentation. OpenACS has most of the infrastructure, now use it. Be liberal with permissions.;
* For a webtool with the best translation toolkit I have seen, it is somewhat of a big surpise the OpenACS website is pretty monolingual. All infrastructure is in place, have the community translate it.
* Give more people CVS access, so they can contribute. The procedure should be easy with little bureaucracy.

Collapse
Posted by Torben Brosten on
Matthew Dodwell,

Creating a project manager role, and giving it the title "Project Manager" does not address any of the issues directly. These create yet another distraction from problem solving and quality initiatives. One might even wonder if it is just a way to pass the responsibility to someone else. Why not just apply the principles of continual improvement directly?

make observations,

plan improvements (including how to verify it works),

implement the plan,

verify the results of the implementation,

make new observations

Collapse
Posted by Torben Brosten on

Joe, let's address your greatest disappointments to start with:

The original implementors of ACS were wrong about everything they ever did. At least, that must be the case, because everything is being re-written by every developer in wholly different ways (it also appears that every current developer is also wrong in every possible way, since everyone writes their own everything from scratch and doesn't document it).

OpenACS 5.2 includes some powerful, new features. The subsystems, including Postgresql and TCL are evolving as well. Development is ongoing to improve the toolkit performance and comply with standards in more ways. Sometimes this means writing new code. Isn't it a good thing to be continually improving the toolkit?

Packages are more broken today than when I started using OACS a year ago.

That depends on your reference point. OpenACS 5.1 is the stable branch. The core packages of OpenACS 5.2 has just been released. This is when work is done to make the other packages work with the new core (if not the new core features). Some cvs changes were made to the 5.1 branch that should have been destined for 5.2. Those can be corrected by reversing changes in the 5.1 branch. The cvs browser ( http://cvs.openacs.org/cvs/openacs-4/packages/ ) has a nice feature that lets you see all the contributions that make up each page (called "annotate"). That can help identify recently made mistaken commits.

One cannot count on any of the packages actually working once installed--even many reasonably new ones are broken in 5.2.0. Apps that were broken in 5.1.x when I started working with OACS are still broken today, and bug reports result in no changes...sometimes even the patches I send take months to be applied or don't get applied at all.

Again, 5.2 is not stable yet. The reality is that some bugs in 5.1 will likely not be fixed, because developers would rather make a "better" fix in 5.2. etc.

The bugtracker needs to be optimized to become really effective in its current role. For example, it's really difficult to get a full, summary count of open bugs. The bugtracker presents counts of all bugs (open and closed), which has limited value for continual improvement. Indeed, I know some think that the bug counts refer to open bugs, yet the count of bugs only increase, even when they are closed!

.ecommerce is still utterly unuseable in the state that it comes down from OACS.org and doesn't work at all in 5.2.0).

Yes. Prior to OpenACS 5.0, ecommerce was published as a toolkit. Now a demo/test site can be setup without having to actually write (much) code.

..it's time for packages that haven't been touched in a year or more to be marked unusable/unmaintained.

There is a package maturity attribute that is being implemented which should help with this. https://openacs.org/doc/current/releasing-package.html In any case, it is in deployers best interests to test everything before implementing.

If I had known the actual state of packages in OACS, and how much time I would spend figuring out that "this one doesn't work in multiple subsites, so we can't use it yet, and this one doesn't work if you have more than one instance, and this one just gives a traceback and hasn't been touched in CVS in two years, and this one doesn't uninstall without a traceback, and this one only supports Oracle despite apparently having code for PostgreSQL", I might not have chosen to use OACS.

I made some bad assumptions about packages early on. Now I beta test everything outside of an implementation site before implementing it or making any commitments about it.

Documentation is copious but wrong. And difficult to read, due to an odd docbook processing quirk that has never been fixed (the doubled up shell output).

You have misidentified a feature as "wrong". The feature was added so that one can copy from the document and paste the code directly into a shell window. See: https://openacs.org/doc/current/install-steps.html#how-to-use

APIs are constantly shifting with no documentation of the changes. After upgrading my devel site to 5.2.0 last night, I became aware of a huge swath of deprecated functions in most of the modules I use. I wasn't aware these functions were being removed.

Me neither. Somehow I missed the warnings in the error log whenever the deprecated functions were called.

Worse, there is no map of what functions replace the deprecated functions.

Actually, there is. It helps to have a development sandbox for each of the different versions one is using or wanting to use. Check the prior version's api-doc (version 5.1 in your case, which happens to be the same as OpenACS' at https://openacs.org/api-doc ) for info about what replaces it, usually under "See Also". For example, see: https://openacs.org/api-doc/proc-view?proc=ad_ssl_available_p

it seems that my focus is quite a bit different than most folks here, since a lot of focus is on dotLRN and major overhauls of the core APIs

Maybe, but I doubt it. Each release is a significant improvement over the last, quite a bit of it in the direction you are wanting to go.

Joe, perhaps the following hints will help:

When you are uncertain, ask. I had to drill that into myself. No one knows your OpenACS struggles and therefore cannot help if you do not make the problems known. How much time I wasted early on, when a simple question or comment to the forums or irc channel could have helped.

Learning OpenACS is about learning how to find answers, and how to use the various subsystems fully as they evolve along with OpenACS. Learning OpenACS style is not a cliff, but an endless spiral of continuous improvement cycles. Pace yourself! Also, a little karma yoga helps! =)

Collapse
Posted by Joe Cooper on
Hi Torben,

Thanks for your detailed response.

I'll take this opportunity to once again say that I'm not bitching because OACS sucks and isn't the most effective tool for building a community website with lots of good services. I couldn't have gotten virtualmin.com running in 3 months with shopping cart, software registrations, forums, bug trackers, FAQs, and user management, with any other system. There is no competition for OpenACS (Plone/Zope is broken enough for me to put it in another class altogether), OpenACS stands alone, so OACS can be as poorly documented and ornery to use as it wants to be. But I'd still like it to be better. 😉

So, to make my following comments a bit softer: OpenACS rules! I love OpenACS! It saved me many months and many American dollars worth of development.

Now, back to the ruthless bashing that everyone by now has come to expect of the ingrateful barbarian from the hills of Texas:

OpenACS 5.2 includes some powerful, new features. The subsystems, including Postgresql and TCL are evolving as well. Development is ongoing to improve the toolkit performance and comply with standards in more ways. Sometimes this means writing new code. Isn't it a good thing to be continually improving the toolkit?

Certainly. If the toolkit is what OpenACS is really about (and I'm in no place to say that it isn't). My complaint is with the fact that the changes are made to the core, with seemingly no regard to breakage in non-core components. About 50% of 5.1.5 application packages are broken in small or large ways, and bug reports about most packages are ignored. Even more packages are broken for 5.2.0 in some way. If the OACS development toolkit is what OpenACS.org is all about and applications are just for fun, the website and the policy needs to say so. I chose OpenACS because it appeared to be a set of pre-written applications that worked together. I wasn't looking for a toolkit (and I passed on Catalyst, Mason, RoR, and several others because they provided the toolkit but not the actual applications), but to some extent that's what I ended up with (though admittedly, there were a lot of almost-there applications that I could start with, which is lacking in most other toolkits). If the OpenACS developers want it to be a development toolkit, say so. It is presented as a set of working solutions based on a common core toolkit. As has been mentioned, the package stability problem is known and the maturity flag in packages is probably a good start for dealing with this problem.

Let me be blunt, so everyone knows where I'm coming from:

The vast selection of packages is what makes OpenACS attractive at all to users like me. There is nothing else in OpenACS that makes me want to use it over something in Perl like Catalyst or Mason, or, if I'm to learn a new language, RoR or Seaside. If the packages do not matter to the OpenACS developers, OACS is not for me. There are exceptions, but the majority of packages appear to be of no concern to any active developer. I am in no place to say what people spend their time on...but if packages don't matter, OACS can't matter to me. I'm not a web application developer...I use OpenACS to sell and support software wholly unrelated to OpenACS, and if I have to make my full-time job working on OpenACS, my actual business is going to fail due to lack of attention.

That depends on your reference point. OpenACS 5.1 is the stable branch. The core packages of OpenACS 5.2 has just been released. This is when work is done to make the other packages work with the new core (if not the new core features). Some cvs changes were made to the 5.1 branch that should have been destined for 5.2. Those can be corrected by reversing changes in the 5.1 branch. The cvs browser ( http://cvs.openacs.org/cvs/openacs-4/packages/ ) has a nice feature that lets you see all the contributions that make up each page (called "annotate"). That can help identify recently made mistaken commits.

My reference point is 5.1.5. That is what runs on my production server, and on my devel server until a couple of days ago. The assertion that 5.1.5 users should use the CVS browser to figure out what code they need to change to make 5.1.5 packages work is a bit frightening to me. I'm not afraid of CVS, and I'm not afraid of patches. But...really? This is standard OACS production server practice? Is there any documentation about how to do this?

I'll refrain from comment on 5.2 from here on out, as I now understand that it is a core library release. My complaints about broken packages apply to 5.1.5 as I downloaded it and the packages from OpenACS.org. I don't know how to do anything other than install the stable release and the packages provided by the software repository, and those packages are often broken and unmaintained.

The bugtracker needs to be optimized to become really effective in its current role. For example, it's really difficult to get a full, summary count of open bugs. The bugtracker presents counts of all bugs (open and closed), which has limited value for continual improvement. Indeed, I know some think that the bug counts refer to open bugs, yet the count of bugs only increase, even when they are closed!

You're saying this summary is not correct?

Open 819
Resolved 399
Closed 1539

I'm not sure how closed could be higher than open if things act as you describe. But, I believe the summary is correct. I've just browsed out to the end of the pages of open bugs (33 pages with 25 per page except on the last page which has 18==818 bugs...one went missing, but close enough).

But, nonetheless, there are bugs that have been open for years with no activity. Sure, a lot of them are not valid bugs, many have probably been fixed but not closed, etc., and some have running commentary as someone addresses small parts of the larger problem reported by the bug. The point is that some go years (years!) without activity.

I have no problem with there being lots of bug reports--every large piece of software has lots of bug reports, and a bug tracker is a great way to keep a todo list. But it is clear that there are some aspects of OpenACS that are simply not maintained. It wouldn't matter (much) if the API were stable, but as it is, many apps don't work on the current stable release of OpenACS and I see no reason to believe 5.2 will make this problem anything but more pronounced.

In summary: I view the steadily increasing breakage of packages as a huge problem with OpenACS. I have no right to demand anyone hold the same opinion I do on the matter...but if anything sends me and my development time/money elsewhere, this will be it.

Yes. Prior to OpenACS 5.0, ecommerce was published as a toolkit. Now a demo/test site can be setup without having to actually write (much) code.

I can live with that. I didn't know that. But I can live with it. My store took about two months to get up and running (some of that was a new package to support software registration that Malte mostly wrote for us). I could have done it faster with other shopping carts, but I wouldn't have gotten the forums and bug-tracker and FAQ and news. Overall, I'm happy. But it would have been nice to know when starting out that ecommerce requires new code to even work.

If I had known the actual state of packages in OACS, and how much time I would spend figuring out that "this one doesn't work in multiple subsites, so we can't use it yet, and this one doesn't work if you have more than one instance, and this one just gives a traceback and hasn't been touched in CVS in two years, and this one doesn't uninstall without a traceback, and this one only supports Oracle despite apparently having code for PostgreSQL", I might not have chosen to use OACS.

I made some bad assumptions about packages early on. Now I beta test everything outside of an implementation site before implementing it or making any commitments about it.

Oh, I have a devel server. Nothing about these broken packages is causing my customers distress (generally). But one can't know the state of a package until one actually installs it and spends time with it. The maturity attribute is an excellent idea, and I hope it will make this problem less irritating.

The point of this particular rant is that I came into OpenACS with big plans of making use of many more applications than I ended up using. It's difficult for a newbie like me to know when a problem is easy or difficult to resolve. Sometimes I've put in the effort to fix the code so that it runs after ten minutes of reading and code prodding. Sometimes I've tried valiantly and failed. And sometimes I've given up as soon as things went south, because the errors looked too scary for me to think about. But a lot of time has been wasted in the process, with not a lot of useful OACS development knowledge gained.

Actually, there is. It helps to have a development sandbox for each of the different versions one is using or wanting to use. Check the prior version's api-doc (version 5.1 in your case, which happens to be the same as OpenACS' at https://openacs.org/api-doc ) for info about what replaces it, usually under "See Also". For example, see: https://openacs.org/api-doc/proc-view?proc=ad_ssl_available_p

Excellent! I didn't know about this. Consider this complaint rescinded. Unless and until I discover that these docs are outdated and/or inaccurate too. 😉

Anyway, OpenACS is without a doubt the best set of integrated (kinda) web applications available, but it's also the only one...so it's hard not to be the best when there's no competition. But, if things continue going the way they have been going since I started using OpenACS and more packages break with each revision, it will become just another web application development toolkit. And with the unfortunate words tcl* and AOLServer* in the description, gathering new users and developers seems unlikely.

*-I have nothing at all against tcl or AOLServer, and in fact, I enjoy tcl now that I've used it. But my python and ruby and perl developer friends literally cringe, groan, or shiver when I say my website is developed entirely in tcl. Bad wrap for tcl, but them's the breaks. And, of course, the letters "A", "O", and "L" are poisonous for most nerds. Again, bad wrap for what is actually pretty darned good software, but it's the reality of the market.

Collapse
Posted by Adam Aggeusz Jaworski on
Attention! yet another barbarian here 😊

Anyone here really interested why OpenACS is still "in shadow", while at the same time has NO competition?

Joe answered above!

The problem is: you have made brilliant, awesome, powerful engine, but you live in your own world, you really don't care about "market" (besides .LRN probably), you completely don't care about people as me, as Joe - end users

so please, stop hoping that better logo, better design and better content will change anything, it will NOT happen

please, don't take it as offense, since all I want is to show you how you are looking in outside world eyes

developers, community leaders, please print below excerpt and read it every morning 😉

" If the OACS development toolkit is what OpenACS.org is all about and applications are just for fun, the website and the policy needs to say so. I chose OpenACS because it appeared to be a set of pre-written applications that worked together. I wasn't looking for a toolkit.
(...)
The vast selection of packages is what makes OpenACS attractive at all to users like me. There is nothing else in OpenACS that makes me want to use it over something in Perl like Catalyst or Mason, or, if I'm to learn a new language, RoR or Seaside. If the packages do not matter to the OpenACS developers, OACS is not for me.
"

BTW:
I love OpenACS!

Collapse
Posted by Michael Steigman on
Playing a bit of devil's advocate here... if all you get with a new release of OpenACS is the core toolkit, you are STILL getting a set or pre-written packages that work together, and a pretty impressive group at that: content repository, subsite, authentication, templating, administration, automated testing, et. al., not to mention a powerful permissions system.

As to the question about packages, I'm sure we all agree that unmaintained and immature packages should not be available through the repository for install. Unmaintained, mature packages are probably another story. This perhaps suggests another piece of package metadata - is the package maintained? I suggest that we either change the semantics of package owner and depend on the presence or lack thereof to tell us whether or not the package maintained OR we add a maintained-p element to track whether the package has an active maintainer (would be the owner). I know, I know, that's another piece of information that will fall out of date oh so quickly.

So, continuing to dream, I will ask whether it is really such a good idea to have critical information like this tied to the package .info file? There are probably dozens of users who could help with tagging package maturity or maintenance data if it were as simple as coming to the site and clicking a couple of buttons. Were that the case, we could integrate ratings/comments for each package, possibly even an overview page showing bugs and recent development. I don't think this would be too difficult to accomplish and could be tied in with the repository to give newcomers and veteran admins alike a more complete picture. Just a thought.

Collapse
Posted by Joe Cooper on
Hey Michael,

I just had to respond to this:

Playing a bit of devil's advocate here... if all you get with a new release of OpenACS is the core toolkit, you are STILL getting a set or pre-written packages that work together, and a pretty impressive group at that: content repository, subsite, authentication, templating, administration, automated testing, et. al., not to mention a powerful permissions system.

That's the definition of a web application development framework, and there are literally dozens of competitors (many written in a language I'm already familiar with). Those aren't end-user applications, and if that was all OACS consisted of, it wouldn't be the right software for me.

I'm not belittling the quality of the core OACS code. It really is nice and getting better, and as I mentioned, OACS is convincing me that I might like tcl. In another thread I've already talked about some of the cool stuff I like about 5.2.0. I'm just pointing out the one and only thing that sets OACS apart from the horde (no pun intended) of other frameworks and toolkits available, and the only reason I chose OACS, was the pre-written set of applications that fit my requirements almost exactly. Without the apps, I wouldn't be here annoying the members of this august body. I would be annoying someone on another forum related to some other project.

In short, I'm aware of those core packages...they just don't matter to me. No matter how good they are and no matter how important they are to the functioning of the high level applications I do care about, if I had to write ecommerce from scratch, I would have done it in Perl using the vast array of well-understood tools provided by CPAN and dozens of libraries and tools available for Perl. I'm here for the applications. Ecommerce, forums, bug-tracker, FAQ, etc. And because those packages exist, I'm developing (or paying Malte to develop) new packages, which are going right back into OpenACS CVS. As soon as I figure this monster out, I will be a pretty regular contributor on every package I use.

I think I'm the kind of user an Open Source project wants to have. I write books about things I love ( http://www.virtualmin.com/support/documentation/thebookofwebmin/ ), I frequently contribute patches (I've got lines of code in Webmin, yum, Squid, and even the Linux kernel, plus a dozen other projects that I've forgotten or aren't worth mentioning), I use the software in a real world environment with a couple thousand active users (growing at 50+ per week), and I'm noisy about bugs. And my real world environment makes money, some of which is going back into OpenACS development.

So, if all of these recent flammable threads are about how to bring people to OpenACS who will use it and help make it better...then I've made an honest attempt to make it clear why I use OpenACS. It's the apps.

Collapse
Posted by Michael Steigman on
I hear you Joe and I'm sure there are many here that agree and wish we could wave a magic wand to make things better. The question is where to go from here and to that end, I'll keep this short as I'd rather discuss package maintenance than split hairs over the definition of a framework :)
Collapse
Posted by Torben Brosten on
Joe, you write "You're saying this summary [of bugs] is not correct?"

Well, the number summarizes all packages and all bug-types combined --including suggestions for improvement (feature requests etc.) and "to do" items as well as actual bugs.

I have not found a way using the bugtracker to get useful code oversight information, such as open bugs by bug-type (and severity) per package.

Collapse
Posted by Malte Sussdorff on
I think we have gotten clear viewpoints why some people have been using OpenACS. As promised we will write a feedback system in OpenACS for 5.3 which will report back data collected in installations, which should give you an idea how many people are using a certain package (okay, have installed a certain package).

Regarding the bugs in bugtracker, it would be great if we could get someone to work on the testservers and figure out if a bug still exists and mark it as still existing somewhere or mark it as fixed in release xyz. I have the slight suspicion that quite some bugs have already been fixed in the packages, though others have been introduced.

Another thing to do would be to have a page on OpenACS which allows people to say “I work on this package” (in addition to the report mentioned above that says “I use this package”) and send them a link every six months to say if they are still working on the package or not. This should give you an idea if a package is actively maintained and who the people working on it are, in contrast to having a maintainer ( I don’t think we can follow through with the concept of maintainers for packages, as too many people work on a package at the same time).

Last but not least, we should ask people:

“What functionality do you miss on OpenACS.org for your daily work”.

Noone says the ideas I mentioned above would be useful and acceptable to people, though the reporting got a TIP, therefore I assume it is useful.

And let me point out one thing: I am thrilled to see this discussion and all the valuable feedback. Thanks a lot for contributing this much. If we as a community manage to put the feedback into actual actions even the better.

Collapse
Posted by Alfred Essa on
"Regarding the bugs in bugtracker, it would be great if we could get someone to work on the testservers and figure out if a bug still exists and mark it as still existing somewhere or mark it as fixed in release xyz. I have the slight suspicion that quite some bugs have already been fixed in the packages, though others have been introduced."

Where is the OpenACS test server? Is it being maintained? By whom?

Collapse
Posted by Ben Koot on
Malte,

The "I work on this package" idea can be achieved with blog. I just added all packages (as listed in bugtracker) to the category sidebar. Rss and email allert creates the warning/update function.

Cheers
Ben

Collapse
Posted by Abelardo Pardo on
"Where is the OpenACS test server? Is it being maintained? By whom?"

Al, Eduardo Pérez is maintaining a test server in:

strauss.gast.it.uc3m.es

It is hosted at UC3M Madrid. It installs from scracth every certain interval the latest version of the code and runs a few tests. It's got 2 instances of 3 branches (stable 5.2, old stable 5.1 and HEAD) per database (Oracle, PostGreSQL).

Eduardo has been collecting tests and fixes over several months. What is very enlightening to me is that due to him monitoring these tests some bugs were found introduced by (expert) developers "improving" packages. He usually ends up notifying the author about the newly introduced bug (which usually he/she is not aware of).

I guess this stresses the fact that unless every single developer regularly uses the testing methodology, it'll be difficult to remove this perception that bugs are never being fixed.

Collapse
Posted by Ben Koot on
Abelardo,

But what if the package maintainers are no longer actively participating? Do these messages reach a central "command post" as well. If not, we may need to work out a better allert system. One solution would be to post them on the blog, so at least the community knows about these issues.

Cheers
Ben

Collapse
Posted by Abelardo Pardo on
Ben, Good idea.

I think this is one of the points that needs to be improved. The error logs are in the server but not easily readable. We will try to create a more explicit web page with these reports so that anyone modifying a package can know right away if anything is broken.

Collapse
Posted by Giampiero Recco on
My few bits:

we choose OpenACS for these reasons:

1) The framework (database structure, templating system, packages, user management, permission, FTS,... ). It's complex but really complete and sooner or later you will take advantage of it. And if you have many developers, having a framework to guide them it's a good thing.

2) The templating system, it's simple and clear.

3) The caching system, we use extensively also the cached db api by Don Baccus, even if not perfect I cannot understand why they are still not on the standard installation. Caching it's a real plus.

4) A good availability of examples and packages.

Then we adapt everything, we use almost no packages as is, but in the end we save a lot of time because almost everything we develop is a stripped down/optimized version of something already well studied and developed in some OpenAcs core or extension packages.

Ciao,
Giampiero