Forum OpenACS Q&A: Re: Why do you use OpenACS and .LRN?
I also maintain a Plone site (http://www.scipy.org), and have for several years. The OpenACS learning cliff is a little less deep and deadly than Plone (another contender in this most recent endeavor). Both OACS and Plone, however, have a cliff rather than a learning curve, and I've never managed to scale either one. Being "easier than Plone" is like being smarter than G.W....the bar isn't very high.
I still have daily error emails from OACS 7 months after deploying the site, because I have no clue where to start to fix the problems. They aren't affecting core parts of my site and they don't appear to affect security, so I haven't lost any sleep over it...but it's frustrating. But not as frustrating as Plone. Plone reinvents everything--database, webserver, relational model, object model, Python grammar and object rules, and more. It was just too much. At least in OACS, if I need to do something stupidly dull like adding web content with a tiny bit of live content, I can figure out how to do it from scratch in an hour or two or three or four by reading the database documentation, the TCL documentation, the AOLServer documentation, and the OACS API documentation (actually, the OACS API documentation is a negative entity...one has to read the code to know anything, since the API docs are very nearly always wrong). In Plone, I never even got that far. If I couldn't add it via the ZMI, I just couldn't do it. Both systems are very poorly documented. Both Plone and OACS are inaccurately documented to the point that following the official tutorials does not result in working code. If I could figure out how to do things the right way, I'd write up documentation (I write a lot of documentation, including a published book...I'm not afraid to help out with docs). But I've never had any luck, even the tutorial was a challenge that ended with me as confused as when I started. I simply have to hire someone who already understands the system when I need new code (thanks, Malte!). This is disappointing, as I'm not a programmer but I've written quite a lot of code in my day (ten years of perl, bash, and bits of C/C++ and Python and lisp, so I'm not helpless)--this stuff shouldn't be rocket science. I really thought I'd be contributing useful code to OACS by now, but I'm not much closer than when I started because I can't devote full-time to developing for OACS. With the only other large modular system I've ever worked on seriously (Webmin), I learned perl and the system well enough to write a working Webmin module in a couple of weeks. It took me that long just to fix a few bugs in some of the packages I'm using in OACS, and I've never written a new package from scratch that worked (and it's not for lack of trying).
Lots of cool apps exist for OACS in various states of completion. Some of them (Project Manager) seem to point the way to a much better OACS. So I have high hopes that things are getting better. I was pleasantly surprised by the quality of some packages. Most were disappointing because they simply don't work in current OACS versions, and there appears to be no active maintenance happening for the majority of them. It has become apparent to me that OACS is a framework (like Rails, like Mason, etc.) and the apps are an accidental side-effect. Like all accidents, some turn out alright, but most are just trouble. I wouldn't want to write an ecommerce application from scratch, but I certainly had to write a lot more code in OACS ecommerce than I expected. The impression I got from OpenACS.org was that many of the packages worked, but this is simply not true--and it gets worse with every release. I've heard it referred to as a community system in a box. It isn't. It is a bunch of cool components that kinda-sorta-work if you squint a bit and don't mind writing some code.
Umm...Ok, so I'm being a bit bitchy about stuff that I got for free. The fact is that I wouldn't have been up and running as fast as I was with any other system that I'm aware of and I'm grateful to have it all running. Ecommerce is making sales for us, the forums and FAQ package are helping us keep our customers informed, and the bug-tracker (which is excellent with a few caveats) help us make our software better.
So what's right about OpenACS?
It has a lot of code, and some of it even works. The size of OACS has been given as a negative, and I understand why it would be to a developer that wants to understand everything and build all custom solutions. But for someone like me, I want to write the fewest lines of code possible. Me writing no lines of code is better than me writing a hundred and a hundred is better than five thousand (even if the five thousand solution is better than the hundred line solution). I only had to write a couple hundred lines of code to get virtualmin.com running and providing most of the services we needed. Malte wrote another couple hundred for us in the ec-serial-numbers package, which provided the last mandatory component (software registration via the ecommerce package). My couple hundred lines were hard-fought and painful, and took weeks to write, but I couldn't have written five thousand lines of Ruby in the same period of time (and it probably would have taken 25,000 loc to start from scratch in RoR to get our minimum functionality and half that in some of the other toolkits that have a few of the necessary components pre-written). I don't really want a great framework. I want apps that work together. I've played with RoR, and it was fun. But I'll never build an entire web application from scratch, so it is simply not the right tool for me.
Performance is excellent. I'm sure someone is reading this and scoffing...but my plone site runs nothing complicated, never has more than a half-dozen users, and a ~500 MB database, and the process size is never less than 1000 MB. I run two OACS instances (devel and production) in about the same amount of memory on Virtualmin.com. CPU usage is never a problem, and we have several times the users and usage of SciPy.org. OACS runs just fine. I'm sure a custom solution would be smaller and more efficient (our problem set is not huge), but having a $300 a month server rental bill is cheaper than spending man-months writing code just so it would fit on a $100 a month server. Not to mention the opportunity cost of not making sales while we were waiting for the app to be finished.
OACS didn't reinvent every wheel. PostgreSQL is a database that I've used many times in the past, and it is extremely reliable, fast, efficient, and well-documented. I like PostgreSQL and I trust it. I never trusted the ZODB, and never will. I'm sure the technology of the object database is fascinating, but I don't care one whit about cool technology. I want a site that works. OACS could stand to stop reinventing so many of its own wheels...at least long enough to make a few of the packages work and the documentation a little bit accurate.
I can see the potential of subsites and groups to make it possible for me to do things with our site that would be really good for our customers. But I've never been able to get it to work. Almost none of the packages that I tried are sub-site aware (or even instance aware), so they can't be used in ways that would be useful to me. So, these things are a really cool idea and I thought I would use them heavily, but they are not reality for someone like me. Maybe when my revenues are significantly higher, I can afford to bring someone in to make the improvements we would need to those apps (classified ads and news and press are the big ones that would be useful but simply don't work right in multiple instances--I've tried to figure out how to fix them, and got some advice on the forums, but the advice just didn't penetrate into my opaque view of OACS, so I gave up).
Permissions are another area that sounds great but appear to be impossible to do correctly on this scale. Plone/Zope uses them in pointless and confusing ways, and so does OACS. The bug-tracker is a great example, where OACS permissions don't even seem to apply. I've tried on a half-dozen occasions to add the ability for normal users to comment on issues that they didn't create (ya know, like every other bug tracker in existence). I've spent hours reading the code and the docs by the package author. I've asked in the forums, twice (as have others), with no response. It seems an insurmountable problem--and I can't even understand why it is such a difficult problem. It ought to be one if-then switch, but the logic is tucked away somewhere invisible to me (and others, since it has come up multiple times).
What's been most disappointing?
The original implementors of ACS were wrong about everything they ever did. At least, that must be the case, because everything is being re-written by every developer in wholly different ways (it also appears that every current developer is also wrong in every possible way, since everyone writes their own everything from scratch and doesn't document it). Packages are more broken today than when I started using OACS a year ago. In short, one should now approach OACS as a toolkit just like RoR. One cannot count on any of the packages actually working once installed--even many reasonably new ones are broken in 5.2.0. Apps that were broken in 5.1.x when I started working with OACS are still broken today, and bug reports result in no changes...sometimes even the patches I send take months to be applied or don't get applied at all (Torben is an exception, and has been very helpful, but ecommerce is still utterly unuseable in the state that it comes down from OACS.org and doesn't work at all in 5.2.0). I understand this is a volunteer effort, and I work on several OSS projects myself, but it's time for packages that haven't been touched in a year or more to be marked unusable/unmaintained. If I had known the actual state of packages in OACS, and how much time I would spend figuring out that "this one doesn't work in multiple subsites, so we can't use it yet, and this one doesn't work if you have more than one instance, and this one just gives a traceback and hasn't been touched in CVS in two years, and this one doesn't uninstall without a traceback, and this one only supports Oracle despite apparently having code for PostgreSQL", I might not have chosen to use OACS.
Documentation is copious but wrong. And difficult to read, due to an odd docbook processing quirk that has never been fixed (the doubled up shell output).
APIs are constantly shifting with no documentation of the changes. After upgrading my devel site to 5.2.0 last night, I became aware of a huge swath of deprecated functions in most of the modules I use. I wasn't aware these functions were being removed, and the only mention of it anywhere was in a forum post in February of 2005 that Torben helpfully pointed out to me. Worse, there is no map of what functions replace the deprecated functions. If I can't count on the API, I can't much count on anything, can I? I'm all for removing dead code, improving functionality, etc. But give me something to work with. I have no clue how to replace the deprecated functions as there is no documentation about what they've been replaced with. Maybe I've just gotten involved in OACS at the wrong time. Maybe in a year all of these problems will be resolved and OACS will be a beautifully crafted object oriented festival of goodness that reads like poetry. Or maybe it will be even worse, because everyone wants to re-write everything and no one wants to document the changes and no one wants to maintain packages that aren't proprietary to their company (I don't mind proprietary components to Open Source projects--that's what Virtualmin.com is selling--but take some responsibility for the core, too). I'd help with documentation if I even knew where to start to comprehend what is changing and how it works.
Anyway, I don't care about the language, or the webserver, or whether the code is object oriented, or includes logic in the database, or has a habit of calling me offensive names in the error logs. I want a base set of applications that:
- Work
- Work together
- Can be extended in small ways without huge amounts of effort--I don't care if the implementation is a bit ugly, if it is documented well enough for me to write the necessary ugly code
- Can be upgraded and maintained without more than a few hours of downtime every 12 months or so
Things that don't matter to me:
- dotLRN
- ERP
- Toolkit coolness
- Object Orientation (or Functional programming, or XP, agile development, or whatever other buzz-word you like). Either the code is good and it works, or it isn't and it doesn't.
I know that everyone wants a different set of packages to work, and so maybe the OACS that the current developers invision is not the OACS that I'd like to have--it seems that my focus is quite a bit different than most folks here, since a lot of focus is on dotLRN and major overhauls of the core APIs. I'd feel better about taking responsibility for the packages I need to work (ecommerce, ec-serial-numbers, forums, bug-tracker) if the API was a bit more stable than quicksand and changes to the API were documented. I guess I'll just have to see how it all shakes out--I'm pretty well committed to the platform for some time, and I'm also pretty certain that at some point I'll get a grasp on OACS enough to make some larger improvements and write new packages, but my enthusiasm for the platform so far has a negative correlation to the amount of time I spend with it (not because it is bad, but because I cannot grasp it and it gets harder to understand with each new revision because the docs reflect reality less and less).
What I'd like to see happen
Clearly I'd like documentation to improve, and the APIs to stop moving (without documentation).
I'd be almost just as happy if someone would tell me a user-facing package (i.e. not a service used by other packages) that I can look at that is correct, uses the new APIs exclusively, and works. That would give me some examples of how I ought to be doing things. I might even be able to turn time spent with such an example into my own working code. I've asked for such examples in the forums, and no example without caveats was forthcoming. The bug-tracker was given as a good example, but with the caveat that it uses a bunch of functionality that is not yet in the core API but will be in a different form in X months. Besides the fact that the bug-tracker code is wholly opaque to me (as discussed earlier), I'd like an example that only uses the OACS officially sanctioned API. Classified ads was given as another good example, but it doesn't support several now-core aspects of OACS (subsites, for example) and is broken when installed along-side another app (user portraits), because one of them used the content repository in an unsupported way.
Weed out packages that are broken and not maintained. They can stick around in CVS. But don't advertise them. If every user wastes as much time on trying to get broken apps to work as I did, it's a lot of manpower going into very unrewarding pursuits--and it leads to whiny belly-aching like this post (and nobody really needs that!). Fix the packages that are supported. Everything in the repository ought to work after installation without more than a few minor configuration tweaks. If it requires code changes to avoid tracebacks, the package is broken. Unfortunately, almost every package I've installed has required some kind of code change to not traceback under some circumstances in 5.1.5.
Everytime I see someone talking about what they'd like to see changed in OACS, it includes some pet API change they really need because RoR or Plone or something has something similar. I'll be reactionary, and say that I'd rather see the existing APIs get some attention in the form of documentation, examples, and working packages that use the existing APIs. I'm sure everyone has an idea for a killer new form validator or the uberest database-object mapper, but all of those things have been invented by half of the OACS developers in one form or another, and we're no better off for it because it isn't documented and it is used in one or two packages and no one else even knows it exists.
I'll stop my whining for a bit, as in several cases I'm just repeating what others have already said. Don't take it as complaining...Ok, it is complaining...but don't take it as criticism. Ummm...Ok, just take it as wishes from someone who really gets excited about the potential of OACS only to be let down on a daily basis by the reality of the incohesive state of the current codebase and documentation.
Finally, I'll repeat that I'm grateful to have these tools. They work well and it shaved months off of my development time. Thanks to all of the developers for the effort despite my being so ready to issue criticism (and forgive me for being critical before being exceedingly useful to the project--I am almost fanatical about never complaining until I've contributed something significant to development, and in this case I've only filed a few bugs, submitted a few small patches, and funded development of ec-serial-numbers, which I believe Malte has checked into CVS...view my complaints as something for me to atone for when I finally do figure out how to develop for OACS).
