Forum OpenACS Development: Re: Ideas for easier installation of OpenACS

Collapse
Posted by Dave Bauer on
This is a good suggestion,

For OpenACS I suppose it means
1) AOLserver + modules
2) PostgreSQL
3) Tcl
4) OpenACS or .LRN (or both...?)

Maybe a first step is just an AOLserver with all the appropriate modules included. It should be possible to distribute the source of AOLserver from OpenACS,org and keep it up to date, it doens't get released too often.

One fix for this particular problem is to make sure you have aolserver4-nspostgres package installed from ubuntu/debian.

Collapse
Posted by Tom Jackson on
I once had a working installer calle app-install. It was mostly for handling the problems with the stack Dave has just listed. I'll describe that in a second, but first a more general point:

OpenACS and the software it relies on are not Apache. The main difference is that you can install one version of Apache and everyone on a machine can use it. It is a system wide service like tar, less or whatever. AOLserver is not a system wide service. You can't build AOLserver with the standard Tcl distribution, you need to enable threads and other things. You may not want AOLserver to rummage around the entire system looking for Tcl packages which may also not have been compiled with the correct switches.

On the other hand, just try to install a new version of Apache on an old version of Linux. Forget it. If you have an older machine which still works very well, but has an old OS, you are totally screwed if you want a new Apache.

AOLserver has very specific, but easy to meet requirements. These requirements are not found in the standard installations. Beyond that, OpenACS and AOLserver are best considered as user applications. A single user installs and maintains all the components, and if another user wants to do the same, there is no problem. If software was installed to a central location, management would eventually, maybe quickly, become impossible.

Now, that was just a description of the basic problem, what is the solution?

App-install, although far from ready for general use, had a particular philosophy: hierarchical installation.

Each component has an associated installation file. The file contained more than just a script for installation. App-install would read the file and step through the instructions. It might prompt the user for data, displaying the default and allowing an override. The override could then affect additional defaults. Once these were set, a series of shell commands could be executed. A number of common operations were represented in the file at a higher level than simple shell commands, like creating and writing file content, changing file modes, sudo for certain operations.

Some installation files were higher up in the hierarchy. These files just contained links to other installation files, and/or the associated tar.gz file. App-install would then simply loop through these handling each in turn. In some cases the default variables in one install are reused, so you don't have to keep track of these details.

So the point was to have component maintainers create these relatively small files, and then a higher level maintainer would combine the parts they like into a new bundle. If one part of the bundle changes, the maintainer only have to change a few lines of one file and create a new version.

Once you create one of these files, you can install the sofware with the same information each time.

Then the maintainer just says which file to grab for a particular cases. When there are problems with an installation, the frustrated user can just post the entire output of the installation, because all of it was recorded to a single file. There is no mystery about what the user did or didn't do, or what failed to work as expected.

Examples of what these install file look like are here:

http://rmadilo.com/files/app-install/

Collapse
Posted by Vic May on
Tom,

It appears that Aolserver and Openacs can work either way -- centralized or user specific. Right now on my Ubuntu system it is installed as centralized but can be user specific by merely pointing the main nsd binary (/usr/bin/aolserver-nsd) to the custom config file.

Among many issues in Linux/Unix world I see two huge issues that could have/should have been solved a long time ago (in my opinion).

One is useless man pages, where every Unix admin I talked to agrees that man pages are mostly useless unless you already more or less know how the application works. Everyone spends enormous amounts of time just to figure out how to use an application, especially where application has a lot of command line options (like "mencoder", for example). It is by far more productive, often shorter and faster to provide a working example of each command line option or each set of options. Nevertheless, following the Unix tradition, programmers keep faithfully creating long, boring and useless man pages. We, the users, are forced to either waste hours upon hours trying to figure out how the application works or spend hours upon hours looking for help on the web. I usually go directly to the web.

Another issue, which is not an issue anymore for most of application but can sometimes be a huge issue at the most inconvenient times is having to compile applications. Imagine, there are millions of Ubuntu users worldwide. Somebody creates an application that has to be compiled. Each user compiles the source code on identical platform, using the same OS as millions of other users. What is the point of doing that when it can be compiled once by one person and others just install the binaries. I understand the intricacies of having different options compiled in etc. That's a stone age mentality. Optional functionality should be dynamically added without the need of recompilation. It is possible and it is being done all the time.

Well, enough of that point. Back to Aolserver/Openacs. I am an end user, not a developer. I don't want to care how installation is done. I just want it to be 100% reliable. The main user base in split between Debian based and Red Hat based distributions. The majority uses x86 processor. Why not make binaries for the majority?

So, why not make a working, statically compiled, self reliant package consisting of Aolserver, Openacs, Postgress and !!!ALL!!! modules that are available. That package should not depend on the location where it is installed, but configured for one initial location (like /opt/aolserver). It is by far much easier to reconfigure and modify a working installation than try to make nonworking one work. If another user needs the package, just copy it to a different directory, modify the configuration to point to that directory and you are done.

Also, it is now easier to fit the OS to the application then the application to the OS by using a virtual machine. For example, if I want to use OpenACS for publishing some content and there is a complete set of binaries available for, lets say Ubuntu 7.10. I just create a virtual machine with the Ubuntu 7.10 and install the binaries. Or, even better, just download the virtual machine file and use it (virtual appliance). That would make it very convenient for many users and OpenACS would be adopted by much larger number of people.

It might be that I am too naive and underestimate the problem but nevertheless that's how I wish things were. :)

Collapse
Posted by Tom Jackson on
Most applications are nowhere nearly as complicated as AOLserver and OpenACS. It is very hard to compare it to other types of precompiled applications. Compiling is the easy part. If you can't get over that minor hump in the road, you will quickly run into the difficulty of providing a secure and useful application.

I cannot stress more that you will quickly run into major trouble if you try to shortcut the process of installing OpenACS. Please consider this: this project is filled with many very talented developers. Yet, even though there are continuous calls for an easier installation, they never get done. There is a reason for this, and it isn't laziness.

Consider one simple example. I decided to compile AOLserver in my home directory to test out some feature. I thought I had discovered a massive bug where the server would run for a few requests then crash with weird errors. I wasted a day tracking down the fact that the library loader was finding the wrong version of Tcl. It used the correct version for building, but found something else at runtime. This could easily happen to anyone.

But configuration is significantly more complicated for all parts of this system. Most software components we use on our servers require no configuration, or very little. Of course as soon as you want to tweak one of these you run into the exact same issues: little documentation, no examples, don't know what to ask for or search for unless you are already an expert.

This is why I gave the example of App-install. It boils down all these decisions into a single file for each installation. The file is supposed to be written by an expert and shared with others. But it is relatively realistic: real developers compile and install even more components than just the basics. The App-install concept is also a form of documentation and an aid to recovery, or more optimistically, to duplication.

I agree about the documentation. I try to write examples of using any code I write. I actually need to do this just to test my code, so it isn't a major problem to provide examples. In fact, I like to write working examples that provide something beyond dry technical usage.

For example, over the last week I wrote a tic-tac-toe game to demonstrate a few features of a tcl templating system, and another few pages to demonstrate the use of [format] in and [assign] in the same system:

http://junom.com/document/twt/view/www/

Each file in that directory can be viewed except the .tcl files. So I simply add on a ~ to the end so your browser will display it as plain text. Examples are nice, working examples are even better.

But when I hear users asking for a download of OpenACS that just runs from a static install, I wonder how much they actually value the product. I know you can go in to Office Depot and buy a bunch of canned contracts for a few dollars, but it isn't really the same as getting good advice from an attorney. Usually the more appealing this seems to a person, the more they need a real professional.

In case I wandered around this too much, I'll stay it briefly: compiling is easy compared to configuration. Configuration is a breeze compared to running or developing an application server. Better to have a few speed bumps along the way, but the process could still be a little easier.

You are still in good company with everyone here wishing that installation was easier (and faster).

Collapse
Posted by Vic May on
RE: ....But when I hear users asking for a download of OpenACS that just runs from a static install, I wonder how much they actually value the product....

Tom,

There is nothing to value if you don't know what it is. ;) I did pass by this project multiple times in the last 3 or so years but never got to the point of installing it. The reason it attracted me is that it is TCL based. I like TCL a lot and was looking for a web/application server that was using it. My options were tclhttpd, aolserver and recently, wub. I did use tclhttpd for a while and liked it for it's simplicity. Then I heard about the great performance of aolserver, so I started looking around and naturally, many links were pointing to openacs. BTW, your name is everywhere where there is talk about aolserver/openacs. I don't know how you find the time. :)
Anyway, I got close to installing several times. I was able to compile Aolserver in the past but it seems there was always something missing that I needed and when I tried to add the missing component, I either got a dead download link, or an empty web page or I can't compile something. I have been trucking and using Linux since it was distributed on floppies and do have some experience with compiling problems. In the past I did try to troubleshoot, try to find missing files, fix dependency problems etc. It is because of that experience that I am reluctant to do so anymore, especially because it is so easy to install applications now days. You never know if it will be 5 minutes or 5 days to compile the application, so, naturally, the excitement isn't there. Well, your remark above intrigued me so I decided to look around for a working installation and found this:
http://www.jsequeira.com/projects/oasisvm/
It's a virtual machine with a complete installation of OpenACS and .LRN. I played with it for several hours and found it to be absolutely amazing. I do sincerely wish I discovered it years ago. I would have known how to use it by now. But it looks like I am hooked. :) Thanks to all the developers for the great work. And thanks to everyone who tried to help. BTW, that virtual machine link should be somewhere on the front page of openacs.org

http://www.jsequeira.com/projects/oasisvm/

Collapse
Posted by Tom Jackson on
Get into the habit of bugging folks here. I started out about the same with AOLserver/ACS/OpenACS: Linux on a stack of floppies. That was kicked off when I decided Windows NT and SQL Server were too expensive and difficult to manage (and too expensive!, did I mention that?)

The 5 minutes or 5 days thing can still occur. It took me a week to find a Linux distribution which enabled most of my laptop features. But x86_64 is new and free software takes time. Interesting thing is that the distribution I found had a semi-full-featured boot cd, so could try before installing the whole thing.

Collapse
Posted by Chris Douce on
Hi Vic,
Yesterday I discovered something similar to what your last post described. For the last couple of days I've been trying to get a native Windows installation of dotLRN up and running (to make a long story short: I had no end of trouble). There was a whole bunch of tips in the fora that I found but I kept getting presented with various flavour of difficulty.

In the end I thought, 'okay, so it's going to be Linux', so I went about finding a virtual machine for my home PC. After a false start, I settled on VMware and on a hunch I Googled (I think) VMWare and dotLRN in the same query which took me to here: http://media.wu-wien.ac.at/download/vdotlrn

Installation and figuring out how to drive the download was dead easy, just as easy as installing some of the other open source environments. If this was linked to on the main install page it could have saved a couple of days of head scratching. This said, I'm happy where I've got to so far in my dotLRN adventure.

Chris

Collapse
Posted by Tom Jackson on
I just installed VMware on Linux. Installation looks very similar to what I had for app-install. It asks a bunch of questions about where to install stuff, and then it compiles and installs, then asks more questions, etc.

But once that is installed, it is easy to startup the Oasis/Linux.vmw and login. I'm sure that on Windows all this works much faster since they seldom give you any choices there.

Overall, pretty cool way to check out stuff, but then what?

It would be interesting to hear what users do once they decide this stuff is cool. Does this provide the motivation to figure out the typical install?

Can you really do an easy native install on Windows?

Collapse
Posted by Malte Sussdorff on
I wrote some scripts which run through on most systems that will give you OpenACS, AOLserver without interfering with your local setup. I did not add PostgreSQL to the mix as this is probably installed already (albeit I have to pray the settings are right and the version works with OpenACS 😊).

Judging from this experience and the fact that I did over 100 installations in the last six months, here is what might work:

Create a script which does the downloading and compiling. It should check if it is able to connect to PG on 5432 and if yes, check that the version works with OpenACS (otherwise stop and return an error). If no, install PG on it's own. I am not a friend of letting the user make decisions in the install process so I would make some default assumptions based on the experience collected so far (e.g. install in /usr/local instead of /opt, use /var/lib/aolserver instead of /opt or /web), openacs as the service name, localhost using the first available port above 8000.

If this would help I can ask my sysadmins to write such a script based on the scripts that I have written so far. Sadly, I am not sure how much support we can guarantee for the script to work on all kinds of platforms, though we would try to make as many "if" statements into it to enable it on even more platforms.

This being said, I totally agree with Tom about the configuration and running part. If all you want is to try out OpenACS and add some more modules using the installer then you should be fine. But it starts with the basic question of "Do you want to install .LRN" which has a significant implication on the install. Furthermore, having the modules installed without and data in them is nice, yet you will wish for more so you can play around with it.

Project/Open ]po[ has gone a nice way here in providing a prefilled dump file with the ability to delete all demo data. That is their way of installing and at the moment they do not have a configuration file which does all the configurations necessary for a normally running system. Same way is what Quest did with their AIMS application. So the next question (assuming you got OpenACS installed), which dumpfile to use. And who is going to provide such a dumpfile for the installation, what should be included? We could probably provide a dump with the minimal components installed, all empty. But will this be enough for the user to get an experience out of OpenACS. And who is the user anyway? Is it the end user evaluating the product for his website like a canned product? Or is it the developer who wants to install this for one of their clients?

The app-install approach works perfect for the developer. As for the end user like Vic says he is, we should probably provide dump files for simple OpenACS, .LRN and maybe whatever combination people are willing to provide dump files for.

Either way, the codebase should be a full checkout of the latest release branch.

Collapse
Posted by Tom Jackson on
The app-install approach works perfect for the developer. As for the end user like Vic says he is, we should probably provide dump files for simple OpenACS, .LRN and maybe whatever combination people are willing to provide dump files for.

Yes, app-install is or was not even good for a single install, multiple ones made it useful. But the basic ideas follow along with Malte's comment: provide the needed files in a known location and let the installer download them. That is a big part of the frustration is just locating the right file, then discovering I need CVS access. Try to locate the cvs instructions, etc.

When it comes to installation scripts, my interest is in developing a hierarchical system for very obvious reasons: divide and conquer. Each tiny piece is relatively easy to install, but if you have to coordinate it with other components, do it for multiple systems and maintain it over time, a mega-script becomes a liability. The hierarchical division of scripts should allow the script maintainers to copy the component scripts, make minor variations as needed for new versions and system differences, and provide expert chosen defaults. And as suggested, record the process so the process can be quickly reviewed by the community instead of needing to ask a bunch of questions or make assumptions about what might have happened.

Malte: does a full checkout of the latest release...you mean the CVS version, not a tar.gz file somewhere? This is a good point, a trap I fell into a month ago. Since then I started using git-cvsimport to maintain an easier to browse repository. One very nice feature of this is the 'snapshot'. It is a tar.gz file of whatever sub-directory you want, minus the CVS directories. It is what I would call 'instant publication.'

Collapse
Posted by Vic May on
Well, I've spent a lot of time playing with different ways of installaing the combination of postgresql/aolserver/openacs. I have been able to install them multiple times including the "Project Open" distribution. I think I got some feel now how things work.
What bugs me now is that not a sigle installation is a fully working bugfree installation. Not counting the apm package links that point to nonexistant repository on openacs.org, there is always a problem with several software packages that either refuse to install or bomb out with SQL/TCL errors when I try to use them. Even the default installation of vmware version of Project Open is broken right from the start.

Question then -- is there a good known way to install packages that works error free? Is there a way to install a bare bones version of OpenACS that I can start adding packages to and test each one if it works?
Is there anything that just works? ;)

Collapse
Posted by Vic May on
Unfortunately aolserver4-nspostgres package does not contain that library file.
Collapse
Posted by Emmanuelle Raffenne on
Hi Vic,

I ran into that missing library problem in Ubuntu too. Ubuntu installs version 4.5 of aolserver and version 4.0-4 of aolserver4-nspostgres (i386). I solved it using the packages from debian/stable instead, that are aolserver 4.0 and nspostgres 4.0-3 (not sure what the difference is for that last one).

I know that it won't solve the main problem (i.e. an easy installation) but that might solve your current one.