Forum OpenACS Development: RFC: automated installer

Posted by Janine Ohmer on
I have been tasked with continuing the work David Kuczek started here at Sloan on an all-inclusive install script for OpenACS.

In looking at his script, which is an excellent start, it seems clear that a truly comprehensive script is going to have to be flexible in several ways:
1.  It needs to support all the major *NIX variants, which will mean that paths and commands will be somewhat OS-specific
2.  It should support customized installs, so that someone who already has Postgres installed, for example, can skip that piece
3.  It should be easy to run it multiple times on an already set-up system without worrying about clobbering anything
4.  Although it should use defaults that mirror our installation documentation, it should allow the user to use different locations and names if s/he chooses to.

David's script does not actually install OpenACS, just everything you need to prepare for the install.  I know that an automated installer for OpenACS/dotLRN exists, because the Collaboraid test servers get reinstalled automagically every night.  I would like it to be possible to invoke that process from this script, when it's ready for prime time.

My thought at this point is to split the install into at least two parts.  The top-level script would allow the user to specify what pieces they want to install, and the second one would ask the user for desired names, locations etc, always offering reasonable defaults which mirror the recommendations in the install docs, and then do the install.  If it's too much to do in one script it could easily be split up into one script per package to be installed.

I'm not sure what to do about OS differences in paths, commands, etc because I'm not sure yet how extensive they will be.  I would prefer to keep it all in one set of scripts, using variables to work around the differences, but it remains to be seen if that approach will work or if it will end up being unreadable.

The consensus when David wrote his TIP for this ( was that the scripts should be written in Bourne shell (sh) so I'll stick with that.

Any comments or suggestions on how I should (or should not) proceed?

Posted by Tom Jackson on


That is a tough problem. One I have been working on for a while. I actually have a tcl based installation script, currently tagged 'app-install', which installs any software that can be installed via a command line.

Tcl is good for a number of reasons, not the least of which is that it _stops_ when it reaches an error. Shell scripts tend to keep chugging along. Currently my script runs through a loop for each application:

  • Query the user to set variables, using a default if available, i.e. install locations, usernames, etc.
  • Run a series of commands. If a command fails, prompt the user for advice: abort, skip, substitute command.
  • Write/append/permission files.

I ran through the entire 5.0 install doc and used the defaults provided for a postgresql install. I found a number of errors in the docs by doing this.

I have to admit it is a bit tedious to setup, but it does work. All the commands which are actually executed are recorded in a log file along with any output from the commands.

I have command scripts for the following:

  • Download/install tcl
  • Download/install daemontools
  • Download/install aolserver
  • Download/install aolserver-modules (including tDOM)
  • Download/install postgresql
  • Download/install init.d script for postgresql
  • Create OpenACS startup scripts -- nsd.tcl, db user, createdb.
  • Download/install OpenACS software.
  • Setup daemontools control directory for service.
  • Initialize cvs repository.

Usually all that is left is to start up the service:

$ ./bin/service-run

Then install OpenACS by filling in the form.

Each application is independent, but they can be chained together and variables set previously default to the already entered values.

The main issue I need to address is to switch over to using sudo. Some commands were executed 'su'. In general it would be nice to use sudo on every command, and allow switching users. Then some configuration of the sudoers file would be required to run the script.

Example script files are at The .rhf files are the configuration files. The application is included with the rmadilo package:

Posted by Joel Aufrecht on
Note also that there is a tcl script already in /etc/install which takes care of everything (database drop and re-add, cvs checkout, daemontools setup or restart, navigating the install web page) AFTER you have a machine with aolserver and postgres or oracle set up.  Its other limitation is that it requires tclwebtest.  Its use is documented at
Posted by Cathy Sarisky on
Hi Janine,

Sounds very useful!

I'll second what Tom said about the importance of the script stopping on error.  We don't want the user to wait on the whole install process only to find out that the script bombed in step one.  Also makes it easier to find the source of the error if it is the last line. :)

When you get to the point of having something ready to test, I can volunteer a fleet of disposable RH7, RH9, and Debian vservers ready to be messed up and deleted.  Takes me about 10 minutes (two command lines and some waiting) to spawn a vserver, and it can be totally goofed up by shell scripts run amok without any problems. :)

Posted by Andrew Piskorski on
You're going to write a big monster script to install all the software OpenACS depends on from source? Doesn't that conflict rather directly with the desire to have those pieces installed using the system's native package management, e.g. rpm?

A more limited script to do automatically do the OpenACS setup tasks after AOLserver, PostgreSQL, etc. are already installed seems more useful to me. Especially if it cleanly supports dropping and re-creating test instances over and over again, that sort of thing. And Joel says above that at least a first draft of this already exists...

If you want to do both the AOLserver etc. install and the OpenACS config stuff, fine, put please make sure it's two separate scripts, and that the second (OpenACS config) script is parameterized enough so that it will work correctly even if you installed AOLserver and everything else from rpm or deb packages rather than from source using the first script.

For these scripts, especially in this community, going with Bourne shell rather than Tcl is a bad idea for maintenance. I've done a lot of Bourne shell programming, and Tcl is just plain better. Perhaps do the first (AOLserver etc. install) in Bourne shell if you really want to. But through its various dependencies I believe OpenACS effectively already requires that tclsh be installed on the box, so at least for the second (OpenACS configure) script I don't see any reason at all to use Bourne shell rather than Tcl.

Posted by Mark Aufflick on
three thoughts:

1) tcl/tk would be a good combo to make a combined text or graphical installet - and since we all have to be tcl experts anyway it makes sense to use what we know

2) i like the way fink works (fink is a MacOS X package manager similar to bsd ports)

on OSX I, for instance, type "fink install postgres" - fink makes sure all the dependancies exist, downloads the source, downloads os X specific patches from it's own server, does the magic and compiles/installs the whole shebang.

3) you need to decide if you are going to use the native platform package manager or not. if so, the way to check dependancies is obvious, albeit different on every platform. you don't have to be locked in to the default packages since you could roll your own, or you could make, eg. an postgres-openfts package that relies on the standard package.

I personally think that it would be best to ignore the package manager since noone will be willing to maintain the up to date packages for every distro.

Instead I would use an autoconf style script to check for the required libraries/versions etc. as a bonus this would give you the paths to everything so you don't need to hardcode specific paths for specific OSs.

Posted by Bart Teeuwisse on
Another interesting approach is to combine Mark's remarks with John's work on portable.nsd. Portable.nsd allows us to run OpenACS code outside AOLserver in a tclsh. See for more information.


Posted by Walter McGinnis on
Here's an interesting side note.  Darwinports (very roughly analogous to FreeBSD ports collection) chose Tcl:

Darwinports is also another target for where openacs should be able to install from.

Posted by Walter McGinnis on
Oh yeah.  It should also be noted that Postgresql 7.4.1 and AOLServer 4 are both already in Darwinports.

Posted by Janine Ohmer on
Just getting back to this...

The reason Sloan wants to build and contribute an automated installer is to make it easier for a new user with no technical background to download and install dotLRN.  The long list of "stuff" that has to be downloaded from here, there and everywhere and installed is thought to be scaring away potential adopters, who don't have much time to spend on this and feel they lack the skills to get through the manual installation.

The question about using Tcl vs Bourne shell boils down to this:  Is tclsh going to be installed on all systems by default?  All of the different flavors of UNIX I have access to already have it, but that's no guarantee.  I don't know how to find out, without trying each one, whether all the major distros include tclsh in a default install and will continue to do so - any sugestions?

I completely agree that the script should stop on error.  I haven't done much shell scripting lately but it seems like it ought to be possible to trap errors somehow;  on Solaris in particular many installers still use Bourne shell, AFAIK. Is there some reason why this can't be done?

My gut feeling is that it would be better to have one script (or set of scripts) that takes the user all the way to installing OpenACS, as Tom describes above, rather than having one set to do the external package installs and another (the tclwebtest-based script) to do the setup.  However, this would mean there would be some duplication between my script and the exisiting one.  Can someone familiar with the existing script comment on how widely used it is, who is supporting it, and what the future plans for it are?  If no-one is really championing it then it's probably not such a big deal to make it potentially obsolete.

Posted by Joel Aufrecht on
We are using and improving the OpenACS install script continuously. Any replacement should offer at least the same functionality (create a new openacs site from scratch, tear down and rebuild an existing site). The only direction forward that really makes any sense to me is to pick one or more distros and create and commit to maintain packages for them. Those packages should specify their dependencies and then re-implement the stuff in the script in the native language of that particular installer platform.

In this scenario, we still need a fairly generic script to serve as the reference across different implementations. Note that the current install script is in fact a bash script, not a tclwebtest script. Tclwebtest is required for a single call, and for some advanced features such as doing a link test after installation.

So the most useful approach would be to pick an existing distribution platform that already has all of the dependencies, and make an OpenACS package on top of that. Out of RPM, debian, bsd ports, fink, etc, which one is closest? What scripting languages do they use? Which is most useful to the broadest range of existing and immediate potential users?

The better solution, tongue in cheek, is to start charging $10,000 for installation services. Then we'd be recognizably enterprise-class.

Posted by Jeff Davis on
I think it's easier to write a portable Tcl installer than a sh based one (it's easy to end up using flags not supported everywhere or depend on commands that don't exist in some OS's). For a Tcl based install we could do it with a tclkit and there are tclkit binaries for pretty much any platform I think people are interested in.

Otoh, you can definitely exit on errors with a shell script

set -e
does that and you can use the trap command to catch the error (although I think that is a bash-ism and you cant trap ERR with vanilla sh).

I would like to see packages for .debs, .rpms, and ebuilds and any of the half dozen other package managers people use.

I suspect the best way to get there would be to have a place in CVS where we keep the source for building with the various package tools (for example there are a set of ebuilds to install all the aolserver stuff + openfts and writing an openacs ebuild would be pretty easy). Maybe we should create an openacs-4/etc/installers/DISTRO directory and under that have the source for generating the various packages/build scripts. If we get them working well then I imagine they will get pulled into more standard distributions.

Another alternative would be to try harder to become the maintainers of aolserver and openacs on various standard distributions. The problem seems to be that the people maintaining aolserver for the various distributions where it is present are not really very good at keeping up to date. I don't know how hard it would be to get commit on gentoo and debian for example, and I don't follow fedora at all.

Anyway, there are a lot of options; I think getting things into standard distributions is important but I don't know how it will happen until some people here get commit to maintain the necessary packages.

Posted by John Sequeira on

If Sloan's need is to demonstrate .LRN to those with limited technical background,  why not skip any installation and use a VMWare image or .LRN LiveCD like the one Malte is working on?

There would be hugely less amount of work to maintain it (no need to select tcl vs. bash,  deb vs rpm vs etc, no need to track  or resolve openacs/tcl/aolserver/tdom version conflicts),  and also way less work/time to install and try it out.

Also,  fwiw I was planning on including a .LRN install on my next Oasis release.

16: OpenACS Live CD (response to 14)
Posted by Malte Sussdorff on
Live CD, i knew I forgot to post something....

This is the OpenACS Knoppix run out of the box, should not use 512MB of RAM (but might), .LRN 2.0 enabled, Jabber running demo live CD.

We have to work a little bit more on it, but are keen to collect feedback and if you want to get your custom made CD don't hesitate to contact Björn Kiesbye (the author), or myself.

17: Re: OpenACS Live CD (response to 16)
Posted by Malte Sussdorff on
You might have to right click and say "save to disk" to download the file and not view it in the browser. Any idea how I could teach my AOLserver to offer this file type as "download only" ?
Posted by Mat Kovach on
There is no reason we can't use TCL.  Obviously if the install system is going to run OpenACS TCL will run on the system.  If we really want to make and installer we might consider using tcl source distribution to create a customize tclsh to do the installation.  The installer would then be a bit easier to control.  We could also look into using starkits to distribute the installer also.

It should be that hard to use a portable TCL solution for an installer I have done with customer projects, successfully, in serveral instances.

Posted by Jun Yamog on
I agree with John, definitely if its for demo.  A vmware image or better yet a LiveCD is the best.
Posted by Janine Ohmer on
Actually, no, demos is not the goal.  The goal is to make it as easy as possible for anyone, even the overworked part-time sys admin at a K-12 school, to try out OpenACS and dotLRN.  The easier it is to install, the more likely they are to attempt it, then succeed, and then use their installation for something cool.

After all this discussion I'm still not sure yet what the best way to accomplish that would be.

Posted by Joel Aufrecht on
The best way to help a random person get a production-quality install is to get OpenACS packaged in compliance with, and then included in, a standard distribution packaging system.  The most popular ones are RPM, Debian packages, BSD ports, and now OS X's fink.  This has been tried before with OpenACS but has failed because a) nobody continues the maintenance b) we didn't try to become a formally accepted package c) the prerequisite packages, especially the heavily patched AOLserver we required until AOLserver 4.0, didn't exist.
Posted by Tom Jackson on

Unfortunately OpenACS relies on many packages which the OpenACS community cannot demand be maintained, and available in rpm, etc. format. To require this is to say no to an auto installer.

Even OpenACS doesn't have a standard which adequately addresses running multiple independent installations on the same machine.

You only install sendmail once, same with qmail, daemontools, and a bunch of other standard packages. It makes sense to have rpms for these singleton packages. However AOLserver and OpenACS and PostgreSQL are often installed in multiple locations on the same machine. Plus you have to address the possibility that a user might not have root access to their machine and might want to install OpenACS with their own privileges.

Even if rpm packages existed, you still need to maintain an installation document to explain how to do the install. Users still have to find and deal with the same issues. What is needed is a tool to combine installations of multiple independent software products.

Also, note that it isn't rocket science to install any individual package. Most of the time is spent figuring out what options to use to ./configure, how to do the post install configuration, etc. The problem is in glueing the packages togeather (plus finding the up-to-date packages is a pita). OpenACS installation itself couldn't be easier, and with 5.0 it seemed to get even easier.

It is possible something as simple as 'configure' and 'gmake' could work. Some AOLserver modules have gmake targets to create release version of the module, so it is obvious it can be used to perform any command line task, handle dependencies, etc.

Posted by Joel Aufrecht on
Agreeing with Tom: the majority of installation errors seem to be problems getting the required setup working.  Once you get to the "install OpenACS" page of the install guide, you are pretty much home free.  That's why I think we need to rely on the packaging systems, and do what we can on our side to require a stable platform.
I've been playing with a multitude of different solutions to the installation in multiple places problem and found one easy solution that would translate easily to an rpm package.

first create all the rpms for the supporting software. (tcl, tdom, nsd-*, aolserver) these packages would all have the appropiate dependancies with each other.

then create the openacs rpm package (again with the appropiate dependancies) This package will not create an installed instance of openacs but create what unix sys-admins refer to as a skeleton directory as well as supplying a bunch of installation-generation scripts.

these scripts would then be used to run the useradd command (or equiv) to create the openacs user, database, directory structure.

the admin would then modify the site specific .conf file and WHamm0! they have openacs installed.

to the admin this could be made extremely easy if we made all the packages avalible from a yum or apt4rpm repo.

this would be the overall view from the admin's perspective:

apt-get install openacs

this downloads all the dependancies, and installs them

acsinit testinstall
vi /srv/openacs/testinstall/etc/config.tcl
svc -u testinstall

where acsinit is the propigation script and it sets up the directory structure into /srv/openacs/<username>

any thoughts to this method?

Posted by Don Baccus on
I think it might be cheaper for us to provide everyone who wants to try it with a $200 PC + our Knoppix CD than to try to maintain all the various package plus Unix/Linux distro variations :)
Posted by Tom Jackson on

I think it would be a mistake to work on something to just help the newbie. While the tool should support a hands off automated install, it should be customizable in an easy way for the expert. Otherwise the expert will never use it and keep it up-to-date.

What would be the benefit for the expert? A record of how a piece of software was installed so it can be repeated when necessary, both for disaster recovery and service duplication.

The installer should focus on software which traditionally has not been available in an rpm or other type of compiled format, or those which are routinely installed in a non-standard location, or when multiple installations per machine is expected. Of course, an installer which could handle this situation, would easily handle installation of compiled, standard packages.

Whatever the installer, it must have two separate parts: the actual installer, and the configuration file. New applications should only require producing a new configuration file to direct the installation. In the case of configure/gmake, you only have to create the configure script and the makefile, no programming required.

The problem with gmake/configure is, I would guess, most developers here don't spend too much time writing configure/gmake scripts. What they do know is the sequence of shell commands required, plus necessary edits of files.

Posted by Don Baccus on
Well, the big problem with configure/gmake is that it is total overkill for the problem we're trying to solve and not terribly user friendly, either.

Unless you're thinking of writing umbrella scripts to configure and make all the C pieces (AOLserver etc).

OpenACS itself has only one configuration choice that's of interest: which DB to use.  Currently we depend on the user configuring their nsd.tcl init script properly, we could have an umbrella tclsh (or whatever) script ask for which database, user names, passwords etc and write the script for the user.

Other than that, though ... is there anything that's hard OTHER THAN installing AOLserver, PG or Oracle, ImageMagick blah blah blah?

I think we should look at defining the problem space a bit more tightly here.  If our goal is to lower the hassle factor for people who want to try it out, do we really need to worry about debian people?  Anyone running Debian is going to know how to install debian packages.

The person running RH or Mandrake or Fedora from an end-user CD set is much more likely to need help.  Maybe we should concentrate on one platform like this first?  Our resources are limited...

And for anyone with a spare machine sitting around, the Knoppix CD approach is a great one, I was only partly joking when I said that it would be cheaper to just ship folks a $200 PC than to try to support every Unix variant out there.

Posted by Joel Aufrecht on
"Currently we depend on the user configuring their nsd.tcl init script properly, we could have an umbrella tclsh (or whatever) script ask for which database, user names, passwords etc and write the script for the user"

We do.  It's part of OpenACS, in /etc/install, and instructions to get it and use it are located at and linked from the top middle part of the home page of

Posted by Tom Jackson on
I was only partly joking when I said that it would be cheaper to just ship folks a $200 PC than to try to support every Unix variant out there.

I for one took you completely seriously, sounded like a reasonable statement. Of course, I'm sure a lot of folks would sign up to try it out if we advertised our solution!

Anyway, I already wrote a tclsh program that works for linux. The only thing missing for the total newbie is a bootstrap script to install the installer.

As you said there isn't anything difficult in the task of installing the system, but my script gathers all the software and installs from scratch in about an hour (mostly postgres compile time). Still left is the boot up of AOLserver to fill out the form for OpenACS install.

The main problem with my program (besides the fact that I wrote it) is it does what you tell it, which could lead to errors, but if the install script is correct, it works. Unlike an installation document, an install script can be tested for correctness.

Posted by John Sequeira on
And for anyone without a spare machine sitting around,  a Knoppix CD works just great with VMWare's free 30 day trial edition.

Does that cover everyone?

Posted by Janine Ohmer on
Tom, I haven't looked at your script yet, but I am envisioning that a useful script would:

- use XML files (probably) to specify the versions of all the pieces to be installed for, say, a functioning 5.0 install and where to get them from
- allow the user to install everything we recommend, or cherry pick the things  s/he wants to install
- use variables for all OS commands, even if we start out only supporting Redhat, to make it easy for someone to get it working on their OS of choice

Among other things not thought of yet. :) Your script probably does not do all of these things currently, but is it written in such a way that it wouldn't be tragic to try to add them?

I still don't have a good feeling for how well supported the tclwebtest script is. Should we rely on it or not?  I see that it is a shell script (at least that is implied by the name, which has been soundly trounced here as a Bad Idea....

Posted by Tom Jackson on


  • xml -- the configuration files are in an xml-like syntax. Therefore the parser reads this syntax and not xml. FWIW, it is a little easier to get the data back out for use.
  • User can choose to skip an installation and continue on to the next one. You can also edit the startup file, or write your own in a few minutes so it only includes the applications you need.
  • use variables -- easy to add variables to the configuration file, then use them later on. Like Ant, but interactively queries the user if they need to alter the default values of the variables. It would be nice to have a mode where you could skip the querying routine (easy but isn't done yet).
  • tclwebtest -- easily work with this by executing a script, which app-install could download. app-install could actually wrap this command line tool so users could easily change the defaults, without trying to figure out what the command line options are.

    My next attempt to improve this will probably be to create a localized copy of all the needed files, and write an updated installation script before installation begins. I also want to use sudo to execute every command as a particular user.

  • Collapse
    Posted by Don Baccus on
    Tclwebtest will be integrated with automated testing, which we're using more and more.  So we'll be supporting it.  We know where Tilmann lives and can send Dirk over to his flat to beat him up if we need to :)

    I think they're planning on doing the integration during the Berlin bug bash.

    Posted by Janine Ohmer on
    It's good to know that we will continue to use tclwebtest, but I am also concerned about the automated install script.  It currently seems sort of experimental, and that might mean it has no real commitment from it's owner /author to keep it updated.  Is it considered important enough that someone will keep maintaining it?
    Posted by Joel Aufrecht on
    Collaboraid will keep maintaining it until an equivalent replacement is available.  It provides these functions:
    1) takes about 5 critical parameters (name, ip, port, directory) and many optional ones
    2) set up full working OpenACS instance or .LRN instance, supervised by daemontools, from CVS checkout of specified tag
    3) drop and rebuild an existing OpenACS instance
    4) run from command line
    5) report success/failure in parseable xml file

    It has been working well for us for several months and is no longer experimental.  It is no more finicky about paths and versions than the standard OpenACS installation, which is to say that it is pretty finicky.

    Posted by Matthew Geddert on
    Derek, your model sounds good to me. My suggestion would be that instead of typing:

        acsinit testinstall
        vi /srv/openacs/testinstall/etc/config.tcl
        svc -u testinstall

    and pre-determining the location of the openacs directory you would rather allow people to specify it like this:

        openacs-init /var/lib/aolserver/<testinstall>


        openacs-init /web/<testinstall>

    The reason I suggest this is that the documentation used to suggest /web/ and now suggests /var/lib/aolserver/ and as you have said you use a different directory, I would guess that it is common for people to use whatever directory they see fit. I would also sugest that the command start with the name of the apt-get able package (i.e. openacs). Lastly, it would be nice if it defaulted to installing openacs, but if the script also allowed for other "prepackaged" systems to be installed such as dotlrn. Maybe a command line option for openacs-init.

    Another idea would be when you run openacs-init or whatever its called it starts an interactive menu that asks with logical defaults then you don't even need to edit config.tcl. I.E.

    what is the servername [default to hostname]?
    what is the database name [default to directory name]?
    what ip address will this server use [localhost]?
    what port will we use for this server [80]?
    should we run an ssl connection [Y]?
      - if YES then: what ssl port should we use [443]?
    what build would you like (openacs,dotlrn,dotwrk,custom) [openacs]?
      - if custom ask for the install.xml file location
    would you like to start openacs now [N]?

    Posted by Derek Carter on
    the name change is trivial... it could be named anything.

    I really like the idea of the interactive init script.

    the reason I suggested /srv/openacs/ was just to be LSB v1.3 compliant as /var/lib/aolserver is LSB v1.2

    but yeah the idea of being able to specify the directory will make those who like to super-customize happy as well.

    Posted by Derek Carter on
    ok hows about a hybrid installer?

    openACS gurus could just drop in the needed switches like this:

    # openacs-init -d /some/random/dir -u username -g groupname -h localhost:8000 ...

    that command would just do the install, populate the needed directories, create the database user, shell user, database etc.
    and the new users/noobies could just run:

    # openacs-init

    And they would be walked through all the questions interactively with all the defaults shown. Then everything would be done.

    Posted by Jade Rubick on
    Please see the current status of installer page:

    If anyone needs write access, write the OCT.

    Posted by xx xx on
    I could cleanup my FreeBSD port, however it seems nobody is interested, so I don't want to waste time.
    Moreover, since my question about the automated installer was never answered, I don't know if it would fit in in the future (
    Posted by Malte Sussdorff on
    HI Aldert, a friend of mine wants to work on including OpenACS into OpenBSD (where he is a core developer). I should get the two of you in touch.
    Posted by Derek Carter on
    Ok I now have a Xen box hosting multiple linux distro vm's:
    SUSE pro
    SUSE enterprise server
    Fedora Core
    Red Hat Enterprise
    Cent OS

    I plan on creating all rpms for all of the above distros as well as testing the install script/installer I may include mandrake if I see a call to support that distro

    I can add a vm for the following distros, however I'm not familiar with creating packages.

    if anyone is interested in developing packages for these distros, please contact me and I'll spool a vm and give you access


    Posted by Bart Teeuwisse on
    Not an installer for OpenACS itself but nevertheless relevant: a set of source RPMs of packages required by OpenACS. See for details.
    Posted by Andrew Piskorski on
    See also recent relevant discussion of OpenACS Linux package repositories and a Windows installer.
    Posted by Derek Carter on
    opps I mis-posted it should go here. but oh well...

    read up here: