Forum OpenACS Q&A: Easier to install

1: Easier to install

Posted by Alfred Werner on 10/05/04 10:14 PM

This was prompted from reading https://openacs.org/forums/message-view?message_id=203708 - but it deserves its own thread rather than get buried in a CMS discussion.

My principal objections/PITA barriers to setting up an aolserver/openacs instance never have to do with the actual openacs - it's the extras that kill you. The decision of this project to stick with the Dan Bernstein tools - daemontools and qmail - are a serious barrier to some. Add in getting all the right versions of tcl, tcllib, postgres installed and it can be a nightmare.

I'd love to see OpenACS work with a vanilla Fedora Core 2 install - with a few additional RPMs for OpenACS and aolserver. Let's ask Dossy to make the RPMs available or give a hand in doing so. Let's make our install compatible with the stock install of the majority platforms (redhat,fedora,suse) and deal with less widely known distributions (yes including debian, gentoo and others) as extra instructions. I would personally cast a vote for fedora as the default platform.

daemontools - replace with normal /etc/init.d/rc conventions
qmail - replace with postfix which is claimed to be a secure alternative - IBM continued development after Wietse left .. and is appearing as default MTA in distros
postgres - stabilize on currently released stable rpm
openfts (see if it comes installed)

By way of contrast - I am currently working on a project to setup monitoring and regression testing of a Peoplesoft instance. I chose to use Cacti/RRDtool which implies PHP/Apache, mysql and net-snmp for monitoring, and tclwebtest for the regression and load testing. I had never installed Cacti, but I installed a FC2 server, chose 'install everything' and only had to download the cacti and rrdtool rpms and run a very straightforward d/b create script in mysql. It was immediately available in the apache instance that was running. Pretty slick I'd say. Tclwebtest was just a download and copy to /usr/local and a few recommended symlinks. It is a combination of many moving parts but all seems to flow nicely into a modern distribution.

If we can achieve that level of install painlessness by NOT engineering ourselves into every 'less common' variant of widely available services, we'll be doing ourselves a HUGE favor! The added bonus of moving to an /etc/init.d type structure is it integrates with the rest of the services on the system rather than us having the one oddball /svc directory and massive incompatibilities to get qmail up and running (I know if you use RPMs you can't un-install sendmail or postfix sometimes without uninstalling cron or other packages that want an MTA or using the force options).

Your thoughts welcome ..

2: Re: Easier to install (response to 1)

Posted by Bart Teeuwisse on 10/05/04 11:12 PM

Alfred,

AFAIK OpenACS isn't dependent on qmail. At least qmail isn't required to run OpenACS. The notification package comes to mind as a packages that can handle incoming emails if qmail is installed. But even then one can easily adjust the processing procs to work with different MTAs.

Still, replacing qmail with an MTA that is actually being maintained is a GOOD idea.

I've been working on some RPMs myself. Including daemontools which I still prefer over inittab. Daemontools doesn't have to break /etc/init.d/rc conventions either. Writing a service script for a service under daemontools control is a snap. Place the script in /etc/init.d, run chkconfig (on Fedora/RedHat) et voila!

One of the biggest advantages of daemontools is the ability to give ordinary users (some) control over the service. Another big win is multilog which prevents log files from filling up the harddrive.

Most components can be made available as RPMs quite easily. Unfortunately, Fedora/RedHat comes with a single threaded Tcl version. Depending on your system configuration, replacing the standard Tcl package with a multi threaded version might lead to dependecy conflicts as some packages require a specific Tcl version.

Once solution to this issue to is release the multi threaded Tcl package under a different name, e.g. tcl-threaded or tcl-aolserver.

On the subject of RPMs I would like to put a vote in for individual packages for each aolserver module.

/Bart

3: Re: Easier to install (response to 2)

Posted by Alfred Werner on 10/06/04 04:05 AM

Bart - I did just look through the latest install docs - things I mention are listed as optional software. Hats off to Joel and Vinod (and many many others) for doing such a good job on the documentation.

Seems to me that the first thing to tackle is to either convince the TCL community to enable threading by default (i.e. no compile option means its on) or to track down the various RPM bundlers for all the distros and ask them to turn on threads. The no flag for TCL is the simplest if they would go for it. I'll look into the TIP process and propose it.

Next after that - aolserver releases in rpm format for the major distros that are threaded-tcl dependency aware.

Next - tDOM.

Assuming postgres is good as is, FTS RPM.

Next .. ?

Other than load - is there any reason not to bundle up all the necessary software into RPMs for a platform or two and let rpmfind.net know about it?

4: Re: Easier to install (response to 1)

Posted by Dave Bauer on 10/06/04 03:03 PM

For full text search on postgresql, best best is to use the postgresql-contrib package that comes with the various distributions and use tsearch2. This works great on debian. I am working on automatic install of tsearch2 based on various default install locations of RPM, DEB and a default source install of Postgresql.

Debian Tcl is thread enabled by default at least.

5: Re: Easier to install (response to 3)

Posted by Alfred Werner on 10/06/04 06:36 PM

I just sent an email to Jeff Hobbs to request that --enable-threads be the default compile behavior. Will report the results.

6: Re: Easier to install (response to 3)

Posted by Bart Teeuwisse on 10/06/04 07:11 PM

Alfred,

I've made an RPM for tDOM. So that one is covered.

/bart

7: Re: Easier to install (response to 1)

Posted by Jade Rubick on 10/06/04 08:16 PM

Could someone create a chart/checklist that shows which RPMs and apt-get packages have to be created, which have been created, who is maintaining them, and what still needs to be done?

If this page were set up, then we could encourage people to help out in chunks, and we could eventually get to a much easier installation.

I can't wait until we get there! This will do a lot for OpenACS.

Any volunteers?

8: Re: Easier to install (response to 7)

Posted by Alfred Werner on 10/07/04 03:30 PM

Since I started this - I'll do it.

9: Re: Easier to install (response to 1)

Posted by Cathy Sarisky on 10/09/04 04:54 AM

Just to re-iterate, there's no need for qmail, even if you want notifications, forum replies, etc. A stock Postfix package (deb or rpm) works fine with a one-line change in the config file to tell Postfix use Maildirs.

10: Re: Easier to install (response to 1)

Posted by Lamar Owen on 10/10/04 03:06 AM

One needs to carefully watch the redistribution license for daemontools. Essentially, it cannot be distributed as a binary in an LSB-compatible form. Quoting DJB's FAQ:
"May we distribute binaries?
You may distribute a precompiled package if

* installing your package produces exactly the same files, in exactly the same locations, that a user would obtain by installing one of my packages listed above;
* your package behaves correctly, i.e., the same way as normal installations of my package on all other systems; and
* your package's creator warrants that he has made a good-faith attempt to ensure that your package behaves correctly.

All installations must work the same way; any variation is a bug. If there's something about a system (compiler, libraries, kernel, hardware, whatever) that changes the behavior of my package, then that platform is not supported, and you are not permitted to distribute binaries for it."

This borders on the ludicrous, since he INSISTS on a TOP LEVEL directory under /!

An RPM that meets LSB cannot be distributed, since the LSB will never allow /service to exist; meaning Red Hat etc will not distribute daemontools or virtually any other DJB tool. And DJB insists it MUST use HIS correct layout or you cannot distribute binaries, period. This is unacceptable to the various Linux distributors; and if a distributor is violating this DJB will try to shut it down, I'm sure.

I say that wearing my "PostgreSQL RPM maintainer" hat, so it's not like I've never built an RPM before (been doing it five years, in fact...)

The relevant standard (proposed; FHS 2.2 is current) is available online at http://www.pathname.com/fhs/pub/fhs-2.3.html

11: Re: Easier to install (response to 10)

Posted by Nick Carroll on 10/10/04 04:50 AM

Maybe we should look into a replacement for daemontools? I did a quick search, and found "runit".

http://www.freshports.org/sysutils/runit/
http://freshmeat.net/projects/runit/?branch_id=29713&release_id=173665

"runit is a daemontools alike replacement for SysV-init and other init schemes. It currently runs on GNU/Linux, OpenBSD, FreeBSD, Mac OS X, and Solaris, and can easily be adapted to other Unix operating systems. runit implements a simple three-stage concept. Stage 1 performs the system's one-time initialization tasks. Stage 2 starts the system's uptime services (via the runsvdir program). Stage 3 handles the tasks necessary to shutdown and halt or reboot."

I will look into how to use runit, and provide documentation on this.

I also have a strong preference for using Postfix for the MTA. Again we don't need to use qmail, therefore removing OpenACS dependencies on DJB software and the "unworkable licences" that come attached.

12: Re: Easier to install (response to 11)

Posted by Nick Carroll on 10/10/04 05:00 AM

runit is also packaged as an RPM and debian package.

13: Re: Easier to install (response to 12)

Posted by Bart Teeuwisse on 10/10/04 09:32 PM

Nick,

sure Daemontools is not LSB compliant but that doesn't prevent the creation and distribution of Daemontools in RPM format.

DJB's requirement that a binary distribution results in the same installation as installation from source is entirely in line with the goals of the RPM format.

Not being LSB compliant might exclude Daemontools from distributions like Fedora, SuSe and others. But with 3rd party repositories becoming main stream thanks to 'yum' and 'apt' there is an attractive alternative.

OpenACS sets up and maintains a repository of binary distributions of all the software required by OpenACS not already included in the OS distributions OpenACS supports. All users would have to do is register the OpenACS repository to use 'yum' or 'apt' to retrieve OpenACS and the packages it depends upon.

I'm not too keen on switching from Daemontools to Runit, which in essence is a clone of Daemontools. It might have more open license but it doesn't address any of the other objects you've raised. Daemontools has proven its worth and has received community support in the form of patches.

You'll find a RPM of Daemontools for Fedora 2 at the Code Mill if you would like to give it a whirl.

/Bart
the Code Mill

14: Re: Easier to install (response to 1)

Posted by Talli Somekh on 10/11/04 12:10 AM

Bart,

Well, since part of the point of this thread is getting OACS available for and/or into mainline distros, having alternative packages in other exotic locales isn't good enough.

If indeed daemontools is that big of a PITA that it hinders installation/and or adoption, then maybe runit ought to be considered for the official solution to maintaining systems. Then people can offer alternatives.

How many other projects have adopted runit, though?

talli

15: Re: Easier to install (response to 14)

Posted by Bart Teeuwisse on 10/11/04 01:02 AM

Talli,

Runit is available in RPM format. AFAIK it is not included in a RPM based distribution. Certainly not Fedora. In fact it isn't in any of the major yum based repositories.

Runit has made it to Debian's testing distribution. It is also included in the unstable Debian distribution. Where as daemontools is included in Debian's stable distribution.

Bart
the Code Mill

16: Re: Easier to install (response to 1)

Posted by Talli Somekh on 10/11/04 01:24 AM

So... in the future, as distros move towards LSB compliance, which tool do you think will have a greater chance of being accepted as a package in the standard distros, the one with the license that allows for flexibility or the one that doesn't?

talli

17: Re: Easier to install (response to 16)

Posted by Patrick Giagnocavo on 10/11/04 01:53 AM

Why don't we just write something in plain old TCL? There will always be a tclsh installed where AOLserver is to be found.

18: Re: Easier to install (response to 17)

Posted by Bart Teeuwisse on 10/11/04 03:04 AM

Patrick,

Why DIY when there are very good alternatives available?

Talli, granted runit's BSD license allows developers to install runit in a LSB compliant fashion. I'm with you that runit might capture market share from daemontools in the long run because of its license.

At the moment though, daemontools has the edge. It has been around longer and is integrated with more Linux distributions (e.g. Debian, Gentoo).

And yes, DJB's license is restrictive. It restricts redistribution of binaries to his LSB incompatible specifications. However, you are allowed to create patches to the source, patches that compile DJB's source code but install the resulting binaries compliant with the LSB standard. Source based Linux distributions like Gentoo can do this. Debian too doesn't distribute daemontools binaries directly, instead there is package to compile and install daemontools.

So there are ways to respect DJB's permissions and be LSB compliant but it is PITA. For now neither daemontools nor runit is included in major Linux distributions. Daemontools RPMs can be obtained if you don't mind that the package isn't LSB compliant. The runit RPMs that I could find are geared towards Mandrake and SuSe, the RPMs for RedHat were for old(er) RedHat versions.

What to do? Continue to depend on Daemontools and make a non LSB compliant binary RPMs available? Or switch to runit and make LSB compliant binary RPMs available? The latter would hopefully make it into standard Linux distributions in the mid to long term.

Bart
the Code Mill

19: Re: Easier to install (response to 1)

Posted by Malte Sussdorff on 10/11/04 06:47 AM

Maybe I'm missing the point, but shouldn't the installation of daemontools be something the OpenACS package provides, if not already there. Seriously, we already have to do so many steps in a package to get OpenACS to run, adding the lines to install daemontools on the way if not already installed should not be a showstopper, should it ?

I'm more eager to see us adopt the installation of additional packages out of the APM. If we have a package provided by Vienna university, there is most definitely the need to install XOTcl. If we enable the spellchecker, we need aspell. If we install Jabber, we need the Jabber package installed. If we want to make good use of webmail, we need TCLLib (which we should add to the default installation anyway, but this is a different issue / TIP), Photo Album -> ImageMagick, Bounce Management -> Change in Postfix configuration (and there the list is not finished). That's why we should have the ability to do these kind of things on installation with the APM and the APM should inform the user about it (this is the reason why the apm-post-install approach, though working, is not a good one).

On the other hand, my eagerness does not go as far as really implementing this.... :).

20: Re: Easier to install (response to 1)

Posted by Torben Brosten on 10/11/04 10:26 AM

What about modifying aD's aolserver keep-alive[1] to work from a cron'd tcl or expect script? There would be less fragmentation of a working openacs system across platforms.

1. http://www.eveandersson.com/arsdigita/free-tools/keepalive-1999.html

21: Re: Easier to install (response to 1)

Posted by Alfred Werner on 10/12/04 06:56 AM

Quick update - Jeff Hobbs says --enable-threads by default is being considered, there are some concerns that expect and few other things may break. I offered to help test whether this is true. He is at the TCL conference this week, so we will probably not hear much for a week or so.

The matrix is coming along - I am going by distrowatch.com to prioritize platforms. BSD, WIndows, and OS/X will have their own entries as well.

22: Re: Easier to install (response to 1)

Posted by Mark Aufflick on 10/13/04 04:10 PM

Really good to see rational discussion of this stuff.

I personally love daemontools. I also love qmail and sawfish - they suit my needs better than the current "trendy" products. But if it takes someone with 10 years of Unix experience a few hours to get the gear all running, most linux users would really not get very far.

Why can't a "default" openacs install use rc.d files etc., and when you want a super duper 1mio hits/day server you can do it all by hand the way you like.

I liked the way our aolserver 3 installs were self-contained. Maybe we could build an aolserver 4 rpm with it's own tcl/tcllib etc. included. That way it's not a problem if the system has a non-threaded tcl installed.

I think email is less of an issue. All we need is:

1/ access to an inject program
2/ access to a maildir format mailbox which gets all the email to the chosen domain

And even (2) is optional.

23: Re: Easier to install (response to 22)

Posted by Bart Teeuwisse on 10/13/04 08:09 PM

Mark,

OpenACS does not depend on a specific MTA. Inject is qmail specific. While inject was once used by OpenACS it no longer is.

To make sure I grepped acs-core for injec, which indeed could not be found.

Bart
the Code Mill