Forum OpenACS Development: Webmin-like control package?

I have a few customers that love OpenACS but are not familiar with
some of the other parts of Unix.  Adding new virtual email accounts,
handling DNS changes, etc. are things that they must rely on me to do
for them.

I am thinking about writing something like a Webmin clone (but much
better of course) that could control either one machine, or using ssh
control several machines.  OpenACS' ability to set permissions on
objects, keep everything in a database etc. makes me think about doing
things like using the CR to keep copies of configuration files which
are  then copied to other places on the Unix machine to reconfigure it.

Is anyone else doing something like this?  I would plan to have it
control both Apache and AOLserver instances, Postgres databases, and
some sort of email control (like being able to add email accounts and
set up basic email aliases).

Actually Jerry, it's the other way around.    I want to replace Webmin, not add to it.

I think that Webmin's interface is painful and it is too easy for an inexperienced user to shoot themselves in the foot.

There is not the fine grained delegation either (eg. let a user modify a DNS record or zone but not delete it; keep a revision history of email aliases and accounts).

Thus the idea that AOLserver + a database is the best place to store configuration data.  OpenACS already has the authentication mechanisms and the ability to restrict or grant certain privs on an object.

You could webmin-ize anything that you want, but the more powerful the system gets, the more finicky the configuration will become... to the point that you will either need a highly in-the-know person to set up the server/cluster appropriately, or you'll have to start offering RPMs that take care of the details for you.  The issue here, to me, seems that everybody has their own take on running aolserver (inittab v. daemontools, /home/aolserver v. /etc/aolserver, all the possible usernames for the nsd process).  More advanced administration of aolserver, PG, etc would likely involve quite a few setuid scripts as well.  Things get even more complicated when you get into issues like setting up accounts and authorized keys for SSH.  As a service to allow people to become more independent in terms of hosting, I think this would be a great idea.  I guess I'm just worried that it'll be really difficult to streamline installation of such a system enough that it would be widely adopted.

Michael's right, John's right, David's right.

Good luck! :)

Jerry's right in the sense that this would be the quickest path.  I'm not familiar with webmin so can't comment on its usability.  Would it be possible to hack the code into something more usable rather than start from scratch?

The right anwer I am most interested in is for the audience that currently does not have nor will ever have the skills to even have an opinion about the best way to set up AOLserver. So a webadmin that would make all the key decisions for the user, but there is some stuff the user just needs to specify--ip address, etc.<br><br>
That would be invaluable to bringing a much wider user base into OACS and would make a "OACS stack" install (Linux, postges, aolserver, OACS) into a realistic possibility for this audience. <br><br>
What can I do to help?

It's been some time since I've been involved with the ACS/Pg ... sorry, OpenACS project, but I'm getting back to it now that a few things have stabilized here.

First, I can make RPM's for a complete initial server stack that work with my existing PostgreSQL RPMset. This is easy -- particularly the way OpenACS is already packaged -- it's basically just an AOLserver installation (separate RPM), the drivers (separate RPMS), config files (whose defaults can be massaged easily enough), and the /web tree.

However, an RPM installation is no panacea. There are a number of things that must be addressed by a prospective OACS RPM set:

1.) How much do we want to automate in the configuration of the install? Install isn't hard anyway -- particularly with the web bootstrap. How much _CAN_ be automated? (Think this through very carefully) Then remember that RPMs are not supposed to write to stdout or read from stdin during installation....

2.) How much do we want to default? After all, one of the biggest steps in the install is figuring out the service name. IMHO, having the service name hardcoded as a directory name isn't the best solution -- what is a reasonable default?

3.) Remember that RPM is a _package_ system and as such is designed for installation, uninstallation, and upgrading of packages. Which brings us to:

4.) How much upgrade functionality do we want? An RPM upgrade is an 'install new version then erase previous version' process, with lots of script munging going on that is not obvious. Of particular note is the obfuscatory way in which the various installation scriptlets run, and their order -- particularly if trigger scripts are involved. The PostgreSQL RPMs have serious issues with upgrades that I have over many months tried to work with, and have had difficulty getting to work right. To the point that I'm about to change fundamental aspects of the package in order to get something a little less broken.

But RPMs need thorough thought before blindly implementing them. And a review of the existing 3.x RPMs is most definitely in order -- if they have reasonable defaults, the principle of least surprise holds. But likewise, is upgrading from them even supportable?

And /web has got to go in the RPM context... /var/spool/openacs, perhaps. Or some other FHS area. But /web is positively begging for trouble on most systems.

All comments welcome, particularly from the one who put together the 3.2.5 RPMset.

Oh, BTW - welcome back, Lamar!

AOLserver's also dependent on libxml2.  OpenACS 4 no longer requires PL/Tcl so we don't need that any more.

How should we handle optional elements, like ImageMagik and Graphviz?  I would assume we shouldn't force folks to have these available in order to install the basic system, but we should document which packages require them (workflow for Graphviz, the photo packages require ImageMagik)

Hey Lamar!

I think we have to be careful not to conflate two different goals
here. The RPMs for 3.x (AFAIK) were created (AFAIK) mostly for
the benefit of existing community members. That's great. But one
of the things we're discussing in this thread is creating
packages for a target audience with far less expertise (and
possibly for use in a fairly well-defined vertical app). If so, then
the choices we make for that goal are likely to be very different.

I would add that, at least for dotLRN, Windows and Mac OS X
installers are probably as important as good linux installers.

Don, to be clear, I was specifically responding to Lamar's
questions about how much to automate and what kinds of
defaults to set. I could still very well be wrong here, but I'm
guessing that you'd want to err on the side of enabling flexibility
for the typical OpenACS developer's RPM's while erring on the
side of ease-of-use for, say a dotLRN install conducted by
somebody who has no clue what values to set for the various
parameters.  I don't know if this makes a practical difference in
how the RPMs are put together, but at the very least we should
consider the various use cases and look for discrepancies.

I didn't mean to imply that ease of installation is not a goal or
was not achieved with the developer-centric RPMs.

Whew...

Anyway, let me address the thoughts brought up.

First, it's good to be able to be back. I am revamping our Intranet site to OACS 4, so work once again realigns with hobby, allowing me the time to do this again. My management here is very excited about workflow in particular, and the ticket tracker as a major secondary. I will be writing packages for several radio-station type applications, including an interface from our automation system to the web page (for a scrolling program guide on our Internet site as required by CARP). There are a few others -- we currently have a major workflow-esque process during our yearly fundraiser, and many things in a radio station's operation can be workflowed right nicely (such as the process of putting a program on the air -- there is scheduling, billing, promo production, intro/lockout production, notification of programmer, etc). So these are exciting times. I have a day or two a week for most of the summer budgeted for it, so I'm looking forward to digging in.

Next, the RPM system allows all sorts of dependency information to be codified. Much of this is completely automatic (such as the AOLserver dependency upon libxml2, which most modern Linux dists include (RH 7.x for instance)). Others, like preventing an OpenACS RPM upgrade must be hardcoded (to do this, a Conflicts: OpenACS < current-version tag would work). And PG's upgrade is very broken. But an AOLserver upgrade shoud be mostly painless, unless AS 4 is radically different from AS 3 (which it is shaping up to be not-quite-radically different.....I'm tracking AS 4 CVS).

Subpackages could easily handle many of these concerns -- in fact, I would _want_ several subpackages in order to cleanly implement the dependencies. As the RPM packager has fine-grained control over the package's dependencies (and provides), the only hard part about this is making a sane subpackaging scheme.

Splitting ACS Packages out into separate RPM's can be done rather easily, and would likely be the best way to handle such dependencies. For instance, I wasn't aware of workflow's dependency upon Graphviz, or the photo stuff's on IM. Those sort of dependencies need to be either applied to the whole dependent package or to the dependent subpackage.

And RPM-based distribution users are used to resolving dependencies -- likewise, if OACS ever made it into a shipping distribution as an install item, accurate package depends are going to be mandatory -- and the various installers (such as RedHat's anaconda) are very much aware of package installation order and dependencies, and automatically handle such.

Although a depends on ImagMagick could be problematic for somd server installations -- but Red Hat is now forcing X on even server installs. Well, the Xlib stuff at least. An X-less server running current RedHat is difficult to obtain at the least.

Now, on to the APM versus RPM issue for ACS packages. I can see the merits of both mechanisms. ACS knows how to handle package installation -- although I had to force some packages to install here due to some odd dependencies on packages that were already installed... But RPM knows how to handle external-to-the-ACS-core dependencies in a sane manner. And, if APM knows how to handle a package 'dropped into' the installed dirs, as Don implies, then maybe this could be a hand-off, allowing RPM to handle dependencies and APM to handle bootstrapping. The core RPM would then Provide: each of the core packages, and the sub-RPM's would have a Requires: on those core packages needed.

A script could be written to parse the APM dependency info into RPM Requires and Provides info during RPM build from the spec file.

But if we're going to do the RPM thing, proper dependency information is a must for sane operation of the systems RPM database.

Next, the question of differing goals comes up. Michael has an excellent point here. I would look towards the Great Bridge (RIP) installer, which, IIRC, is still available on gborg.postgresql.org, as a model installation system that uses RPMs as the backend package. The Loki installer (source still available from CodeWeavers at least) is a nice, open source graphical installer -- I don't know about Win32 and MacOS, though.

RPM is designed to assist in installation, not replace a good GUI installer. An AOLserver-backed web-based installer might be the right direction, with the service name and all the AOLserver options available from there.

RPM install automation != a GUI installer's automation. RPM cannnot have an interactive install (unlike Debian). Install first, then run a script or GUI for configuration. If you want a good 'vertical' installer, then write such an animal with RPM bindings so that you can install the packages as RPM's, with all of RPM's nice dependency tracking, but with the GUI layered above, allowing configuration immediately after the RPM's are installed. Python is one language that (thanks to anaconda) has excellent RPM/librpm bindings to allow direct installation and configuration. The Loki installer can be a shell archive (CodeWeaver's 'CrossOver Office' and 'CrossOver Plugin" are distributed this way) that the user simply executes, bringing up whatever window(s) is/are necessary. This is a wheel already invented.

The amount of automation would be determined by the answer to the question 'What is the minimum set of operating data that is absolutely required to be entered by the target sysadmin?' And I _do_ mean sysadmin -- a webserver installation is one not likely to be tackled by any but a sysadmin.