Forum OpenACS Development: Webmin-like control package?

Posted by Patrick Giagnocavo on
I have a few customers that love OpenACS but are not familiar with
some of the other parts of Unix.  Adding new virtual email accounts,
handling DNS changes, etc. are things that they must rely on me to do
for them.

I am thinking about writing something like a Webmin clone (but much
better of course) that could control either one machine, or using ssh
control several machines.  OpenACS' ability to set permissions on
objects, keep everything in a database etc. makes me think about doing
things like using the CR to keep copies of configuration files which
are  then copied to other places on the Unix machine to reconfigure it.

Is anyone else doing something like this?  I would plan to have it
control both Apache and AOLserver instances, Postgres databases, and
some sort of email control (like being able to add email accounts and
set up basic email aliases).

Posted by Jerry Asher on
Webmin comes with modules that claims to do most of this already (including postgres, apache, and qmail.)

Why not start off by creating a webmin plugin/module for AOLserver and the OACS?

Posted by Patrick Giagnocavo on
Actually Jerry, it's the other way around.    I want to replace Webmin, not add to it.

I think that Webmin's interface is painful and it is too easy for an inexperienced user to shoot themselves in the foot.

There is not the fine grained delegation either (eg. let a user modify a DNS record or zone but not delete it; keep a revision history of email aliases and accounts).

Thus the idea that AOLserver + a database is the best place to store configuration data.  OpenACS already has the authentication mechanisms and the ability to restrict or grant certain privs on an object.

Posted by Michael Feldstein on
Having a simple, clear, and unified web admin interface for
handling the day-to-day tasks of managing AOLServer, the db,
and the mail server would be a huge win. Having to administer
all the various unfamiliar pieces of the OpenACS stack is a
significant barrier to entry for those potential users who are
thinking about an in-house installation (as opposed to managed
hosting). This is a particularly important with vertical apps (like
dotLRN), where the client is buying what's supposed to be a
unified packaged solution.

It does sound like a hard job, though. Exactly what tasks could
be handled through a web interface? How hard would it be to
come up with a clear UI for those tasks? Which ones would be
the most useful to have first? And what tasks are admins likely to
need to perform that could *not* be handled through a web
admin interface?

Posted by John Mileham on
You could webmin-ize anything that you want, but the more powerful the system gets, the more finicky the configuration will become... to the point that you will either need a highly in-the-know person to set up the server/cluster appropriately, or you'll have to start offering RPMs that take care of the details for you.  The issue here, to me, seems that everybody has their own take on running aolserver (inittab v. daemontools, /home/aolserver v. /etc/aolserver, all the possible usernames for the nsd process).  More advanced administration of aolserver, PG, etc would likely involve quite a few setuid scripts as well.  Things get even more complicated when you get into issues like setting up accounts and authorized keys for SSH.  As a service to allow people to become more independent in terms of hosting, I think this would be a great idea.  I guess I'm just worried that it'll be really difficult to streamline installation of such a system enough that it would be widely adopted.
Posted by David Kuczek on
It would be nice if that package could integrate the OACS backup and vacuum tasks...


Posted by Don Baccus on
Michael's right, John's right, David's right.

Good luck! :)

Jerry's right in the sense that this would be the quickest path.  I'm not familiar with webmin so can't comment on its usability.  Would it be possible to hack the code into something more usable rather than start from scratch?

Posted by Michael Feldstein on
The right answer probably depends on whose problem you're
trying to solve. If the idea is to clone Webmin for current
OpenACS users, then creating a Webmin plugin and getting
OpenACS community folks to use it and whine about it for a
while might be the best way to start scoping out a replacement.
If, on the other hand, you are more tightly focused on reducing
the barrier to entry for potential OpenACS or OpenACS-based
vertical app users, then you probably want to identify the specific
tasks that are (a) important for them to be able to handle and (b)
realistic to expect them to be able to handle and build a
no-compromises UI with tightly focused functionality (and maybe
one that sets some defaults that are likely to work in typical
cases at the expense of some flexibility).
Posted by David Geilhufe on
The right anwer I am most interested in is for the audience that currently does not have nor will ever have the skills to even have an opinion about the best way to set up AOLserver. So a webadmin that would make all the key decisions for the user, but there is some stuff the user just needs to specify--ip address, etc.<br><br>
That would be invaluable to bringing a much wider user base into OACS and would make a "OACS stack" install (Linux, postges, aolserver, OACS) into a realistic possibility for this audience. <br><br>
What can I do to help?
Posted by John Mileham on
Sounds to me like making an out-of-the-box OACS stack installer is a task best suited to a set of RPMs (something that somebody is hopefully thinking about doing once stable releases start getting cut?)... a webmin-like interface could be packaged up as yet another RPM.  There are really two different issues here: easy setup of a vanilla server stack and simplified configuration of a running server stack.  Both of these tools would be valuable independently, and both projects would benefit from working together (or at least the community would benefit as a result of such a collaboration), but they don't seem like the same project to me.
11: RPM Caveats. (response to 1)
Posted by Lamar Owen on
It's been some time since I've been involved with the ACS/Pg ... sorry, OpenACS project, but I'm getting back to it now that a few things have stabilized here.

First, I can make RPM's for a complete initial server stack that work with my existing PostgreSQL RPMset. This is easy -- particularly the way OpenACS is already packaged -- it's basically just an AOLserver installation (separate RPM), the drivers (separate RPMS), config files (whose defaults can be massaged easily enough), and the /web tree.

However, an RPM installation is no panacea. There are a number of things that must be addressed by a prospective OACS RPM set:

1.) How much do we want to automate in the configuration of the install? Install isn't hard anyway -- particularly with the web bootstrap. How much _CAN_ be automated? (Think this through very carefully) Then remember that RPMs are not supposed to write to stdout or read from stdin during installation....

2.) How much do we want to default? After all, one of the biggest steps in the install is figuring out the service name. IMHO, having the service name hardcoded as a directory name isn't the best solution -- what is a reasonable default?

3.) Remember that RPM is a _package_ system and as such is designed for installation, uninstallation, and upgrading of packages. Which brings us to:

4.) How much upgrade functionality do we want? An RPM upgrade is an 'install new version then erase previous version' process, with lots of script munging going on that is not obvious. Of particular note is the obfuscatory way in which the various installation scriptlets run, and their order -- particularly if trigger scripts are involved. The PostgreSQL RPMs have serious issues with upgrades that I have over many months tried to work with, and have had difficulty getting to work right. To the point that I'm about to change fundamental aspects of the package in order to get something a little less broken.

But RPMs need thorough thought before blindly implementing them. And a review of the existing 3.x RPMs is most definitely in order -- if they have reasonable defaults, the principle of least surprise holds. But likewise, is upgrading from them even supportable?

And /web has got to go in the RPM context... /var/spool/openacs, perhaps. Or some other FHS area. But /web is positively begging for trouble on most systems.

All comments welcome, particularly from the one who put together the 3.2.5 RPMset.

Posted by Don Baccus on
Upgrades can be restricted to the AOLserver and PG RPMs.  OpenACS 4 includes a package manager and - after we get our shit together and have a proper repository set up here - folks will be able to grab new packages and updates and use the APM to upgrade or install datamodels, etc.
Posted by Don Baccus on
Oh, BTW - welcome back, Lamar!

AOLserver's also dependent on libxml2.  OpenACS 4 no longer requires PL/Tcl so we don't need that any more.

How should we handle optional elements, like ImageMagik and Graphviz?  I would assume we shouldn't force folks to have these available in order to install the basic system, but we should document which packages require them (workflow for Graphviz, the photo packages require ImageMagik)

Posted by Roberto Mello on
Welcome back Lamar,

It's great that you brought up the RPMs issue. I'm working on packaging OpenACS 4 for Debian and have been thinking about these very issues. We could try to work together on getting installation scripts and such for the packages.

Another question I have is whether we should package the different OpenACS packages or just the core and let the user pick whatever other packages he/she wants through the APM. Don seems to have implied that we should let the APM do it.

Posted by Michael Feldstein on
Hey Lamar!

I think we have to be careful not to conflate two different goals
here. The RPMs for 3.x (AFAIK) were created (AFAIK) mostly for
the benefit of existing community members. That's great. But one
of the things we're discussing in this thread is creating
packages for a target audience with far less expertise (and
possibly for use in a fairly well-defined vertical app). If so, then
the choices we make for that goal are likely to be very different.

I would add that, at least for dotLRN, Windows and Mac OS X
installers are probably as important as good linux installers.

Posted by Don Baccus on
I don't think the comment about RPMs is true, Michael.  RPMs are as close as you get to painless installs in the RedHat/Mandrake world, and the RPMs put together for 3.x allow for just dropping everything in and firing up a new instance.

We do of course need windows installers if we do indeed succeed with native Windows support.  Mac OS X ... I wonder if anyone here knows anything about standard installs for that platform?

Roberto - we should load up all the existing packages that we distribute as part of the tarball and dump them in the installed /packages directory.  The APM will take over from there.  What I meant was that I don't see any need to track interpackage dependencies, etc, at the RPM level.  The APM will make sure the right stuff's installed and will sort out dependencies.

I don't think we want to autoload all packages because it takes a long, long time to do so due to the reference data that's bundled in.
Timezones in particular.  We've talked about bulk loading of that data but it's never happened and if it does, it will be for a future release.

Posted by Michael Feldstein on
Don, to be clear, I was specifically responding to Lamar's
questions about how much to automate and what kinds of
defaults to set. I could still very well be wrong here, but I'm
guessing that you'd want to err on the side of enabling flexibility
for the typical OpenACS developer's RPM's while erring on the
side of ease-of-use for, say a dotLRN install conducted by
somebody who has no clue what values to set for the various
parameters.  I don't know if this makes a practical difference in
how the RPMs are put together, but at the very least we should
consider the various use cases and look for discrepancies.

I didn't mean to imply that ease of installation is not a goal or
was not achieved with the developer-centric RPMs.

Posted by Walter McGinnis on
The idea of having multiple targeted installation paths is
interesting, but also problematic.


We could have a "demo" distribution for marketing purposes,
that is never intended to be a real site and is ultra simple to set
up.  It should come pre-configured and only takes a minimum
parameters to get it started.

Things that we learn while creating a demo installation may be
generally useful.  Such as how to pre-configure (i.e. loading data
to set up the site rather than stepping through the install) cleanly
could be a big potential time saver for developers if it leads to a
re-usable export/import scheme that allows developers to "save"
a baseline install that they can return to after corrupting a
development site's data, etc.


Constructing multiple installation paths splits focus and
potentially drains quality of development from larger installation

Maintaining multiple installation paths is a pain.

BTW, Mac OS X has a .pkg system for distribution.  Its pretty
general, has a standard system for building packages
(PackageBuilder included with the free Developer Tools), and
has a nice GUI installer that it works with.  Other than that I don't
really know anything about it.  Perhaps MarkD (borkware
anyone?) or Russell M. more.

I'm pretty groggy at this point in the morning.  I wonder how bad
my writing is... I think editing might be a nice feature.

19: Responses to RPM stuff. (response to 1)
Posted by Lamar Owen on

Anyway, let me address the thoughts brought up.

First, it's good to be able to be back. I am revamping our Intranet site to OACS 4, so work once again realigns with hobby, allowing me the time to do this again. My management here is very excited about workflow in particular, and the ticket tracker as a major secondary. I will be writing packages for several radio-station type applications, including an interface from our automation system to the web page (for a scrolling program guide on our Internet site as required by CARP). There are a few others -- we currently have a major workflow-esque process during our yearly fundraiser, and many things in a radio station's operation can be workflowed right nicely (such as the process of putting a program on the air -- there is scheduling, billing, promo production, intro/lockout production, notification of programmer, etc). So these are exciting times. I have a day or two a week for most of the summer budgeted for it, so I'm looking forward to digging in.

Next, the RPM system allows all sorts of dependency information to be codified. Much of this is completely automatic (such as the AOLserver dependency upon libxml2, which most modern Linux dists include (RH 7.x for instance)). Others, like preventing an OpenACS RPM upgrade must be hardcoded (to do this, a Conflicts: OpenACS < current-version tag would work). And PG's upgrade is very broken. But an AOLserver upgrade shoud be mostly painless, unless AS 4 is radically different from AS 3 (which it is shaping up to be not-quite-radically different.....I'm tracking AS 4 CVS).

Subpackages could easily handle many of these concerns -- in fact, I would _want_ several subpackages in order to cleanly implement the dependencies. As the RPM packager has fine-grained control over the package's dependencies (and provides), the only hard part about this is making a sane subpackaging scheme.

Splitting ACS Packages out into separate RPM's can be done rather easily, and would likely be the best way to handle such dependencies. For instance, I wasn't aware of workflow's dependency upon Graphviz, or the photo stuff's on IM. Those sort of dependencies need to be either applied to the whole dependent package or to the dependent subpackage.

And RPM-based distribution users are used to resolving dependencies -- likewise, if OACS ever made it into a shipping distribution as an install item, accurate package depends are going to be mandatory -- and the various installers (such as RedHat's anaconda) are very much aware of package installation order and dependencies, and automatically handle such.

Although a depends on ImagMagick could be problematic for somd server installations -- but Red Hat is now forcing X on even server installs. Well, the Xlib stuff at least. An X-less server running current RedHat is difficult to obtain at the least.

Now, on to the APM versus RPM issue for ACS packages. I can see the merits of both mechanisms. ACS knows how to handle package installation -- although I had to force some packages to install here due to some odd dependencies on packages that were already installed... But RPM knows how to handle external-to-the-ACS-core dependencies in a sane manner. And, if APM knows how to handle a package 'dropped into' the installed dirs, as Don implies, then maybe this could be a hand-off, allowing RPM to handle dependencies and APM to handle bootstrapping. The core RPM would then Provide: each of the core packages, and the sub-RPM's would have a Requires: on those core packages needed.

A script could be written to parse the APM dependency info into RPM Requires and Provides info during RPM build from the spec file.

But if we're going to do the RPM thing, proper dependency information is a must for sane operation of the systems RPM database.

Next, the question of differing goals comes up. Michael has an excellent point here. I would look towards the Great Bridge (RIP) installer, which, IIRC, is still available on, as a model installation system that uses RPMs as the backend package. The Loki installer (source still available from CodeWeavers at least) is a nice, open source graphical installer -- I don't know about Win32 and MacOS, though.

RPM is designed to assist in installation, not replace a good GUI installer. An AOLserver-backed web-based installer might be the right direction, with the service name and all the AOLserver options available from there.

RPM install automation != a GUI installer's automation. RPM cannnot have an interactive install (unlike Debian). Install first, then run a script or GUI for configuration. If you want a good 'vertical' installer, then write such an animal with RPM bindings so that you can install the packages as RPM's, with all of RPM's nice dependency tracking, but with the GUI layered above, allowing configuration immediately after the RPM's are installed. Python is one language that (thanks to anaconda) has excellent RPM/librpm bindings to allow direct installation and configuration. The Loki installer can be a shell archive (CodeWeaver's 'CrossOver Office' and 'CrossOver Plugin" are distributed this way) that the user simply executes, bringing up whatever window(s) is/are necessary. This is a wheel already invented.

The amount of automation would be determined by the answer to the question 'What is the minimum set of operating data that is absolutely required to be entered by the target sysadmin?' And I _do_ mean sysadmin -- a webserver installation is one not likely to be tackled by any but a sysadmin.