Forum OpenACS Development: Automated Install, Configuration, and Test
I'm trying to wrap my head around automated installation, configuration, and testing but I'm not finding much (any) usable doc for acs-automated-testing or install.xml capabilities. I'm also looking at the dotlrn autoinstall scripts that utilize tclwebtest to perform after-install UI-based actions. This is pretty sparse too.
Is anybody out there interested in moving these issues forward? I see several posts indicating that we should use these utilities/features but I've had no response to the several posts I've made asking for information, pointers, examples, etc.
I assume that someone(s) have analyzed and used these features and must have some notes, examples, code or cheatsheets on implementation.
So, I'll ask once again if anyone wishes to share? If not, I'll start at square one by digging through the source.
Thanks in advance.
This post describes the install.xml file - it's a way to automatically install packages and set parameters at bootstrap time, rather than going through the tedium of the package-manager.
tclwebtest is a HTTP testing tool written by Til and hosted at sourceforge. The documentation is pretty good and it was pretty easy to get tclwebtest to automate an OpenACS bootstrap (basically: load the index page and click 'Next' a couple times).
I tried at one point to get my head around acs-automated-testing and failed. I couldn't figure out what I was supposed to do with it. I also couldn't figure out how I was supposed to have tclwebtest interact with acs-automated-testing.
I'm sure you knew all this info already, so hopefully others will chime in. And if no one does, I'd be happy to cheerlead you as you dive through the source.
I wrote the tclwebtests for installing OpenACS and .LRN from scratch and setting up .LRN demo data. We are going to be using the OpenACS parts of these scripts for daily creation of integration servers during the External Authentication project we are doing right now at Collaboraid. Joel Aufrecht has been asking me to improve the documentation for the scripts and I hope I'll get around to that. An extension we'd like to make is to get the scripts to run all acs-automated-testing tests on the server as well.
The acs-automated-testing package is for Tcl API tests (similar to JUnit or XUnit for those who are familiar with those). We need more of those tests. The tests reside in a package in the dir tcl/test and a test is defined by invoking aa_register_case. There are enough cases in the OpenACS codebase now to serve as examples that I think anyone could get started writing some tests for his package. Writing those tests really pays off.
As far as the tclwebtest tests go, take a look at the files with the .test extension in the .LRN setup scripts (obtain with cvs -d :pserver:mailto:email@example.com:/cvsroot checkout dotlrn-install). You can run those tests like this:
tclwebtest -config_file path_to_server_config_file test_file.test
What we need is more tests for the various application packages, such as forums. For example tests creating forums and forum postings and different settings for forums and logging in with different users etc. A lot of the ground work for doing those tests is already in the .LRN tests in the form of a Tcl API in the file tclwebtest-functions.tcl (such as logging in users, setting APM parameters etc.).
A page summarizing all the testing tools that we have at our disposal for OpenACS would be nice. Maybe Joel can help us fit this into the documentation?
I would be happy to answer more questions on this topic.
Andrew: Thank you also for the reference. Important feature.
Peter: I have just started to look at the dotlrn installation scripts. I have an old copy so I'll update before I move too far forward. Thanks for pointing out the automation procs. I'll also start looking through the source tree for automated test definitions.
Here's a short blurb describing what I've determined since I began this thread last night.
The following utilities and features are
helpful when automating the tasks of installation, configuration,
development, and testing of an OACS system or package. These tasks are
repetitively performed hundreds of times while developing and testing
OACS packages and subsystems. While the tools described below may be
used for other purposes, their extreme value can be realized by
developers as they develop consistent, quality, error-free extensions
and modifications of an OACS system. Automating the error-prone, time
consuming, boring tasks of re-installation and initial configuration of
a system allows the developer to focus on the creative, non-repetitive
tasks required in the development process. This automation along with
the provision of an automated package TCL API tester *will* result in
higher quality and lower unit development cost. In addition, a
well-defined set of TCL API test instructions will result in lower
maintenance cost when unit modifications or extensions are necessary. Lastly (and most importantly), the existence of a package TCL API test suite will describe
the intended usage of a package's TCL API!
- The standalone tclwebtest tool is useful in manipulating the system at the HTML, or web UI, level. As such, it can be used both to test and configure the system. tclwebtest will retrieve pages, search for text in the response, fill-in and submit forms, and manipulate client cookies. Since this tool is written in TCL, the coder can utilize the power of this scripting language in his own test scripts.
- The acs-automated-testing package is useful in manipulating the system at the TCL API level. As such, it can be used in regression testing of packages and subsystems.
- The internal install.xml feature is useful when installing
a custom system. It defines application(s?) and describes settings and
actions associated with the application. This feature is an extension of
the OACS installation bootstrapping process. Its customization actions
are performed upon completion of OACS core installation, but prior to
the completion of the installation process.
Install.xml: Customized InstallationThe following discussion describes actions performed in the acs-bootstrap-installer located in the /packages/acs-bootstrap-installer/installer directory. Unless otherwise noted, files discussed below are located relative to this directory. The install.xml file, located in the system's pageroot direcory, is processed in file index.tcl which places elements in an nsv named acs_application. These elements contained in this nsv are later processed in packages-install.tcl just after the core packages are installed with a call to apm_mount_core_packages. If the install.xml file is sucessfully parsed, a message is printed to the browser indicating that "The installer will automatically install the [specified] application after the basic OpenACS tookit has been installed"
The topmost install.xml element is application and, even though it looks possible that multiple application nodes may be presented in the install.xml file, only the first application node is processed. Also note that comments, delimited by the standard html comment markers (<!-- and -->), are allowed after the <?xml version> element and before the <application> element.
Analysis of packages-install.tcl indicates that the following xml structure is valid within the install.xml file:
<application (although multiple application elements may exist, only first is processed)
name= Used in displayed message during install (ex. "dotlrn")
pretty-name= Used in displayed message during install (ex. ".LRN")
home= Used in displayed message during install (ex. "https://openacs.org/projects/dotlrn")
<actions> (only a single actions element is allowed)
<text (all content within this element is ignored) />
package= A glob pattern, expanded at /packages/ (ex. "*" installs all packages)
<mount (instantiates and mounts a package in site map at /mount-point)
package= Package_key of target package (ex. "dotlrn")
mount-point= Site node name. OK if node exists w/o package mounted (ex. "dotlrn")
instance-name= Site node instance name (ex. "dotLRN")
<set-parameter (sets either a URL OR a package parameter)
<url= A valid mounted site map package instance (ex. "/")
name= A valid parameter for specified site node (ex. "DefaultMaster")
value= A value for specified parameter (ex. "/packages/dotlrn/www/dotlrn-master")
<package= A valid package_key (ex. "acs-kernel")
name= A valid parameter for specified package (ex. "IndexRedirectURL")
value= A value for specified parameter (ex. "/dotlrn/index")
In the description above (actually based on the dotLRN application install), the dotLRN application is installed and configured by:
- All packages in the /packages/ directory are installed (<install package="*"/>)
- The dotlrn package is instantiated and mounted in the site map at /dotlrn (<mount package="dotlrn" mount-point="dotlrn" instance-name="dotLRN"/>)
- The site DefaultMaster template is set to a the dotlrn-specific template file (<set-parameter url="/" name="DefaultMaster" value="/packages/dotlrn/www/dotlrn-master"/>). Note that the url form of the set-parameter action is used to set a parameter in specific instance of acs-subsite (the main subsite).
- The default site index page is set to /dotlrn/index by (<set-parameter package="acs-kernel" name="IndexRedirectUrl" value="/dotlrn/index"/>)
- The community member root URL is set to /dotlrn/community-member by (<set-parameter package="acs-kernel" name="CommunityMemberURL" value="/dotlrn/community-member"/>)
- The community member admin URL is set to /dotlrn/admin/user by
(<set-parameter package="acs-kernel" name="CommunityMemberAdminURL"
I thought I would post a real example of using the install.xml file that I use to ease the pain of re-installing my development environment.
<!-- This install.xml file installs Randy's Development Server Instance
****SAMPLE ACTIONS**** (everything is case-sensitive)
<mount package="acs-subsite" mount-point="nmon" instance-name="nmon"/>
<set-parameter package="some-package" name="parameter-name" value="parameter-value"/>
<set-parameter url="/some/mounted/package/url" name="parameter-name" value="parameter-value"/>
<application name="devel-instance" pretty-name="Randy's Development Instance" home="http://localhost.localdomain/projects/devel">
<mount package="acs-developer-support" mount-point="devsup" instance-name="devsup"/>
<set-parameter package="acs-developer-support" name="DatabaseEnabledP" value="1"/>
<set-parameter package="acs-developer-support" name="UserSwitchingEnabledP" value="1"/>
<set-parameter package="acs-developer-support" name="EnabledIPs" value="*"/>
<set-parameter package="acs-developer-support" name="EnabledOnStartupP" value="1"/>
<set-parameter package="acs-developer-support" name="ShowCommentsInlineP" value="1"/>
<mount package="monitoring" mount-point="mon" instance-name="mon"/>
<set-parameter package="monitoring" name="TopOptions" value="-bn1"/>
<set-parameter package="monitoring" name="WatchDogFrequency" value="0"/>
<set-parameter package="monitoring" name="TopLocation" value="/usr/bin/top"/>
<set-parameter package="monitoring" name="TopFrequency" value="5"/>
<mount package="bulk-mail" mount-point="bmail" instance-name="bmail"/>
<mount package="acs-automated-testing" mount-point="test" instance-name="test"/>
<mount package="search" mount-point="search" instance-name="search"/>
<set-parameter url="/search" name="SearchIndexerInterval" value="300"/>
All this detail is really useful. Couple questions.
- Is it possible to install just a few packages, rather than all the packages? I would guess you could do this by using multiple <install> sections.
- Is the <text> section just for comments? Does it display in the browser?
[glob -nocomplain "[acs_root_dir]/packages/[apm_required_attribute_value $action package]/*.info"]
2. The 'text' section doesn't show. If you're thinking that you need to display something, looks like you could put html or text in 'prettyname' and index.tcl will show it. Of course, index.tcl could also be modified to display the text section...
The same test file works fine when I use tclwebtest standalone. I assume I have loaded the various .tcl files incorrectly when OpenACS starts up or some changes need to be made to make all this work well together.
Any ideas how to get this working?
What provides the http::config proc? Does it live in a package /tcl directory, and is it named something-procs.tcl? If not you may have to use the tcl "package require" command although I've never used it. Actually, I've never even looked at it.
If I run the tclwebtest standalone it now runs for http. I am working on https next.
My tester package allows the creation of one or more site monitors, starts (schedules a proc to run) and stops the proc, puts result data in the db, notifies people of results.
I have all but the proc calling the tclwebtest connection and sending messages, which are next. (You have to customize some files as I built it quickly.)
If you want a copy when I am done let me know.
Dave McBride (mailto:firstname.lastname@example.org)
installtag in install.xml globs for files and you can have multiple instances of the tag. The "*" example was probably copied from .LRN, where the release tarball only includes the packages the application needs, which means that installing all of them is the right thing to do.
It is true that currently only one
application tag is processed. My notion when I wrote this was that we would later extend the installer to read the XML file, and offer a select widget on the initial OpenACS install page if more than one application has been defined in install.xml.
But I've been putting that off until we actually *have* more than one vertical application to distribute :)
Dave: am I understanding correctly that you're using the sh/tclsh front-end to Til's tclwebtest inside of openacs somehow? How is it controlled? Do you have it hooked to run once on site install? Or, is there some way to define multiple command/response (test) files and run tclwebtest within openacs from a web admin UI?
The http.tcl lib (on my RH9 system) is located in:
I am using tclwebtest via a unix command call at the moment. I catch the result, and store it with a date stamp. I would like to use it just by calling run_test, but that is not working yet. It fails when it hits the http::config call. Thanks for letting me know of the location of the http package.
I created a UI to create, update, delete a site monitor. It can add, update, delete people to notify, specify the test file to run, and you can set the interval in seconds. It can be started and stopped by an admin user, it is controlled by ad_schedule_proc, and a user can view the on-going results.
It re-starts all active monitors by scheduling the procs upon restart of the aol server. It is subsite and multi-instance aware. No monitors are set up when OpenACS is first installed.
I will provide examples of test files. This is great for testing any website, verifying uptime for your ASP site, etc., and proving your site is up. You can re-use the same testfile for different intervals to notify other people less often for instance.
I have all this working using the unix shell tclwebtest for http sites. It currently hangs when I try https sites. I would prefer to call run_test directly.
your scripts sound interesting. We checked in the OpenACS and .LRN setup scripts yesterday, see
The scripts underwent some changes by Joel yesterday and we will debug them today. Maybe you could check in your package to /contrib?
Concerning the install.xml file I think we should remove the install=* instruction and instead make sure the dependencies in the dotlrn info file are up-to-date.
By default this will create a user called service0 and an openacs server called service0 in /var/lib/aolserver/, and start and install the site at http://localhost:8000.
cvs -z3 -d :pserver:email@example.com:/cvsroot co -d install openacs-4/etc/install sh install/install.sh
There are probably lots of bugs and I left many TODOs in the code (which is Peter's work and my butchery), but I know enough other people are interested that I thought it better to get this out and get help and feedback on the method than to wait while we get around to polishing.
Known bugs: ignores command-line-specified config file; creates an orphan daemontools process.
As I was looking at (and using) tclwebtest over the past few weeks, I came across a warning that tclwebtest doesn't work on non-port-80 connections. Don't know if that's still true or not.
The deamontools orphan is probably caused by moving the service directory before forcing supervise to totally relenquish control. I analyzed this same issue when I tested and modified the older version of the automatic .lrn install scripts. I solved the issue I identified as follows (copied from another forum article). Note that my setup uses supervise and multilog to manage the error log also. Just disregard the log stuff if that's not the case in your setup.
- remove the service's symlink from /service (svscan no longer restarts supervise)
- svc -xd /home/servers/service0/log ('log' supervise exits and service stops)
- svc -xd /home/servers/service0 ('aolserver' supervise exits and service stops)
- recreate the service's symlink in /service (svscan retarts supervise processes)
- 'svscan' should now automagically restart everything
I call tclwebtest externally, which is probably good in that the package uses tclwebtest without any alterations. I put the https part (tls1.4) for tclwebtest into the tclwebtest-0.3/lib directory.
The best thing to do is try tclwebtest from the command line with the test of choice first. If the test file can run, it should run from inside the tester package. I did notice any sourced files could not be found, so the test file has have all the source you will use to run the test. I did not use the full path when I sourced include files though.
I have to create a package parameter for the tclwebtest path before I finish the package development. And I have to port it to Postgresql, write those xql files and the such.
The test files are put in [acs_root_dir]/www/test_files so the package can be host node ready. I created an examples directory for a few test files that worked for me.
Anyone who wants it, let me know.
thanks for posting this solution to the orphaned supervise problem. The scripts currently solve the problem the brutal way with kill -9 of the supervise process, but your approach seems much more kosher. The scripts should be updated.