Forum OpenACS Development: Automated Install, Configuration, and Test

o acs-automated-testing
o install.xml
o tclwebtest

I'm trying to wrap my head around automated installation, configuration, and testing but I'm not finding much (any) usable doc for acs-automated-testing or install.xml capabilities. I'm also looking at the dotlrn autoinstall scripts that utilize tclwebtest to perform after-install UI-based actions. This is pretty sparse too.

Is anybody out there interested in moving these issues forward? I see several posts indicating that we should use these utilities/features but I've had no response to the several posts I've made asking for information, pointers, examples, etc.

I assume that someone(s) have analyzed and used these features and must have some notes, examples, code or cheatsheets on implementation.

So, I'll ask once again if anyone wishes to share? If not, I'll start at square one by digging through the source.

Thanks in advance.

Randy

Collapse
Posted by Vinod Kurup on
Great questions, Randy. I'm interested in any responses as well.

This post describes the install.xml file - it's a way to automatically install packages and set parameters at bootstrap time, rather than going through the tedium of the package-manager.

tclwebtest is a HTTP testing tool written by Til and hosted at sourceforge. The documentation is pretty good and it was pretty easy to get tclwebtest to automate an OpenACS bootstrap (basically: load the index page and click 'Next' a couple times).

I tried at one point to get my head around acs-automated-testing and failed. I couldn't figure out what I was supposed to do with it. I also couldn't figure out how I was supposed to have tclwebtest interact with acs-automated-testing.

I'm sure you knew all this info already, so hopefully others will chime in. And if no one does, I'd be happy to cheerlead you as you dive through the source. :-)

Collapse
Posted by Andrew Piskorski on
I don't know. One thing I can add though, is that tclwebtest implements handling for client side cookies, which I found quite useful.
Collapse
Posted by Peter Marklund on
Randy,
I wrote the tclwebtests for installing OpenACS and .LRN from scratch and setting up .LRN demo data. We are going to be using the OpenACS parts of these scripts for daily creation of integration servers during the External Authentication project we are doing right now at Collaboraid. Joel Aufrecht has been asking me to improve the documentation for the scripts and I hope I'll get around to that. An extension we'd like to make is to get the scripts to run all acs-automated-testing tests on the server as well.

The acs-automated-testing package is for Tcl API tests (similar to JUnit or XUnit for those who are familiar with those). We need more of those tests. The tests reside in a package in the dir tcl/test and a test is defined by invoking aa_register_case. There are enough cases in the OpenACS codebase now to serve as examples that I think anyone could get started writing some tests for his package. Writing those tests really pays off.

As far as the tclwebtest tests go, take a look at the files with the .test extension in the .LRN setup scripts (obtain with cvs -d :pserver:mailto:anonymous@dotlrn.collaboraid.net:/cvsroot checkout dotlrn-install). You can run those tests like this:

tclwebtest -config_file path_to_server_config_file test_file.test

What we need is more tests for the various application packages, such as forums. For example tests creating forums and forum postings and different settings for forums and logging in with different users etc. A lot of the ground work for doing those tests is already in the .LRN tests in the form of a Tcl API in the file tclwebtest-functions.tcl (such as logging in users, setting APM parameters etc.).

A page summarizing all the testing tools that we have at our disposal for OpenACS would be nice. Maybe Joel can help us fit this into the documentation?

I would be happy to answer more questions on this topic.

Collapse
Posted by Randy O'Meara on
Thank you, Vinod. I was aware of the references you listed, and I appreciate your time and offer to cheerlead. I would sure like to see *everyone* post their own notes here. I'm certain that, if someone has used the tools and features, their input here would add value here.

Andrew: Thank you also for the reference. Important feature.

Peter: I have just started to look at the dotlrn installation scripts. I have an old copy so I'll update before I move too far forward. Thanks for pointing out the automation procs. I'll also start looking through the source tree for automated test definitions.

Here's a short blurb describing what I've determined since I began this thread last night.

_____________________________________________________________

The following utilities and features are helpful when automating the tasks of installation, configuration, development, and testing of an OACS system or package. These tasks are repetitively performed hundreds of times while developing and testing OACS packages and subsystems. While the tools described below may be used for other purposes, their extreme value can be realized by developers as they develop consistent, quality, error-free extensions and modifications of an OACS system. Automating the error-prone, time consuming, boring tasks of re-installation and initial configuration of a system allows the developer to focus on the creative, non-repetitive tasks required in the development process. This automation along with the provision of an automated package TCL API tester *will* result in higher quality and lower unit development cost. In addition, a well-defined set of TCL API test instructions will result in lower maintenance cost when unit modifications or extensions are necessary. Lastly (and most importantly), the existence of a package TCL API test suite will describe the intended usage of a package's TCL API!

  • The standalone tclwebtest tool is useful in manipulating the system at the HTML, or web UI, level. As such, it can be used both to test and configure the system. tclwebtest will retrieve pages, search for text in the response, fill-in and submit forms, and manipulate client cookies. Since this tool is written in TCL, the coder can utilize the power of this scripting language in his own test scripts.
  • The acs-automated-testing package is useful in manipulating the system at the TCL API level. As such, it can be used in regression testing of packages and subsystems.
  • The internal install.xml feature is useful when installing a custom system. It defines application(s?) and describes settings and actions associated with the application. This feature is an extension of the OACS installation bootstrapping process. Its customization actions are performed upon completion of OACS core installation, but prior to the completion of the installation process.
Collapse
Posted by Randy O'Meara on
Here's another installment that describes the install.xml feature

Install.xml: Customized Installation

The following discussion describes actions performed in the acs-bootstrap-installer located in the /packages/acs-bootstrap-installer/installer directory. Unless otherwise noted, files discussed below are located relative to this directory. The install.xml file, located in the system's pageroot direcory, is processed in file index.tcl which places elements in an nsv named acs_application. These elements contained in this nsv are later processed in packages-install.tcl just after the core packages are installed with a call to apm_mount_core_packages. If the install.xml file is sucessfully parsed, a message is printed to the browser indicating that "The installer will automatically install the [specified] application after the basic OpenACS tookit has been installed"

The topmost install.xml element is application and, even though it looks possible that multiple application nodes may be presented in the install.xml file, only the first application node is processed. Also note that comments, delimited by the standard html comment markers (<!-- and -->), are allowed after the <?xml version> element and before the <application> element.

Analysis of packages-install.tcl  indicates that the following xml structure is valid within the install.xml file:

<application (although multiple application elements may exist, only first is processed)
        name= Used in displayed message during install (ex. "dotlrn")
        pretty-name= Used in displayed message during install (ex. ".LRN")
        home= Used in displayed message during install (ex. "https://openacs.org/projects/dotlrn")
>
        <actions> (only a single actions element is allowed)
                <text (all content within this element is ignored) />
                <install
                        package= A glob pattern, expanded at /packages/ (ex. "*" installs all packages)
                />
                <mount (instantiates and mounts a package in site map at /mount-point)
                        package= Package_key of target package (ex. "dotlrn")
                        mount-point= Site node name. OK if node exists w/o package mounted (ex. "dotlrn")
                        instance-name= Site node instance name (ex. "dotLRN")
                />
                <set-parameter (sets either a URL OR a package parameter)
                        <url= A valid mounted site map package instance (ex. "/")
                                name= A valid parameter for specified site node (ex. "DefaultMaster")
                                value= A value for specified parameter (ex. "/packages/dotlrn/www/dotlrn-master")
                        <package= A valid package_key (ex. "acs-kernel")
                                name= A valid parameter for specified package (ex. "IndexRedirectURL")
                                value= A value for specified parameter (ex. "/dotlrn/index")
                />
        </actions>
</application>

In the description above (actually based on the dotLRN application install), the dotLRN application is installed and configured by:
  1. All packages in the /packages/ directory are installed (<install package="*"/>)
  2. The dotlrn package is instantiated and mounted in the site map at /dotlrn (<mount package="dotlrn" mount-point="dotlrn" instance-name="dotLRN"/>)
  3. The site DefaultMaster template is set to a the dotlrn-specific template file (<set-parameter url="/" name="DefaultMaster" value="/packages/dotlrn/www/dotlrn-master"/>). Note that the url form of the set-parameter action is used to set a parameter in specific instance of acs-subsite (the main subsite).
  4. The default site index page is set to /dotlrn/index by (<set-parameter package="acs-kernel" name="IndexRedirectUrl" value="/dotlrn/index"/>)
  5. The community member root URL is set to /dotlrn/community-member by (<set-parameter package="acs-kernel" name="CommunityMemberURL" value="/dotlrn/community-member"/>)
  6. The community member admin URL is set to /dotlrn/admin/user by (<set-parameter package="acs-kernel" name="CommunityMemberAdminURL" value="/dotlrn/admin/user"/>)
Collapse
Posted by Randy O'Meara on
oops. One slight correction: the install.xml file resides in the installroot, not the pageroot.
Collapse
Posted by Randy O'Meara on
I don't know how many are using the install.xml feature of oacs, but maybe more will if they see a real example.

I thought I would post a real example of using the install.xml file that I use to ease the pain of re-installing my development environment.

<?xml version="1.0"?>

<!-- This install.xml file installs Randy's Development Server Instance
-->

<!--
    ****SAMPLE ACTIONS**** (everything is case-sensitive)

    <install package="package-to-instantiate-and-mount"/>
    <mount package="acs-subsite" mount-point="nmon" instance-name="nmon"/>
    <set-parameter package="some-package" name="parameter-name" value="parameter-value"/>
    <set-parameter url="/some/mounted/package/url" name="parameter-name" value="parameter-value"/>
-->

<application name="devel-instance" pretty-name="Randy's Development Instance" home="http://localhost.localdomain/projects/devel">

  <actions>

    <install package="*"/>

    <mount package="acs-developer-support" mount-point="devsup" instance-name="devsup"/>
    <set-parameter package="acs-developer-support" name="DatabaseEnabledP" value="1"/>
    <set-parameter package="acs-developer-support" name="UserSwitchingEnabledP" value="1"/>
    <set-parameter package="acs-developer-support" name="EnabledIPs" value="*"/>
    <set-parameter package="acs-developer-support" name="EnabledOnStartupP" value="1"/>
    <set-parameter package="acs-developer-support" name="ShowCommentsInlineP" value="1"/>

    <mount package="monitoring" mount-point="mon" instance-name="mon"/>
    <set-parameter package="monitoring" name="TopOptions" value="-bn1"/>
    <set-parameter package="monitoring" name="WatchDogFrequency" value="0"/>
    <set-parameter package="monitoring" name="TopLocation" value="/usr/bin/top"/>
    <set-parameter package="monitoring" name="TopFrequency" value="5"/>

    <mount package="bulk-mail" mount-point="bmail" instance-name="bmail"/>

    <mount package="acs-automated-testing" mount-point="test" instance-name="test"/>

    <mount package="search" mount-point="search" instance-name="search"/>
    <set-parameter url="/search" name="SearchIndexerInterval" value="300"/>

  </actions>

</application>

Collapse
Posted by Vinod Kurup on
Randy,

All this detail is really useful. Couple questions.

  1. Is it possible to install just a few packages, rather than all the packages? I would guess you could do this by using multiple <install> sections.
  2. Is the <text> section just for comments? Does it display in the browser?
Thanks again
Collapse
Posted by Randy O'Meara on
1. You can mix-n-match... multiple 'action's are allowed, and the value is globbed.
  [glob -nocomplain "[acs_root_dir]/packages/[apm_required_attribute_value $action package]/*.info"]

2. The 'text' section doesn't show. If you're thinking that you need to display something, looks like you could put html or text in 'prettyname' and index.tcl will show it. Of course, index.tcl could also be modified to display the text section...

Randy

Collapse
Posted by Dave McBride on
I have all of the functionality working for the site tester except the full running of the test file. It fails to find the proc 'http::config' when a do_request is executed inside a tclwebtest file.

The same test file works fine when I use tclwebtest standalone. I assume I have loaded the various .tcl files incorrectly when OpenACS starts up or some changes need to be made to make all this work well together.

Any ideas how to get this working?

Dave

Collapse
Posted by Randy O'Meara on
Dave,

What provides the http::config proc? Does it live in a package /tcl directory, and is it named something-procs.tcl? If not you may have to use the tcl "package require" command although I've never used it. Actually, I've never even looked at it.

Randy

Collapse
Posted by Dave McBride on
It seems that the require statement is in the tclwebtest shell script. The tclwebtest starts out as a shell and is converted into a tcl shell. If I include only the tcl code from it, the require statement fails to find the http package upon loading. I looked around and could not find the http package.

If I run the tclwebtest standalone it now runs for http. I am working on https next.

Dave

Collapse
Posted by Dave McBride on
I am currently creating a automated test harness that runs a tclwebtest test file (to see if a site is up and if specific pages are available). I recently created a test suite to test each page for a client and now they want to see if the site is up periodically.

My tester package allows the creation of one or more site monitors, starts (schedules a proc to run) and stops the proc, puts result data in the db, notifies people of results.

I have all but the proc calling the tclwebtest connection and sending messages, which are next. (You have to customize some files as I built it quickly.)

If you want a copy when I am done let me know.

Dave McBride (mailto:dave@crimsoncode.com)

Collapse
Posted by Randy O'Meara on
Sure, Dave. When you're ready I would like to look.
Collapse
Posted by Don Baccus on
The install tag in install.xml globs for files and you can have multiple instances of the tag. The "*" example was probably copied from .LRN, where the release tarball only includes the packages the application needs, which means that installing all of them is the right thing to do.

It is true that currently only one application tag is processed. My notion when I wrote this was that we would later extend the installer to read the XML file, and offer a select widget on the initial OpenACS install page if more than one application has been defined in install.xml.

But I've been putting that off until we actually *have* more than one vertical application to distribute :)

Collapse
Posted by Randy O'Meara on
Thanks for telling us your intent, Don. BTW, thanks for creating this feature. In addition to your original intent, it is really helpful in configuring a newly reinstalled devel site with minimum keystrokes. Something I sometimes do 25 times in one day. This feature along with the .LRN tclwebtest install scripts makes it a 2 minute task.

Dave: am I understanding correctly that you're using the sh/tclsh front-end to Til's tclwebtest inside of openacs somehow? How is it controlled? Do you have it hooked to run once on site install? Or, is there some way to define multiple command/response (test) files and run tclwebtest within openacs from a web admin UI?

The http.tcl lib (on my RH9 system) is located in:

    /usr/share/tcl8.3/http2.4
Collapse
Posted by Dave McBride on
My intent was to create a site monitor that stored the results of the test in the Oracle db. Sort of a site uptime / page verify package, I call it tester. The tclwebtest program allows great flexibility in what can be verified for clients/customers.

I am using tclwebtest via a unix command call at the moment. I catch the result, and store it with a date stamp. I would like to use it just by calling run_test, but that is not working yet. It fails when it hits the http::config call. Thanks for letting me know of the location of the http package.

I created a UI to create, update, delete a site monitor. It can add, update, delete people to notify, specify the test file to run, and you can set the interval in seconds. It can be started and stopped by an admin user, it is controlled by ad_schedule_proc, and a user can view the on-going results.

It re-starts all active monitors by scheduling the procs upon restart of the aol server. It is subsite and multi-instance aware. No monitors are set up when OpenACS is first installed.

I will provide examples of test files. This is great for testing any website, verifying uptime for your ASP site, etc., and proving your site is up. You can re-use the same testfile for different intervals to notify other people less often for instance.

I have all this working using the unix shell tclwebtest for http sites. It currently hangs when I try https sites. I would prefer to call run_test directly.

Dave

Collapse
Posted by Peter Marklund on
Dave,
your scripts sound interesting. We checked in the OpenACS and .LRN setup scripts yesterday, see

http://cvs.openacs.org/cvs/openacs-4/etc/install/

The scripts underwent some changes by Joel yesterday and we will debug them today. Maybe you could check in your package to /contrib?

Concerning the install.xml file I think we should remove the install=* instruction and instead make sure the dependencies in the dotlrn info file are up-to-date.

Collapse
Posted by Joel Aufrecht on
Automatic install script is complete and checked in. It checks out and installs an OpenACS (and optionally dotLRN) OpenACS server on a machine that already has a database, aolserver, and daemontools. The configuration file lets you change the server name, run without daemontools, use dotLRN, and control the paths. To get it,
cvs -z3 -d :pserver:anonymous@openacs.org:/cvsroot co  -d install openacs-4/etc/install
sh install/install.sh
By default this will create a user called service0 and an openacs server called service0 in /var/lib/aolserver/, and start and install the site at http://localhost:8000.

There are probably lots of bugs and I left many TODOs in the code (which is Peter's work and my butchery), but I know enough other people are interested that I thought it better to get this out and get help and feedback on the method than to wait while we get around to polishing.

Known bugs: ignores command-line-specified config file; creates an orphan daemontools process.

Collapse
Posted by Randy O'Meara on
Joel: A comment and a possible fix for the supervise orphan.

As I was looking at (and using) tclwebtest over the past few weeks, I came across a warning that tclwebtest doesn't work on non-port-80 connections. Don't know if that's still true or not.

The deamontools orphan is probably caused by moving the service directory before forcing supervise to totally relenquish control. I analyzed this same issue when I tested and modified the older version of the automatic .lrn install scripts. I solved the issue I identified as follows (copied from another forum article). Note that my setup uses supervise and multilog to manage the error log also. Just disregard the log stuff if that's not the case in your setup.

  • remove the service's symlink from /service (svscan no longer restarts supervise)
  • svc -xd /home/servers/service0/log ('log' supervise exits and service stops)
  • svc -xd /home/servers/service0 ('aolserver' supervise exits and service stops)
  • recreate the service's symlink in /service (svscan retarts supervise processes)
  • 'svscan' should now automagically restart everything
Randy
Collapse
Posted by Dave McBride on
The tester package is fully operational now. The problem must have been in the test file I wrote for https.

I call tclwebtest externally, which is probably good in that the package uses tclwebtest without any alterations. I put the https part (tls1.4) for tclwebtest into the tclwebtest-0.3/lib directory.

The best thing to do is try tclwebtest from the command line with the test of choice first. If the test file can run, it should run from inside the tester package. I did notice any sourced files could not be found, so the test file has have all the source you will use to run the test. I did not use the full path when I sourced include files though.

I have to create a package parameter for the tclwebtest path before I finish the package development. And I have to port it to Postgresql, write those xql files and the such.

The test files are put in [acs_root_dir]/www/test_files so the package can be host node ready.  I created an examples directory for a few test files that worked for me.

Anyone who wants it, let me know.

Dave (mailto:dave@crimsoncode.com)

Collapse
Posted by Peter Marklund on
Randy,
thanks for posting this solution to the orphaned supervise problem. The scripts currently solve the problem the brutal way with kill -9 of the supervise process, but your approach seems much more kosher. The scripts should be updated.
Collapse
Posted by Randy O'Meara on
There is a *very* related thread regarding how to create an acs-automated-test script at https://openacs.org/forums/message-view?message_id=121426
Collapse
Posted by Brian Fenton on
Just bumping this thread cos I found it very useful. Thanks for all your efforts Randy!

Brian