Forum OpenACS Development: Re: tclwebtest with openacs-4/etc/install tests -- help getting started

hi, I'm trying on tclwebtest for acceptance tests as well, and like you I like to run it outside of aolserver.
I got the server parameters in this (really ugly) way:

# determine the server address
# open the config file to determine the server parameters
set file [open "../../../etc/config.tcl" r]
while {[gets $file line]!=-1} {
    if [regexp {set address} $line] {
        eval $line
        # address is set
    }
    if [regexp {set httpport} $line] {
        eval $line
        # httpport is set
    }
    if [regexp {set port} $line] {
        eval $line
        # port is set
    }
    if [regexp {set server} $line] {
        eval $line
        # server is set
    }
}
if {![info exists address] || ![info exists httpport]} {
  if {[info exists server] && [info exists address($server)] && [info exists port($server)]} {
      set server "http://[set address($server)]:[set port($server)]"
  } else {
      error "Error parsing config file!"
  }
} else {
    set server "http://${address}:${httpport}"
}

And I did not put my files into automated-testing, but in packages/<my-package>/tests/ instead so it is bundled with the package it's meant for. Surely this will have to move once there is a real testing framework in place for the whole of oacs.
As for the contents of the testsuite I just kind of "click around" through the pages, seeing that I cover all the pages, all the functionalities, some/most of the error messages on wring input etc.
The main concern I have which made me using this is breaking some code somewhere without noticing it, like you change a proc somewhere or mass update something in 20 files and in 1 page it breaks for some obscure reason. I don't want the client to find that out.

Hi Kolja,

Thanks for sharing your setup + use of the new testing stuff. My setup and use right now pretty closely resembles yours. My only new finding is that I can install tclwebtest to /usr/local/src and just write a short shell script in my packages/my-package/tests dir that runs all my tests by first cd'ing to /usr/local/src/tclwebtest. I like this better since I don't have to muddy by OACS source tree with tclwebtest source.


####  Set these to your settings.

set tclwebtestDir = "/usr/local/tclwebtest"

####  Stop here.

set testsDir = `pwd`
echo "Running tests from $testsDir"

# Get in tclwebtest dir to run tests.
cd $tclwebtestDir

tclwebtest -config_file ${testsDir}/nsd-config.tcl ${testsDir}/inbox-procs.test
Right now I only have the one test file but I can obviously loop here later and run all tests for this package from this shell script.

Then I added the full path to the new OACS /etc/install test suite to my local *.test scripts so I can re-use all the OACS testing procs.

    # Set full path to OACS tclwebtest harness scripts.
    set oacs_script_dir {/usr/local/src/oacs/OpenACS/openacs-4/etc/install/tcl}

    # Source OACS tclwebtest harness scripts.
    # We use their ::twt::* procs.
    source ${oacs_script_dir}/test-procs.tcl

So it is working nicely for me right now. I know as you say Surely this will have to move once there is a real testing framework in place for the whole of oacs.. So I am trying to keep good comments with this stuff in my CVS repository!

One thing that I noticed is that you said I did not put my files into automated-testing and that may be something I am doing wrong. Do the unit tests (for acs-automated-testing) all go in packages/acs-automated-testing/tcl/test or do they go in packages/my-package/tcl/test on a per package basis? This might explain why I have to watch my unit test tcl files in the APM to get them loaded in the acs-automated-testing package UI ! Glad you mentioned that.

Also I am thinking of moving tclwebtest tests to a new directory under server_root/etc/integration-tests -- at the level of the tclwebtest install suite -- instead of in per package subdirectories.