Having tests is only half of the solution. Having a way to distribute the tests, run the tests, and collect the results is the other half. As of OpenACS 5.1 (or possibly 5.2 - we did some of the work on HEAD many months ago and it still isn't released yet), the basics of the second half are in place. You can use any framework you like to write tests, and still run the tests from within OpenACS and distribute them as standard parts of the packages. If your test require tclwebtest, then the test will simply not run on machines without those prerequisites. Writing tclwebtest tests is
documented.
If your test requires other frameworks, then we need to extend OpenACS testing to handle them, but this can be done fairly simply by extending the -libraries flag of
aa_register_case.
So, yes, I do feel that OpenACS needs some real regression testing. I've outlined a path to add it via tclwebtest. I am not at all opposed to other test platforms. If someone wants to add another test platform, they should produce (and I'll help):
- Documentation to install the test package (example)
- Documentation to write tests (example)
- At least one test for each of the core packages
- Extention to aa_register_case for the new package (example from aa_register_case:
# run library specific code
foreach library $libraries {
if { $library == "tclwebtest" } {
# kludge: until tclwebtest installs itself in the proper
# place following the tcl way, we use this absolute path
# hack.
set tclwebtest_absolute_path "/usr/local/tclwebtest/lib"
if { ![info exists ::auto_path] || [lsearch $::auto_path $tclwebtest_absolute_path] == -1 } {
lappend ::auto_path $tclwebtest_absolute_path
}
if { [catch {
package require tclwebtest
package require http
} err] } {
set error_level metatest
append case_error "tclwebtest is not available. Not registering this test case.\n\nError message: $err\n\n"
}
}
}
