Forum OpenACS Development: Re: tclwebtest with openacs-4/etc/install tests -- help getting started

I think the theory behind unit tests is that each of them tests a single encapsulated functionality and thus should be able to run independently from each other and not require a specific order to run. To set up prerequisites that a unit test depends on there is some functionality in acs-automated-testing as far as I remember.

Client emulation testing naturally requires a much longer series of actions to be performed in a specific order. When you run tclwebtest from the command line it runs the tests of a directory in sorted order, so you could call your tests sth like 00-register.test, 01-login.test etc.

How to deal with that when integrating tclwebtest in acs-automated-testing I don't know - maybe add a sorting facility, or an optional dependency information for each test, or instruct test writers to create long, self-contained tclwebtest tests that don't have any outside dependencies (I wouldn't like that).

Hi, I think what you said about unit testing is great as a concept and would like to work towards it:

I think the theory behind unit tests is that each of them tests a single encapsulated functionality and thus should be able to run independently from each other and not require a specific order to run. To set up prerequisites that a unit test depends on there is some functionality in acs-automated-testing as far as I remember.

But I have a few issues implementing such a test suite!

I want my unit tests to be independent and just based on the object (namespace-mimicked) that they are written for but... am running into some questions of dependencies and valid/useful assumptions to make.

Let me try to explain. Say I write unit tests for all my "objects" in my package. I would still want to be able to be assured that the unit tests for acs-kernel were run prior to my package unit tests being run. Though they are somewhat encapsulated functionality of my package objects, they do rely on at least the kernel being functioning.

Then the next lower level. Within a package, well it's easiest to do with an example so let me pretend I'm writing unit tests for the CR. I would want to have the unit test for the cr_folder complete before I ran my unit test for the cr_item and then the cr_revision. I can't even create a cr_revision without first having a cr_item, which itself requires a cr_folder to exist. And I'd personally, want to see the "right" test fail first: e.g. I would want cr_folder unit test to fail first if I can't create a folder, not the cr_item test -- it'd just be more obvious and hopefully easier to debug.

Also, my unit tests would still want to load data that assumed other packages were loaded and tested first. Like users have to be created to do just about anything worthwhile since acs_objects have creation_users. And I'd want the test users to be there for all my package's unit tests and then get deleted.

So I would want to be able to:

1. Specify package unit test suite execution order.
2. Specify data load/teardown proc per package test suite. (data is available until all unit tests in package suite have completed)
3. Specify object test order within my package test suite.
4. Specify data load/teardown/config proc per combinations of object tests within my package test suite. (think this is what aatest init-classes are for -- though I don't think this works if the unit tests are not in the same tcl file.)

I am not familiar with any commercial test frameworks and I'm just getting starting with the concept of unit testing... so anyone wants to suggest resources or recommended reading (RTFM), fine. Tips or examples how someone has addressed similar issues in another test environment would be swell!

I did just order a test driven development book so there is hope if I'm totally off base here;)