Forum OpenACS Q&A: OpenACS testing Framework

Collapse
Posted by Carl Coryell-Martin on
Here at Civilution, we are dipping our toes into the Extreme Programming discipline. It seems very appropriate for the kinds of sites that we like to do.

One of the fundamental tenents of XP is to have unit tests and acceptance tests cover all the desired functionality. I am currently thinking about clever ways of unit testing openACS flavored pages, and I wondered if anyone would like to share things that work for them.

Write now I am thinking about some tools that help me test tcl procs, tools that use the simple templating system and ad_page_contract to test the inputs and outputs of pages.

Cheers,

Collapse
Posted by Don Baccus on
Coverage testing is a fundamental tenant of software engineering, whether or not one uses a current buzzword :)

Testing, automated and manual, is a sore point with the ACS in both Classic and OpenACS form.  Ideally, aD - the company with dozens of software engineers who are actually PAID to work on the toolkit - would do adequate testing and also provide tools to help automate those bits of testing that can be automated (automated testing of web sites is far more difficult than for many classes of software, there are other threads on the issue here and at web/db).  We could then depend on our ACS Classic base being solid and concentrate on ferreting out porting bugs.  As it is, each of our release has included many bug fixes to the basic toolkit, which slows us down.  The fact that we find and fix so many ACS Classic bugs while doing our  ports is an indication to me that our core porting team believes that  testing is important.

aD has made a lot of noise over their belated recognition that testing  is important, and have made some steps to do more of it, but they still release broken software.  Of course, all software's broken by definition, but ACS Classic is too frequently released with brokeness that even simple testing would uncover.

If you or someone else wants to step up to the plate and organize and improve testing of the OpenACS toolkit, above and beyond what the core  porting team does while porting and users while using, you'd be deemed a hero.  I'm serious - testing is far too often treated as a second-class activity, and testers as second-class contributors, but I don't think you'd find that attitude prevalent here.

Especially given the fact that we've all had lots of frustrating experience over the past year or so with untested, flakey software (the various ACS Classic releases).

Collapse
Posted by Lars Pind on
Take a look at the ACS Test Harness package for 4.0, which you'll find at http://www.arsdigita.com/acs-repository/one-package?package_id=42. It's a framework intended to attack exactly this problem.

Only problem is that I haven't had the time to finish it. (I know, it's ironic and sad.) But if anyone in the community feels like taking ownership of the package, it would benefit both OpenACS and ACS Classic.

Collapse
Posted by Michael Feldstein on
One idea we've discussed here before is to embed a bug-report link into every ACS-backed page. Ideally, it would be group-scoped and subsite-aware so you could turn it on only for the right people on the right pages. In an ideal world, the bug report feature would automatically log who submitted the bug, when, what page they were on, and what page they just came from, in addition to having fields that the users could fill in to describe the bug.

This, obviously, would be a compliment to an automated testing scheme, not a replacement for one. But if you tied it into the SDM and the workflow manager in 4.0, you could greatly increase the chances that a given site is properly and thoroughly reviewed before launch.