Forum OpenACS Development: Xowiki Regression Test

Collapse
Posted by Dave Bauer on
Hi. I recently upgraded to xowiki 0.50 with xotcl-core 0.48.

I tried running xowiki/admin/regression_test.tcl and got the following error

During this test:

Create and Render Index Page
OK: object name parsed - passed (0 ms)
OK: method passed from package initialize - passed (1 ms)
ERROR: index page resolved returned '0' ne '1'
ERROR: we have a page object returned '0' ne '1'

invalid command name "::"
while executing
"::$page_item_id item_id"
("uplevel" body line 1)
(procedure "?" line 1)
invoked from within
"? {expr {[::$page_item_id item_id]>0}} 1 "item_id given""
("uplevel" body line 126)
invoked from within
"uplevel {

If i run test.tcl I get this

Check Permissions based on default policy
OK: user_id is guest - passed (0 ms)
OK: the public cannot delete this page - passed (0 ms)
OK: the public cannot admin this package - passed (1 ms)
New Query: /XOWIKI-TEST/en/index

during '::xo::cc m view'
::xo::cc ::xotcl::Object->configure
::xo::ConnectionContext ::xotcl::Class->create
invoked from within
"my create ::xo::cc -package_id $package_id -parameter_declaration $parameter -user_id $user_id -actual_query $actual_query -url $url"
(procedure "require" line 16)
::xo::ConnectionContext->require
invoked from within
"ConnectionContext require -package_id $package_id -user_id $user_id -parameter $parameter -url $url -actual_query $actual_query"
(procedure "initialize" line 14)
::xowiki::Package ::xo::PackageMgr->initialize
invoked from within
"::xowiki::Package initialize -parameter $index_vuh_parms -package_id $info(package_id) -url /$instance_name/en/index -actual_query "" -user_id 0"
("uplevel" body line 152)
invoked from within
"uplevel {

Do we need both of those files? If one more up to date than the other?

Collapse
2: Re: Xowiki Regression Test (response to 1)
Posted by Gustaf Neumann on
no. just test.tcl.... i have updated the tests today, they run fine now on my installations.
Collapse
3: Re: Xowiki Regression Test (response to 1)
Posted by Dave Bauer on
Hmmm.

I doubled checked and I have the most up to date test.tcl.

I restarted aolserver also and it did not help I still get this error:

"ConnectionContext require -package_id $package_id -user_id $user_id -paramete
r $parameter -url $url -actual_query $actual_query"
(procedure "initialize" line 14)
::xowiki::Package ::xo::PackageMgr->initialize
invoked from within
"::xowiki::Package initialize -parameter $index_vuh_parms -package_id $info(pac
kage_id) -url /$instance_name/en/index -actual_query "" -user_id 0"
("uplevel" body line 152)
invoked from within
"uplevel {
# regression test for xowiki

My previous install of xowiki seems to work OK, its just not passing the tests. I am not sure if this is a problem that will come up later so I would like to figure out what's causing the test to fail.

Collapse
4: Re: Xowiki Regression Test (response to 3)
Posted by Gustaf Neumann on
Dave, I have provided a fix for this in CVS head. I could reproduce the behavior you reported above with the released version of XOTcl. It was a strange bug with argument passing and shimmering, most only reproducible with the regression test.

While testing, i found another strange problem with the site-nodes. There seem to be cases in the version of openacs in cvs head, where "site_node::exists_p" returns false, but the entry exists in the the site_note table. The new version of the regression test tries to fix such cases as well...

Collapse
5: Re: Xowiki Regression Test (response to 4)
Posted by Dave Bauer on
Thank you Gustaf. It works perfectly. As always you assisstance is appreciated.
Collapse
6: Re: Xowiki Regression Test (response to 1)
Posted by Don Baccus on
So why does xowiki have its own unique way of doing regression testing rather than use the existing automated test system?

We just finally got consensus on trying to get selenium testing integrated with existing testing rather than have a brand-new package ...

Collapse
7: Re: Xowiki Regression Test (response to 6)
Posted by Gustaf Neumann on
don,

I did not have time yet to integrate my tests with automated testing. The "new framework" is not intended for the public, is about one page of code, and is much simpler than the automated testing notation. XOTcl (the language) uses the same "framework" since the beginning. It is simply the case that i am much faster developing tests this way. Nobody has to learn it, it is just for me. Read it this way: this file escaped from my environment, it is not documented, dave just saw it by observing the changelog of the cvs commits, he was curious, what it does and run it. Most probably, the test cases (containing the "framework") should be removed from cvs in order to avoid confusions.

Btw., these test cases of xowiki cannot be done via selenium. It seems that i have missed the consensus (most likely my fault), and i do not understand what "integrating" means (Selenium Core?, Selenium IDE?, Selenium RC?, all of these? with automated testing or just by-packing?). In my personal opinion, Selenium IDE is no big win over tclwebtest, since it can easily conflict with javascript based GUIs, ajax etc., which are becoming more important these days.

Collapse
8: Re: Xowiki Regression Test (response to 1)
Posted by Dave Bauer on
Gustaf,

I mostly agree that to test functionality, tclwebtest is the way to go and easier to use with acs-automatd testing.

You can actually enter text into the richtext textareas with tclwebtest. Of course you can't test an javascript features with it.

Is is easier to train clients to record their own tests with selenium-ide.

The intergration with selenium is the idea of one admin ui to run and see test results. Hamilton wrote a little code to capture selenium test results and we will try to figure out a way to make this work within the acs-automated testing framework.

Collapse
Posted by Don Baccus on
Selenium and tclwebtest are not mutually exclusive, as dave's pointing out above.

"integration" in this context mostly means providing a way to access selenium test from the automated test package, so those involved in evaluating code for release can go to one place to review test results. This will also be convenient when we (hopefully) resurrect the nightly build/test cycle we had implemented about three years ago.

Rolling your own is almost always going to be easier than doing things within the toolkit framework. Go for it. As long as you don't mind your code not being accepted into the toolkit as being part of a standard release of, oh, say, .LRN for instance.

Collapse
Posted by Gustaf Neumann on
don baccus wrote: "Go for it. as log as you don't mind..."

why are you poking around on that? Some people and users are happily using xowiki with its portlet in dotlrn even without xowiki having "standard regression tests". It's the task of the dotlrn consortium to decide what to include in dotlrn, and if this decision boils down to the fact, whether or not the regression test is part of automated testing, i am sure some institution or person - maybe even myself - will invest the resources (two hours?) to integrate it. why making live more complicated as it is?