Forum OpenACS Q&A: How do I speedup tclwebtest ?

Hi,

In general I find tclwebtest a nice tool, however sometimes it has a problem with speed, or can it be that I use it the wrong way ?

Example: I use tclwebtest to create an assessment. The assessment package displays some quite large and complex forms.
I use the tclwebtest command form find ~n {xxx} to locate the form, and then process the elements with field fill yy, field select -index zz etc.

The problem is that the form find ~n {xxx} command takes about 70 seconds to return for the page where I create the assessment. It seems to be the case, that a complex form with e.g. many hidden fields increases the time. The creation of an assessment with one section and 10 questions of different types takes about 30 minutes.
As far as I have tested the other tclwebtest commands do not have this problem.

I tried to find the problem in the tclwebtest library, and I think I have traced it to the procs extract_forms_if_necessary and deal_with_field (The latter is called by the first one.)
I am not 100% sure, but I suspect some extensive use of regexp in these procs to cause the loss of performance.

Is this a known problem, and does anyone know a workaround ?
Does tcl and regexp have a performance issue ?

Thanks.

Åsmund

Thanks for the tips Dave.

I kind of like tclwebtest, because it integrates well into the openacs framework, and it run on the serverside. But the thing about performance is annoying ..

Åsmund

Yes, maybe it has to to with timeouts, but I don't think so.
I have tried to look into this, and it seems to me that all the loading of data happens either from the do_request or link follow method.

The form find simply parse the received information.

I don't know if tclwebtest runs http 1.0 or 1.1, and as far as I know you can't configure this either.

Sometimes the do_request and link follow methods takes a long time to (but not every time) and this could probably have something to do with timeouts etc.

I think the form find method use so longe time because of several nested regexp. Some of them use up to 1 or 2 seconds to complete but most of them are quite quick. But many small in total add up to a quite long time in the end.

The problem turned out to be a few inefficient loops with regular expressions, which seem to have at least a quadratic behavior depending on the size of the analyzed web-page. This was especially bad on assessment.

i made a couple of changes to tclwebtest, which still passes the regression test and is on its regression test more than 6 times faster on user time:

before: 4.647u 0.310s 0:05.40 91.6% 0+0k 0+0io 0pf+0w
after: 0.696u 0.230s 0:01.15 80.0% 0+0k 0+4io 0pf+0w

The new version is on a few examples from assessment more than 60 (!) times faster (wall clock time) than before. One can get the version form here:

http://media.wu-wien.ac.at/download/tclwebtest.tcl

I have sent a separate mail to tilman singer, who is the main developer of tclwebtest. hopefully, he can incoportate these changes in future versions.

Åsmund has written a very nice recorder for tclwebtest as a firefox plugin (XUL application). the recorder allows to create tclwebtest files from a user session by clicking on links and filling out forms etc. The tclwebtest files can be nicely run in threads with openacs, which is nice for regression testing (no issues with missing ad_conn infos). If there is enough interest, we might be able to make this available.

Åsmund has written the recorder as a byproduct of his bachelor work, since writing long tclwebtest scripts was to boring. It will need some cleanup and beautification, so it will take a little bit more time until it is released.

yesterday, we were discussion the dependency on localization. Currently, the approach will only work for one locale. There are two problem areas:

• detecting page failure: When use site specific "pretty" return error pages (e.g. for 404 not found), the http status code will be 200 (try the error message on openacs.org)

• dynamic links: when new items are created, it is much easier to find these depending on the link label than on the ids, which will change. In some cases, the links will be fine.

The first problem could be solved by either by changing/programming around this behavior of the aolserver (see discussions on the aolserver list) or by adding hidden fields to the return pages. Currently the recorder checks the title of the page, which works pretty well but is locale dependent. The second problem is more serious. Introducing IDs for every anchor etc. does not seem realistic.

Do the tclwebtester or selenium testers have some thoughts / best practice experiences about this?