Forum .LRN Q&A: Problems with assessments

Hello!

We have still not solved the problems with
assessments. Sniff.

We have applied all the patches in our installation
commented in the forums.

The scene:
One assessment for 40 students with 20 test questions
in one section.
On student submit is nearly 30 seconds (very slowly!).
Lock in exclusive mode the table acs_objects. We get "Idle in transaction" in Postgres DB. The platform is blocked.
We have to kill the transaction (the postgres process).

I think of is very importante for the future of dotlearn
one working assessment. We can test the application for
that numbers (and greater). We like do assessments for
grups up to 200 students.

Regards,
Agustin

Collapse
Posted by Gustaf Neumann on
Which versions of the CR and pg are you using exactly?

The history:

  • for OpenACS versions up to 5.2, no locking was performed,
    but that lead to deadlocks which were detected by PostgreSQL, killing automatically the conflicting queries.

  • OpenACS 5.3 introduced locks on the Tcl level (in content::item::new and content::revision::new).

  • Malte had some problems with locks, which were induced by psql calls calling the content_item__new() and content_revision__new() stored procedures directly (see
    https://www.openacs.org/forums/message-view?message_id=499911).
    i got the impression that malted committed a version to HEAD.

  • We stress tested assessment in autumn with the locks on
    the Tcl level, and it worked quite well. then we observed the problems like Malte from other apps, and moved the locks into the stored procedures. However, two weeks ago, we had exactly your symptoms, caused by a stored procedure running for an hour.

Are you using the version with the locks in the Tcl layer or in the stored procedures?

Collapse
Posted by Jose Agustin Lopez Bueno on
Hi, Gustaf!

We are using, in production, Postgres 8.1.4.

Related with tcl / xql files:
-We have the next line:
db_dml lock_objects "LOCK TABLE acs_objects IN SHARE
ROW EXCLUSIVE MODE"
in
acs-content-repository/tcl/content-item-procs.tcl
acs-content-repository/tcl/content-revision-procs.tcl

-We have in psql the next procs locking acs_objects:
content_item__new (overloaded 20, 17 and 16 parms)
content_revision__new (13 parameters)

Any help, please?

(We have observed counted more than 150 locks (pg_locks)
for every assessment answer inserted in Pg!)

Are there any Pg conf parameter to tune these numbers?

Regards,
Agustin

Collapse
Posted by Gustaf Neumann on
Locking on both levels is not a good idea. It is no perfect solution, but for keep you going, i would recommend to remove the locks from content_item__new() and content_revision__new() stored procedures. From my experience, your should be able to run the assessment without reaching the "idle in transaction" status.

The assessment has certain weaknesses in design. I am afraid, to reach a significant better performance, the assessment should be re-engineered.

Thanks again!

But what numbers are you telling?

How many simultaneous assessments are you trying
in your center?

Regards,
Agustín

Collapse
Posted by Gustaf Neumann on
For students answering online exercises, we do not use assessment, but our own system https://nm.wu-wien.ac.at/research/publications/b318.pdf . We have up to about 1600 concurrent users, most of the solving exercises.

For course evaluation, we use assessment module, but only in a few parts. Currently, we are running evaluations for 1200 courses reaching most of our students (20000). We do this currently not online, but based on a paper & scanner based solution. We have quite a complex system tailored to the needs of our university. We have plans to offer the teachers in addition to the paper based system online assessments but this will need some work to run smoothly in this scale.

In autumn we were running a few smaller assessments (a few questionnaires for a few hundred students each) with mixed experiences.

Our 'learn' system predated .LRN, but was merged when .LRN came out in parts. Last time, i checked, our system differs from OpenACS/.LRN in about 15.000 changes.

We are not only running the learn system, but have about 20 OpenACS/.LRN sites within the university for various purposes, and similar numbers for customer sites, supported through our university spin off Knowledge Markets https://www.km.co.at. Knowledge Markets runs a site for customer assessments https://www.evaluate.at/ using the assessment module. It runs there with the mentioned changes, but i cant say, how many simultaneous assessments are usually filled out there.