Let me give your first question a try Stan.
>Those people in dotLRN land who are actively
>implementing "testing" in real curricula: how do you
>understand "revisions" of such tests?
Right now, we revise a test and it effects all people that have taken that instance of the test (e.g. change a question or an answer to a question and all people who have taken the test in the past are effected by the change).
>What is the distinction between a modification
>of the "same" question and a modification that
>implies that you now have an entirely "new" question?
Let me try to distinguish the two: the modification of the "same" question is the modification of a question that was/is actually used in an exam (e.g. it was discovered that the question and answer pairs where not matched up correctly and once they are corrected the results of this instance of the exam change). A totally "new" question when the content changes independent of a specific instance of an exam (e.g. change of actual knowledge: once it was believed that the primary cause of peptic ulcers was hypersecretion of acid, now Helicobacter pylori is the peptic ulcer star but because this has changed in the books over the past couple of years does not mean that the results of an exam in 1985 are changed)?
>Is there a distinction?
I am not sure. Do not think the examples I used did a good job of distingushing the two above, because they both could be solved using "revisions" in the content repository (revisions of a single .
>Is there a definable transition point?
>If so, what is it?
The point we actually ask the user if they want to create a "new" question?
>If not, are we nuts to talk about "revisions"?
Far from nuts, but keep it up you are getting closer.
So in summary: I tend to want to leave it up to the user (test admin) to define when a "new" question should be created (with warnings). Revisions can and should be used when a question that is in use or has been used in the past is changed.