The assessment module in it's simplest form is a dynamic information gathering tool. This can be clearly seen in the first group of use cases, which deal with surveys (one form of assessment, e.g. for quality assurance or clinical trials). An extension of this information gathering the possibility to conduct an evaluation on the information given, as we show in the second group of use cases (testing scenarios). Last but not least, the assessment tool should be able to provide it's information gathering features to other packages within the OpenACS framework as well.
It is very important to note, that not all parameters and features mentioned in this use case should be displayed to the user at all times. Depending on the use case, a good guess with pre determined parameters should be made for the user (e.g. no need to let the user fill out correct answers to questions, if the question is not used in a test). Some use cases like elections require special parameters not necessary anywhere else (like counting system).
Survey scenario
The survey scenarios are the basic use cases for the use of the assessment system.Simple survey
An editor wants to conduct surveys on his site. For this purpose he creates questions which are stored in a question catalogue. From this question catalogue, the editor choose the questions he wants to use in his current survey along with the style the survey should be presented to the user. Once satisfied he can make the survey public or test it first. Once the survey is public subjects (users) of the site can take the survey by filling out the generated form with all the questions the author added to the survey.Quality Assurance
A company wants to get feedback from users about it's product. It creates a survey which offers branching (to prevent users from filling out unnecessary data, e.g. if you answered you have never been to Europe the question "Have you seen Rome" should not show up) and multi-dimensional likert scales (To ask for the quality and importance of a part of the product in conjunction).Professional data entry
A clinic wants to conduct a trial. For this research assistants are asked to interview the patients and store the answers in the assessment on behalf of the client. For meeting FDA requirements it is mandatory to prove exactly who created any datum, when, whether it is a correct value, whether anyone has looked at it or edited it and when along with other audit trails. As mistakes might happen, it is important that the system runs checks on the plausibility of the entered data and the validity of it (area code should be five digits, if the age of the patient is below 10, no need to ask for credit card information, ...).University survey
A Professor wants to create a test by searching through the question database and selecting old questions. He searches the database for a specific keyword or browses by category. The System presents him all questions which have the keyword and/or category in it. The Professor is able to preview every question and may then decide which question he will transfer into the survey.Internal Evaluation
An institution wants to survey students to compare the quality of specific courses, teachers, or other factors effecting the quality of their education and level of happiness. It should be possible for the person who takes the survey to submit the survey anonymously and only be able to take the survey once.It should also be able to show the results of a survey to a group of users (e.g. a specific department evaluated). The results should be able to be displayed in a way that give a department a ranking compared with other departments.
Reuse of questions
The author of multiple choice question decides that the provided answers are not good for differentiating the knowledge of the subjects and changes some of them. All editors using this question should be informed and asked, if they want to use the changed version or the original one. If the decision is made to switch, it has to be guaranteed that a distinction between subjects that answered the original and the new version is kept. In addition the editor should be able to inform all subjects that have taken the question already, that it has changed (and that they might (have to) re-answer).Multiple languages
The quality assurance team of the company mentioned above realizes that the majority of it's user base is not native English speakers. This is why they want to add additional translations to the questions to broaden the response base. For consistency, the assessment may only be shown to the subject if all questions used have been translated. Furthermore it is necessary to store the language used along with the response (as a translation might not be as good as the original).The poll
An editor wants to conduct a poll on the site with immediate publication of the result to get a feeling how users like the new design of the website. The result can be displayed in an includelet (see the below for details) on any page the editor wants.The election
The OpenACS community wants to conduct a new election on the OCT. On creation the names of the contestants have to be available along with a list of all users allowed to vote. Depending on the election system, the users have one or multiple votes (ranked or not), which are calculated in a certain way. Once the election is over the result is published.Collective Meeting planing
The sailing club needs to find meeting time for all skippers to attend. Given a number of predefined choices, each skipper can give his/her preference for the time slots. The slot with the highest approval wins and is automatically entered into the calendar of all skippers and a notification send out.Testing scenario
Especially in the university environment it is important to be able to conduct tests. These help the students to prepare for exams but also allow Professors to conduct exams. In addition to the data collection done in a survey scenario testing adds checks and instant evaluation to assessment.Proctored Exam
A Professor wants to have a proctored test in a computer room. He wants to create the test using question that he has added and are already in the database. The only people allowed to take the test are the people that have actually showed up in the room (e.g. restricting the exam to specific IP-subnet and/or an exam password which he will give the students in the room at the time of the test that gives them access to the exam). Additional security measures include:- Students have to submit the survey signed with their PGP key (which has been verified by the university) at the end.
- Students have to print out their test and sign every page to make sure the answers in the system are identical to the ones the student has given.
- In a purely multiple choice environment, the Test might be printed out on a sheet of paper for each user along with a return sheet which needs the answers to be ticked off. A scanner system scans this return sheet and stores the data for the student in the system.
The Mistake
A Professor has created a test from the question pool and have administered the exam to a group of students. The test has been taken by some of his students already. He discovers that the answer to one of the questions is not correct. He modifies the test and should be given the option to change the results of exams that have already been completed and the option to notify students who have taken the test and received a grade that their results have changed.Discriminatory power
A Professor has created a test which is taken by all of his students. The test results should be matched with the individual results to create the discriminatory power and the reliability of the questions used in the test. The results should be stored in the question database and be accessible by every other professor which has the privileges to access the database of this professor.[A Question improves the test in reliability if it differentiates in the context of the test. This is happening if it has discriminatory power. The Question has discriminatory power if it is splitting good from bad students within the question in the same way they passes the test as good and bad students. The discriminatory power tells the professor if the question matches the test. Example: A hard question with a high mean value should be answered by good students more often right than by bad students. If the questions is answered same often by good and bad students the discriminatory power tells the professor that the question is more to guess than to know]
The vocabulary test
A student wants to learn a new language. While attending the class, he enters the vocabulary for each section into the assessment system. If he wants to check his learned knowledge he takes the vocabulary test which will show him randomized words to be translated. Each word will have a ranking stating how probable it is for the word to show up in the test. With each correct answer the ranking goes down, with each wrong answer it goes up. Once a section has been finished and all words have been translated correctly, the student may proceed to the next section. Possible types of questions:- Free text translation of a word
- Free text translation of a sentence
- Multiple choice test
- Fill in the blanks