Sequencing

Along with Data Validation and Versioning, probably the most vexing problem confronting the Assessment package is how to handle conditional navigation through an Assessment guided by user input. Simple branching has already been accomplished in the "complex survey" package via hinge points defined by responses to single items. But what if branching/skipping needs to depend on combinations of user responses to multiple items? And how does this relate to management of data validation steps? If branching/skipping depends not merely on what combination of "correct" or "in range" data the user submits, but also on combinations of "incorrect" or "out of range" data, how the heck do we do this?

One basic conceptual question is whether Data Validation is a distinct process from Navigation Control or not. Initially we thought it was and that there should be a datamodel and set of procedures for checking user input, the output of which would pipe to a separate navigation datamodel and set of procedures for determining the user's next action. This separation is made (along with quite a few other distinctions/complexities) in the IMS "simple sequencing" model diagrammed below). But to jump the gun a bit, we think that actually it makes sense to combine these two processes into a common "post-submission user input processing" step we'll refer to here as Sequencing. (Note: we reviewed several alternatives in the archived prior discussions here.

So here is our current approach. We note that there are two scopes over which Sequencing needs to be handled:

• intra-item: checks pertaining to user responses to a single item
• inter-item : checks pertaining to user responses to more than one item; checks among multiple items will be built up pairwise

So how might we implement this in our datamodel? Consider the "sequencing" subsystem of the Assessment package:

Specific Entities

• Item-checks (as_item_checks) define 1..n ordered evaluations of a user's response to a single Item. These can occur either via client-side Javascript when the user moves focus from the Item, or server-side once the entire html form comes back. They are associated (related) to as_items.

  The goal is to have a flexible, expressive grammar for these checks to support arbitrary types of checks, which will be input validation ("Is the user's number within bounds?"; "Is that a properly formatted phone number?"). One notion on check_sql. Instead of using comparators we store the whole SQL command that makes up this check with a predefined variable "value" that contains the response of the user to the item the item_check is related to. If we want to make sure the value is between 0 and 1 we store "0 < :value < 1" with the check. Once an item is submitted, the system looks up the related checks for this item and replaces in each of them ":value" with the actual response.
  

  Item Checks thus will have these attributes:

  • item_check_id
  • cr:name - identifier
  • cr:description - Explanation what this check does
    
  • check_location - client-side or server-side
  • javascript_function - name of function that gets called when focus moves
  • user_message - optional text to return to user if check is true
    
  • check_sql - The sql that contains the check
• Inter-Item-checks (as_inter_item_checks) are similar to Item-Checks but operate over multiple Items. They are server sided checks that are associated with as_sections defining if a section should be displayed or with as_items, defining if an item should be displayed.
  
  The goal is to have a way of telling if a section (or an item within a section) shall be displayed or not depending on the section-checks. This way you could say that you only display this section if the response to item(1234) "Color of your eye" was "blue" and the response to item(4231) "Color of your hair" was "red". Sadly we can't use such an easy way of checking the ":value" as we do with item_checks, as we do not know which item this refers to. Instead we store the item_id like this ":item_1234". This way the check_sql would look like ":item_1234 == 'blue' AND :item_4231 == 'red'". Additionally other variables might be defined by the API at a later stage,  e.g. ":percent_score", which would be replaced by the current percentage value (aka score) that subject had in the test so far (taken from the as_session_table). It might be interesting to pass these variables along in the API, this remains to be seen when actually implementing the system.
  
  The Inter Item Checks also allow post section navigation (in contrast to the pre section / item navigation mentioned above). If post_check_p is true, the check will be done *after* the user has hit the submit button. Depending on the result of the check (if it is true) the user will be taken to the section given by section_id or to the item given by item_id, depending whether this assessment is section based (questions are displayed in sections) or item based (each item will be displayed on a separate page). If there are multiple post checks for a given section/item the order will be defined by the relationship (which means this relationship has to have a sort_order attribute).
  
  • inter_item_check_id
  • cr:name - identifier
  • cr:description - Explanation what this check does
  • user_message - optional text to return to user
  • check_sql - The sql that contains the check
    
  • post_check_p - Is this a check that should be executed after the user hit the submit button while answering the item / section.
    
  • section_id - Section to call if we are in a section mode (all items will be displayed in sections) and it is a post_check.
    
  • item_id - Item to call if we are in "per item" mode (all items will be displayed on a separate page) and it is a post_check.
  • Potential extension: item_list - list of item_ids that are used in the check_sql to speed up the check.