Forum OpenACS Development: Survey - PreSpec discussion

Sloan has identified internally the potential for many survey-like packages: simple survey for classes/communities, course evaluation, leadership feedback form, elections, etc. Add to that Berkelee's needs: testing, imbedded polls in online content, etc. and basic elearning assessments needs and you end up with quite an array of features. Many require branching. Most require changes to permissions (who can answer, how many times, who can read answers) and many require various types of notifications (notifying prof when test is taken, sending feedback from to the person being evaluated).

In my research on the Bboards I was struck by the following quote about Workflow from Lars:

A very quick outline of what I envisioned with workflow: I want to let people build packages like the ticket-tracker, like bug tracker, or other packages that involve a workflow as a significant component, to easily build that, and get a superb user interface almost for free. How's that for a goal? :)

Which I could easily use for my vision for Survey as:

A very quick outline of what I envisioned with survey: I want to let people build packages like the poll, test, feedback, simple survey, or other packages that involve asking users questions as a significant component, to easily build that, and get a superb user interface almost for free. How's that for a goal? :)

Sloan is considering how to phase and approach our enhancements to Survey, and though the spec is not yet done we want to post publicly on some high level issues.

A: Do we just hack the current survey module until it works for August then deal with more complicated issues or do we build a base package with greater functionality then implement "refinements" of it to create different functionalities?

Our timeline is that we definitely need "simple survey" by August and we need branching by mid fall.

B: If we do decide to build a solid base functionality the following technical questions come up

1. Should we base it on Workflow?
2. Should questions and/or answers be stored in the Content Repository?
3. How do we manage the "Refinements," i.e., Polls, tests, course evaluations, elections, in such a way that they are easiest to build and maintain and have minimum repeated code.

Our thanks to those who have helped us clarify the issues and come up with suggestions and we encourage people who have been communicating with us by email to post your ideas here too.

This sounds like it might have potential for applications (employment or other) as well.

Workflow does seem like a nice base for this. The applications I work with are extremely subject to branching and require a lot of checking of the data and from what I have seen of workflow it is a nice basis for that kind of system.

One of my requirements is that the system never redirect. So if you finish page 1 and hit submit, the system will perform the required checks and show either page 1 or page 2 depending on whether or not you pass the required checks.

Another issue I run into is being sent an html file and having to break it apart and make it into a dynamic application page so I see the ability to easily template the system as important.

First of all, thanks for posting this, Caroline.

Given the relatively tight schedule and the fact that dotLRN is still something of a moving target (though stabilizing rapidly), the fact that you folks will be busy with testing, data migration, getting bugs fixed, etc, it might be wise to just make what exists today work for your August 1 deadline.  While I imagine you intend to farm out the project, you folks are going to have a very busy summer.  In my experience working with Sloan (and other clients) things go best when there's a lot of communication back-and-forth, and it just seems a relatively straightforward first effort would be best.  Meaning communications and coordination won't be overly stressed during what I think will be a fairly stressful time.

Of course I may be wrong.  You may already have testing and data migration in the bag :)  But I do know your August 1 deadline is solid.

The first step towards the larger picture would seem to be an analysis of the current datamodel to see if it is flexible enough to serve as a base for further development.  I haven't looked at OF's changes yet - the basic need is to be able to attach a survey to any object in the system (much as we're talking about with "general ratings") and I know the original datamodel didn't support that.

As far as workflow ... the concept's great but IMO the design goal's not been fully met, yet.  It's important to OpenACS that we do meet that goal but I wouldn't depend on our doing so in this timeframe.  In my experience it's harder to incorporate than I first expected.

Don't take this as a vote against using it - eventually I want to see workflow so easy to use that it we do so almost everywhere.  For now, though, I'd think carefully before deciding either way.  If workflow would help in defining the flow through different forms of surveys then using it is probably a plus.  If you think that there may be some need to make changing that flow relatively easy to customize down the road, then workflow can definitely be a plus if it is used correctly.

Michael -- At Berklee we've decided to follow IMS whereever it seems to be useful.  At the moment, we're looking at the Content Packaging spec for course content, supported with a data model we've layered on top of the content repository.  I haven't read the testing specification yet, but if it's similar, our design should be flexible enough to support importing and exporting quizzes into/from the CR.

I wasn't even aware of such standards.  Now that I am, yes, standards compliance would seem extremely important.

I have responded to Ben's point on standards in a new thread:

https://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0004oW&topic_id=12&topic=OpenACS%204%2e0%20Design

I lean in Lars's direction a bit, although not quite as much as
Pind's Rule of Five. My biggest concerns is that there's huge risk
of a second system effect on survey. Everyone has seen the
current module, and everyone has grand ideas for how it should
work next. There are probably a number of goals between all of
us:

- branching: a huge amount of work for such a small word.

- a generic survey engine: to provide a base for other packages
that are based on a survey-like process (quizzes, etc...).

- more answer types.

- better results processing / correlation.

- ability to link surveys to other objects in the system, make
certain actions dependent upon the completion of a survey.

(and I'm sure there's plenty more)....

There's a good reason to be conservative here. One solid path
might consist of focusing on a clean architecture to start and *
only then* adding features in an iterative manner. Given the
current state of the survey package (horrendous), just rebuilding
a clean architecture will be plenty of work in and of itself.

Hi all,

Caroline, if you're convinced of the need for branching (and I'm with Ben on the amount of work it will be), I think it may be worth at least speccing the datamodel for it now; if you don't, I'd guess it will be a significant amount of work to backport scoring from a branching system to a non-branching one.

Taking another spin on it, The simple survey module is up and running right now on ACS 4x, for instance, you can try it out at commongood.ybos.net. (In development, so be gentle..) I believe this is the same module that's downloadable at developer.ybos.net/modules. If you really just need a quick start, this might do the trick, and should integrate with dotlrn pretty easily.

So, I guess my question is: will the branching survey scores, (presumably used for testing?) need to integrate with the simple survey scores that you're rolling out in August? That might be a critical question. If the answer is no, I'd roll out simple survey quickly, and start working now on the branching system, paying extra attention to architectural decisions which will help you write a good admin interface; the admin interface for simple survey is, frankly, only okay. Adding branching support, expecting professors, et al. to use that interface would be pretty terrible.

Thoughts from others?

Don, incidentally, I don't know where your general ratings development is at, but we have a general ratings module, here http://developer.ybos.net/modules/stored_module-detail?module_id=2133, which you might want to use. It includes procs and graphics to make friendly ratings graphics. (like 3 stars out of 5).

Sorry, broken link.. commongood.ybos.net

Malte did branching for simple survey on ACES and ACS 3.4.5+... Don't know how hard it would be to port his code to ybos' 4.x simple survey. Sounds like a nice start though!

https://openacs.org/bboard/q-and-a-fetch-msg.tcl?msg_id=0002Q

Interesting observations on the distinction between surveys and assesments, Samir. We'd be primarily interested in using such a package for large survey projects and our additions to the package's requirements point to a couple of other areas where differences might come into play - participant management and notification. Here's our short list:

• Summary page for respondents to review/print answers.
• Notifications for survey author (for following up with participants who did not respond) and/or a nice interface for author to track respondents.
• Initial participant notification. It must be easy to notify large participant groups. Anyone using the package for surveys will likely have large groups of participants (who aren't likely to be users on the system) to stick into the DB and notify.

Also, does anyone have experience with any of the commercial options out there? A brief competitive analysis or discussion of features might be useful.