Forum OpenACS Development: dotLRN and standards

Collapse
Posted by Michael Feldstein on

The issue has come up around the degree to which dotLRN needs to worry about eLearning industry standards. Since I'm picking up this conversation from another thread, let me start by quoting Ben Adida's concerns:

Standards definitely merit a discussion of their own. I am all for standards. But I want to warn against thinking that IMS/SCORM are the panacea that people are expecting. The standards right now are fairly complex, and I have yet to see two "SCORM-compliant" systems actually talk to one another.

Some Background: SCORM is a standard that came about from the desire on the part of the US Federal goverment (especially the military) to be able to create large volumes of high-quality online learning content , have that content be able to interoperate seemlessly with server-side eLearning infrastructure made by a variety of vendors, and be as easy to re-use for other courses as possible.

Ben is correct that in the past "standards compliance" has not meant a whole lot. To begin with, most vendors were pretty selective in their implementation of the standards. On top of that, the standards were missing critical pieces that were necessary in order for them to be useful beyond very basic interoperability.

However, it is critical to be aware that the situation has changed. First of all, the various vendors have received a lot of pressure from their clients to be more completely standards-compliant. And they have responded. I work with SCORM-based systems quite a bit and I can tell you from firsthand experience that they are a lot better than they were a year ago.

Second, the biggest missing piece is of the spec is supposedly about to be filled in with 1.3 release, due out next week. Will it be sufficient? I don't know. It hasn't been released yet. But we need to take it very seriously, if for no other reason than the entire industry does.

AFAICT, the biggest problem with SCORM is that there very few vendors who are implementing them in interesting ways. There's a real opportunity for dotLRN to take a leadership role here.

Collapse
Posted by John Mileham on
Our approach, as Patrick stated before in the survey thread, is to make sure we can model anything that the prevalent standards can model.  Actual standards-compliant interfaces are a secondary concern for us for the time being, but it IS absolutly crucial that we be able to represent everything that the major standards to.  The questions, then, are: What standards should be implemented?  What pieces of each standard are considered to be the "best-of-show" (most useful, backed by the largest potential user base)?

This leads to the question of what the requirements are for your project.  The needs of distance learning are significantly different from those of on-site corporate training.  Then there's the difference between self-paced study vs. instructor-led and community-oriented teaching.  Berklee's needs fall strongly in the distance education, instructor led category, and we'll most likely be putting time into the specs that cover those areas at the beginning.  That said, we're very interested in helping the system grow to meet needs beyond our own as time progresses, especially if they further the reach and acceptance of dotLRN itself.

Collapse
Posted by Michael Feldstein on
John, you make a great point. The needs for higher ed and
corporate are somewhat different. That having been said, as
somebody who has done work in both worlds, I can tell you that
they are converging. The corporate world, which has largely been
focused on self-paced, is slowly coming to see the value of a
blended approach. Likewise, universities are coming to
understand that they need to create content assets (the online
equivalent of textbooks and workbooks) to use with their
instructor-facilitated courses.

To me, the best thing to do would be to put in the plumbing for
the standards now and let people hook into that plumbing on an
as-needed basis. I'm not technical enough to know whether
SCORM and IMS could usefully be implemented as service
contracts, but it may be worth thinking about it.

Off the top of my head, the IMS Enterprise specification (for
exchange of student ID info and similar stuff) is probably the
most immediately applicable across the various markets.
SCORM (which includes the IMS packaging and metadata
specificaitons), the IMS question & test spec (which may be
included in SCORM 1.3, though I'm not sure) and the simple
sequencing spec (which I'm pretty sure will be in SCORM 1.3)
are also important (though it's worth noting that the sequencing
spec is not yet final).

Collapse
Posted by Peter Vessenes on
Michael, one funny from the IMS whitepaper is that I can see they send answers to questions in the xml that they're sending to be rendered. This is from their multiple choice IEEE question in the whitepaper. Stuff like this makes me wonder what the priorities are for the standards group.

I'm mostly just curious, since I am not an expert on SCORM or IMS, but I can't help thinking that sending, say MIT students the answers to their multiple choice questions in XML might not be a good idea. In particular, it would mean that you couldn't use a web browser to render this stuff, you'd need something a little more closed. And even then, a packet sniffer would let you get 100% every time.

What are your thoughts on this? Can you summarize the overall mission and priorities of these institutions? It would probably help me get situated thinking about them.

Collapse
Posted by Michael Feldstein on
Peter, forgive my limited understanding of the coding issues, but
are there more secure ways to approach this problem? I've run
into variations on this before (all the time, actually) but nobody
hasn't given me a satisfactory solution yet. At the end of the day,
people usually just shrug their shoulders and say "Well, as long
as it's not so easy to cheat that the average person has an
overwhelming temptation, it's good enough." Keep in mind that
most testing of this kind is not taking place in a monitored room,
so even people with no web skills at all can cheat by, say,
consulting their notes if they really want to do so. I'm not saying
that the problem isn't worth solving; I'm just saying that many
users of online testing care less than you might think about this
issue.

In answer to your question, the IMS and ADLNet (the latter
produces the SCORM standard, which is mostly derived from the
IMS standards) are generally highly regarded by folks in all
facets of the eLearning world. Their primary role is to set
standards that promote interoperability of eLearning elements
(lessons, test questions, exercises, etc.) on the most granular
level possible.

They are often neutral with respect to implementation, i.e., they
try to avoid getting into discussions about how these systems
should actually be programmed. The IMS mostly provides (XML)
metadata specifications for many of the various elements in
eLearning courses. SCORM currently goes further by specifying
APIs for communication between the client (course-in-browser)
and the server (Learning Management System, or LMS).

Both standards bodies try to strike a balance between providing
leadership and specifying only those elements that are both (a)
important for interoperability at a granular level and (b) relatively
non-contentious.

Again, both standards bodies are highly regarded and closely
watched by vendors and clients alike in both the public and
private sectors. The catch is that I'm not sure what the quality of
technical oversight is. In general, my sense is that the folks on
the standards bodies are smart. However, I've also found that
the current tools on the market are very badly designed, the
vendors are usually much more interested in adding bullet
points to their brochures than in providing technical excellence,
and clients (who are generally not IT folks) often don't have the
technical expertise to know what's right.

So overall, I think dotLRN cannot afford to ignore IMS and
SCORM. Not being standards-compliant will have a strong
negative impact on adoption in the corporate world and in the
Federal government. It's probably a lot less important for higher
ed, but not necessarily completely unimporant.

But that doesn't mean we shouldn't examine the standards with
a critical eye. There are folks at MIT who are connected with both
standards bodies and who (I believe) are in communication with
the dotLRN folks, so there may be an opportunity to influence the
direction that the standards take in the future.

Collapse
Posted by Michael Serio on
It's my understanding that these XML specs are a method for exchanging data between systems. How data is stored on a given system is a separate issue. We can be SCORM, ADL-compliant because we have a data model and code that will export/import XML according to these specs to/from our system. Support for these specs are a consideration when designing a data model and determining a feature set. But our system will still use a relational database (PostgreSQL in our case) to store the information.
Collapse
Posted by Michael Feldstein on

It's my understanding that these XML specs are a method for exchanging data between systems. How data is stored on a given system is a separate issue. We can be SCORM, ADL-compliant because we have a data model and code that will export/import XML according to these specs to/from our system. Support for these specs are a consideration when designing a data model and determining a feature set. But our system will still use a relational database (PostgreSQL in our case) to store the information.

That's exactly right. The specifications difine (minimum) features and metadata labels, partially define a data model, and in some cases the APIs for putting and calling the relevant data. They do not eliminate the need for application design. But I think they can provide important hints for thinking through hard problems like branching, testing functionality, etc.

Collapse
Posted by Michael Feldstein on
Sorry, I meant "putting and getting" in that last post. I guess I've
been trading to many options on the stock market lately.