Forum .LRN Q&A: Adaptive learning

Collapse
Posted by Michael Feldstein on
I'm moving this over from another thread for the sake of clarity.

Here's what Staffan Hannson wrote:

Michael, whether you are actually saying this or not, I get the feeling that you are saying: "Curriculum is not so important that it is worth implementing right away; nevertheless it is so important that it demands several months of further research." This conclusion actually makes sense if we presuppose (1) that the users and promoters of dotLRN have displayed no interest in Curriculum's functionalities and (2) that the technical solution of these functionalities will affect the very foundation of the groupware, because it demands a dotLRN that is an LMS.

Both of these presuppositions, and thereby the conclusion derived from them, are wrong. In fact, there is a clearly confirmed interest in implementing branched sequencing of learning activities; and the implementation of this service, though not a piece of cake, will not affect any core service of OpenACS. Hence, I believe you're overestimating the technical influence of Curriculum on OpenACS and dotLRN, while underestimating its importance to the users. I'm quite sure that by claiming (following IMS terminology) that we are developing an LMS (alternatively LTS, learning technology system, in IMS talk), we have tricked you and others into this belief. If that is the case, it's our fault.

You're absolutely right when you say that IMS specifications are not likely to provide us with any sort of guidance to usability. Even IMS agree with that. As the IMS Simple Sequencing Specification makes perfectly clear: "The nature of the control and communication interfaces, and the mechanisms for mediating interactions between the a learner and a LTS, are not part of this Specification. In addition, issues such as look and feel, presentation style, and placement of navigation controls are not defined by this Specification." These are matters that the community and other vendors will be able to develop to their liking as soon as the basic structure is implemented. And I have great confidence that six or nine months down the road, you'll bring the implementation to new levels of perfection. You’re certainly the community’s expert in this field.

I'm kind of curious to know where your skepticism toward IMS comes from. I don't know what it is you expect IMS to solve for you, which they have failed to do, that has made you disappointed in them. Their Simple Sequencing Specification certainly provides us with the architectural blueprint we seek for our purposes. And as even NATO's ADL SCORM turn to IMS for standards, I can picture us doing the same. Whether or not IMS is a complete and satisfying authority for turning dotLRN into an LMS (as the industry defines the term) I have not the slightest idea, and will gladly leave that analysis to those who are actually working on such a task.

At any rate, the implementation of Curriculum does not affect the question of user profiles.

Collapse
2: Re: Adaptive learning (response to 1)
Posted by Michael Feldstein on

OK, let me take this one paragraph at a time.

Michael, whether you are actually saying this or not, I get the feeling that you are saying: "Curriculum is not so important that it is worth implementing right away; nevertheless it is so important that it demands several months of further research." This conclusion actually makes sense if we presuppose (1) that the users and promoters of dotLRN have displayed no interest in Curriculum's functionalities and (2) that the technical solution of these functionalities will affect the very foundation of the groupware, because it demands a dotLRN that is an LMS.

Nope, that's not what I'm saying. Here are the points I'm trying to make:

  1. From the perspective of evangelism and of driving wider adoption (somewhat redundant, I know), implementing Simple Sequencing, which is part of the upcoming SCORM 1.3, before implementing SCORM 1.2, is putting the cart before the horse. This in no way means that I'm discouraging community members from scratching their own itches. If somebody out there is saying, "I can use this right now and I'm willing to put my resources toward it, then far be it from me to interfere.
  2. In addition to the fact that, all else being equal, I think implementation of SCORM 1.2 will be far more important in the short term for driving wider adoption than Simple Sequencing will be, I also, frankly, don't believe that the IMS approach to standards in general and the approach they've defined in Simple Sequencing in particular is the right solution to the adaptive learning problem. I'm speaking from experience, having just written an adaptive learning course and being in the process of writing another one right now. I'm not eager to rush out and implement a specification that I think is deeply flawed. That having been said, since it *has* been adopted by ADL for SCORM 1.3, I would add that being partway to SCORM 1.3 compliance would not be a bad thing.
  3. I certainly did not intend to imply that implementing curriculum using Simple Sequencing would somehow do damage to the dotLRN core. My reasons for advising caution are the onese I state above. But I repeat that if people need a SS-based curriculum module now and are willing to invest their own resources then more power to them.

You're absolutely right when you say that IMS specifications are not likely to provide us with any sort of guidance to usability. Even IMS agree with that. As the IMS Simple Sequencing Specification makes perfectly clear: "The nature of the control and communication interfaces, and the mechanisms for mediating interactions between the a learner and a LTS, are not part of this Specification. In addition, issues such as look and feel, presentation style, and placement of navigation controls are not defined by this Specification." These are matters that the community and other vendors will be able to develop to their liking as soon as the basic structure is implemented. And I have great confidence that six or nine months down the road, you'll bring the implementation to new levels of perfection. You’re certainly the community’s expert in this field.

Thanks for the vote of confidence, but it's not usability of the UI that I'm worried about. I'm worried about usability of the schema in terms of supporting course design. What is simple sequencing all about? It's about interactivity. It's about teaching your course to respond to the learner's needs. It gets to the heart of good teaching and of everything that's hard to do in self-paced online learning. I don't think that a pre-defined taxonomy of interaction types--any taxonomy of pre-defined interaction types--is going to solve this problem. I'd prefer to use more organic methods like those that underly the semantic web concept.

Now, it's possible that I'm underestimating the flexibility of the IMS specification. You have undoubtedly spent more time looking at it than I have. But I'm fundamentally skeptical about their approach and I don't see anything in what I've puzzled through of the spec over the last several months that encourages me to change my mind.

I'm kind of curious to know where your skepticism toward IMS comes from. I don't know what it is you expect IMS to solve for you, which they have failed to do, that has made you disappointed in them. Their Simple Sequencing Specification certainly provides us with the architectural blueprint we seek for our purposes. And as even NATO's ADL SCORM turn to IMS for standards, I can picture us doing the same. Whether or not IMS is a complete and satisfying authority for turning dotLRN into an LMS (as the industry defines the term) I have not the slightest idea, and will gladly leave that analysis to those who are actually working on such a task.

I am not expressing global skepticisim toward the IMS. In fact, if you look at my posts on these boards, you'll find that I have been a vocal advocate for aggressive adoption of their standards. I even have been an advocate of simple sequencing until recent experience changed my mind. I just don't think that they have the right approach to solve this problem. (They face a similar challenge, with similarly questionable results, on the test question standard, BTW.) Not all educational standards problems are equally easy and not all are equally well served by the same solution approach. Sequencing is one of the hardest problems we have. People have been writing about "intelligent tutoring systems," which are essentially adaptive learning systems, for at least 30 years. How many of these have you actually seen outside of some university's comp sci laboratory?

At any rate, the implementation of Curriculum does not affect the question of user profiles.

I agree.

Collapse
3: Re: Adaptive learning (response to 1)
Posted by Staffan Hansson on
Michael, I now realize how much we've missed you and your insights in this area. You are the obvious guru and you have a lot to teach us. No doubt, by the end of this thread we'll stand on a firm base from which we are much better equipped to critically examine the IMS specification; what's bad about it and what's good about it, and what conclusions to draw from this analysis when implementing Curriculum. You (and of course the entire community) can certainly help making the OpenACS solution to the problem of adaptive learning the best so far, by raising a critical voice whenever needed and by offering constructive solutions.

There is just one problem; by the end of this thread, when we have reached that new level of adaptive learning enlightenment, the once so keen financers will have followed your expert advice and dropped out, and we will have no practical use for our insights. We'll probably laugh at this irony in the long run. However, though this laugh would hopefully prolong our lives, the lack of food on our table in the short run is likely to prevent us from ever experiencing that long-run benefit. When you say that you wouldn't want those who feel a need for Curriculum to be discouraged by you, everyone knows that the damage is already done. You have "weight" and I haven't. Please don't take your (temporarily?) pessimistic outlook out on us. Keep your skepticism coming, though.

You're making Simple Sequencing out to be a rocket science. Having studied the IMS specifications, we cannot agree with that portrayal. It's a complex material, yes, but rocket science, no. If it is flawed, those flaws will be detected. If the flaws are as fundamental as you say, they'll be sorted out on a fundamental level. We immediately realized, for example, that without considerable caching the implementation would truly suck. However flawed, IMS offer universal standards for Simple Sequencing, adopted by SCORM. Their specifications are more or less exact instructions that can and will be tested and improved. Your organics are certainly important considerations when making Simple Sequencing useful, but they don't constitute an architectural blueprint for developers.

In regard to the 30 years of failed attempts to produce adaptive learning systems, what can I say? It is indeed a depressing anecdote. I wasn't born then, but from what Discovery teaches me, back in those days rocket science was actually a "rocket science", whereas today it is more of an industry. I can imagine that a corresponding development has taken place in the field of Simple Sequencing. With IMS standards being adopted by major players in the industry, we are now experiencing a period in time when people are actually working in the same direction, or within a common framework. This raises my hopes that we won't be telling each other that discouraging anecdote 30 years from now.

Now, over to the real arguments for avoiding Simple Sequencing - and those I projected onto you. You claim that there is a horse and a cart, SCORM 1.2 and SCORM 1.3. Thus you want to postpone the Curriculum implementation until other more basic functionalities are in place. This was the reasoning I was trying to track and pin down, semi-successfully. I was assuming that your horse was of a fundamental sort, with the cart being dependent on it. To me, there is no horse and no cart. Curriculum is a standalone package; it does not make a case for or even imply SCORM this or that and in this or that order. If there is a horse that has to be before our cart, it is OpenACS. I assumed that this was your horse; hence presupposition (2). But nope.

In my mind, this leaves us with the "no acute need for"-argument for postponing Curriculum. I will not pretend to know what rank or rating Simple Sequencing has on the evangelization chart, and hence its priority for development. I only know that people within and without the OpenACS/dotLRN world take an interest in it. I also know that it is quite possible to take interest in several things at the time. Now that dotLRN, along with its community of developers and vendors, is in a phase where it has to expand or stay forever obscure, it may even be the thing to do. Strategically, it can pay off to support initiatives from wannabe professional developers rather than to remove the carpet from under their feet, especially when the institutional users/promoters actually are interested in their proposals.

Let's drop the pessimism. Let's start off this project and you'll have the chance of your life to break that 30-year-old spell that has plagued your industry. Let's chase those demons away!

Collapse
4: Re: Adaptive learning (response to 1)
Posted by Dave Bauer on
Steffan,

A technical question... are you considering using any part of the new workflow package? It seems like it might be helpful in implementing sequencing.

Collapse
5: Re: Adaptive learning (response to 1)
Posted by Staffan Hansson on
Hi Dave,

Yes, we would definitely like to use Workflow. Most likely not for the sequencing part, though, but for the course design part of the package, i.e., the part that has to do with the administration of the tree of learning activities and their rules etc. For the actual sequencing engine that controls the adaptive traversal of the tree (the part that is specified by IMS), however, Workflow won’t be specific enough for that well-defined and well-regulated task, I think.

Lars himself has recommended against using Workflow for highly customized and optimized tasks. It has something to do with Pind's Rule of Five 😊

Collapse
6: Re: Adaptive learning (response to 1)
Posted by Talli Somekh on
I'm usually not the best one to call for calm and sanity, and while I have no opinion as to the bulk of this thread I kinda feel that a line was crossed here.

Steffan I am also the head of a company in the OpenACS community and I have to ask that marketing and the technical decisions the OpenACS community makes be separate.

Michael is a member of the dotLRN governance and it is his part of his responsibility to decide where is the most appropriate place for funding. He has proven his integrity in this regard. You do have a strong argument, though, and I'm sure the appropriate people are listening.

Whether or not the language gets heated in this forum is up to the posters to decide, but I don't think anyone is trying to take anyone's food off of anyone's tables.

talli

Collapse
7: Re: Adaptive learning (response to 1)
Posted by Michael Feldstein on
Talli, while I greatly appreciate your rallying to my defense, I actually feel that Staffan is being quite civilized about the (legitimate) conerns that he's raising. Furthermore, the User Advisory Board is called an "advisory" board for a reason. We advise the community on certain issues. We don't direct.

I think I can offer some perspective here that will resolve the potential conflicts here. My concerns are regarding adaptive learning writ large. There is a large set of problems that falls under the heading of "adaptive learning" and I simply don't believe that that the IMS approach is well suited to ultimately solve some of the most difficult (and interesting, and valuable) problems within this problem set.

OTOH, the "Curriculum" module does not necessarily seek to solve the full problem set of adaptive learning. If current or potential dotLRN users see a need for the functionality that the Curriculum module will provide, and if Staffan's team believes that the Simple Sequencing spec is well suited as a means for providing that functionality, then I would say to any potential funders that there is no reason not to go ahead. On the contrary, it most certainly be helpful for us to be able to say that we have implemented Simple Sequencing which, whatever my opinion, is now a stardard that has been adopted by the two most widely respected e-learning standards bodies in the industry.

All that having been said, one of my roles *is* to educate the dotLRN/OpenACS development community on the broader and deeper issues in online learning. We have to be very careful about how we sling the lingo and what we claim dotLRN can (and cannot) do. However the IMS officially describes LMS's, for example, the most common (though not universal) understanding of what an "LMS" is is a system that is at least partly compliant with AICC, SCORM 1.x, or both. We can (and should) brag about what dotLRN can do; we can (and should) be agressive about going head-to-head Blackboard and  WebCT, too. However, we should not call dotLRN an LMS when we do so. Likewise, we should be agressive about building out a curriculum module if we think it offers functionality that is immediately useful. In the process, though, we should be careful and conservative about any claims we make regarding adaptive learning. And we should be looking *beyond* the standards as well as *to* them in order to see where we can provide leadership.

I am in no way, shape, or form discouraging the building of the curriculum module. I do not claim to speak for the needs of the community and would never discouraging people from building what they need. That would be antithetical to everything we are trying to do here. In fact, I'll go even further. Staffan, if you would find my input on the functional spec for Curriculum to be helpful, please email me. I would be delighted to help you move the project forward in any way that I can.

Without contracting any of that, though, let me renew my plea for the community to look very seriously at implementing SCORM 1.2 sooner rather than later and let me also strongly caution the community to be very careful about using e-learning industry terminology without making sure that we have shared understanding of what the industry means when it uses these terms. Staffan, true adaptive learning is hard. Really and truly. It's not hard because of the technology. It's hard because boiling down the principles of good, interactive teaching into a set of algorythms is really hard. I'm sure that Simple Sequencing is a fairly straightforward spec. I'm much less sure, though, that it solves the problems of true adaptive learning.

Collapse
8: Re: Adaptive learning (response to 1)
Posted by Alfred Essa on
Rafael wrote: "I would like dotLRN to eventually be an Adaptive Learning Environment (or what I prefer to call a Intelligent LMS). An ILMS would change the way it presents its courses based on the information it has about the student. The student profile is then a key issue here."

I agree with this statement. dotLRN currrently provides functionality for Course Management and Online Communities. We eventually need to extend the feature set to allow capabilities associated with Learning Management Systems and Content Management Systems. In this regard the Curriculum Project and Adaptive Learning Environments are relevant. I expect that we will be paying a lot of attention to these topics in the coming months.

Collapse
10: Re: Adaptive learning (response to 1)
Posted by Michael Feldstein on
<rant>

OK, I think it was precisely the "Intelligent LMS" comment that set off my danger meter regarding the adaptive learning stuff. I am a very, very big fan of adaptive learning. I'm pushing my company to put a lot of energy into it right now because I think it's critically important. But I get nervous when we start using (or making up) buzzwords so far out from implementation or even implementable definitions.

To begin with, dotLRN doesn't even have an LMS, never mind an "intelligent" one. And having worked with a number of LMS's recently, I can tell you that it will be a major industry advance to have non-intelligent LMS that simply works well.

Second, there is currently no standard or model that comes close to showing us how to build an "intelligent LMS." If you look at this IMS document [http://www.imsproject.org/simplesequencing/v1p0pd/imsss_bestv1p0pd.html], you will see that even the IMS iteself claims that the sequencing spec only gets you "limited" adaptive learning. Beyond "limited" (the next box beyond their scope on the chart they show) is "full." Beyond *that* is "intelligent."

I have no issue whatsoever with the curriculum module. I also have no issue whatsoever with setting ambitious goals. But I have a big issue with aiming for buzzwords rather than  well-defined educational goals. I don't even know what "intelligent LMS" means, really.

</rant>

OK, now that I have that out of my system, let me not make any assumptions about what other people have in mind. Rafael, putting the buzzwords aside for a minute, what is the long-term goal that you were thinking of when you used the phrase "intelligent LMS"? What would that look like? How might it work? Let's hash this out into a vision that's concrete enough to be turned into an implementation plan.

Collapse
9: Re: Adaptive learning (response to 1)
Posted by Staffan Hansson on
Talli, I appreciate your concern for keeping the atmosphere in the community a friendly one. You certainly contribute to this with your good-humored posts. Me, I wasn't using a heated language according to my own standards, but I did grow up in a rough neighborhood, so... I too deeply respect integrity, even though I might have a slight problem with authority (but who hasn't these days?). It was indeed because of Michael's strong integrity I felt I could speak my mind. What I do regret is the doubt I expressed about the financers' integrity and hence ability to make an independent judgment of the actual facts in this case. I apologize.

Michael, I think you have just landed this discussion on the level where it belongs. Thanks.

By now we all know that the Curriculum module will not revolutionize dotLRN; its stated goal is not to solve the problems of true adaptive learning but to offer a simple sequencing that fully meets the top standard. We further know that we have the human resources within the community to address the issue of adaptive learning and eventually offer services the world has never seen. And, finally, we know that the community might want to aim at making dotLRN as a whole an implementation of SCORM 1.2 (why not 1.3?).

Let me try to explain why we never claim that Curriculum with its simple sequencing will be an implementation of SCORM. If we eat salad, and rabbits are known to eat salad, this doesn't make us rabbits. Similarly, if we follow IMS simple sequencing standards, and SCORM 1.3 is known to follow the same standards, this doesn't make us followers of SCORM standards. We are vegetarians: we eat salad, not rabbits; we follow IMS, not SCORM. So, Curriculum itself does not imply SCORM 1.3, but neither does it prevent dotLRN as a whole from following SCORM 1.3 - or 1.2, for that matter. We have no opinion about dotLRN as a whole.

About the need for common definitions; Curriculum will indeed be partly compliant with SCORM, making it all right in IMS's view for us to call it an LMS. Is it the community's opinion that we, nevertheless, should refrain from calling it an LMS? We may simply state that Curriculum offers simple sequencing without claiming it is an LMS. Whichever causes the least confusion and puts OpenACS and dotLRN in the best marketing position is fine by us. Fortunately, we have never used the term adaptive learning in any of our texts on Curriculum. To be honest, I didn't even know the term until it became the title of this thread...

Collapse
11: Re: Adaptive learning (response to 1)
Posted by Michael Feldstein on
On the issue of SCORM 1.2 vs. 1.3, SCORM 1.3 *is* SCORM 1.2 + the IMS Simple Sequencing Spec. So basically, if we already are implementing Simple Sequencing, then essentially adding SCORM 1.2 compliance should bring us up to compliance with SCORM 1.3. (There may be some other minor tweaks to the spec; I'm not sure. But Simple Sequencing is the whole reason that 1.3 exists.)

Regarding the LMS issue, I urge us not to use that term until we have a full SCORM implementation. We're working in a hype-filled industry here; it's best to build a reputation for trustworthiness by being very conservative about what we say that dotLRN provides.

Collapse
12: Re: Adaptive learning (response to 1)
Posted by Warwick Bailey on
We take the view here that there is substantial consensus around content
packaging standards and we can safely move forward using the IMS Specifications
(which are extended in SCORM). There is little consensus around user tracking
with some using server-side tracking, some using SCORM client side tracking and
Microsoft doing their own version of client side tracking with .LRN. We are
currently working on a model that does server side tracking unless presented
with a SCORM object when it will use the client side tracking. In
general, our experience is that client side tracking is 'fragile' and very
platform sensitive.

There appear to be 2 generally accepted, but different, definitions of an LMS.
One focusses on scope for tracking students and the other on enterprise
integration (e.g. with Student Records and Finance). We tend to use the former,
but I always ask anyone who says they have an LMS... 'what do you mean by that?'

Collapse
13: Re: Adaptive learning (response to 1)
Posted by Rafael Calvo on
Hi,
I agree with Michael about the difficulties of adaptive learning.

Regarding with what ILMS means, I do not have a precise answer. In fact, because it is a somewhat an open question is that I will be chairing a workshop in July this year entitled "Towards Inteligent Learning Management Systems" at the Artificial Intelligence in Education Conference, in Sydney July this year.
The workshop's aim is basically to have a lot of researhcers discuss and hopefully answer that question.
The conference will bring around 500 researchers from around world, and I expect to have about 50 in the workshop.
Please visit: http://www.cs.usyd.edu.au/~aied/ for more information. I will be able to give more details about the workshop in a few days.

I do believe that dotLRN has a great chance of being ahead in this game. The reason is that since it is open source all the researchers (as the ones I mentioned above) should find it a great tool to work with.

Collapse
14: Re: Adaptive learning (response to 1)
Posted by Staffan Hansson on
Michael wrote:
Regarding the LMS issue, I urge us not to use that term until we have a full SCORM implementation. We're working in a hype-filled industry here; it's best to build a reputation for trustworthiness by being very conservative about what we say that dotLRN provides.
Amen to that! I have now removed all claims that we're making an LMS from our texts on Curriculum.
Collapse
15: Re: Adaptive learning (response to 1)
Posted by Staffan Hansson on
Our overall model for simple sequencing, a translation of the IMS Simple Sequencing Specification into OpenACS talk, is up for public viewing now.