This question is not easy to anwer, since it is hard to measure the effects of an elearning platform in a scientific clean way. One has to separate two questions, the improvements of the learning process (from a student and a teacher point of view) and the improvement of the learning quality (as well from students and teachers and universities point of view; i have a couple of slides in this regards about learn@wu). ... and then, it is hard to attribute the effects to a certain e-learning environment.
For our university the following factors were important:
- Without substantial e-learning-content, our system would not be by far as successful. Good content requires easy content creation, maintenance and accessability. ... requires an academic staff willing to provide content.
- Target on teachers: they must provide good content, they know what they need in their classes. It is important to provide the right tools for their needs, provide best practices, provide e-learning education for teachers. e-learning should make the life of a teacher easier.
- Target on university management: although we achieved substantial cost-reductions in our setup it is hard to talk about these to a chief financial officer and to attribute these to the investments in our system. Without a commitment from the university management, we would have a handful of e-learning courses, and not 1200 (as my current statistics says). With rich and transparent online materials it is much easier to provide quality assurance.
- Alignment with the university processes and infrastructure (course/class generation, checking prerequirements for enrolling classes, flexible course catalogs, reflecting organisational structures and identity management, authentification); this means roughly integration with legacy systems, SAP etc.
The sitation might be quite different for other universities and shools. For example, we developed an e-learning environment based on .lrn and our extensions for the highshools of an austrian county: for them, it was most important to use a proven environment (our university has a very good standing) and to provide means to raise the activity level in their notebook classes. So we provided online quizzes with evaluations for the end of their lectures (which was exactly fulfilling their needs) and provided means to reflect their organizational strucures...
These factors were not at all cruical for my university.
To your original point: we did some investigations on measuring the effectiveness of e-learning environments (containg learning materials and teachers and students).
We found it very hard to relate "the quality of an elearning environment" to e.g. student success at exams, because this has a large range of influence factors. From our experience it is better to investigate on "satisfaction" or on "percieved knowlege increase", which are still somewhat fuzzy. However, we did not make an comparative evaluation of different platforms.
Look into http://www.sloan-c.org/effective/ for a quick intro into such approaches, which are pretty much in line with our experiences.