Forum .LRN Q&A: Question about dotlrn database management

I'm studying how dotlrn works for a possible installation by my university. One of the things i noticed is that we can't completely remove users from the database, and the same thing for courses and classes. I think that this make the database more dificult. Every year hundreds of new students come to the university and hundreds finish the courses and leave, courses and classes are shutdown permanently. This way, i think, the database size only grows and grows whit the time. My question is, i'm seeing things wrong? If not, is there any strategy or solution to get ride off this useless data (controling the size of the database) from the database and make the management of the database more easy?
Thanks for any help :)
Collapse
Posted by Tilmann Singer on
There have been discussions about this before here on these forums, with the consensus that it does not make sense to delete old stuff, given the complexity of relations between the various database objects.

When you try to compute how much storage a user or a class actually consumes on the hard disk you'll propably find out that it is irrelevant.

Collapse
Posted by Janine Ohmer on
Also, the upside to this behavior is that you can convert your graduating students to alumni in the system, and offer services that keep them coming back to the site.  If done right,  your alumni office (the one that tries to get donations to the school) will love you for this! :)
Collapse
Posted by Alfred Essa on
Jorge, You have identified a very important issue for .LRN. Instititutions do need a systematic way of expiring items such as user accounts and classes. One of the weaknesses currently also of .LRN is that we have not developed a roadmap for integration with external systems. Student information and, therefore, student accounts would typically come from an external system such as a registrar system. Let me flag your question as something that requires more thinking and needs to be part of our roadmap very soon for .LRN.
Collapse
Posted by jorge freitas on
One of the things people ask me is how we can take the information already existent in the institution database (information about the users, courses and classes) and put that information in dotlrn, preventing to introduce that information manually. This is a subject of very interest to me since i have to find a solution to this situation in order to advance for a future installation of dotlrn in my university. If anyone have a idea of what i can do or where to start, i will apreciated.
Thanks :)
Collapse
Posted by Dirk Gomez on
Transfering data: Connect to the other database(s) via well-known gateways, e.g. with Oracle to Oracle with database links and transfer the data. You can make tables in other databases accessible via Views, that way you don't need to import data, but will always work on fresh data.

Growing data: More entropy with every semester - that is right. Why is "old" data useless? What DotLrn may need - addressed in this thread already - is an archiving function, but getting that right is highly non-trivial.

Collapse
Posted by Carl Robert Blesius on
.LRN 2.0 (release within a month) will have multiple ways to sync user data with external sources.

* Batch account synchronization though OACS-specified format
* Batch account synchronization through IMS 1.1
* Real-time account synchronization framework
* Real-time account sync driver for LDAP (not sure how far we are on this, but it is funded and Greenpeace is going to be using it)

Maybe we will find a champion to look into group (course and community) synchronization for .LRN 2.1

Disk space is getting cheaper on a monthly basis and often the value of "retired" user contributions warrants the fractions of a cent it may cost to save it. Deactivation fits our relational data model better than deletion, but if we are forced to removed a user and all dependent contributions it might make sense to actually leave something in the DB (e.g. "users contributions deleted") so that other members of the community are not confused (e.g. if your contributions to this discussion disappeared, it would make it confusing for a new user).

Collapse
Posted by Raad Al-Rawi on
Just thought I'd throw in a couple of points...

We are currently carrying out a survey of over 3000 PhD students, for which we are using the survey package under our main dotLRN instance.
The point here is that these students are not existing users, nor will we expect them to necessarily use our dotLRN system once the survey has completed (we use the community aspects, not the course management features). When the survey is finished, we will have all these users in the system with no way to remove them permanently. Not a huge issue, but some bits of the system might get sluggish with a large enough number of users.

I guess the process of getting students created and registered with the appropriate class (using existing data) is what most people/universities moving to dotLRN would be interested in doing?
If so, I noticed that bulk registration of users in dotLRN is quite slow, so even if you could pick out the relevant details (user, course, class) to use, moving a large number of students into dotLRN could be lengthy!

Admittedly we are using an old version of dotLRN, and I would be interested in how the latest version compares. It takes about 20 seconds to create the user, register them with dotLRN, set the browse permissions and add them to a community/group.

Overall, I think the issue with keeping users may be more about performance than about storage space, plus the point about contributions is a valid one. Perhaps there could be some way to delete users who don't have any content associated with them (like ad_remove_user for ACS)?

<Raad>