Forum .LRN Q&A: New user tracking package

Collapse
Posted by David Ortega on
Hi all,

In the scope of the E-Lane project [http://e-lane.org/] we are going to develop a new package whose main functionality will be user tracking.

The purpose behind this development is to ensure we track and retrieve knowledge about how end users are making use of the system. The main objective of this package will be to track user behaviour and activities, so that Professors and administrators can use this to understand how the system is used and make improvements in the learning process.

Briefly, our package will be able to generate reports about: users, classes, communities... and also about departments or subjects (because It will be possible to add the information available about each class).

Scalability is an important design consideration for us, so we are going to use a log analyzer, awstats [http://awstats.sourceforge.net/] to obtaint most of the data about: objects views, user logging tracking, sessions history ... We are also going to query the database to get data about creation of objects such as: forums, messages, news, comments...

Any suggestions or comments are more than welcome!

Greetings,

David

Collapse
Posted by Malte Sussdorff on
If I understand correctly you need to keep track of:
  • user_id (who has done an action)
  • package_id (where was the action done)
  • url (what was the action)
  • object_id (on what object was the action done)
  • date (when was the action done)
To improve performance you might want to add
  • community / class_id
  • package_type
I'm not entirely sure where you want to store this information for each request. But I'd suggest to keep it seperate from your main database. Either store it in a flat file (like you approach with appending values to the logged information in the logfile suggests) or use a different database with seperate database connections on a seperate server (read: data warehouse). The latter would allow you to make use of this information even if you are loadbalancing the system. Furthermore you could execute long queries on the second database without an impact on the main database serving the site.

As we are in the lucky position to have a smart AOLserver, you can actually start with the creation of a new database pool (call it e.g. "user_log") connecting to a seperate tablespace (or user with postgres), which does not require you to have yet another physical server for logging especially in the development phase, but allow you to make use of this seperation at a later stage when load is high.

At one point in time it might even make sense to dublicate some tables (e.g. users, groups), as this would prohibit you from having to call the real database if you want to get the user's name out of the user_id.

Collapse
Posted by Malte Sussdorff on
Oh, and I totally forgot: Great news you work on this. I'm pretty sure some institutions will make very good use of this tool.

One last thing: You might be legally able to collect all this information, but I assume you are not allowed to make use of it in every way. So it would be great if you could disable each bit of data collection and reporting on a seperate administration page.

Collapse
Posted by David Ortega on
Hi Malte, thanks for your comments.

Our idea is to add user_id and community_id to each request stored in the logfiles. Awstats will use REGEXP to filter the URL and then to recognize objects visited, e.g. when a user wants to see a message of a forum, it can be identified by message-view?message_id=XXXXXX.

As first approach, we are going to use the data provided by awstats. This software, extracts the important information of logfiles, and stores it in formatted text or xml files.

Collapse
Posted by Don Baccus on
i think extending the log file will work well and of course will scale very nicely.
Collapse
Posted by Orzenil Silva Junior on
Hi David,

Congratulations to E-lane people initiative in providing a user tracking package for openacs/dotlrn.

I'm working with oacs-5-1 branch with dotlrn - thanks Tracy to tag it with alpha ! - because of LORS and made some presentations here in Brazil for people interested in LMS systems. They really agree SCORM is a great feature in dotlrn - I made some presentation with reload scorm editor - but all of them claims about lacking of user tracking system.

I read your report and if you and Rocael want count on me to some help - maybe alfa/beta tester, documentation, improvement on tcl or adp front, testing data models on postgresql ...

BTW, do you know who i could contact in UNICAMP/Brazil about E-lane? I saw UNICAMP is a partner of E-lane here in Brazil and i would be informed about what they are doing.

Collapse
Posted by Vinh Chung on
Hi David,

Im an undergraduate student of Uni of Sydney and im currently doing my thesis which is similar to the project you have at hand. My goals is to report users details in .LRN to the site's professor or administrator so that they could provide better contents or services to the students. Currently i would like to report on: how many times has the user visit a certain page, their last visit date, how many users logged on each week, and how many registered users in the class.

Is there anyway that i can help?

Cheers,

Vinh

Collapse
Posted by David Ortega on
Hi Orzenil and Vinh,

Before nothing, thanks for your interest and for your offer to help ourselves.

Nowadays we are taking our first steps regarding the development of the package. Our first aims are to modify the perl script of awstats to adapt it to our needs and to create the infrastructure of the package. We will tell you when we advance a little more and then all of us can see how we can collaborate.

Orzenil, we spoke to UNICAMP about your interest so I suppose that they will contact with you.

Best regards,

David

Collapse
Posted by xx xx on
Is this package going to be generic? Could it become an important addition system usability testing of .LRN and openACS?