Forum OpenACS Q&A: Monitoring dynamic pages viewed when logged in?

OpenACS already has a feature that allows you to monitor the static
content users have viewed when logged in, but does it have a way to
monitor the dynamic content they have viewed?

Hi James,
I am also interested in the answer... What I have used for tracking Real audio useage from another server is do inserts in a log table with the user_id, cut and datetime before redirecting the user to the audio file.

I want to expand this to include specific (dynamic) page views by specific users. There are two possible ways of doing this:

  1. Continue same model with INSERTS into the log table after the page is rendered with
    ns_return 200 text/html ....
  2. Or use UPDATES into table (Inserts if new) based on user_id and have an integer field that is incremented to show number of times that user has viewed the page and restamp the datetime.

So, which method is more efficient? Are there better ways to do this that will allow for moderate to high usage?

TIA
-Bob

Collapse
Posted by Stephen . on
This is a problem begging for a common solution. There are many places in the toolkit that need to record who viewed what when information (clickthrough, banner ads/ideas, bboard new stuff, new stuff in general, static pages, session tracking...).

Ad fixed up the AOLserver nslog module to support logging arbitrary information. Good candidates would be session_id, user_id, package_id, node_id etc.  Periodicaly load the log file into the database for analysis or use by packages. There's a data wharehouse package somewhere...

Some packages have real time requirements for their logging. Maybe all you can provide is a common interface for logging with an -immediate switch, and a common back end for storage and analysis.

James, the first question you need to ask yourself is, "How quickly do I really need this access log data?" In other words, do you really need to log the info into your RDBMS immeditely, or can you you write it to a log file, and then parse it out later and load it into your RDBMS in a batch process?

Stephen, I hadn't heard of any aD enhancements to the ns_log module. Can you fill us in, please?

Back when we were discussing and implementing a user tracking feature for a site I worked on at aD, the consensus was that logging to the AOLserver acess log and then parsing the info out later is almost always the way to go. (This should always give you better scaleability than trying to stuff non-time-critical user tracking into your RDBMS on the fly.) Vadim Nasardinov came up with a simple (if non-intuitive) way to log arbitrary stuff like user_id to the AOLserver access log, but that didn't require any changes to the stock ns_log module that came with AOLserver 3.2+ad12.

There is no ACS data wharehouse package! There is Richard Li's old pre-alpha Clickstream package (and his Master's thesis), but while that's definitely worth learning from, you probably wouldn't want to run it as is.

Perhaps he's referring to Rob's support to log extendedheaders:
ns_section ns/server/${server}/module/nslog
ns_param   extendedheaders  host
This directive within the config.tcl will direct nslog to log the contents of the host header if it is present.

What you can do is add your own headers to the request headers of a conn within a registered proc or page, and have nslog log those. Something like this:

. . .
ns_set put [ns_conn headers] user_id $user_id
ns_set put [ns_conn headers] shopping_cart_total $shopping_cart_value
. . .
and
ns_section ns/server/${server}/module/nslog
ns_param   extendedheaders  user_id
ns_param   extendedheaders  shopping_cart_total
Should cause the user_id and shopping_cart_total to be placed into the log for your later analysis.

Along those lines, I patched my nslog module to look for the header X-NO-LOG. If found, it merely returns without logging the request at all. I then have various registered procs and pages add that header at times when I don't want to log a request. I use this to keep various keepalives, robots, and other kinds of entries out of my logs to help keep them a bit more tuned to my needs.

Collapse
Posted by Stephen . on
The LogExtendedHeaders option doesn't seem to be mentioned in the config file reference or the docs...  Takes a comma separated list of header names (input headers, from ns_conn), so not exactly like I remembered simply loging Tcl vars, but the same result.

Yes, the Clickstream package. A Data Wharehouse 'package' came with ACS3. Neither are plug n play...