Forum OpenACS Development: Users who have requested a page in the last 10 min. ? (acs-core)

In Acs 3.x their was a Page Whos-online.tcl wich stored ,the user_id's with time stamps in a global nsv_set,  of those user who requested  the Whos-online.tcl.
To store the user_id's of everyone who has requested ANY Page from the Server, a peace of code (from the Whos-online.tcl, the peace that fils up the nsv_set) had to be placed some where in the acs-code wich is proccessed every time a user requests a (ANY) Page (like the  request-proccessor) from the Server.
The user_id's in the nsv_set should last for about 10 min.
(time stamp).
Is their a possibility to do somthing like this in OACS 4.x , to know who is 'online' at the Webserver. ?

I put the peace of code from acs 3.x below, so you can have an impression how we did it.

This function is from the tcl-library file (3.x)

packages/education/education-procs.tcl

it is called every time a user requests a (any) page.
Here we fill up the nsv_set, we clean up the nsv_set (removing of user_id's wich have a time stamp older then 10 min. ) on the whos-online.tcl ,but this could be done in the core acs too.
Maybe with an scheduled proc so the nsv_set doesn't need to be cleand up every time some one requests a page.

proc edu_redirect_nonlogged_in_users_to_registration_page {conn args why} {
    set url(1) [lindex [ns_conn urlv] 0]

    if { [empty_string_p $url(1)] } {
        set user_id [ad_verify_and_get_user_id]
        if { $user_id==0 } {
            # if user is not logged in, redirect him to the registration page
            ad_returnredirect /register/
        } else {
###########################################################

            # log the hit for "whos-online"
            nsv_set last_hit $user_id [ns_time]
            # if user is logged in, redirect him to his home page

###########################################################
            ad_returnredirect "[portals_get_portal_property base_url [portals_get_portal_id user $user_id [portals_domain_id_from_domain_name\
users]]]"
        }
        return filter_return
    }

    set url(1) [lindex [ns_conn urlv] 0]
    if { [string compare $url(1) register]==0 } {
        # we want to allow non-logged in users to register
        return filter_ok
    }

    if { [string compare $url(1) users]==0 } {
        # all users are able to see user homepages
        return filter_ok
    }

    if { [string compare $url(1) mail]==0 } {
        # all users are able to see user homepages
        return filter_ok
    }

    if { [string compare $url(1) survey]==0 } {
        # all users are able to see surveys
        return filter_ok
    }
  if { [string compare $url(1) graphics]==0 || [string compare $url(1) /education/graphics]==0 } {
        # graphics should no be checked
        return filter_ok
    }

    if { [ad_verify_and_get_user_id]==0 } {
        set return_url [ns_conn url]
        ad_redirect_for_registration
        return filter_return
    } else {

##########################################################

        # log the hit for "whos-online"
        nsv_set last_hit [ad_get_user_id] [ns_time]
        return filter_ok

##########################################################
    }
}

Thank's for your help.

I would like to have this feature back in OpenACS 4.x, yes, I miss it from 3.x, and the request processor would be the place to do it.

How do others feel?

If folks agree, would you be willing to work up a patch, Bjoern, including a display page for the admin pages?

Instead of an nsv I think we should use an ns_cache with a timeout, then the pruning would be handled automatically.
Wouldn't it make sense to store the subsite as well, if possible? So that one cannot only ask for the site-wide list of online users, but also for a list of online users for a specific subsite?

This would propably involve a call to [site_node_closest_ancestor_package "acs-subsite"] - which doesn't access the db, just digs in the site_nodes nsv - per request. Acceptable?

Should we maybe be using util_memoize for the caching so that clustering (multiple web servers) is properly handled?
Looking into util_memoize, I wonder where it deals with server clusters - I just see calls to ns_cache. Did this functionality disappear or am I looking in the wrong place?
Til, iirc it only sync's flushes (look for ncf.send in the code).
There is a whos-online page in 4.x in acs-subsite/www/shared/ but it checks the last_visit field from the users table, which I don't think gets updated on a page request, but only on login.
This is right the last_visit field get's updated on Login only, thats our problem.

I couldn't start the work on the patch to day, I'll do it tomorrow. I thing  doing it with an ns_cache object is an good idea.
How can I find out how many  different subsites (the URL's they are running on) are on the System?
Having an ns_cache for each subsite could increas the performance I thing, the subsite check would only needed to be doen once  (to find the right ns_cach object), instead of checking the URL for every user(_id) in one single ns_cache object.
We then would have one ns_cache object
online_subsite_users
Wich holds the ns_cache objects (values) for the different subsite Url's (Key's),  with user_id (key) and timestamp (value).

I just checked the ns_cache doc again an it locks like the timeout (if secified) is ment for the whole ns_cache object. Meaning the whole ns_cache object will be cleand up after 10 min. not just the enrys older then 10 min.

I may be completely off here, but should we maybe update the last_visit column on request (not on every request, maybe only if last recorded value is older than say a few minutes), and get the last visits of the latest 10 minutes from the db wrapped in a util_memoize. That way we have a solution that works with clustering.
O.k. I did it much like we did it in acs 3.x..

Updating the db and working with a util_memoiz is a fine idead, but we don't want to hit the db for every single request.
To let this work with clusters we would have to put this information in the db, for standalone servers this patch will work.

In case their is more interest to let it run on clusters too, we might just append the Body of

proc ad_get_online_user_ids
proc ad_remove_offline_users_from_last_hit

to check wether it's a standalone server (doing it with an  nsv_set)or a cluster (query the db , using a util_memoize)

Patch:
Add to the bottom of the file
/web/server/acs-tcl/utilities-procs.tcl

The folowing two proc's:

proc ad_get_online_user_ids
#returns a tcl list with all user_id's from those users wich have requested a page in the last 10 min.

proc ad_remove_offline_users_from_last_hit
#will be called by a scheduled proc every 1 hour it removes old entrys from the nsv_set

ad_proc ad_remove_offline_users_from_last_hit {} {
      Removing all user_id's from the last_hit (nsv_set) wich have a time Stamp older than 10 min.
    } {


        array set last_hit [nsv_array get last_hit]
        set onliners_out [list]
        set oldtime [expr [ns_time] - [ad_parameter LastVisitUpdateInterval "" 600]]

        for {set search [array startsearch last_hit]} {[array anymore last_hit $search]} {} {
            set user [array nextelement last_hit $search]
            set time $last_hit($user)
            if {$time<$oldtime} {
                lappend onliners_out $user
            }
        }

        array donesearch last_hit $search

        for {set i 0 } { $i < [llength $onliners_out]} {incr i} {

            nsv_unset last_hit [lindex $onliners_out $i]
        }


    }

ad_proc ad_get_online_user_ids {} {
                    This function returns a list of user_ids from users wich have requested a page
                    from this Server in the last 10 min
} {


        array set last_hit [nsv_array get last_hit]
        set onliners [list]
        set oldtime [expr [ns_time] - [ad_parameter LastVisitUpdateInterval "" 600]]


        for {set search [array startsearch last_hit]} {[array anymore last_hit $search]} {} {
            set user [array nextelement last_hit $search]
            set time $last_hit($user)
            if {$time>$oldtime} {
                lappend onliners $user
            }
      }

        array donesearch last_hit $search

      return $onliners

}






In file
/web/server/packages/acs-tcl/request-processor-procs.tcl

in function 'ad_proc -private rp_handler {}'
Line 712
add this lines

#log the hit for "whos-online"
  set user_id [ad_conn user_id]
#ensure that the user has a user_id
  if { $user_id != "0"} {  nsv_set last_hit $user_id [ns_time] }

to update the nsv_set on every http request.

In file
/web/server/packages/acs-tcl/request-processor-init.tcl

at the beginning of function 'ad_after_server_initialization procs_register {'
insert the following lines

#schedule the proc to cleanup old entrys from the nsv_set last_hit
ns_schedule_proc -thread ad_remove_offline_users_from_last_hit 3600

Save changes and restart the server.

Here is a littel tcl/adp script to check wether every thing worked.

tcl:

multirow create onliners user_id

array set last_hit [nsv_array get last_hit]

multirow create onliners_all user_id time

for {set search [array startsearch last_hit]} {[array anymore last_hit $search]} {} {
  set user [array nextelement last_hit $search]
    set time $last_hit($user)
    multirow append onliners_all $user $time

}
array donesearch last_hit $search

set user_ids [ad_get_online_user_ids]

for {set i 0} {$i < [llength $user_ids]} { incr i} {

    multirow append onliners [lindex $user_ids $i]

}

ad_remove_offline_users_from_last_hit

ADP:

<master>
<property name ="title">Jabber</property>
<property name="context"></property>

All users in nsv_set
<ul>
<multiple name=onliners_all>
<li>USER_ID: @onliners_all.user_id@ TIME: @onliners_all.time@</LI>
</multiple>
</ul>

only online
<ul>
<multiple name=onliners>
<li>USER_ID: @onliners.user_id@ TIME: @onliners.user_id@</LI>
</multiple>
</ul>

Hi,
I posted a bug (#332) and a patch(#110) to the bug tracker on openacs.org.
I've rewritten the patch wi now work with the namespace util::whos_online, as well it's possible for users to set them self invisible.

-public
util::whos_online::user_ids          (returns a list of user_id's of those users wich have requested a page in the last 10min.)
util::whos_online::all_user_ids      (like ::user_ids but even those user_ids wich are invisible are returned)
util::whos_online::all_invisible    (return all user_ids wich are invisible and online )
util::whos_online::set_invisible  $user_id        (Set the user_id to invisible, this state alters when a user gets offline )
util::whos_online::unset_invisible $user_id      (Resets the user_id to visible)
util::whos_online::check_invisible $user_id      ( returns 'true' if user_id is invisible and 'false' otherwise)

-private
util::whos_online::flush    (Cleaning the internal used nsv_sets, run by a Scheduled proc every hour)

The new patch has number #126 .