Forum OpenACS Development: Re: Get number of visitors currently browsing the site

Collapse
Posted by Gustaf Neumann on
Malte, the two xotcl files should not be loaded
by the RP, but during startup of the server.
if you make a "make install-aol" from the xotcl sources,
a file xotcl.tcl is installed that handles .xotcl files
in the modues/tcl/ directory.

yes, for the thread::mutex, Zoran's Tcl Thread Library
is needed (as indicated in the README file).
If you use the code on recent linux kernels,
make sure to use a recent tcl threads version (2.6),
earlier versions caused troubles at least with FC2.
i added the mutex lock just recently, it is just
needed for very busy sites, where the server receives
a high number of requests immediately after startup.
Before that we used the code for a long time without
noticing the problem.

We use the tcl thread library since a few years with
the aolserver, it never caused any troubles or
instabilities.

After providing the package, i added on a few more things:
- caching of the dns lookup. successful dns lookups are
already cached by the core, but the lookup for
non-resolveable names can be quite slow.
- we want to prevent user-id sharing to some degree.
Therefore i added some code to track whether a
uses accesses the site within a time window
simultanous from multiple sites. A single switch
happens frequently, multiple switches are suspicious.
- for measuring a users activity i added a exponential
fitted value. This might lead to a simpler and better
way to block overactive clients..
This stuff is not finished yet...

season greetings
-gustaf

Collapse
Posted by Orzenil Silva Junior on
Hi Gustaf,

I'm trying load aolserver4 with libthread2.6.so from tcl threads 2.6 and these errors are occurring (aolserver error log):

[30/Dec/2004:15:41:43][8945.1024][-main-] Notice: modload: loading '/usr/local/aolserver/lib/thread2.6/libthread2.6.so'
[30/Dec/2004:15:41:43][8945.1024][-main-] Warning: modload: could not find Ns_ModuleInit in /usr/local/aolserver/lib/thread2.6/libthread2.6.so
[30/Dec/2004:15:41:43][8945.1024][-main-] Fatal: modload: failed to load module '/usr/local/aolserver/lib/thread2.6/libthread2.6.so'

threads2.6 was installed as a loadable module for the AOLserver with this config options:

aoldir="/usr/local/aolserver"
../configure --enable-threads --with-aolserver=$aoldir \
--prefix=$aoldir --exec-prefix=$aoldir

In my aolserver config.tcl I have:

ns_param libthread ${homedir}/lib/thread2.6/libthread2.6.so

Maybe this error is related with this warning in threads CONFIG:

#
# AOLserver uses its own package loading mechanism.
# To load, just do "ns_eval package require Thread"
# at the AOLserver startup or later from any thread.
#

but when i add "ns_eval package require Thread" to aolserver4 config.tcl the server doesnt load and doesnt write to log file.

What am i missing?

Collapse
Posted by Gustaf Neumann on
Orzenil,

when a c module is loaded via the config-file,
it calls after loading Ns_ModuleInit. This was the only
way to load a c module for aolserver3. For aolserver4,
this is less important, not to say deprecated, since
one should be able to use a package require. So,
proceeding as malte suggests should be ok, as long the
package require happens "early engough".

I am still puzzeled about you messages, since
when i do the same configure, -DNS_AOLSERVER=1 is
added to the list of defines, and Ns_ModuleInit
is part of the .so file. (nm libthread2.6.so|grep Module)

-gustaf

Gustaf,

It is very strange! The only way i could have libthread2.6 compiled with Ns_ModuleInit was changing by hand generic/aolstub.cpp to

/* #ifdef NS_AOLSERVER */
#include "/usr/local/aolserver/include/ns.h"

althroug my CONFIG was passing parameter "--with-aolserver" the var NS_AOLSERVER was not setting. I do not know why.

This way i could have as a result from "nm libthread2.6.so|grep Module":

00001654 T Ns_ModuleInit
00011b6c D Ns_ModuleVersion

and libthread2.6.so module was loaded correct by aolserver when declaring "ns_param libthread ${homedir}/lib/thread2.6/libthread2.6.so" in modules section.

Problem now is with throttleThread 😟

When I put your modules/tcl/*xotcl files in my aolserver dir modules/tcl/ and reload aolserver thats ok. Server is up but every page i browse nothing happens and this message is written in aolserver log errors:

"throttleThread do ::throttle check 200.181.95.53 200.181.95.53"
("eval" body line 1)
invoked from within
"eval [my server] do [self] $args"
(procedure "do" line 3)
::throttle ::THREAD::Client->do
invoked from within
"my do check $requestor $pa"
(procedure "check" line 11)
::throttle->check
invoked from within
"my check"
(procedure "postauth" line 4)
::throttle->postauth
invoked from within
"throttle postauth"

(xotcl 1.3.4 is installed with make-aol; xotcl.tcl is in $aoldir/modules/tcl/ and *xotcl files are sourced because when i remove them the website pages display fine without any error):

Maybe errors in throttle_mod.xotcl script (as Malte observed)?

Thank you for your patience.

Collapse
Posted by Gustaf Neumann on
To shorten this thread, i hacked together my first apm
file
http://media.wu-wien.ac.at/download/xotcl-stuff-0.3.apm
that should work for some more people. It still assumes
that XOTcl and the Tcl Threads 2.6 are installed in place,
the two .xotcl files are changed into -procs.tcl files.

yes, there was a problem with the old version and the standard RP,
it was just working with plain AS and the
modified RP of learn@wu. I just installed on my notebook
a fresh version of OACS, and it seems to work fine....

the name is not polished, but it indicates that this
package contains different things (xotcl-thread support,
request monitor with throttleling support, value-added whos-online).
At least the first one should be moved into
a different package...

happy new year
-gustaf

Collapse
Posted by Jade Rubick on
Gustaf: are you willing to release this package in CVS?
Collapse
Posted by Gustaf Neumann on
Sure. First it should run without much tweaking,
then it should be splitted into two parts (xotcl
tread handling should not be part of a request
monitor), then adding some configuration options
and finally put it into cvs... and well, the docu
should be improved, and we have to find a way to
integrate the docs with the api browsers.

For the first part, Zoran has released a version
of libthread 2.6 that has the configuration
problems fixed (called 2.6.1). With that
the thread issues like the ones from Orzenil
should be resolved (got a positive feedback from
him).

http://prdownloads.sourceforge.net/tcl/

There is a version with improved functionality
(e.g. recording site switching, user tracking):

http://media.wu-wien.ac.at/download/xotcl-stuff-0.4.apm

Best regards
-gustaf

Collapse
Posted by Orzenil Silva Junior on
Jade: after some coordination with Gustaf he helps me to get xotcl-stuff package working. I think features included in xotcl-stuff-0.4 are very useful and Gustaf is working on it to get it better finding a way to document xotcl classes/objects/methods similar to ad_proc and add parameters to the package.

Thanks, Gustaf.

I am working now in a brief tutorial about how Gustaf help me to install requirements for xotcl-stuff and soon i will post it in this forum. Maybe other people could try it.

\Orzenil

Orzenil,

i am still puzzled about the -DNS_AOLSERVER.
it looks, as we are working with different versions.
my version is from sept 2004 or so from CVS, the
newest update is from aug. 2004. Can you check yours
as well?

thanks, -gustaf

localhost:~/import/thread2.6> find . -type f -exec fgrep RCS: \{} \;
...
* RCS: @(#) $Id: threadPoolCmd.c,v 1.26 2004/08/02 20:27:11 vasiljevic Exp $
...

Collapse
Posted by Orzenil Silva Junior on
Gustaf,

My threadPoolCmd.c version is newer than v 1.26

* RCS: @(#) $Id: threadPoolCmd.c,v 1.28 2004/11/27 06:11:02 vasiljevic Exp $

i downloaded threads2.6 from tcl.sf.net: http://prdownloads.sourceforge.net/tcl/thread2.6.tar.gz?download

following changes were introducing in v 1.28 over v 1.26 (directly from cvs):

http://cvs.sourceforge.net/viewcvs.py/tcl/thread/generic/threadPoolCmd.c?r1=1.28&r2=1.26&diff_format=u

More info about my environment:
debian gnu/linux ( linux kernel: 2.6.5-1-686)
tcl8.4.5
aolserver4.0.0r2

Unfortunately xotcl-stuff still doesnt work in my openacs-5-1-3 + dotlrn-2-1b install. Errors:

- http://servername/status/index

invalid command name "seconds"
while executing
"seconds set trend"
invoked from within
"thread::send $tid "$o $args""
(procedure "do" line 37)
::throttleThread ::THREAD->do
invoked from within
"throttleThread do $t set trend"
(procedure "counterTable" line 4)
invoked from within
"counterTable Views [list seconds Second minutes Minute hours Hour]"
invoked from within
"set views_trend [counterTable Views [list seconds Second minutes Minute hours Hour]]"
("uplevel" body line 102)
invoked from within
"uplevel {

# present usage statistics, active users, etc

...

- http://servername:8000/status/whos-online

invalid command name "Users"
while executing
"Users totalInfo"
invoked from within
"thread::send $tid "$o $args""
(procedure "do" line 37)
::throttleThread ::THREAD->do
invoked from within
"throttleThread do Users totalInfo"
invoked from within
"foreach element [throttleThread do Users totalInfo] {
foreach {user_id pa timestamp hits} $element break
if {[string first . $user_id] > 0} {
..."
("uplevel" body line 49)
invoked from within
"uplevel {
ad_page_contract {
Displays who's currently online

thanks,

Happy new year

.: My wish for 2005: learn xotcl ("exotickle" 😊 )

Gustaf, the reason I talk about the RP is the fact that I want to take your code and make a decent OpenACS Package out of it, which can be installed using the APM (taking into account the prerequisits).

I too experience the problem of Orzenil, though I managed to get the thread::mutex command to work by just adding the package require in to the 0-acs-init.tcl. And this is the way it should be with AOLserver4 (using package require instead of loading library files manually).

Sadly though, .xotcl files are sourced before the .tcl files in /tcl, so I'm stuck again, but I managed to get this to work by adding the packagerequire to your thread_mod.xotcl right at the beginning.

So far so good, but now I'm stuck as you have a typo in your code due to the fact that you seem to have forgotten to open a paranthesis correctly "} persistent" is the culprit.

And now I got a command not found ::xotcl:: and after manually copying the code of xotcl.tcl over to my webserver I get "invalid command name throttle".

So, assuming that you have a new version anyway, I will give up at this point and look forward to see your code in the new year.

Malte, i see. look into modules/tcl/xotcl.tcl to understand,
what happens during loading. The .xotcl files are not
sourced after the .tcl files in general, since they are sourced by a .tcl file.
There is no big magic involved. If you have xotcl.tcl still
under modules, and you load libthread at some later time,
and you source thread_mod.xotcl before that, it won't work.

It should be possible to source xotcl.tcl in 0-acs-init.tcl
after the "package require Thread" (if this is the right
place, and the files there are ns_evaled). It should also
be possible to add the "package require Thread" to the
begin of thread_mod.xotcl (with the disadvantage that it
will cause problems with aolserver 3.*).

i can't diagnostic the other error messages. i have also
downgraded my notebook to the files of the 0.3-tar file,
but i did not get the parenthesis mismatch...

-gustaf