Forum OpenACS Q&A: Server Log Format

Collapse
Posted by MaineBob OConnor on
What format does AOLServer use for it's server
logs and does it conform to one of these formats?

My friend Paul has NetTracker....

"NetTracker is configured to allow users to select from the following
log formats: IIS Standard / Extended Log Format, Lotus Domino / IBM
ICSS Log Format, NCSA Common / Combined Log Format, Netscape
Flexible Log Format, Open Market Extended Log Format, W3C Extended
Log Format, and the WebSite Log Format. NetTracker can also process
GZipped log files."

Also,  do I need a cron job to copy the log files from the protected
area to ./www web accessable area?

I'm using: OpenACS 3.2.2, AOLServer 3.0, Postgres 7.0.1.
Has anyone used NetTracker or another program like this?
Any hints for tracking visitors coming to our site?

TIA
-Bob

Collapse
Posted by S. Y. on

What format does AOLServer use for it's server logs and does it conform to one of these formats?

I don't remember what format AOLserver nslog defaults to, but you can certainly set it explicitly under the nslog section of your nsd.tcl file.

    ns_section "ns/server/yourserver/module/nslog"
    ns_param logcombined true
    

which corresponds to NCSA combined log format.

I'm currently running Apache and freeware Webalizer and there's no need to copy server log files to a web-accessible area of the html document root. My log analysis cron job runs as root, so it really doesn't matter where the output files land. With Apache, you can alias directories (e.g., /usr/local/webalizer/htdocs is aliased as "/usage/" for the web server). I forget if AOLserver can do this, but you can always symlink to the target directory 9 (i.e., ln -s /web/yourserver/www/admin/monitor/webalizer /usr/local/webalizer/htdocs).

Any hints for tracking visitors coming to our site?

You will have to diddle with the settings yourself. You may want to group everything from the *.aol.com domain. There are probably sections for ignoring *.jpg, *.gif, and *.css files. You may need to ignore certain referrers (e.g., localhost, if you do development and production on the same box or if you run a search engine like htdig). You may want to ignore certain IP addresses and hosts (e.g., your company's firewall, your home DSL box, etc.).

The best is to set it up, let it run, and tweak the configuration over a week or two until you're getting the kind of reports that you/your client wants. Run the log analysis when traffic is lowest (for me, that's 4 A.M. Pacific Time).

Good luck.

Collapse
Posted by Krzysztof Kowalczyk on
Sean, did you have any problems with Webalizer? I remember that when I was trying to use it, despite giving it my best, I failed to define aliases (reporting different web sites as one) so I switched to Analog but it was a few months ago so maybe that has changed or maybe I was just a looser.

I don't know which format AOLserver uses but if I don't specify format in nsd.tcl and don't specify format in Analog's config file (I use Analog 4.11), Analog does the analysis just fine (I run it once a day from a cron job).

Collapse
Posted by S. Y. on

Sean, did you have any problems with Webalizer?

I have had no problems with Webalizer. The DNS cache works, the gd-based PNG graphing works, the aliasing works (GroupSite function for Webalizer), the MangleAgent function works. I looked at analog, but decided that I liked Webalizer's spiffier looking reports.

I don't know which format AOLserver uses but if I don't specify format in nsd.tcl and don't specify format in Analog's config file (I use Analog 4.11), Analog does the analysis just fine (I run it once a day from a cron job).

Okay, I read the documentation (nslog.html) provided with the AOLserver source (in the nslog directory). Verbatim from the Theory of Operation paragraph:

"The nslog module implements Common Log Format access logging. This format can be used by any web analyzer tool. It optionally supports NCSA Combined Log Format and supports log file rolling. The log files are stored in the server/server1/modules/nslog directory."

The Combined Log Format records accesses, referers and agents (a.k.a. browser software). At least with Apache, you can split this up into separate logs. If you're having problems getting Webalizer to process AOLserver access logs, you have a configuration problem. The data in the log (combined log format) is identical to that of Apache. I've had Webalizer process logs from both server software.

Collapse
Posted by Janine Ohmer on
We (furfly) have been using Nettracker for a couple of years now.
It's not open source but it's spiffy enough that I'm willing to
overlook that. :)

We've never had any trouble with it reading AOLserver access
logs.  You can tell Nettracker where to look for log files (via an
absolute path) so as long as they are on the same system and
the user running Nettracker has permission to read them, you
don't need to copy them anywhere.