Forum OpenACS Development: aolserver-errors

Unlike the creation of 100,000+ lines of Tcl code to form a "thin layer over an RDBMS" which we now call "OpenACS 4.5"!

AFAIK aolserver-errors.pl hasn't changed an iota since it was written.  One of our folks fixed the infinite loop bug (search the archives, as I must do soon so I can include it in our OpenACS 4.5 release).  You probably won't make the same mistake in Tcl, which after all is cleaner, less obtuse, doesn't come with a puzzle book, etc ...

I guess one must ask: are their Unices out there that come with Perl but not with Tcl?  I assume that Solaris is *not* one of them since Tcl was originally developed by Sun.

That's interesting, I wonder if there's any truth to the claim of tclsh insecurity?  It sounds like it could simply be a perl religionist's FUD attack.

Are any of the folks out there running non-critical OpenACS 4 sites able to plug this patch in and give it a whirl?

I wrote the perl version of watchdog.

The original version of watchdog was written all in tcl as a stand-alone aolserver service (like rollover). It was terribly slow. Processing a 1-meg span of error log took nearly 15 minutes. Rewritten as a perl script, that dropped to around 1 second.

This was back in the tcl 7.6 days, and the problem was that the old code relied on long lists (which were slow in 7.6), and many regexps, at least one for every line being read. Also, tcl 7.6 didn't pre-compile, which is a performance killer for an app that is basically a collection of regexps that get executed in a small loop many times.

Don said this is the type of job that tcl is made for. This is basically a text-munging job, and when I think of text-munging, I think of perl (I don't think I'm alone.). I knew perl didn't suffer from the two weakness that made the old code slow.

Another reason it's in perl is because I didn't know about the performance improvements that existed in 8.x at the time. Like most people at aD, my only interaction with Tcl was through AOLserver, so I never considered writing it as a tcl script to be exec'ed by a 8.x tclsh (which seems odd, but was probably the only way to get acceptable performance out of tcl for this task at the time.)

And regarding Jon's comment...

The original reason that this was written in Perl was that someone told the MIT boys that "perl==secure tcl==security flaw"

I've never met Jon, and I've certainly never talked to him about watchdog. I'm not sure where he gets his information.

I would appreciate it if someone could take the time to convert it or, if not, I'll
release a 4.5 patch when I get some time.

I don't think it does. I'm not really sure what the aolserver-errors.pl script does in that case, either.

Anyway, probably 99% of the time if the log has rolled it will be smaller than before. But that's not entirely certain. Is there any way to directly detect the fact that the log has rolled?

Here's a new code snippet with what I think is a partial solution to this log rolling issue:

if { $num_bytes > 0 } {
   set log_file_size [file size $log_file]
   set sizediff [expr {$log_file_size - $lastread}]

   # If the log has been rolled, presumably it will now be smaller
   # in size than it was last time.  So in that case, always read
   # from the beginning of the file.
   #
   # TODO: But, it isn't GUARANTEED that it will always be smaller.
   # Is there any other way for us to detect if the log has been
   # rolled or otherwise fooled with?  --atp@piskorski.com,
   # 2002/04/09 13:13 EDT

   if { $sizediff < 0 } {
      set lastread 0
      set sizediff [expr {$log_file_size - $lastread}]
   }

   if { $sizediff > $num_bytes } {
      set lastread [expr {$log_file_size - $num_bytes}]

      append output "Log file grew by [expr {round(100.0 * [expr {$sizediff / 1024.0 / 1024.0}]) / 100.0}] megabytes.
Reporting on the last $num_bytes bytes of log.
"

   } 
}

<pre>    if {![info exists lastread] || $lastread > [file size $log_file]} {
        set lastread 0
        set lastread_time "First Run"
    }
</pre>

I personally never really use the packages/monitoring/www/watchdog/index.tcl page at all, so fixing this isn't a priority for me, but as a stopgap you can get the correct functionality by, on that page, changing the call to wd_errors to use the old aolserver-errors.pl script instead, like so:

wd_errors -external_parser_p 1 -num_minutes $num_minutes -num_bytes $bytes  

This is now in CVS. I ported the monitoring package to PG as best as I can, given that much of it is Oracle specific (Cassandracle, etc). I added the new watchdog-procs file posted by Andy above. I've been using it for the past couple days (Oracle and PG) and it's reporting errors properly so far.

In regards to the monitoring package, I had to add a parameter to specify command-line options to send to 'top'. On my version (procps v. 2.07), I had to send it '-b -n 1'. I also had to adjust the top-scraper code, cuz the columns were in a different order. It should still work on whatever version it used to work (Solaris?).

Things to do:

1. figure out why ad_monitoring_analyze_tables isn't working in Oracle
2. try to create a PG replacement for Cassandracle
3. Use templating (currently it just spits out HTML in the tcl pages)