Forum OpenACS Q&A: Re: My production site is down

I will begin with the assumption that daemontools has been installed and configured to run the target site. I will fill in more of the basic daemontools details if requested. Rest assured that this is not as complicated as it looks. This could probably be explained with far fewer words than I use below.

If you think this idea is useful, it could be generalized and incorporated into the system logging procedures very easily. This would allow selectively logging any package's messages to a separate file.

Do yourself a favor and install the daemontools man pages if you have not already done so. Get them from http://smarden.org/pape/djb/manpages/daemontools-0.76-man.tar.gz.

So, you should already have a /service directory, and there should also be a symlink /service/site that points at the site's root directory. Following this symlink and listing the directory contents should show that a 'run' file exists. This 'run' file contains the deamontools startup script for your site.

BASIC MULTILOG CONFIGURATION

Create a symlink to your physical logging location at the root of your site, and configure the log directory and run script:

# cd /service/site
# ln -s /var/log/site log
# mkdir log/error
# cd log
# touch run
# chmod 775 run
# emacs run

======== run file content ========

#!/bin/sh
exec setuidgid aolserverUsr \
aolserverGrp \
s3999999 ./error

(note that aolserverUsr/Grp are the user and group you've configured to run the site service. Older install docs suggest nsadmin:web, but the newer docs suggest a more secure setup)

Restart the site, set supervise ownership, and verify that basic logging is working as expected:

# svc -d /service/site
# svc -u /service/site
# chown -R aolserverUsr.aolserverGrp .
# chmod -R g+rwx supervise

You should now check the contents of the log/error directory. The 'current' file will grow with error log content and will automatically roll somewhere around 3999999 bytes. By default, multilog will keep the latest 10 log files but this number is configurable.

You can roll the logs manually, if you wish:

# svc -a /service/site/log

PACKAGE-SPECIFIC MULTILOG EXTENSION

This setup will allow you to log package-specific messages to a separate log file. This is in addition to the main log file configured above. The key to making this work dependably is to format your log messages with a unique prefix string so that multilog can then find them and place them in a separate file. This is extremely useful in developing a package since you don't have to peruse voluminous, unrelated log messages looking for the ones you've logged from your package. My setup includes the following.

Two package parameters:

debug default = "1"
logPrefix default = "!pkgmain"

nsv accessor procedures:

pkg::debug
pkg::logPrefix

At package initialization time, I place the debug and logPrefix values into package-specific nsv variables so that they can be changed dynamically at runtime.

A log procedure:

pkg::log

I wrap my log/debug messages inside a check for debug:

if {[pkg::debug]} {pkg::log -msg ...}

Note that, if you wish, you can turn debug logging on/off and make the prefix unique for *each* instance of a mounted package by changing the parameters for any mounted instance in the site-map.

Change the log/run file:

======== run file content ========
#!/bin/sh
exec setuidgid aolserverUsr \
aolserverGrp \
s3999999 +* ./error \
-* +*!pkg*:* ./pkg

(note that this will log all messages containing the wildcard string "!pkg*:" to the directory log/pkg. Active logging will be to file 'current', and these logs will roll just like the log/error directory does.)

SUPPORT CODE

========= pkg-init.tcl =========

nsv_set pkg_global logPrefix [pkg::parameter -name logPrefix -default "!pkgmain"]
nsv_set pkg_global debug [pkg::parameter -name debug -default 0]

========= pkg-procs.tcl =========

ad_proc -public pkg::log {
{-msg:required}
} {
Logs pkg messages with a specified logPrefix,
where logPrefix is an pkg package parameter settable at the Site Map
package mount point. 

NOTE: if !pkg is used as the first part of logPrefix and multilog (part of
Daemontools package) is being used to manage the service error log, then a line like the
following can be used to append all pkg log messages to separate "pkg" log: 

exec setuidgid nsadmin multilog s3999999 +* ./error -* +*!pkg*:* ./pkg

@param msg the string to be logged.
@return nothing.
} {
ns_log notice "[nsv_get pkg_global logPrefix]: $msg"
}

ad_proc -public pkg::debug {
} {
Returns the state of global pkg debug boolean (actually stored in pkg_global nsv).
This value is used throughout the pkg application to turn on verbose debug logging.
This value can be set through the APM interface as 'debug' is an pkg parameter.

@return true (1) or false (0) depending on the pkg 'debug' parameter setting.
} {
return [nsv_get pkg_global debug]
}

ad_proc -public pkg::parameter {
{-name:required}
{-default ""}
} {

} {
return [parameter::get -package_id [get_package_id] -parameter $name -default $default]
}