Forum OpenACS Q&A: Aolserver 4.5 multithreading

Most likely you have an older version of Linux that you run 3.13 on, and have a newer version of Linux that you run 4.5 on.

Read the man page on the "ps" command and look for info about threads.

You can run "ps -eLf" to see information about each thread, for instance. Running "ps -eLf | grep nsd" will show you that multiple threads are in fact running.

This article from IBM DeveloperWorks will have more information: http://www-128.ibm.com/developerworks/linux/library/l-threading.html?ca=dgr-lnxw07LinuxThreadsAndNPTL

Thanks
Jay

Problem:
Server chokes under havy load and clients getting time out.

We have added few days ago entries in order to increase default pool min and maxthreads as you will notice in the config file.

What we want to achieve:

We would like to be able to create new threadpools and map them to certain sections of the site. ie. have a dedicated pool for /admin to be able to access server even when is nonresponsive to others.

Also have dedicated fast pool for images, if this makes sense and separate pools for POST from GET.

Here is the config.tcl file we would like to configure properly.

=======================================

ns_log notice "nsd.tcl: starting to read config file..."

######################################################################
#
# Instance-specific settings
# These default settings will only work in limited circumstances
# Two servers with default settings cannot run on the same host
#
######################################################################

#---------------------------------------------------------------------
# change to 80 and 443 for production use
set httpport 80
set httpsport 443
# If setting port below 1024 with AOLServer 4, read comments in file:
# /var/lib/aolserver/service0/packages/etc/daemontools/run

# The hostname and address should be set to actual values.
# setting the address to 0.0.0.0 means aolserver listens on all interfaces
set hostname mywebsite.com
#set address [ns_info address]
set address xxxxxxxxxxxxxxxxxxxx

# Note: If port is privileged (usually < 1024), OpenACS must be
# started by root, and, in AOLserver 4, the run script have a
# '-b address' flag which matches the address according to settings (above)

set server "myserver"
set servername "myserver Community"

set serverroot "/var/lib/aolserver/${server}"

#---------------------------------------------------------------------
# which database do you want? postgres or oracle
set database postgres

set db_name $server
set db_user $server
set db_password "xxxxxxxxxxxxxx"
set db_host localhost
set db_port 5432

#---------------------------------------------------------------------
# if debug is false, all debugging will be turned off
set debug false
#set debug true

set homedir /usr/local/aolserver45
set bindir [file dirname [ns_info nsd]]

set max_file_upload_mb 20
set max_file_upload_min 5

######################################################################
#
# End of instance-specific settings
#
# Nothing below this point need be changed in a default install.
#
######################################################################

#---------------------------------------------------------------------
#
# AOLserver's directories. Autoconfigurable.
#
#---------------------------------------------------------------------
# Where are your pages going to live ?
set pageroot ${serverroot}/www
set directoryfile index.adp,index.html,index.htm
#set charset iso-8859-2

ns_section ns/parameters
ns_param serverlog ${serverroot}/log/error.log
ns_param home $homedir
# maxkeepalive is ignored in aolserver4.x
ns_param maxkeepalive 30
ns_param logroll on
ns_param maxbackup 5
ns_param debug $debug
ns_param mailhost localhost

# # setting to Unicode by default
# # see http://dqd.com/~mayoff/encoding-doc.html
ns_param HttpOpenCharset utf-8
ns_param URLCharset utf-8
ns_param HackContentType 1
ns_param OutputCharset iso-8859-2

#---------------------------------------------------------------------
# Thread library (nsthread) parameters
#---------------------------------------------------------------------
ns_section ns/threads
ns_param mutexmeter true ;# measure lock contention
# The per-thread stack size must be a multiple of 8k for AOLServer to run under MacOS X

ns_param stacksize [expr 128 * 8192]

#
# MIME types.
#
ns_section ns/mimetypes
# Note: AOLserver already has an exhaustive list of MIME types:
# see: /usr/local/src/aolserver-4.{version}/aolserver/nsd/mimetypes.c
# but in case something is missing you can add it here.
ns_param Default text/plain
ns_param NoExtension text/plain
ns_param .pcd image/x-photo-cd
ns_param .prc application/x-pilot
ns_param .xls application/vnd.ms-excel
ns_param .doc application/vnd.ms-word

#---------------------------------------------------------------------
#
# Server-level configuration
#
# There is only one server in AOLserver, but this is helpful when multiple
# servers share the same configuration file. This file assumes that only
# one server is in use so it is set at the top in the "server" Tcl variable
# Other host-specific values are set up above as Tcl variables, too.
#
#---------------------------------------------------------------------

ns_section ns/servers
ns_param $server $servername

#
# Server parameters
#
ns_section ns/server/${server}
ns_param directoryfile $directoryfile
ns_param pageroot $pageroot
ns_param maxconnections 250 ;# max connections to put on queue
ns_param maxdropped 0
ns_param maxthreads 60
ns_param minthreads 20
ns_param threadtimeout 120 ;# Idle threads die at this rate
ns_param globalstats false ;# Enable built-in statistics
ns_param urlstats true ;# Enable URL statistics
ns_param maxurlstats 1000 ;# Max number of URL's to do stats on
# ns_param directoryadp $pageroot/dirlist.adp ;# Choose one or the other
# ns_param directoryproc _ns_dirlist ;# ...but not both!
# ns_param directorylisting fancy ;# Can be simple or fancy
#jd adding from v.3.13
ns_param ConnsPerThread 125 ;# after this number thread will exit good for getting around mem leaks

#jd new 4.5 entries
ns_pools set default -minthreads 20
ns_pools set default -maxthreads 60
ns_pools set default -maxconns 125
ns_pools set default -timeout 120
ns_log Notice "Default Pool: [ns_pools get default]"
#/jd

#---------------------------------------------------------------------
#
# ADP (AOLserver Dynamic Page) configuration
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/adp
ns_param map /*.adp ;# Extensions to parse as ADP's
# ns_param map "/*.html" ;# Any extension can be mapped
ns_param enableexpire false ;# Set "Expires: now" on all ADP's
ns_param enabledebug $debug ;# Allow Tclpro debugging with "?debug"
ns_param defaultparser fancy
#adding for tuning purposes
ns_param cache true ;# in memory cache of ADP's
ns_param cachesize [expr 1000*1024]

ns_section ns/server/${server}/adp/parsers
ns_param fancy ".adp"

ns_section ns/server/${server}/redirects
ns_param 404 "global/404"
ns_param 403 "global/forbidden.html"
ns_param 500 "global/500"

#
# Tcl Configuration
#
ns_section ns/server/${server}/tcl
ns_param library ${serverroot}/tcl
ns_param autoclose on
ns_param debug $debug
#jd adding nsvbucktes for server tuning
ns_param nsvbuckets 33 ;#No. of buckets to hold nsv's

#---------------------------------------------------------------------
#
# Rollout email support
#
# These procs help manage differing email behavior on
# dev/staging/production.
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/acs/acs-rollout-support

# EmailDeliveryMode can be:
# default: Email messages are sent in the usual manner.
# log: Email messages are written to the server's error log.
# redirect: Email messages are redirected to the addresses specified
# by the EmailRedirectTo parameter. If this list is absent
# or empty, email messages are written to the server's error log.
# filter: Email messages are sent to in the usual manner if the
# recipient appears in the EmailAllow parameter, otherwise they
# are logged.

# ns_param EmailDeliveryMode redirect
# ns_param EmailRedirectTo mailto:somenerd@yourdomain.test, mailto:othernerd@yourdomain.test
# ns_param EmailAllow mailto:somenerd@yourdomain.test,mailto:othernerd@yourdomain.test

#---------------------------------------------------------------------
#
# WebDAV Support (optional, requires oacs-dav package to be installed
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/tdav
ns_param propdir ${serverroot}/data/dav/properties
ns_param lockdir ${serverroot}/data/dav/locks
ns_param defaultlocktimeout "300"

ns_section ns/server/${server}/tdav/shares
ns_param share1 "OpenACS"
# ns_param share2 "Share 2 description"

ns_section ns/server/${server}/tdav/share/share1
ns_param uri "/dav/*"
# all WebDAV options
ns_param options "OPTIONS COPY GET PUT MOVE DELETE HEAD MKCOL POST PROPFIND PROPPATCH LOCK UNLOCK"

#ns_section ns/server/${server}/tdav/share/share2
# ns_param uri "/share2/path/*"
# read-only WebDAV options
# ns_param options "OPTIONS COPY GET HEAD MKCOL POST PROPFIND PROPPATCH"

#---------------------------------------------------------------------
#
# Socket driver module (HTTP) -- nssock
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/module/nssock
ns_param timeout 120 ;#120
ns_param address $address
ns_param hostname $hostname
ns_param port $httpport
# setting maxinput higher than practical may leave the server vulnerable to resource DoS attacks
# see http://www.panoptic.com/wiki/aolserver/166
ns_param maxinput [expr {$max_file_upload_mb * 1024 * 1024}] ;# Maximum File Size for uploads in bytes
ns_param recvwait [expr {$max_file_upload_min * 60}] ;# Maximum request time in minutes

#---------------------------------------------------------------------
#
# Access log -- nslog
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/module/nslog
ns_param debug false
ns_param dev false
ns_param enablehostnamelookup false
ns_param file ${serverroot}/log/${server}.log
ns_param logcombined true
ns_param extendedheaders COOKIE
# ns_param logrefer false
# ns_param loguseragent false
ns_param logreqtime true
ns_param maxbackup 1000
ns_param rollday *
ns_param rollfmt %Y-%m-%d-%H:%M
ns_param rollhour 0
ns_param rollonsignal true
ns_param rolllog true

#---------------------------------------------------------------------
#
# nsjava - aolserver module that embeds a java virtual machine. Needed to
# support webmail. See http://nsjava.sourceforge.net for further
# details. This may need to be updated for OpenACS4 webmail
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/module/nsjava
ns_param enablejava off ;# Set to on to enable nsjava.
ns_param verbosejvm off ;# Same as command line -debug.
ns_param loglevel Notice
ns_param destroyjvm off ;# Destroy jvm on shutdown.
ns_param disablejitcompiler off
ns_param classpath /usr/local/jdk/jdk118_v1/lib/classes.zip:${bindir}/nsjava.jar:${pageroot}/webmail/java/activation.jar:${pageroot}/webmail/java/mail.jar:${pageroot}/webmail/java

#---------------------------------------------------------------------
#
# CGI interface -- nscgi, if you have legacy stuff. Tcl or ADP files inside
# AOLserver are vastly superior to CGIs. I haven't tested these params but they
# should be right.
#
#---------------------------------------------------------------------
#ns_section "ns/server/${server}/module/nscgi"
# ns_param map "GET /cgi-bin/ ${serverroot}/cgi-bin"
# ns_param map "POST /cgi-bin/ ${serverroot}/cgi-bin"
# ns_param Interps CGIinterps

#ns_section "ns/interps/CGIinterps"
# ns_param .pl "/usr/bin/perl"

#---------------------------------------------------------------------
#
# PAM authentication
#
#---------------------------------------------------------------------
ns_section ns/server/${server}/module/nspam
ns_param PamDomain "pam_domain"

#---------------------------------------------------------------------
#
# OpenSSL for Aolserver 3.3 and 4
#
#---------------------------------------------------------------------
if { [ns_info version] < 4} {

#---------------------------------------------------------------------
# OpenSSL for Aolserver 3.3
#---------------------------------------------------------------------

ns_section "ns/server/${server}/module/nsopenssl"

ns_param ModuleDir ${serverroot}/etc/certs

# NSD-driven connections:
ns_param ServerPort $httpsport
ns_param ServerHostname $hostname
ns_param ServerAddress $address
ns_param ServerCertFile certfile.pem
ns_param ServerKeyFile keyfile.pem
ns_param ServerProtocols "SSLv2, SSLv3, TLSv1"
ns_param ServerCipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
ns_param ServerSessionCache true
ns_param ServerSessionCacheID 1
ns_param ServerSessionCacheSize 512
ns_param ServerSessionCacheTimeout 300
ns_param ServerPeerVerify false
ns_param ServerPeerVerifyDepth 3
ns_param ServerCADir ca
ns_param ServerCAFile ca.pem
ns_param ServerTrace false

# For listening and accepting SSL connections via Tcl/C API:
ns_param SockServerCertFile certfile.pem
ns_param SockServerKeyFile keyfile.pem
ns_param SockServerProtocols "SSLv2, SSLv3, TLSv1"
ns_param SockServerCipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
ns_param SockServerSessionCache true
ns_param SockServerSessionCacheID 2
ns_param SockServerSessionCacheSize 512
ns_param SockServerSessionCacheTimeout 300
ns_param SockServerPeerVerify false
ns_param SockServerPeerVerifyDepth 3
ns_param SockServerCADir internal_ca
ns_param SockServerCAFile internal_ca.pem
ns_param SockServerTrace false

# Outgoing SSL connections
ns_param SockClientCertFile certfile.pem
ns_param SockClientKeyFile keyfile.pem
ns_param SockClientProtocols "SSLv2, SSLv3, TLSv1"
ns_param SockClientCipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
ns_param SockClientSessionCache false
ns_param SockClientSessionCacheID 3
ns_param SockClientSessionCacheSize 512
ns_param SockClientSessionCacheTimeout 300
ns_param SockClientPeerVerify false
ns_param SockServerPeerVerifyDepth 3
ns_param SockClientCADir ca
ns_param SockClientCAFile ca.pem
ns_param SockClientTrace false

# OpenSSL library support:
# ns_param RandomFile /some/file
ns_param SeedBytes 1024

} else {

#---------------------------------------------------------------------
# OpenSSL for Aolserver 4
#---------------------------------------------------------------------
ns_section "ns/server/${server}/module/nsopenssl"

# Note this portion of the configuration is not perfect, and you
# will get errors in the your error.log. However, it does
# work. Fixes welcome.

# this is used by acs-tcl/tcl/security-procs.tcl to get the
# https port.
ns_param ServerPort $httpsport
# setting maxinput higher than practical may leave the server vulnerable to resource DoS attacks
# see http://www.panoptic.com/wiki/aolserver/166
# must set maxinput for nsopenssl as well as nssock
ns_param maxinput [expr {$max_file_upload_mb * 1024 * 1024}] ;# Maximum File Size for uploads in bytes

# We explicitly tell the server which SSL contexts to use as defaults when an
# SSL context is not specified for a particular client or server SSL
# connection. Driver connections do not use defaults; they must be explicitly
# specificied in the driver section. The Tcl API will use the defaults as there
# is currently no provision to specify which SSL context to use for a
# particular connection via an ns_openssl Tcl command.
ns_section "ns/server/${server}/module/nsopenssl/sslcontexts"
ns_param users "SSL context used for regular user access"
# ns_param admins "SSL context used for administrator access"
ns_param client "SSL context used for outgoing script socket connections"

ns_section "ns/server/${server}/module/nsopenssl/defaults"
ns_param server users
ns_param client client

ns_section "ns/server/${server}/module/nsopenssl/sslcontext/users"
ns_param Role server
ns_param ModuleDir ${serverroot}/etc/certs
ns_param CertFile certfile.pem
ns_param KeyFile keyfile.pem
# ns_param CADir ca-client/dir
# ns_param CAFile ca-client/ca-client.crt
# for Protocols "ALL" = "SSLv2, SSLv3, TLSv1"
ns_param Protocols "SSLv3, TLSv1"
ns_param CipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
ns_param PeerVerify false
ns_param PeerVerifyDepth 3
ns_param Trace false

# following from bartt's nsd4.tcl, might help stablize openssl connections?
# http://www.mail-archive.com/aolserver@listserv.aol.com/msg07092.html
ns_param SessionCache true
ns_param SessionCacheID 1
ns_param SessionCacheSize 512
ns_param SessionCacheTimeout 300

# ns_section "ns/server/${server}/module/nsopenssl/sslcontext/admins"
# ns_param Role server
# ns_param ModuleDir /path/to/dir
# ns_param CertFile server/server.crt
# ns_param KeyFile server/server.key
# ns_param CADir ca-client/dir
# ns_param CAFile ca-client/ca-client.crt
# for Protocols "ALL" = "SSLv2, SSLv3, TLSv1"
# ns_param Protocols "All"
# ns_param CipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
# ns_param PeerVerify false
# ns_param PeerVerifyDepth 3
# ns_param Trace false

ns_section "ns/server/${server}/module/nsopenssl/sslcontext/client"
ns_param Role client
ns_param ModuleDir ${serverroot}/etc/certs
ns_param CertFile certfile.pem
ns_param KeyFile keyfile.pem
# ns_param CADir ${serverroot}/etc/certs
# ns_param CAFile certfile.pem
# for Protocols "ALL" = "SSLv2, SSLv3, TLSv1"
ns_param Protocols "SSLv2, SSLv3, TLSv1"
ns_param CipherSuite "ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP"
ns_param PeerVerify false
ns_param PeerVerifyDepth 3
ns_param Trace false

# following from bartt's nsd4.tcl, might help stablize openssl connections?
# http://www.mail-archive.com/aolserver@listserv.aol.com/msg07092.html
ns_param SessionCache true
ns_param SessionCacheID 1
ns_param SessionCacheSize 512
ns_param SessionCacheTimeout 300

# SSL drivers. Each driver defines a port to listen on and an explitictly named
# SSL context to associate with it. Note that you can now have multiple driver
# connections within a single virtual server, which can be tied to different
# SSL contexts.
ns_section "ns/server/${server}/module/nsopenssl/ssldrivers"
ns_param users "Driver for regular user access"
# ns_param admins "Driver for administrator access"

ns_section "ns/server/${server}/module/nsopenssl/ssldriver/users"
ns_param sslcontext users
# ns_param port $httpsport_users
ns_param port $httpsport
ns_param hostname $hostname
ns_param address $address
# following added per
# http://www.mail-archive.com/aolserver@listserv.aol.com/msg07365.html
# Maximum File Size for uploads:
ns_param maxinput [expr {$max_file_upload_mb * 1024 * 1024}] ;# in bytes
# Maximum request time
ns_param recvwait [expr {$max_file_upload_min * 60}] ;# in minutes

# ns_section "ns/server/${server}/module/nsopenssl/ssldriver/admins"
# ns_param sslcontext admins
# ns_param port $httpsport_admins
# ns_param port $httpsport
# ns_param hostname $hostname
# ns_param address $address
}

#---------------------------------------------------------------------
#
# Database drivers
# The database driver is specified here.
# Make sure you have the driver compiled and put it in {aolserverdir}/bin
#
#---------------------------------------------------------------------
ns_section "ns/db/drivers"
if { $database == "oracle" } {
ns_param ora8 ${bindir}/ora8.so
} else {
ns_param postgres ${bindir}/nspostgres.so ;# Load PostgreSQL driver
}

if { $database == "oracle" } {
ns_section "ns/db/driver/ora8"
ns_param maxStringLogLength -1
ns_param LobBufferSize 32768
}

# Database Pools: This is how AOLserver ``talks'' to the RDBMS. You need
# three for OpenACS: main, log, subquery. Make sure to replace ``yourdb''
# and ``yourpassword'' with the actual values for your db name and the
# password for it, if needed.
#
# AOLserver can have different pools connecting to different databases
# and even different different database servers. See
# https://openacs.org/doc/openacs-5-1/tutorial-second-database.html

ns_section ns/db/pools
ns_param pool1 "Pool 1"
ns_param pool2 "Pool 2"
ns_param pool3 "Pool 3"

ns_section ns/db/pool/pool1
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose $debug
ns_param extendedtableinfo true
ns_param logsqlerrors $debug
if { $database == "oracle" } {
ns_param driver ora8
ns_param datasource {}
ns_param user $db_name
ns_param password $db_password
} else {
ns_param driver postgres
ns_param datasource ${db_host}:${db_port}:${db_name}
ns_param user $db_user
ns_param password $db_password
}

ns_section ns/db/pool/pool2
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose $debug
ns_param extendedtableinfo true
ns_param logsqlerrors $debug
if { $database == "oracle" } {
ns_param driver ora8
ns_param datasource {}
ns_param user $db_name
ns_param password $db_password
} else {
ns_param driver postgres
ns_param datasource ${db_host}:${db_port}:${db_name}
ns_param user $db_user
ns_param password $db_password
}

ns_section ns/db/pool/pool3
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose $debug
ns_param extendedtableinfo true
ns_param logsqlerrors $debug
if { $database == "oracle" } {
ns_param driver ora8
ns_param datasource {}
ns_param user $db_name
ns_param password $db_password
} else {
ns_param driver postgres
ns_param datasource ${db_host}:${db_port}:${db_name}
ns_param user $db_user
ns_param password $db_password
}

ns_section ns/server/${server}/db
ns_param pools "*"
ns_param defaultpool pool2

### LDAP settings

ns_section "ns/ldap/pool/ldap"
ns_param user "cn=admin,ou=SysAdmin,dc=myserver,dc=net"
ns_param password "xxxxxxxxxxxxxx"
ns_param host "box-2.mywebsite.com"
ns_param connections 2
ns_param verbose On

ns_section "ns/ldap/pools"
ns_param ldap "LDAP Pool"

ns_section "ns/server/${server}/ldap"
ns_param pools "*"
ns_param defaultpool ldap

#---------------------------------------------------------------------
# which modules should be loaded? Missing modules break the server, so
# don't uncomment modules unless they have been installed.
ns_section ns/server/${server}/modules
ns_param nssock ${bindir}/nssock.so
ns_param nslog ${bindir}/nslog.so
ns_param nssha1 ${bindir}/nssha1.so
ns_param nscache ${bindir}/nscache.so
ns_param libthread /usr/local/aolserver/lib/thread2.6.4/libthread2.6.4.so

# openacs versions earlier than 5.x requires nsxml
# ns_param nsxml ${bindir}/nsxml.so

#---------------------------------------------------------------------
# nsopenssl will fail unless the cert files are present as specified
# later in this file, so it's disabled by default
# ns_param nsopenssl ${bindir}/nsopenssl.so

# authorize-gateway package requires dqd_utils
# ns_param dqd_utils dqd_utils[expr {int($tcl_version)}].so

# Full Text Search
# ns_param nsfts ${bindir}/nsfts.so

# LDAP authentication
ns_param nsldap ${bindir}/nsldap.so

# PAM authentication
# ns_param nspam ${bindir}/nspam.so


# These modules aren't used in standard OpenACS installs
# ns_param nsperm ${bindir}/nsperm.so
# ns_param nscgi ${bindir}/nscgi.so
# ns_param nsjava ${bindir}/libnsjava.so
# ns_param nsrewrite ${bindir}/nsrewrite.so

if { [ns_info version] >= 4 } {
# Required for AOLserver 4.x
ns_param nsdb ${bindir}/nsdb.so
} else {
# Required for AOLserver 3.x
ns_param libtdom ${bindir}/libtdom.so
}

# nsthread library which should become standard in 5.3
# if {[file exists ${homedir}/lib/thread2.6.4/libthread2.6.4[info sharedlibextension]]} {
# ns_param libthread ${homedir}/lib/thread2.6.4/libthread2.6.4[info sharedlibextension]
# }
#ns_param libthread /usr/local/aolserver/lib/thread2.6.4/libthread2.6.4.so

ns_log notice "nsd.tcl: using threadsafe tcl: [info exists tcl_platform(threaded)]"
ns_log notice "nsd.tcl: finished reading config file."

FILE Start===============

ns_section ns/db/drivers
ns_param postgres /usr/local/aolserver/bin/nspostgres.so

ns_section ns/db/pool/pool1
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose false
ns_param extendedtableinfo true
ns_param logsqlerrors false
ns_param driver postgres
ns_param datasource localhost:5432:myserver
ns_param user myserver
ns_param password xxxxxxxxxxxxxxxxxxxx

ns_section ns/db/pool/pool2
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose false
ns_param extendedtableinfo true
ns_param logsqlerrors false
ns_param driver postgres
ns_param datasource localhost:5432:myserver
ns_param user myserver
ns_param password xxxxxxxxxxxxxxxxxxxx

ns_section ns/db/pool/pool3
ns_param maxidle 600
ns_param maxopen 2400
ns_param connections 20
ns_param verbose false
ns_param extendedtableinfo true
ns_param logsqlerrors false
ns_param driver postgres
ns_param datasource localhost:5432:myserver
ns_param user myserver
ns_param password xxxxxxxxxxxxxxxxxxxx

ns_section ns/db/pools
ns_param pool1 {Pool 1}
ns_param pool2 {Pool 2}
ns_param pool3 {Pool 3}

ns_section ns/ldap/pool/ldap
ns_param user cn=admin,ou=SysAdmin,dc=myserver,dc=net
ns_param password xxxxxxxxxxxxxxxxxxxx
ns_param host box-2.myserver.net
ns_param connections 2
ns_param verbose On

ns_section ns/ldap/pools
ns_param ldap {LDAP Pool}

ns_section ns/mimetypes
ns_param Default text/plain
ns_param NoExtension text/plain
ns_param .pcd image/x-photo-cd
ns_param .prc application/x-pilot
ns_param .xls application/vnd.ms-excel
ns_param .doc application/vnd.ms-word

ns_section ns/parameters
ns_param serverlog /var/lib/aolserver/myserver/log/error.log
ns_param home /usr/local/aolserver45
ns_param maxkeepalive 30
ns_param logroll on
ns_param maxbackup 5
ns_param debug false
ns_param mailhost localhost
ns_param HttpOpenCharset utf-8
ns_param URLCharset utf-8
ns_param HackContentType 1
ns_param OutputCharset iso-8859-2

ns_section ns/server/myserver
ns_param directoryfile index.adp,index.html,index.htm
ns_param pageroot /var/lib/aolserver/myserver/www
ns_param maxconnections 250
ns_param maxdropped 0
ns_param maxthreads 60
ns_param minthreads 20
ns_param threadtimeout 120
ns_param globalstats false
ns_param urlstats true
ns_param maxurlstats 1000
ns_param ConnsPerThread 125

ns_section ns/server/myserver/acs/acs-rollout-support

ns_section ns/server/myserver/adp
ns_param map /*.adp
ns_param enableexpire false
ns_param enabledebug false
ns_param defaultparser fancy
ns_param cache true
ns_param cachesize 1024000

ns_section ns/server/myserver/adp/parsers
ns_param fancy .adp

ns_section ns/server/myserver/db
ns_param pools *
ns_param defaultpool pool2

ns_section ns/server/myserver/ldap
ns_param pools *
ns_param defaultpool ldap

ns_section ns/server/myserver/module/nsjava
ns_param enablejava off
ns_param verbosejvm off
ns_param loglevel Notice
ns_param destroyjvm off
ns_param disablejitcompiler off
ns_param classpath /usr/local/jdk/jdk118_v1/lib/classes.zip:/usr/local/aolserver/bin/nsjava.jar:/var/lib/aolserver/myserver/www/webmail/java/activation.jar:/var/lib/aolserver/myserver/www/webmail/java/mail.jar:/var/lib/aolserver/myserver/www/webmail/java

ns_section ns/server/myserver/module/nslog
ns_param debug false
ns_param dev false
ns_param enablehostnamelookup false
ns_param file /var/lib/aolserver/myserver/log/myserver.log
ns_param logcombined true
ns_param extendedheaders COOKIE
ns_param logreqtime true
ns_param maxbackup 1000
ns_param rollday *
ns_param rollfmt %Y-%m-%d-%H:%M
ns_param rollhour 0
ns_param rollonsignal true
ns_param rolllog true

ns_section ns/server/myserver/module/nsopenssl
ns_param ServerPort 443
ns_param maxinput 20971520

ns_section ns/server/myserver/module/nsopenssl/defaults
ns_param server users
ns_param client client

ns_section ns/server/myserver/module/nsopenssl/sslcontext/client
ns_param Role client
ns_param ModuleDir /var/lib/aolserver/myserver/etc/certs
ns_param CertFile certfile.pem
ns_param KeyFile keyfile.pem
ns_param Protocols {SSLv2, SSLv3, TLSv1}
ns_param CipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP
ns_param PeerVerify false
ns_param PeerVerifyDepth 3
ns_param Trace false
ns_param SessionCache true
ns_param SessionCacheID 1
ns_param SessionCacheSize 512
ns_param SessionCacheTimeout 300

ns_section ns/server/myserver/module/nsopenssl/sslcontext/users
ns_param Role server
ns_param ModuleDir /var/lib/aolserver/myserver/etc/certs
ns_param CertFile certfile.pem
ns_param KeyFile keyfile.pem
ns_param Protocols {SSLv3, TLSv1}
ns_param CipherSuite ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP
ns_param PeerVerify false
ns_param PeerVerifyDepth 3
ns_param Trace false
ns_param SessionCache true
ns_param SessionCacheID 1
ns_param SessionCacheSize 512
ns_param SessionCacheTimeout 300

ns_section ns/server/myserver/module/nsopenssl/sslcontexts
ns_param users {SSL context used for regular user access}
ns_param client {SSL context used for outgoing script socket connections}

ns_section ns/server/myserver/module/nsopenssl/ssldriver/users
ns_param sslcontext users
ns_param port 443
ns_param hostname myserver.net
ns_param address xxx.xxx.xxx.xxx
ns_param maxinput 20971520
ns_param recvwait 300

ns_section ns/server/myserver/module/nsopenssl/ssldrivers
ns_param users {Driver for regular user access}

ns_section ns/server/myserver/module/nspam
ns_param PamDomain pam_domain

ns_section ns/server/myserver/module/nssock
ns_param timeout 120
ns_param address xxx.xxx.xxx.xxx
ns_param hostname myserver.net
ns_param port 80
ns_param maxinput 20971520
ns_param recvwait 300

ns_section ns/server/myserver/modules
ns_param nssock /usr/local/aolserver/bin/nssock.so
ns_param nslog /usr/local/aolserver/bin/nslog.so
ns_param nssha1 /usr/local/aolserver/bin/nssha1.so
ns_param nscache /usr/local/aolserver/bin/nscache.so
ns_param libthread /usr/local/aolserver/lib/thread2.6.4/libthread2.6.4.so
ns_param nsldap /usr/local/aolserver/bin/nsldap.so
ns_param nsdb /usr/local/aolserver/bin/nsdb.so

ns_section ns/server/myserver/redirects
ns_param 404 global/404
ns_param 403 global/forbidden.html
ns_param 500 global/500

ns_section ns/server/myserver/tcl
ns_param library /var/lib/aolserver/myserver/tcl
ns_param autoclose on
ns_param debug false
ns_param nsvbuckets 33

ns_section ns/server/myserver/tdav
ns_param propdir /var/lib/aolserver/myserver/data/dav/properties
ns_param lockdir /var/lib/aolserver/myserver/data/dav/locks
ns_param defaultlocktimeout 300

ns_section ns/server/myserver/tdav/share/share1
ns_param uri /dav/*
ns_param options {OPTIONS COPY GET PUT MOVE DELETE HEAD MKCOL POST PROPFIND PROPPATCH LOCK UNLOCK}

ns_section ns/server/myserver/tdav/shares
ns_param share1 OpenACS

ns_section ns/servers
ns_param myserver {myserver Community}

ns_section ns/threads
ns_param mutexmeter true
ns_param stacksize 1048576

=========File End

as for mapping.
Is there a way to find out which threadpool is actualy taking your request?
and lastly should myDefault be mapped with { GET /} or {GET /*} ?

Thanks

The problem is that even that threadpools seem to work, or at least are loaded (see below), when I run "ps -eLf" I get only 15 threads open.

1. In my understanding there in fact should be at least as many as the sum of all threadpools minthreads in my case around 90.

2. When looking at network stats with "netstat -s" (see below) the number of active connections is strangely low compared to passive conns and high number of failed connections.

Tcp:
20778 active connections openings
1798957 passive connection openings
574669 failed connection attempts
98681 connection resets received
112 connections established
45683111 segments received
57018897 segments send out
866619 segments retransmited
667 bad segments received.
220484 resets sent

========config-info===============
ns_section ns/server/myserver/pool/admin
ns_param maxconnections 100
ns_param minthreads 2
ns_param maxthreads 2
ns_param threadtimeout 120
ns_param map {GET */admin/*}
ns_param map {POST */admin/*}
ns_param map {GET /acs-admin/*}
ns_param map {POST /acs-admin/*}

ns_section ns/server/myserver/pool/fast
ns_param maxconnections 200
ns_param minthreads 10
ns_param maxthreads 20
ns_param threadtimeout 120
ns_param map {GET /*.jpg}
ns_param map {GET /*.swf}
ns_param map {GET /image/*}

ns_section ns/server/myserver/pool/mainpool
ns_param maxconnections 200
ns_param minthreads 80
ns_param maxthreads 80
ns_param threadtimeout 120
ns_param map {GET /*}
ns_param map {POST /}

ns_section ns/server/myserver/pools
ns_param fast {fast pool}
ns_param admin {admin pool}
ns_param mainpool {mainpool pool}

Could you confirm on your system that setting min (and max)threads does work. In my case the number of open threads stays same regardless. As I mentioned in my post above I should see around 90 open threads and there are only 15 and the number never goes up, users are getting 'time-outs' and no new threads are created.

This worked fine in 3.13 and if this is not working in 4.5 then 4.5 in my view is not commercialy usable.

As I said setting threadpools did nothing in terms of number of threads actualy started, until I have entered the following lines into config.tcl file.

ns_pools set default -minthreads 80
ns_pools set default -maxthreads 200
ns_pools set default -maxconns 200
ns_pools set default -timeout 120
ns_log Notice "Default Pool: [ns_pools get default]"

Suddenly I have got 80 threads opened when checked with "ps -eLF"
This however lasted only some time and threads started to die below the -minthreads setting, as low as 13.
This tells me that -minthreads works only at startup and then when threads die it is not checked anymore, which looks like a bug to me.

Maintaining at least minthreads could become difficult. I think you could consider it a bug, but maybe not. The result is this: your server will regulate threads down to the number required, a 'high water mark' less than or equal to maxthreads.

The best way to test this is to use something like apache bench. Set the concurrency above the minthreads and see if you can get it to go back over that amount. (Then let it rest and see threads exit, during testing you can set a low thread timeout.) Then set concurrency to something above maxthreads and see how it performs. You can test pages which are slow and see what happens. You will probably also need to keep pushing up the number of request to run (-n 500, or whatever), so that each thread gets multiple requests, and the queue gets a little full. You can also set the maxconnections to different values (very low and very high) and test that. Threads are started very quickly, but their startup cost is spread over maxconnections requests.

I'm guessing that what you will discover is that minthreads is a good hint at startup, but may be too many. If it is always too many, then you are wasting memory. Only by watching a production system can you determine a reasonable value for minthreads. If you have massive fluctuations in the number of requests over a time larger than thread timeout, then minthreads is pretty useless, you can expect it to drop below that soon after startup.

-gustaf neumann