Forum OpenACS Q&A: Support for multibyte: .tcl ok .adp not
https://openacs.org/bboard/q-an d-a-fetch-msg.tcl?msg_id=0001CE&topic_id=12&topic=OpenACS%204%2e0%20De sign and applied the patches to Openacs 3.2.4(download from cvs).
The handling of multibyte (Big5, a Tradictional Chinese encoding) works fine in form post/get and all .tcl pages until I tested against ecommerce module which use .adp templates. The templated .adp page did not display the encoding correctly.
Then I traced down to the code in ad_return_template which is in ad-style.tcl. In ad_return_template proc, it calls ns_adp_parse which is defined in adp.c in aolserver source. It looks like ns_adp_parse might be the cause of the problem. Does that mean one should avoid using ad_return_template when one needs an encording other than ascii ? any pointers would be appreciated.
Kenny Chiang
This line:
eval "uplevel { ns_write "$http_header[ns_adp_parse -file" $fully_qualified_template_filename"]" }"for this couple of lines:
set content_type "text/html" eval "uplevel { ns_write "$http_header[ns_startcontent -type $content_type][ns_adp_parse -file "$fully_qualified_template_filename"]" }"It works for latin1 characters, i guess it will be the same for Big5.
Now, could you please ask me a question?
I asked in his thread (https://openacs.org/forums/message-view?message_id=18629 )
if anybody could post his configuration of a working system. Could you please tell me how have you compiled POstgreSQL, if your tables are created as Unicode, and what encoding system
are you using within AOLserver?
I'd really apreciate it. If you read my question in the thread above, i've got problems with upper/lower chars.
I don't even know if Big5 has anything similar to upper/lower chars, but if it does, are they working for you?
Anyway, could you post your configuration schema?
Thanks a lot.
I have some patches which instead load the adp file into a Tcl string, using the encoding conversion of your choice (as specified by the mime type that you assign to .adp files in your server config file). It then calls ns_adp_parse with the -string option and passes that string. The patch also sets the appropriate output encoding so that AOLserver converts the file from UTF8 to the correct encoding when it returns it to the browser.
My patches are at http://www.ai.mit.edu/people/hqm/openacs
My .tcl config file is shown below. We are running Japanese OpenACS sites with OpenACS3.2.5 and shiftJIS encoding. Note that it is key to turn off the default mapping of .adp files to the adp parser, that I think the default OpenACS config file has. Instead, we use the big registered proc that passes everything under /* to ad_handle_abstract_url
The db's are created using the "createdb --encoding=unicode" flag
# # AOLserver/OpenNSD Sample Configuration File # # Improved with lots of comments and things you'll need for # running OpenACS. # # by Roberto Mello (rmello@fslc.usu.edu) # $Id: nsd.txt,v 1.6 2001/02/15 06:29:43 rmello Exp $ ns_log notice "nsd.tcl: starting to read config file..." # # If httpport is set to 80, you'll have to start AOLserver as root and pass the user # AOLserver will run as in the command line. (e.g. ./nsd -u nsadmin -g nsadmin -t ../nsd.tcl) # (assuming you're starting AOLserver from the {aolserverdir}/bin directory. set httpport 80 set httpsport 443 # # Make sure your /etc/hostname is setup right, with your full domain. If you want AOLserver # to listen to www.foobar.com instead of foobar.com then hard code the appropriate domain # in the line below (e.g. set hostname www.foobar.com). # set hostname [ns_info hostname] set address [ns_info address] # # You can name your server whatever you want, but you'll need a directory with that # name under {aolserverdir}/servers/ writable by you AOLserver user. # # For example: if your AOLserver is in /usr/local/aolserver and your server is "server1" # you will need a /usr/local/aolserver/servers/server1 directory. # set server "openacs325" set servername "openacs325" # # AOLserver's home and binary directories. Autoconfigurable. # set homedir [file dirname [ns_info config]] set bindir [file dirname [ns_info nsd]] # # Where are your pages going to live ? # set pageroot /web/${server}/www set directoryfile index.adp,index.tcl,index.html,index.htm # # nsssl: Only loads if keyfile.pem and certfile.pem exist. # If you are using SSL, make sure you have these dirs and files (refer # to the AOLserver docs) set sslkeyfile ${homedir}/servers/${server}/modules/nsssl/keyfile.pem set sslcertfile ${homedir}/servers/${server}/modules/nsssl/certfile.pem # # Global server parameters # ns_section "ns/parameters" ns_param home $homedir ns_param debug false ns_param mailhost localhost ns_param ServerLog ${homedir}/log/${server}-error.log # ns_param LogRoll on ns_param HackContentType 1 ns_param URLCharset shift_jis ns_param OutputCharset shift_jis ns_param HttpOpenCharset shift_jis # # Thread library (nsthread) parameters # ns_section "ns/threads" ns_param mutexmeter true ;# measure lock contention #ns_param stacksize [expr 128*1024] ;# Per-thread stack size for hungry C modules # # MIME types. # # Note: AOLserver already has an exhaustive list of MIME types, but in # case something is missing you can add it here. # ns_section "ns/mimetypes" ns_param default "*/*" ;# MIME type for unknown extension ns_param noextension "*/*" ;# MIME type for missing extension #ns_param ".xls" "application/vnd.ms-excel'' ns_param .html "text/html; charset=shift_jis" ns_param .tcl "text/html; charset=shift_jis" ns_param .adp "text/html; charset=shift_jis" # # Tcl Configuration # ns_section "ns/server/${server}/tcl" ns_param autoclose "on" ns_param debug "false" # This is where this server's private Tcl library is. All .tcl files in this directory # are parsed when AOLserver starts. The crucial procedures used in OpenACS are # defined through this library # ns_param library "/web/${server}/tcl" ############################################################ # # Server-level configuration # # There is only one server in AOLserver, but this is helpful when multiple # servers share the same configuration file. This file assumes that only # one server is in use so it is set at the top in the "server" Tcl variable # Other host-specific values are set up above as Tcl variables, too. # ns_section "ns/servers" ns_param $server $servername # # Server parameters # ns_section "ns/server/${server}" ns_param directoryfile $directoryfile ns_param pageroot $pageroot ns_param globalstats false ;# Enable built-in statistics ns_param urlstats false ;# Enable URL statistics ns_param maxurlstats 1000 ;# Max number of URL's to do stats on ns_param enabletclpages true ;# Parse tcl files in pageroot (dangerous) # # If you want to customize AOLserver's response to things like ``Internal Server Error'' # and other HTTP responses. OpenACS comes with some in the directory www/global. ns_param NotFoundResponse "/global/file-not-found.html" ns_param ServerBusyResponse "/global/busy.html" ns_param ServerInternalErrorResponse "/global/error.html" ns_param ForbiddenResponse "/global/forbidden.html" ns_param UnauthorizedResponse "/global/unauthorized.html" # Directory listings -- use an ADP or a Tcl proc to generate them. # #ns_param directoryadp $pageroot/dirlist.adp ;# Choose one or the other ns_param directoryproc _ns_dirlist ;# ...but not both! ns_param directorylisting fancy ;# Can be simple or fancy # for backport of ad_proc -- hqm ns_param acs_root_dir "/web/${server}" # # ADP (AOLserver Dynamic Page) configuration # #ns_section "ns/server/${server}/adp" # ns_param map "/*.adp" ;# Extensions to parse as ADP's #ns_param map "/*.html" ;# Any extension can be mapped # ns_param enableexpire false ;# Set "Expires: now" on all ADP's # ns_param enabledebug false ;# Allow Tclpro debugging with "?debug" # ADP special pages #ns_param errorpage ${pageroot}/errorpage.adp ;# Pretty-print ADP scripting errors # # ADP custom parsers -- see adp.c # ns_section "ns/server/${server}/adp/parsers" ns_param fancy ".adp" # # Socket driver module (HTTP) -- nssock # ns_section "ns/server/${server}/module/nssock" ns_param port $httpport ns_param hostname $hostname ns_param address $address # # Socket driver module (HTTPS) -- nsssl # # nsssl does not load unless sslkeyfile/sslcertfile exist (above). # ns_section "ns/server/${server}/module/nsssl" ns_param port $httpsport ns_param hostname $hostname ns_param address $address ns_param keyfile $sslkeyfile ns_param certfile $sslcertfile # # Database drivers # The database driver is specified here. PostgreSQL driver being loaded. # Make sure you have the driver compiled and put it in {aolserverdir}/bin # ns_section "ns/db/drivers" ns_param postgres ${bindir}/postgres.so ;# Load PostgreSQL driver # # Database Pools: This is how AOLserver ``talks'' to the RDBMS. You need three for # OpenACS: main, log, subquery. Make sure to replace ``yourdb'' and ``yourpassword'' # with the actual values for your db name and the password for it. # AOLserver can have different pools connecting to different databases and even different # different database servers. # ns_section "ns/db/pools" ns_param main "OpenACS Main Pool" ns_param log "OpenACS Log Pool" ns_param subquery "OpenACS Subquery Pool" ns_section "ns/db/pool/main" ns_param Driver postgres ns_param Connections 5 ;# 5 is a good number. Increase according to your needs ns_param DataSource localhost::openacs325 ;# Replace 'yourdb' with the name of your database in PG ns_param User nsadmin ;# User and password AOLserver will use to connect ns_param Password "nsadmin123" ns_param Verbose Off ;# Set it to On to see all queries. Good for debugging SQL. ns_param LogSQLErrors On ns_param ExtendedTableInfo On # ns_param MaxOpen 1000000000 ;# Max time to keep idle db connection open # ns_param MaxIdle 1000000000 ;# Max time to keep active db connection open ns_section "ns/db/pool/log" ns_param Driver postgres ns_param Connections 5 ns_param DataSource localhost::openacs325 ns_param User nsadmin ns_param Password "nsadmin123" ns_param Verbose On ns_param LogSQLErrors On ns_param ExtendedTableInfo On # ns_param MaxOpen 1000000000 # ns_param MaxIdle 1000000000 ns_section "ns/db/pool/subquery" ns_param Driver postgres ns_param Connections 2 ns_param DataSource localhost::openacs325 ns_param User nsadmin ns_param Password "nsadmin123" ns_param Verbose On ns_param LogSQLErrors On ns_param ExtendedTableInfo On # ns_param MaxOpen 1000000000 # ns_param MaxIdle 1000000000 ns_section "ns/server/${server}/db" ns_param Pools "*" ns_param DefaultPool "main" # # nscp: AOLserver Control Port - very useful for testing and evaluating. # Uncomment the sample password below and do a "telnet localhost 9999" # log in with "nsadmin", password "x", type "ns_crypt newpassword salt" # and paste the new encrypted string below. # # Sample User="nsadmin", password="x" set nscp_user "nsadmin:t2GqvvaiIUbF2:" # # Control port -- nscp # nscp does not load unless nscp_user is a valid user. # ns_section "ns/server/${server}/module/nscp" ns_param port 9999 ns_param address "127.0.0.1" ;# LOCALHOST IS RECOMMENDED ns_section "ns/server/${server}/module/nscp/users" ns_param user $nscp_user # # Access log -- nslog # ns_section "ns/server/${server}/module/nslog" ns_param rolllog true ;# Should we roll log? ns_param rollonsignal true ;# Roll log on SIGHUP ns_param rollhour 0 ;# Time to roll log ns_param maxbackup 5 ;# Max number to keep around when rolling # ns_param file ${homedir}/log/${server}.log ns_param EnableHostnameLookup Off ns_param LogCombined On ns_param LogRefer Off ns_param LogUserAgent Off # # nsjava - aolserver module that embeds a java virtual machine. Needed to # support webmail. See http://nsjava.sourceforge.net for further # details. # ns_section "ns/server/acs-pg/module/nsjava" ns_param EnableJava "off" ;# Set to on to enable nsjava. ns_param VerboseJvm "off" ;# Same as command line -debug. ns_param LogLevel "Notice" ns_param DestroyJvm "off" ;# Destroy jvm on shutdown. ns_param DisableJITCompiler "off" ns_param ClassPath "/usr/local/jdk/jdk118_v1/lib/classes.zip:/usr/local/aolserver/bin/nsjava.jar:/home/nsadmin/mirror/openacs325/www/webmail/java/activation.jar:/home/nsadmin/mirror/openacs325/www/webmail/java/mail.jar:/home/nsadmin/mirror/openacs325/www/webmail/java" # # CGI interface -- nscgi, if you have legacy stuff. Tcl or ADP files inside # AOLserver are vastly superior to CGIs. You don't actually need the Interps # if your script calls the appropriate interpreter itself. # ns_section "ns/server/${server}/module/nscgi" ns_param map "GET /cgi-bin /web/$server/cgi-bin" ns_param map "POST /cgi-bin /web/$server/cgi-bin" ns_param Interps CGIinterps ns_section "ns/interps/CGIinterps" ns_param .pl "/usr/bin/perl" ns_param .doit "/bin/bash" # # Modules to load # ns_section "ns/server/${server}/modules" ns_param nssock ${bindir}/nssock.so ns_param nslog ${bindir}/nslog.so ns_param nsxml ${bindir}/nsxml.so # ns_param nsperm ${bindir}/nsperm.so ns_param nscgi ${bindir}/nscgi.so # ns_param nsjava ${bindir}/libnsjava.so # ns_param ats Tcl # ## nsssl: loads only if requisite files already exist (see top of this # file). # if { [file exists $sslcertfile] && [file exists $sslkeyfile] } { ns_param nsssl ${bindir}/nsssle.so } else { ns_log warning "nsd.tcl: nsssl not loaded because key/cert files do not exist." } # nscp: loads only if nscp_user is set (see top of this file). if { $nscp_user != "" } { # ns_param nscp ${bindir}/nscp.so } else { ns_log warning "nsd.tcl: nscp not loaded because user/password is not set." } #ns_section "ns/server/${server}/ats" #ns_param ResourcePath ${homedir}/modules/tcl/ats/resources # # To Source OpenACS Config File. You need a "yourservername.tcl" file in the # OpenACS parameters dir. e.g.: say your server is "server1" then you'd need a # file named /web/server1/parameters/server1.tcl # (Hint: Rename ad.tcl to yourservername.tcl and edit it.) # source /web/${server}/parameters/${server}.tcl ns_log notice "nsd.tcl: finished reading config file." ns_log notice "HOMEDIR = $homedir"
I compiled postgres as follows:
./configure --enable-locale --enable-recode --enable-multibyte --enable-unicode-conversion --with-maxbackends=64 --with-tcl --with-tclconfig=/usr/local/lib --with-perl --with-python --with-openssl --with-CXX --enable-syslog
and created db's in UNICODE. My configuration file is same as the one posted by Henry except I changed the shift_jis to big5.
There is no upper/lower case in big5 (in fact, in Chinese). So I can not be helpful to test out your problem. sorry about that.
I did try patch the ad_return_template as you suggested but it still output juck characters as before. I think it is caused by the "ns_adp_parse -file" which does not honor the charset conversion as Henry suggested.
Henry,
I removed the default adp mapping to adp parser as you suggested and the "pure" .adp's work fine with this change. But the problem remains in the ad_return_template which is in use in serveral modules in openacs, such as ecommerce,gp and sdm etc. Do you have any suggestions on what to change to make use on the templated .adp pages called by ad_return_template ?
Contiune on testing the openacs modules, I found ad_return_top_page proc is not patched to handle charset conversion in your patches. ad_return_top_page is not used in a lot of modules, but for completness you may want to include in your patch set. The patch I did is as follows:
proc_doc ad_return_top_of_page {first_part_of_page{content_type "text/html; charset=big5"}} "Returns HTTP headers plus the top of the user-ivisible page. Saves a TCP packet (and therefore some overhead) compared to using ReturnHeaders and an ns_write." { ns_startcontent -type $content_type set all_the_headers "HTTP/1.0 200 OK MIME-Version: 1.0 Content-Type: $content_type " util_WriteWithExtraOutputHeaders $all_the_headers $first_part_of_page }
Would you please coordinate this with him and come up with one tarball that we can keep on the contrib section of openacs.org? I will give you permissions to write over it. I made a readme for Hafeez's patch on the tarball that's in the contrib code right now. It explains what the files are and how to install them. You could included that in the tarball too.
My tarball is at the /new-file-storage on openacs.org.
Thanks.
[02/May/2001:12:09:52][1649.4101][-conn0-] Error: can't read "user_is_logged_on" : no such variable can't read "user_is_logged_on": no such variable while executing "if { $user_is_logged_on } { set welcome_message "Welcome back $user_name! If you're not $user_name, click <a href="$register_url">here</a>." } ..." invoked from within chunk: 1 of adp: ns_adp_parse [02/May/2001:12:09:52][1649.4101][-conn0-] Error: can't read "welcome_message": no such variable can't read "welcome_message": no such variable while executing "ns_adp_puts -nonewline $welcome_message " invoked from within chunk: 2 of adp: ns_adp_parse [02/May/2001:12:09:52][1649.4101][-conn0-] Error: can't read "search_widget": no such variable can't read "search_widget": no such variable while executing "ns_adp_puts -nonewline $search_widget " invoked from within chunk: 3 of adp: ns_adp_parse [02/May/2001:12:09:52][1649.4101][-conn0-] Error: can't read "recommendations_if_there_are_any": no such variableShouldn't the ns_adp_parse be eval in "uplevel 1" ?
I changed the code as follows but had some difficulty to get the quote working. can someone point out what is wrong in the code:
set mimetype [ns_guesstype $fully_qualified_template_filename] set encoding [ns_encodingfortype $mimetype] set fd [open $fully_qualified_template_filename r] fconfigure $fd -encoding $encoding set template [read $fd] close $fd if { $cache_p } { # build the http header, with a no cache pragma if cache_p is 0 set http_header "HTTP/1.0 200 OK MIME-Version: 1.0 Content-Type: $mimetype Pragma: No-Cache " eval "uplevel { ns_write "$http_header[ns_adp_parse -string "$template"]" }" } else { eval "uplevel { ns_return 200 $mimetype [ns_adp_parse -string "$template"] }" }
The error message of above code look like this:
[02/May/2001:13:52:30][1984.8201][-conn4-] Error: extra characters after close-quote extra characters after close-quote while executing "ns_write "HTTP/1.0 200 OK MIME-Version: 1.0 Content-Type: text/html; charset=big5 Pragma: No-Cache [ns_adp_parse -string "Welcome to [ec_s..." ("uplevel" body line 1) invoked from within "uplevel { ns_write "HTTP/1.0 200 OK MIME-Version: 1.0 Content-Type: text/html; charset=big5 Pragma: No-Cache [ns_adp_parse -string " Welcom..." ("eval" body line 1) invoked from within "eval "uplevel { ns_write "$http_header[ns_adp_parse -string "$template"]" }"" (procedure "ad_return_template" line 68)
Now, let's try to solve this problem:
Henry, you're right when you say that you can call ns_adp_parse on files directly, that you must convert them to string before.
The problem is that, i don't know why, AOLserver seems to parse
ns_adp_parse -string
with the plain parser, and
ns_adp_parse -file
with the fancy parser.
Also, you forgot the ns_startcontent.
And the ecommerce module requires the fancy parser (it uses its own tags).
So i've come up with this piece of code. It works, but it is quite ugly.
Anyway, here is the code, and let's see if somebody smarter than me can tell if it is ok.
set mimetype [ns_guesstype $fully_qualified_template_filename] set encoding [ns_encodingfortype $mimetype] set fd [open $fully_qualified_template_filename r] fconfigure $fd -encoding $encoding set template [read $fd] close $fd set http_header "HTTP/1.0 200 OK MIME-Version: 1.0 Content-Type: $mimetype Pragma: No-Cache " if { $cache_p } { eval "uplevel { ns_write "$http_header[ns_startcontent -type $mimetype][ns_adp_eval -parser fancy {$template}]" }" } else { eval "uplevel { ns_return 200 $mimetype [ns_adp_eval -parser fancy {$template}] }" }
<p>
Thanks for your code, it works for me too except I need to make a bit of modification as follows:
<p>
<pre>
if { $cache_p } {
eval "uplevel { ns_write "$http_header[ns_startcontent -type "$mimetype"][ns_adp_eval -parser fancy {$template}]" }"
} else {
eval "uplevel { ns_return 200 "$mimetype" [ns_adp_eval -parser fancy {$template}] }"
}
</pre>
<p>
since my mimetype is "text/html; charset=big5" that needs to be enclosed in double quote.
<p>
Thanks again.
nsd8x. Kenny - you say post/get in Big5 encoding works fine, can you
confirm that the posted values are correctly stored in the db - do
they look as expected when queried with psql?
The error that I experience does _not_ appear in the browser: the
umlauts will be displayed correctly, but stored as two funny
characters in the db, according to the server log and to direct
querying with psql.
Here is a list of the steps that I took - please tell me what I am
missing.
Downloaded aolserver3_2+ad12_src.tar.gz
Downloaded aolserver-ad12-japanese-patches.tgz from
http://www.ai.mit.edu/people/hqm/openacs/ and followed the
instructions in the contained README file:
applied acs-aolserver-ad12.patch
copied 8bit.enc
make, make install
applied ad12-tcl.patch to the modules/tcl directory of the compiled
aolserver
Got openacs-3.2.5.tgz from openacs.org
created a db with createdb --encoding=UNICODE and loaded the openacs
datamodel
downloaded openacs-japanese-patches from
http://www.ai.mit.edu/people/hqm/openacs/ and replaced all occurencies
of shift_jis with iso-8859-1 as well as the line on the beginning
(which will go into tcl/00-ad-preload.tcl):
encoding system shiftjis
with
encoding system iso8859-1
applied that modified patch to the freshly unpacked OpenACS 3.2.5
Set those configuration parameters in my nsd.tcl:
ns_section "ns/parameters"
ns_param HackContentType 1
ns_param URLCharset iso-8859-1
ns_param OutputCharset iso-8859-1
ns_param HttpOpenCharset iso-8859-1
ns_section "ns/mimetypes"
ns_param .html "text/html; charset=iso8859-1"
ns_param .tcl "text/html; charset=iso8859-1"
ns_param .adp "text/html; charset=iso8859-1"
Now when I start the aolserver and do a POST that contains german
umlauts (tested with /pvt/basic-info-update-2.tcl) the umlauts will be
converted into two funny characters. They do already show up in the
server log:
[07/May/2001:19:42:55][1541.9222][-conn2-] Notice: Querying 'update users
set first_names = 'Günther',
last_name = 'Schmidt',
[...]
(This is an uppercase A with tilde on it followed by a quarter sign in
first_names, in case this posting gets somehow converted)
before that there is this notice in the server log:
[07/May/2001:19:42:55][1541.9222][-conn2-] Notice: ns_getform using
encoding iso8859-1 for charset iso-8859-1
I also tried changing the line $AOLSERVER/modules/tcl/init.tcl
encoding system [ns_config ns/server/[ns_info server] SystemEncoding
utf-8]
to:
encoding system [ns_config ns/server/[ns_info server] SystemEncoding
iso-8859-1]
Don't know what effect that actually has but it was mentioned
somewhere on the forum.
I use the following shell script to start the aolserver:
#!/bin/sh LD_LIBRARY_PATH=/usr/local/pgsql/lib:usr/local/lib export LD_LIBRARY_PATH PGCLIENTENCODING=UNICODE export PGCLIENTENCODING LANG=zh_TW.Big5 export LANG umask 022 exec /home/nsadmin/bin/nsd $*
and set the environment as below in my profile to run psql:
PATH=$PATH:/usr/local/pgsql/bin:~/bin export PATH LANG=zh_TW.Big5 export LANG PGCLIENTENCODING=BIG5 export PGCLIENTENCODING
In this way you have nsd talks to the postgres in UNICODE and have psql speaks BIG5 (or in your case iso-8859-1). Since the data you POST thru aolserver is actually stored in postgres in UNICODE encoding and that is what you saw in the server log. OTH, if you want to query the db thru psql, then you have to set PGCLIENTENCODING to ISO8859_1 (I'm not sure if the spelling is correct, you have to check on postgres documents) to have the postgres client library to convert UNICODE to ISO8859-1, so you can display your charset correctly in your terminal.
-- Kenny Chiang
with psql and had nothing to do with aolserver.
psql wants PGCLIENTENCODING set to LATIN1 to convert to iso8859-1
The error I get concerns this:
<pre>
extra characters after close-quote
while compiling
"eval "uplevel { ns_write "
("if" then script line 2)
while compiling
"if { $cache_p } {
eval "uplevel { ns_write "$http_header [ns_startcontent -type $mimetype] [ns_adp_eval -parser fancy {$template}]" }"
} else {
..."
("if" else script line 26)
while compiling
"if { [llength $available_templates] == 0 } {
ad_return_error "No template available" "We can't find any template for presenting the output of this sc..."
(compiling body of proc "ad_return_template", line 34)
invoked from within
"ad_return_template
"
("uplevel" body line 154)
invoked from within
"uplevel 1 $code"
(procedure "source_with_encoding" line 11)
invoked from within
"source_with_encoding $ad_conn(file)"
</pre>
Which I get regardless of whether I use Gaizka's suggested change, or Kenny Chiang's modification. The target encoding is shift_jis. The other modules that we "need" work, but they are not templated like ecommerce is.
Any help welcome. I will be on #openacs on irc.openprojects.net for those of you with IRC access during this evening (Sept 27th, EST).
The error I get concerns this:
extra characters after close-quote while compiling "eval "uplevel { ns_write " ("if" then script line 2) while compiling "if { $cache_p } { eval "uplevel { ns_write "$http_header [ns_startcontent -type $mimetype] [ns_adp_eval -parser fancy {$template}]" }" } else { ..." ("if" else script line 26) while compiling "if { [llength $available_templates] == 0 } { ad_return_error "No template available" "We can't find any template for presenting the output of this sc..." (compiling body of proc "ad_return_template", line 34) invoked from within "ad_return_template " ("uplevel" body line 154) invoked from within "uplevel 1 $code" (procedure "source_with_encoding" line 11) invoked from within "source_with_encoding $ad_conn(file)"
Which I get regardless of whether I use Gaizka's suggested change, or Kenny Chiang's modification. The target encoding is shift_jis. The other modules that we "need" work, but they are not templated like ecommerce is.
Any help welcome. I will be on #openacs on irc.openprojects.net for those of you with IRC access during this evening (Sept 27th, EST).
config file, and tell me what version of AOLserver and ACS (and what patches) you are using, I might be able to help out.
.adp files are generally parsed assuming a UTF8 encoding unless you use
my hack of reading the file into Tcl as a string first, with the encoding set specifically on the file descriptor. I have patches to
make ACS load .adp files in this way. You could also convert your
shift_jis .adp files into UTF8. I personally like to keep everything
in files in shift_jis though, because most tools like editors can
deal with it, whereas UTF8 is not widely supported by the backend tools. On the other hand, storing everything in UTF8 in the database
is the best way to go, hands down, in my opinion.
I am using 3.2.5 with your patches, 3.3.1+ad12 version of AOLserver.
I *think* the problem is in the part where the template is returned. I have modified ad-style.tcl per Kenny Chiang's message above. I have tried both ns_adp_parse -string and ns_adp_eval -parser fancy in the ad_return_template proc definition, both give the same result.
You can see the error message at http://205.247.253.66/ecommerce .
In my server.log, I see
[28/Sep/2001:14:28:18][23963.6798848][-conn44-] Notice: ns_getform using encoding shiftjis for charset shift_jis
followed by the error messages.
My nsd.tcl config file (passwords and comments removed for brevity):
ns_log notice "nsd.tcl: starting to read config file..." set httpport 80 set httpsport 443 set hostname 205.247.253.66 set address 205.247.253.66 set server "server1" set servername "somewhere.ne.jp" set homedir [file dirname [ns_info config]] set bindir [file dirname [ns_info nsd]] set pageroot /home/akira/openacs-3.2.5/www set directoryfile index.adp,index.html,index.tcl,index.htm set sslkeyfile ${homedir}/servers/${server}/modules/nsssl/keyfile.pem set sslcertfile ${homedir}/servers/${server}/modules/nsssl/certfile.pem ns_section "ns/parameters" ns_param home $homedir ns_param debug false ns_param MailHost localhost ns_param ServerLog ${homedir}/log/server.log ns_param LogRoll on ns_param maxthreads 20 #ns_param minthreads 7 ns_param HackContentType 1 ns_param URLCharset shift_jis ns_param OutputCharset shift_jis ns_param HttpOpenCharset shift_jis ns_section "ns/threads" ns_param mutexmeter true ;# measure lock contention ns_param stacksize [expr 128*1024] ;# Per-thread stack size for hungry C modules ns_section "ns/mimetypes" ns_param default "*/*" ;# MIME type for unknown extension ns_param noextension "*/*" ;# MIME type for missing extension #ns_param ".xls" "application/vnd.ms-excel'' ns_param .html "text/html; charset=shift_jis" ns_param .tcl "text/html; charset=shift_jis" ns_param .adp "text/html; charset=shift_jis" ns_section "ns/server/${server}/tcl" ns_param autoclose "on" ns_param debug "false" ns_param library "/home/akira/openacs-3.2.5/tcl" ns_section "ns/servers" ns_param $server $servername ns_section "ns/server/${server}" ns_param directoryfile $directoryfile ns_param pageroot $pageroot ns_param globalstats true ;# Enable built-in statistics ns_param urlstats true ;# Enable URL statistics ns_param maxurlstats 1000 ;# Max number of URL's to do stats on ns_param enabletclpages true ;# Parse tcl files in pageroot (dangerous) ns_param NotFoundResponse "/global/file-not-found.html" ns_param ServerBusyResponse "/global/busy.html" ns_param ServerInternalErrorResponse "/global/error.html" ns_param ForbiddenResponse "/global/forbidden.html" ns_param UnauthorizedResponse "/global/unauthorized.html" ns_param directoryproc _ns_dirlist ;# ...but not both! ns_param directorylisting fancy ;# Can be simple or fancy # # ADP custom parsers -- see adp.c # ns_section "ns/server/${server}/adp/parsers" ns_param fancy ".adp" ns_section "ns/server/${server}/module/nssock" ns_param port $httpport ns_param hostname $hostname ns_param address $address ns_section "ns/db/drivers" ns_param postgres /usr/local/aolserver/bin/postgres.so ; # Load PostgreSQL driver # # Database Pools: This is how AOLserver ``talks'' to the RDBMS. You need three for # OpenACS: main, log, subquery. Make sure to replace ``yourdb'' and ``yourpassword'' # with the actual values for your db name and the password for it. # AOLserver can have different pools connecting to different databases and even different # different database servers. # ns_section "ns/db/pools" ns_param main "OpenACS Main Pool" ns_param log "OpenACS Log Pool" ns_param subquery "OpenACS Subquery Pool" ns_section "ns/db/pool/main" ns_param Driver postgres ns_param Connections 5 ;# 5 is a good number. Increase according to your needs ns_param DataSource localhost:5432:db ns_param User username ;# User and password AOLserver will use to connect ns_param Password "password" ns_param Verbose On ;# Set it to On to see all queries. Good for debugging SQL. ns_param LogSQLErrors On ns_param ExtendedTableInfo On ns_param MaxOpen 1000000000 ;# Uncommenting these two cause AOLserver to keep the ns_param MaxIdle 1000000000 ;# db connection open. Can be a good thing, up tp your needs. ns_section "ns/db/pool/log" ns_param Driver postgres ns_param Connections 5 ns_param DataSource localhost::db ns_param User username ns_param Password "password" ns_param Verbose Off ns_param LogSQLErrors On ns_param ExtendedTableInfo On ns_param MaxOpen 1000000000 ns_param MaxIdle 1000000000 ns_section "ns/db/pool/subquery" ns_param Driver postgres ns_param Connections 4 ns_param DataSource localhost::db ns_param User username ns_param Password "password" ns_param Verbose Off ns_param LogSQLErrors On ns_param ExtendedTableInfo On ns_param MaxOpen 1000000000 ns_param MaxIdle 1000000000 ns_section "ns/server/${server}/db" ns_param Pools "*" ns_param DefaultPool "main" ns_section "ns/server/${server}/module/nslog" ns_param rolllog true ;# Should we roll log? ns_param rollonsignal true ;# Roll log on SIGHUP ns_param rollhour 0 ;# Time to roll log ns_param maxbackup 360 ;# Max number to keep around when rolling ns_section "ns/server/${server}/modules" ns_param nssock ${bindir}/nssock.so ns_param nslog ${bindir}/nslog.so ns_param nsperm ${bindir}/nsperm.so # ns_param nscgi ${bindir}/nscgi.so # ns_param nsjava ${bindir}/libnsjava.so # ## nsssl: loads only if requisite files already exist (see top of this # file). # if { [file exists $sslcertfile] && [file exists $sslkeyfile] } { ns_param nsssl ${bindir}/nsssle.so } else { ns_log warning "nsd.tcl: nsssl not loaded because key/cert files do not exist." } # # To Source OpenACS Config File. You need a "yourservername.tcl" file in the # OpenACS parameters dir. e.g.: say your server is "server1" then you'd need a # file named /web/server1/parameters/server1.tcl # (Hint: Rename ad.tcl to yourservername.tcl and edit it.) # source /home/user/openacs-3.2.5/parameters/acs.tcl ns_log notice "nsd.tcl: finished reading config file."