Forum OpenACS Q&A: Response to Taking data fm asp page and storing in to unicode form in database

You are using Postgres? Make sure you compiled with unicode support,

 ./configure  --enable-locale --enable-recode --enable-multibyte --enable-unicode-conversion --with-maxba
ckends=64 --with-tcl --with-perl --with-openssl --with-CXX --enable-syslog --with-java
#

and when you create your database add the unicode encoding flag:

/usr/local/pgsql/bin/initdb  --encoding unicode  -D /usr/local/pgsql/data

The trick now is to make sure that users browsers are allowing them to enter form text as Unicode. Browsers will generally set input encoding to the same as the page's encoding, so make sure to set the MIME type of the page to "text/html; charset=UTF8" (or is it UTF-8?)

Then, you need to make sure that AOLserver reads the form data in the proper encoding (i.e., utf8). You can set the default encoding with the param

ns_section "ns/parameters"
  
# Set output charset encoding and default form input encoding to ShiftJIS
        ns_param   HackContentType 1
        ns_param   URLCharset      shift_jis
        ns_param   OutputCharset   shift_jis
        ns_param   HttpOpenCharset shift_jis

(substitute UTF8 for shift_jis above, or maybe "UTF-8", I forget whether it wants MIME encoding name or AOLserver's encoding names)

Or else, you can call ns_getform with a specific encoding parameter. Look at the source code, it is optional argument: proc ns_getform {{charset ""}}

Everything should just work 😊

I have done this kind of thing with Japanese text without any real difficulty (well OK, maybe a little difficulty). The trick is to remember that AOLserver uses UTF8 internally, the Postgres driver passes it to Postgres, and the only trouble is when AOLserver tries to apply charset conversions when it is either reading data from the broswer, or outputting data to the browser over the network stream.