Forum OpenACS Development: Re: Charset difference with NaviServer and OpenACS 5.9:

Not so sure it's a headers issue as the Content-Type header is correct for the given link/page. I would first check to make sure the data stored in the db is correct.

That said, ad_return_top_of_page uses ReturnHeaders that, in turn, uses ns_startcontent, which sets the connection outputEncoding (see Ns_ConnSetEncoding in conn.c). The default connection outputEncoding is suppose to be utf-8 anyway.

FWIW, the outputEncoding is utilized in Ns_ConnWriteVChars (in connio.c) to convert the response bytes using Tcl_UtfToExternalDString (if outputEncoding is not set to utf-8) i.e. converting each byte of a utf-8 character before it is sent to the browser.

Thank you Gustaf, we are in agreement. My reply was meant to put more emphasis on the function of ns_startcontent (as opposed to the response headers) as it sets the connection outputEncoding and if that is not set to utf-8 (the default for outputCharset parameter) naviserver converts the actual response (via Tcl_UtfToExternalDString) before it is sent back to the browser. Converting a utf-8 string to external, converts each byte of a utf-8 character thus leading to the awkward result. Finally, I suggest checking the data in the db before delving deeper into encodings.

Hi!

Thanks for the quick replies first of all.

I spent a few hours producing sample code and unsuccessfully trying to understand character encoding in AOL/NaviServer. Until I found out that I simply have to replace ns_startcontent with ReturnHeaders. Sorry, this didn't become clear from your discussion.

- Maybe you want to update the documentation of ns_headers. It does't say anything about "configuring the streams", which it should. I understood it's just a variant of ns_write.
- Why did't you update ns_startcontent to behave consistently in NaviServer?

Even following the SF discussion on ns_headers I don't understand why you had to break something, even if it is "old" or "inconsistent".

This is all handled automatically now, and ns_startcontent should not be used.

Breaking stuff means increased costs for us (and others) to update. If you break stuff, then we'll need to fall back to the policy to update only if absolutely necessary. The last time we managed to stay 7 years in that mode...

Cheers,
Frank

Thanks a lot!!

Frank