Forum OpenACS Development: Re: Ain’t No ☀shine

5: Re: Ain’t No ☀shine (response to 1)
Posted by Gustaf Neumann on
What happens is the following:
  • tcl8.6 has a more aggressive sharing of Tcl_Objs than tcl8.5
  • in tcl8.6, $v passed to ns_return is of type bytearray (meaning "binary data")
  • the conversion to binary happens in the xowiki function, when the content of $v is written via "-encoding binary".
  • interestingly, the length of the bytearray is reported from Tcl via Tcl_GetByteArrayFromObj() is 1, although the "black sun" is actually 3 bytes long.
  • the conversion to the bytearray can be done more easily with the roundtrip over base64 than writing to a file via xowiki (see below)

One can easily test the behavior by commenting in/out the base64 round-trip in the code below:

set v "☀"
#set v "*"
set v [binary decode base64 [binary encode base64 $v]]
ns_return 200 "text/plain" $v
i am not sure, what the best solution is, and whether this is a bug in tcl8.6 (the intended semantics of "number of chars" vs. "number of bytes" is not always clear to me). i will look closer, when time permits.

What can you do in the meanwhile? The easiest is to use "*" instead of "☀" :), or can certainly force a conversion from bytearray by string append operations (as you showed above).

7: Re: Ain’t No ☀shine (response to 5)
Posted by Gustaf Neumann on

if possible, get the tip version of NaviServer from bitbucket. it should work with your example with tcl 8.7a1 and probably as well with tcl8.6.*. My example did not work for other reasons, the example below is supposed to work. Handling binary data in tcl is still a pain (and a moving target).

set v "☃"
set v [encoding convertfrom utf-8 [binary decode base64 [binary encode base64 [encoding convertto utf-8 $v]]]]
ns_return 200 "text/plain" $v