Forum OpenACS Development: Re: Ain’t No ☀shine
The surprising thing in my "fconfigure" example is, that the variable v is not "written", but only used as input source for the file writing process. And this fact alone changes the state of the internal representation in a way, that leads to the problematic behavior.It may be surprising because you were presented a broken string plus file download. As I tried to explain, this is/was a bad interaction between not shaping the value (bytearray) properly on its way out of Tcl and NaviServer sniffing on the presence of a bytearray, but because of its brokeness presenting an octet-stream.
The fact the values in Tcl (the value Tcl_Obj referenced by variable v) is transmogrified into different internal representations (e.g., different bytearrays without you noticing) with or without an external (string) representation, this is at the heart of Tcl.
Some of the surprise will go away when using [encoding convertto] as shown below because it will take care of not touching any shared value:
The value being written out to the file will distinct from the one ending up in ns_return. As I said, one must follow the protocol.
set v "\u2600"; # ☀ ::xowiki::write_file /tmp/sun [encoding convertto utf-8 $v] ns_return 200 text/plain $v
Actually, the developer who faced this bug in the first place, tried to fix the problem using "convert", and even thought she had fixed it (using [encoding convertfrom [ns_conn encoding] $result]). However, this "fixed" the problem only for German umlauts, not for the three-byte sun (which was not part of the initially observed problem).
During code review I saw this "conversion-based fix" and immediately had the suspicion that there must be something wrong at some other place. Now that we have found the bug, or at least a solution within the framework, everything is fine! Thank you!