i18n.xml
Delivered as text/xml
[ hide source ] | [ make this the default ]
File Contents
<?xml version='1.0' ?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
<!ENTITY % myvars SYSTEM "../variables.ent">
%myvars;
]>
<chapter id="i18n">
<title>Internationalization</title>
<authorblurb>
<para>
By <ulink url="mailto:peter@collaboraid.biz">Peter Marklund</ulink>
and <ulink url="mailto:lars@collaboraid.biz">Lars Pind</ulink>
</para>
</authorblurb>
<sect1 id="i18n-overview">
<title>Internationalization and Localization Overview</title>
<table id="i18n-l10n-process">
<title>Internationalization and Localization Overview</title>
<tgroup cols="3">
<colspec colname="step"/>
<colspec colname="description"/>
<colspec colname="who"/>
<thead>
<row>
<entry colname="step">Stage</entry>
<entry colname="description">Task</entry>
<entry colname="who">Who</entry>
</row>
</thead>
<tbody>
<row>
<entry colname="step">Internationalization</entry>
<entry>Package Developer uses the acs-lang tools to replace all visible text in a package with <emphasis>message keys</emphasis>. (<link linkend="i18n-introduction">More information</link>)</entry>
<entry>Package Developer</entry>
</row>
<row>
<entry morerows="1" colname="step">Release Management</entry>
<entry>The newly internationalized package is released.</entry>
<entry>Package Developer</entry>
</row>
<row>
<entry>The translation server is updated with the new package.</entry>
<entry>Translation server maintainers</entry>
</row>
<row>
<entry colname="step">Localization</entry>
<entry>Translators work in their respective locales to write text for each message key. (<link linkend="i18n-translators">More information</link>)</entry>
<entry>Translators</entry>
</row>
<row>
<entry morerows="2" colname="step">Release Management</entry>
<entry>The translated text in the database of the translation server is compared to the current translations in the OpenACS code base, conflicts are resolved, and the new text is written to catalog files on the translation server.</entry>
<entry>Translation server maintainers</entry>
</row>
<row>
<entry>The catalog files are committed to the OpenACS code base.</entry>
<entry>Translation server maintainers</entry>
</row>
<row>
<entry>A new version of OpenACS core and/or affected packages is released and published in the OpenACS.org repository.</entry>
<entry>Release Manager</entry>
</row>
<row>
<entry morerows="1" colname="step">Upgrading</entry>
<entry>Site Administrators upgrade their OpenACS sites, either via the automatic upgrade from the Repository or via tarball or CVS </entry>
<entry>Site Administrators</entry>
</row>
<row>
<entry>Site Administrators import the new translations. Existing local translations, if they exist, are not overwritten.</entry>
<entry>Site Administrators</entry>
</row>
</tbody>
</tgroup>
</table>
</sect1>
<sect1 id="i18n-introduction">
<title>How Internationalization/Localization works in OpenACS</title>
<para>
This document describes how to develop internationalized OpenACS
packages, including writing new packages with
internationalization and converting old packages. Text that
users might see is "localizable text"; replacing monolingual text
and single-locale date/time/money functions with generic
functions is "internationalization"; translating first
generation text into a specific language is "localization." At
a minimum, all packages should be internationalized. If you do
not also localize your package for different locales, volunteers
may use a public "localization server" to submit suggested text.
Otherwise, your package will not be usable for all locales.
</para>
<para>
The main difference between monolingual and internationalized
packages is that all user-visible text in the code of an internationalized
package are coded as "message keys." The message keys
correspond to a message catalog, which contains versions of the
text for each available language. Script files (.adp and .tcl and .vuh), database files (.sql), and APM parameters are affected.
</para>
<para>
Other differences include: all dates read or written to the
database must use internationalized functions. All displayed
dates must use internationalized functions. All displayed
numbers must use internationalized functions.
</para>
<para>
Localizable text must be handled in ADP files, in Tcl files, and
in APM Parameters. OpenACS provides two approaches, message
keys and localized ADP files. For ADP pages which are mostly
code, replacing the message text with message key placeholders
is simpler. This approach also allows new translation in the
database, without affecting the filesystem. For ADP pages
which are static and mostly text, it may be easier to create a
new ADP page for each language. In this case, the pages are
distinguished by a file naming convention.
</para>
<sect2 id="i18n-content">
<title>User Content</title>
<para>OpenACS does not have a general system for supporting multiple, localized versions of user-input content. This document currently refers only to internationalizing the text in the package user interface.</para>
</sect2>
<sect2 id="i18n-locale-templates">
<title>Separate Templates for each Locale</title>
<para>If the request processor finds a filenamed <computeroutput>filename.locale.adp</computeroutput>, where locale matches the user's locale, it will process that file instead of <computeroutput>filename.adp</computeroutput>. For example, for a user with locale <computeroutput>tl_PH</computeroutput>, the file <computeroutput>index.tl_PH.adp</computeroutput>, if found, will be used instead of <computeroutput>index.adp</computeroutput>. The locale-specific file should thus contain text in the language appropriate for that locale. The code in the page, however, should still be in English. Message keys are processed normally.</para>
</sect2>
<sect2 id="i18n-message-catalog">
<title>Message Catalogs</title>
<sect3 id="i18n-message-catalog-adps">
<title>Message Keys in Template Files (ADP Files)</title>
<para>
Internationalizing templates is about replacing human readable
text in a certain language with internal message keys, which
can then be dynamically replaced with real human language in
the desired locale. Message keys themselves should be in
ASCII English, as should all code. Three different syntaxes
are possible for message keys.
</para>
<para>
"Short" syntax is the recommended syntax and should be used
for new development. When internationalizing an existing
package, you can use the "temporary" syntax, which the APM can
use to auto-generate missing keys and automatically translate
to the short syntax. The "verbose" syntax is useful while
developing, because it allows default text so that the page is
usable before you have done
localization. </para>
<itemizedlist>
<listitem>
<para>
The <emphasis role="strong">short</emphasis>:
<computeroutput>#<replaceable>package_key.message_key</replaceable>#</computeroutput>
</para>
<para>
The advantage of the short syntax is that it's short. It's
as simple as inserting the value of a variable. Example:
<computeroutput>#<replaceable>forum.title</replaceable>#</computeroutput>
</para>
</listitem>
<listitem>
<para>
The <emphasis role="strong">verbose</emphasis>: <computeroutput><trn
key="<replaceable>package_key.message_key</replaceable>"
locale="<replaceable>locale</replaceable>"><replaceable>default
text</replaceable></trn></computeroutput>
</para>
<para>
The verbose syntax allows you to specify a default text in
a certain language. This syntax is not recommended
anymore, but it can be convenient for development, because
it still works even if you haven't created the message
in the message catalog yet, because what it'll do is
create the message key with the default text from the tag
as the localized message. Example: <emphasis><trn
key="forum.title" locale="en_US">Title</trn></emphasis>
</para>
</listitem>
<listitem>
<para>
The <emphasis role="strong">temporary</emphasis>:
<computeroutput> <#<replaceable>message_key</replaceable>
<replaceable>original text</replaceable>#></computeroutput>
</para>
<para>
This syntax has been designed to make it easy to
internationalize existing pages. This is not a syntax that
stays in the page. As you'll see later, it'll be replaced
with the short syntax by a special feature of the APM. You
may leave out the message_key by writing an underscore (_)
character instead, in which case a message key will be
auto-generated by the APM. Example: <emphasis><_ Title></emphasis>
</para>
</listitem>
</itemizedlist>
<para>
We recommend the short notation for new package development.
</para>
</sect3>
<sect3 id="i18n-message-catalog-tcl">
<title>Message Keys in Tcl Files</title>
<para>
In adp files message lookups are typically done with the syntax
<computeroutput>\#package_key.message_key\#</computeroutput>. In Tcl
files all message lookups *must* be on either of the following formats:
</para>
<para>
<itemizedlist>
<listitem><para>Typical static key lookup: <computeroutput>[_ package_key.message_key]</computeroutput> - The message key and package key used here must be string literals, they can't result from variable evaluation.</para>
</listitem>
<listitem><para>
Static key lookup with nondefault locale: <computeroutput>[lang::message::lookup $locale package_key.message_key]</computeroutput> - The message key and package key used here must be string literals, they can't result from variable evaluation.</para>
</listitem>
<listitem>
<para>
Dynamic key lookup: <computeroutput>[lang::util::localize $var_with_embedded_message_keys]</computeroutput> - In this case the message keys in the variable <computeroutput>var_with_embedded_message_keys</computeroutput> must appear as string literals <computeroutput>\#package_key.message_key\#</computeroutput> somewhere in the code. Here is an example of a dynamic lookup:
<computeroutput>set message_key_array {
dynamic_key_1 \#package_key.message_key1\#
dynamic_key_2 \#package_key.message_key2\#
}
set my_text [lang::util::localize $message_key_array([get_dynamic_key])]
</computeroutput>
</para>
</listitem>
</itemizedlist>
</para>
<para>
Translatable texts in page Tcl scripts are often found in page titles,
context bars, and form labels and options. Many times the texts are
enclosed in double quotes. The following is an example of grep commands
that can be used on Linux to highlight translatable text in Tcl files:
</para>
<screen>
# Find text in double quotes
<userinput>find -iname '*.tcl'|xargs egrep -i '"[a-z]'</userinput>
# Find untranslated text in form labels, options and values
<userinput>find -iname '*.tcl'|xargs egrep -i '\-(options|label|value)'|egrep -v '<#'|egrep -v '\-(value|label|options)[[:space:]]+\$[a-zA-Z_]+[[:space:]]*\\?[[:space:]]*$'</userinput>
# Find text in page titles and context bars
<userinput>find -iname '*.tcl'|xargs egrep -i 'set (title|page_title|context_bar) '|egrep -v '<#'</userinput>
# Find text in error messages
<userinput>find -iname '*.tcl'|xargs egrep -i '(ad_complain|ad_return_error)'|egrep -v '<#'</userinput>
</screen>
<para>
You may mark up translatable text in Tcl library files and Tcl pages
with temporary tags on the <#key text#> syntax.
If you have a sentence or paragraph of text with
variables and or procedure calls in it you should in most cases
try to turn the whole text into one
message in the catalog (remember that translators is made easier the longer the phrases to translate are). In those cases, follow these steps:
</para>
<itemizedlist>
<listitem>
<para>
For each message call in the text, decide on a variable name and replace
the procedure call with a variable lookup on the syntax %var_name%. Remember
to initialize a Tcl variable with the same name on some line above the text.</para></listitem>
<listitem><para>If the text is in a Tcl file you must replace variable lookups
(occurrences of $var_name or ${var_name}) with %var_name%</para></listitem>
<listitem><para>You are now ready to follow the normal procedure and mark up the text using a
tempoarary message tag (<#_ text_with_percentage_vars#>) and run the action
replace tags with keys in the APM.</para></listitem>
</itemizedlist>
<para>
The variable values in the message are usually fetched with upvar, here is an example from dotlrn:
<computeroutput>
ad_return_complaint 1 "Error: A [parameter::get -parameter classes_pretty_name]
must have <em>no</em>[parameter::get -parameter class_instances_pretty_plural] to be deleted"
</computeroutput>
was replaced by:
<computeroutput>
set subject [parameter::get -localize -parameter classes_pretty_name]
set class_instances [parameter::get -localize -parameter class_instances_pretty_plural]
ad_return_complaint 1 [_ dotlrn.class_may_not_be_deleted]
</computeroutput>
</para>
<para>
This kind of interpolation also works in adp files where adp variable values will be inserted into the message.
</para>
<para>
Alternatively, you may pass in an array list of the variable values to be interpolated into the message so that
our example becomes:
</para>
<screen>
<userinput>set msg_subst_list [list subject [parameter::get -localize -parameter classes_pretty_name] class_instances [parameter::get -localize -parameter class_instances_pretty_plural]]
ad_return_complaint 1 [_ dotlrn.class_may_not_be_deleted $msg_subst_list]
</userinput>
</screen>
<para>
When we were done going through the Tcl files we ran the following
commands to check for mistakes:
</para>
<screen>
# Message tags should usually not be in curly braces since then the message lookup may not be
# executed then (you can usually replace curly braces with the list command). Find message tags
# in curly braces (should return nothing, or possibly a few lines for inspection)
<userinput>find -iname '*.tcl'|xargs egrep -i '\{.*<#'</userinput>
# Check if you've forgotten space between default key and text in message tags (should return nothing)
<userinput>find -iname '*.tcl'|xargs egrep -i '<#_[^ ]'</userinput>
# Review the list of Tcl files with no message lookups
<userinput>for tcl_file in $(find -iname '*.tcl'); do egrep -L '(<#|\[_)' $tcl_file; done</userinput>
</screen>
<para>
When you feel ready you may visit your package in the
<ulink url="/acs-admin/apm">package manager</ulink>
and run the action "Replace tags with keys
and insert into catalog" on the Tcl files that you've edited to
replace the temporary tags with calls to the message lookup
procedure.
</para>
<sect4 id="i18n-date-time-number">
<title>Dates, Times, and Numbers in Tcl files</title>
<para>
Most date, time, and number variables are calculated in Tcl files. Dates and times must be converted when stored in the database,
when retrieved from the database, and when displayed. All dates
are stored in the database in the server's timezone, which is an
APM Parameter set at
<computeroutput>/acs-lang/admin/set-system-timezone</computeroutput>
and readable at
<computeroutput>lang::system::timezone.</computeroutput>. When
retrieved from the database and displayed, dates and times must
be localized to the user's locale.
</para>
</sect4>
</sect3>
<sect3 id="i18n-message-apm-params" xreflabel="Multilingual APM Parameters">
<title>APM Parameters</title>
<para>
Some parameters contain text that need to be localized. In
this case, instead of storing the real text in the parameter,
you should use message keys using the short notation above,
i.e. <emphasis
role="strong">#<emphasis>package_key.message_key</emphasis>#</emphasis>.
</para>
<para>
In order to avoid clashes with other uses of the hash
character, you need to tell the APM that the parameter value
needs to be localized when retrieving it. You do that by saying:
<emphasis role="strong">parameter::get -localize</emphasis>.
</para>
<para>
Here are a couple of examples. Say we have the following two
parameters, taken directly from the dotlrn package.
</para>
<informaltable frame="all">
<tgroup cols="2" colsep="1" rowsep="1">
<colspec colname="c1"/>
<colspec colname="c2"/>
<thead>
<row>
<entry>Parameter Name</entry>
<entry>Parameter Value</entry>
</row>
</thead>
<tbody>
<row>
<entry>class_instance_pages_csv</entry>
<entry><computeroutput>#<replaceable>dotlrn.class_page_home_title</replaceable>#</computeroutput>,Simple 2-Column;<computeroutput>#<replaceable>dotlrn.class_page_calendar_title</replaceable>#</computeroutput>,Simple 1-Column;<computeroutput>#<replaceable>dotlrn.class_page_file_storage_title</replaceable>#</computeroutput>,Simple 1-Column</entry>
</row>
<row>
<entry>departments_pretty_name</entry>
<entry><computeroutput>#<replaceable>departments_pretty_name</replaceable>#</computeroutput></entry>
</row>
</tbody>
</tgroup>
</informaltable>
<para>
Then, depending on how we retrieve the value, here's what we get:
</para>
<informaltable frame="all">
<tgroup cols="2" colsep="1" rowsep="1">
<colspec colname="c1"/>
<colspec colname="c2"/>
<thead>
<row>
<entry>Command used to retrieve Value</entry>
<entry>Retrieved Value</entry>
</row>
</thead>
<tbody>
<row>
<entry>parameter::get <emphasis role="strong">-localize</emphasis> -parameter class_instances_pages_csv</entry>
<entry>Kurs Startseite,Simple 2-Column;Kalender,Simple 1-Column;Dateien,Simple 1-Column</entry>
</row>
<row>
<entry>parameter::get <emphasis role="strong">-localize</emphasis> -parameter departments_pretty_name</entry>
<entry>Abteilung</entry>
</row>
<row>
<entry>parameter::get -parameter departments_pretty_name</entry>
<entry><computeroutput>#<replaceable>departments_pretty_name</replaceable>#</computeroutput></entry>
</row>
</tbody>
</tgroup>
</informaltable>
<para>
The value in the rightmost column in the table above is the
value returned by an invocation of parameter::get. Note that
for localization to happen you must use the -localize flag.
</para>
<para>
The locale used for the message lookup will be the locale of
the current request, i.e. lang::conn::locale or ad_conn
locale.
</para>
<para>
Developers are responsible for creating the keys in the message
catalog, which is available at <computeroutput>/acs-lang/admin/</computeroutput>
</para>
</sect3>
</sect2>
</sect1>
<sect1 id="i18n-convert">
<title>How to Internationalize a Package</title>
<tip>
<para>
For multilingual websites we recommend using the UTF8
charset. In order for AOLserver to use utf8 you need to set
the config parameters OutputCharset and
URLCharset to utf-8 in your AOLserver config file (use the etc/config.tcl
template file). This is the default for OpenACS 5.1 and later. For sites running on Oracle you need to make
sure that AOLserver is running with the NLS_LANG environment
variable set to .UTF8. You should set this variable in the
nsd-oracle run script (use the
acs-core-docs/www/files/nds-oracle.txt template file).
</para>
</tip>
<orderedlist>
<listitem>
<formalpara>
<title>Replace all text with temporary message tags</title>
<para>From<computeroutput>/acs-admin/apm/</computeroutput>, select a
package and then click on
<computeroutput>Internationalization</computeroutput>, then
<computeroutput>Convert ADP, Tcl, and SQL files to using the
message catalog.</computeroutput>. This pass only changes the adp files; it does not affect catalog files or the catalog in the database.</para>
</formalpara>
<mediaobject>
<imageobject>
<imagedata fileref="images/i18n-1.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
<para>You will now be walked through all of the selected adp pages. The UI shows you the intended changes and lets you edit or cancel them key by key.</para>
<mediaobject>
<imageobject>
<imagedata fileref="images/i18n-2.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
</listitem>
<listitem>
<formalpara>
<title>Replace the temporary message tags in ADP files</title>
<para>From the same <computeroutput>Convert ADP ...</computeroutput> page in <computeroutput>/acs-admin/apm</computeroutput> as in the last step, repeat the process but deselect <computeroutput>Find human language text ...</computeroutput> and select <computeroutput>Replace <# ... #> tags ...</computeroutput> and click OK. This step replaces all of the temporary tags with "short" message lookups, inserts the message keys into the database message catalog, and then writes that catalog out to an XML file.</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Replace human-readable text in Tcl files with temporary tags</title>
<para>Examine all of the Tcl files in the packages for human-readable text and replace it with temporary tags. The temporary tags in Tcl are slightly different from those in ADP. If the first character in the temporary tag is an underscore (<computeroutput>_</computeroutput>), then the message keys will be auto-generated from the original message text. Here is an unmodified Tcl file:</para>
</formalpara>
<programlisting>
set title "Messages for $a(name) in $b(label)"
set context [list [list . "SimPlay"] \
[list [export_vars -base case-admin { case_id }] \
"Administer $a(name)"] \
"Messages for $a(name)"]
</programlisting>
<para>... and here is the same file after temporary message tags have been manually added:</para>
<programlisting>
set title <#admin_title Messages for %a.name% in %b.label%#>
set context [list [list . <#_ SimPlay#>] \
[list [export_vars -base case-admin { case_id }] \
<#_ Administer %a.name%#>] \
<#_ Messages for %a.name%#>]
</programlisting>
<para>Note that the message key <computeroutput>case_admin_page_title</computeroutput> was manually selected, because an autogenerated key for this text, with its substitute variables, would have been very confusing
</para>
</listitem>
<listitem>
<formalpara>
<title>Replace the temporary message tags in Tcl files</title>
<para>Repeat step 2 for Tcl files. Here is the example Tcl file after conversion:</para>
</formalpara>
<programlisting>
set title [_ simulation.admin_title]
set context [list [list . [_ simulation.SimPlay]] \
[list [export_vars -base case-admin { case_id }] \
[_ simulation.lt_Administer_name_gt]] \
[_ simulation.lt_Messages_for_role_pre]]
</programlisting>
</listitem>
<listitem>
<formalpara>
<title>Internationalize SQL Code</title>
<para>If there is any user-visible Tcl code in the .sql or .xql files, internationalize that the same way as for the Tcl files.</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Internationalize Package Parameters</title>
<para>
See <xref linkend="i18n-message-apm-params"/>
</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Internationalize Date and Time queries</title>
<para></para>
</formalpara>
<orderedlist>
<listitem>
<para>Find datetime in .xql files. Use command line tools to find suspect SQL code:</para>
<programlisting>grep -r "to_char.*H" *
grep -r "to_date.*H" *
</programlisting>
</listitem>
<listitem>
<para>In SQL statements, replace the format string with the ANSI standard format, <computeroutput>YYYY-MM-DD HH24:MI:SS</computeroutput> and change the field name to *_ansi so that it cannot be confused with previous, improperly formatting fields. For example,</para>
<programlisting>to_char(timestamp,'MM/DD/YYYY HH:MI:SS') as foo_date_pretty</programlisting>
<para>becomes</para>
<programlisting>to_char(timestamp,'YYYY-MM-DD HH24:MI:SS') as foo_date_ansi</programlisting>
</listitem>
<listitem>
<para>In Tcl files where the date fields are used, convert the datetime from local server timezone, which is how it's stored in the database, to the user's timezone for display. Do this with the localizing function <computeroutput><ulink url="/api-doc/proc-view?proc=lc_time_system_to_conn">lc_time_system_to_conn</ulink></computeroutput>:</para>
<programlisting>
set foo_date_ansi [lc_time_system_to_conn $foo_date_ansi]</programlisting>
<para>When a datetime will be written to the database, first convert it from the user's local time to the server's timezone with <computeroutput><ulink url="/api-doc/proc-view?proc=lc%5ftime%5fconn%5fto%5fsystem">lc_time_conn_to_system</ulink></computeroutput>.
</para>
</listitem>
<listitem>
<para>When a datetime field will be displayed, format it using the localizing function <computeroutput><ulink url="/api-doc/proc-view?proc=lc_time_fmt">lc_time_fmt</ulink></computeroutput>. lc_time_fmt takes two parameters, datetime and format code. Several format codes are usable for localization; they are placeholders that format dates with the appropriate codes for the user's locale. These codes are: <computeroutput>%x, %X, %q, %Q, and %c.</computeroutput></para>
<programlisting>set foo_date_pretty [lc_time_fmt $foo_date_ansi "%x %X"]</programlisting>
<para>
Use the <computeroutput>_pretty</computeroutput> version in your ADP page.
</para>
<itemizedlist>
<listitem>
<para>
%c: Long date and time (Mon November 18, 2002 12:00 AM)
</para>
</listitem>
<listitem>
<para>
%x: Short date (11/18/02)
</para>
</listitem>
<listitem>
<para>
%X: Time (12:00 AM)
</para>
</listitem>
<listitem>
<para>
%q: Long date without weekday (November 18, 2002)
</para>
</listitem>
<listitem>
<para>
%Q: Long date with weekday (Monday November 18, 2002)
</para>
</listitem>
</itemizedlist>
<para>
The "q" format strings are OpenACS additions; the rest follow unix standards (see <computeroutput>man
strftime</computeroutput>).
</para>
</listitem>
</orderedlist>
</listitem>
<listitem>
<formalpara>
<title>Internationalize Numbers</title>
<para>
To internationalize numbers, use <computeroutput>lc_numeric $value</computeroutput>, which formats the number using the appropriate decimal point and thousand separator for the locale.
</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Internationalizing Forms</title>
<para>When coding forms, remember to use message keys for each piece of text that is user-visible, including form option labels and button labels.</para>
</formalpara>
</listitem>
<listitem>
<formalpara id="catalog-consistency-check">
<title>Checking the Consistency of Catalog Files</title>
<para>
This section describes how to check that the set of keys used in
message lookups in tcl, adp, and info files and the set of keys in
the catalog file are identical. The scripts below assume that
message lookups in adp and info files are on the format
\#package_key.message_key\#, and that message lookups in Tcl files
are always is done with one of the valid lookups described above. The script further assumes
that you have perl installed and in your path. Run the script like
this:
<computeroutput>
acs-lang/bin/check-catalog.sh package_key
</computeroutput>
</para>
</formalpara>
<para>
where package_key is the key of the package that you want to
test. If you don't provide the package_key argument then all
packages with catalog files will be checked.
The script will run its checks primarily on en_US XML catalog files.
</para>
</listitem>
</orderedlist>
<sect2>
<title>Avoiding common i18n mistakes</title>
<itemizedlist>
<listitem>
<formalpara>
<title>Replace complicated keys with longer, simpler keys</title>
<para>When writing in one language, it is possible to create clever code to make correct text. In English, for example, you can put an <computeroutput>if</computeroutput> command at the end of a word which adds "s" if a count is anything but 1. This pluralizes nouns correctly based on the data. However, it is confusing to read and, when internationalized, may result in message keys that are both confusing and impossible to set correctly in some languages. While internationalizing, watch out that the automate converter does not create such keys. Also, refactor compound text as you encounter it.</para>
</formalpara>
<para>The automated system can easily get confused by tags within message texts, so that it tries to create two or three message keys for one long string with a tag in the middle. In these cases, uncheck those keys during the conversion and then edit the files directly. For example, this code:</para>
<programlisting> <p class="form-help-text"><b>Invitations</b> are sent,
when this wizard is completed and casting begins.</p></programlisting>
<para>has a bold tag which confuses the converter into thinking there are two message keys for the text beginning "Invitations ..." where there should be one:</para>
<mediaobject>
<imageobject>
<imagedata fileref="images/i18n-3.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
<para>Instead, we cancel those keys, edit the file manually, and put in a single temporary message tag:</para>
<programlisting> <p class="form-help-text"> <#Invitations_are_sent <b>Invitations</b> are sent,
when this wizard is completed and casting begins.#>
</p></programlisting>
<para>Complex if statements may produce convoluted message keys that are very hard to localize. Rewrite these if statements. For example:</para>
<programlisting>Select which case <if @simulation.casting_type@ eq "open">and
role</if> to join, or create a new case for yourself. If you do not
select a case <if @simulation.casting_type@ eq "open">and role</if>
to join, you will be automatically assigned to a case <if
@simulation.casting_type@ eq "open">and role</if> when the
simulation begins.</programlisting>
<para>... can be rewritten:</para>
<programlisting><if @simulation.casting_type@ eq "open">
Select which case and role to join, or create a new case for
yourself. If you do not select a case and role to join, you will
be automatically assigned to a case and role when the simulation
begins.
</if>
<else>
Select which case to join, or create a new case for
yourself. If you do not select a case to join, you will
be automatically assigned to a case when the simulation
begins.
</else></programlisting>
<para>Another example, where bugs are concatenated with a number:</para>
<programlisting><if @components.view_bugs_url@ not nil>
<a href="@components.view_bugs_url@" title="View the @pretty_names.bugs@ for this component">
</if>
@components.num_bugs@
<if @components.num_bugs@ eq 1>
@pretty_names.bug@
</if>
<else>
@pretty_names.bugs@
</else>
<if @components.view_bugs_url@ not nil>
</a>
</if>
<if @components.view_bugs_url@ not nil>
<a href="@components.view_bugs_url@" title="#­bug-tracker.View_the_bug_fo_component#">
</if>
@components.num_bugs@
<if @components.num_bugs@ eq 1>
@pretty_names.bug@
</if>
<else>
@pretty_names.bugs@
</else>
<if @components.view_bugs_url@ not nil>
</a>
</if>
</programlisting>
<para>It would probably be better to do this as something like:</para>
<programlisting><if @components.view_bugs_url@ not nil>
<if @components.num_bugs@ eq 1>
<a href="@components.view_bugs_url@" title="#­bug-tracker.View_the_bug_fo_component#">#­bug-tracker.one_bug#</a>
</if><else>
<a href="@components.view_bugs_url@" title="#­bug-tracker.View_the_bug_fo_component#">#­bug-tracker.N_bugs#</a>
</else>
</if></programlisting>
</listitem>
<listitem>
<formalpara>
<title>Don't combine keys in display text</title>
<para>Converting a phrase from one language to another is usually more complicated than simply replacing each word with an equivalent. When several keys are concatenated, the resulting word order will not be correct for every language. Different languages may use expressions or idioms that don't match the phrase key-for-key. Create complete, distinct keys instead of building text from several keys. For example:</para>
</formalpara>
<para>Original code:</para>
<programlisting>multirow append links "New [bug_tracker::conn Bug]" </programlisting>
<para>Problematic conversion:</para>
<programlisting>multirow append links "[_ bug-tracker.New] [bug_tracker::conn Bug]"</programlisting>
<para>Better conversion:</para>
<programlisting>set bug_label [bug_tracker::conn Bug]
multirow append links "[_ bug-tracker.New_Bug]" "${url_prefix}bug-add"</programlisting>
<para>... and include the variable in the key: <computeroutput>"New %bug_label%"</computeroutput>. This gives translators more control over the phrase.</para>
<para>In this example of bad i18n, full name is created by concatenating first and last name (admittedly this is pervasive in the toolkit):</para>
<programlisting><a href="@past_version.maintainer_url@" title="#­bug-tracker.Email# @past_version.maintainer_email@">
@past_version.maintainer_first_names@ @past_version.maintainer_last_name@</a></programlisting>
</listitem>
<listitem>
<formalpara>
<title>Avoid unnecessary duplicate keys.</title>
<para>When phrases are exactly the same in several places, use a single key.</para>
</formalpara>
<para>For common words such as
Yes and No, you can use a library of keys at <ulink
url="/acs-lang/admin/message-list?package%5fkey=acs%2dkernel&locale=en%5fUS">acs-kernel</ulink>.
For example, instead of using
<computeroutput>myfirstpackage.Yes</computeroutput>, you
can use <computeroutput>acs-kernel.Yes</computeroutput>.
You can also use the <ulink
url="/acs-lang/admin/package-list?locale=en%5fUS">Message Key Search</ulink> facility to find duplicates.
Be careful, however, building up sentences from keys
because grammar and other elements may not be consistent
across different locales.</para>
<para>Additional discussion: <ulink
url="http://openacs.org/forums/message-view?message_id=164973">Re:
Bug 961 ("Control Panel" displayed instead of
"Administer")</ulink>, <ulink
url="http://openacs.org/forums/message-view?message_id=125235">Translation
server upgraded</ulink>, and <ulink url="http://openacs.org/forums/message-view?message_id=158580">Localization questions</ulink>.</para>
</listitem>
<listitem>
<formalpara>
<title>Don't internationalize internal code words</title>
<para>Many packages use code words or key words, such as "open" and "closed", which will never be shown to the user. They may match key values in the database, or be used in a switch or if statement. Don't change these.</para>
</formalpara>
<para>For example, the original code is </para>
<programlisting>workflow::case::add_log_data \
-entry_id $entry_id \
-key "resolution" \
-value [db_string select_resolution_code {}]</programlisting>
<para>This is incorrectly internationalized to </para>
<programlisting> workflow::case::add_log_data \
-entry_id $entry_id \
-key "[_ bug-tracker.resolution]" \
-value [db_string select_resolution_code {}]</programlisting>
<para>But <computeroutput>resolution</computeroutput> is a keyword in a table and in the code, so this breaks the code. It should not have been internationalized at all. Here's another example of text that should not have been internationalized:</para>
<programlisting>{show_patch_status "open"}</programlisting>
<para>It is broken if changed to</para>
<programlisting>{show_patch_status "[_ bug-tracker.open]"}</programlisting>
</listitem>
<listitem>
<formalpara>
<title>Fix automatic truncated message keys</title>
<para>The automatic converter may create unique but crytic message keys. Watch out for these and replace them with more descriptive keys. For example:</para>
</formalpara>
<programlisting>
<msg key="You">You can filter by this %component_name% by viisting %filter_url_string%</msg>
<msg key="You_1">You do not have permission to map this patch to a bug. Only the submitter of the patch
and users with write permission on this Bug Tracker project (package instance) may do so.</msg>
<msg key="You_2">You do not have permission to edit this patch. Only the submitter of the patch
and users with write permission on the Bug Tracker project (package instance) may do so.</msg></programlisting>
<para>These would be more useful if they were, "you_can_filter", "you_do_not_have_permission_to_map_this_patch", and "you_do_not_have_permission_to_edit_this_patch". Don't worry about exactly matching the english text, because that might change; instead try to capture the meaning of the phrase. Ask yourself, if I was a translator and didn't know how this application worked, would this key and text make translation easy for me?
</para>
<para>Sometimes the automatic converter creates keys that don't semantically match their text. Fix these:</para>
<programlisting><msg key="Fix">for version</msg>
<msg key="Fix_1">for</msg>
<msg key="Fix_2">for Bugs</msg></programlisting>
<para>Another example: <computeroutput>Bug-tracker component maintainer</computeroutput> was converted to <computeroutput>[_ bug-tracker.Bug-tracker]</computeroutput>. Instead, it should be <computeroutput>bug_tracker_component_maintainer</computeroutput>.</para>
</listitem>
<listitem>
<formalpara>
<title>Translations in Avoid "clever" message reuse</title>
<para>Translations may need to differ depending on the context in which
the message appears.
</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Avoid plurals</title>
<para>Different languages create plurals differently. Try to avoid keys which will change based on the value of a number. OpenACS does not currently support internationalization of plurals. If you use two different keys, a plural and a singular form, your application will not localize properly for locales which use different rules or have more than two forms of plurals.</para>
</formalpara>
</listitem>
<listitem>
<formalpara>
<title>Quoting in the message catalog for tcl</title>
<para>Watch out for quoting and escaping when editing text that is also code. For example, the original string </para>
</formalpara>
<programlisting>set title "Patch \"$patch_summary\" is nice."</programlisting>
<para>breaks if the message text retains all of the escaping that was in the Tcl command:</para>
<programlisting><msg>Patch \"$patch_summary\" is nice.</msg></programlisting>
<para>When it becomes a key, it should be:</para>
<programlisting><msg>Patch "$patch_summary" is nice.</msg></programlisting>
<para>Also, some keys had %var;noquote%, which is not needed since those
variables are not quoted (and in fact the variable won't even be
recognized so you get the literal %var;noquote% in the output).</para>
</listitem>
<listitem>
<formalpara>
<title>Be careful with curly brackets</title>
<para>Code within curly brackets isn't evaluated. Tcl uses curly brackets as an alternative way to build lists. But Tcl also uses curly brackets as an alternative to quotation marks for quoting text. So this original code</para>
</formalpara>
<programlisting>array set names { key "Pretty" ...} </programlisting>
<para>... if converted to </para>
<programlisting>array set names { key "[_bug-tracker.Pretty]" ...} </programlisting>
<para>... won't work since the _ func will not be called. Instead, it should be </para>
<programlisting>array set names [list key [_bug-tracker.Pretty] ...]</programlisting>
</listitem>
</itemizedlist>
</sect2>
</sect1>
<sect1 id="i18n-design">
<title>Design Notes</title>
<para>User locale is a property of ad_conn, <computeroutput>ad_conn locale</computeroutput>. The request processor sets this by calling <computeroutput>lang::conn::locale</computeroutput>, which looks for the following in order of precedence:</para>
<orderedlist>
<listitem><para>Use user preference for this package (stored in ad_locale_user_prefs)</para></listitem>
<listitem><para>Use system preference for the package (stored in apm_packages)</para>
</listitem>
<listitem><para>Use user's general preference (stored in user_preferences)</para></listitem>
<listitem><para>Use Browser header (<computeroutput>Accept-Language</computeroutput> HTTP header)</para></listitem>
<listitem><para>Use system locale (an APM parameter for acs_lang)</para></listitem>
<listitem><para>default to en_US</para></listitem>
</orderedlist>
<para>For ADP pages, message key lookup occurs in the templating engine. For Tcl pages, message key lookup happens with the <computeroutput>_</computeroutput> function. In both cases, if the requested locale is not found but a locale which is the default for the language which matches your locale's language is
found, then that locale is offered instead.</para>
</sect1>
<sect1 id="i18n-translators">
<title>Translator's Guide</title>
<para>Most translators use the <ulink url="http://translate.openacs.org">OpenACS Public Translation Server</ulink>, because the process of getting new message keys onto the server and getting new translations back into the distribution are handled by the maintainers of that machine. You can also do translation work on your own OpenACS site; this makes your own translations more readily available to you but also means that your work will not be shared with other users unless you take extra steps (contacting an OpenACS core developer or submitting a patch) to get your work back to the OpenACS core.</para>
<para>The basic steps for translators:</para>
<itemizedlist>
<listitem>
<para>Go to the <ulink url="/acs-lang">Localization</ulink> page and choose the locale that you are translating to. If the locale is not present you need to visit <ulink url="/acs-lang/admin">Administration of Localization</ulink> and create the locale.</para>
</listitem>
<listitem>
<formalpara>
<title>Translating with Translator Mode</title>
<para>To translate messages in the pages they appear, <ulink url="http://localhost:8008/acs-lang/admin/translator-mode-toggle">Toggle Translator Mode</ulink> and then browse to the page you want to translate. Untranslated messages will have a yellow background and a red star that you click to translate the message. Translated messages have a green star next to them that is a hyperlink to editing your translation. There is a history mechanism that allows you to see previous translations in case you would want to revert a translation.</para>
</formalpara>
<mediaobject>
<imageobject>
<imagedata fileref="images/translator-mode.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
<para>While in Translator mode, a list of all message keys appears at the bottom of each page.</para>
<mediaobject>
<imageobject>
<imagedata fileref="images/translations.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
</listitem>
<listitem>
<formalpara>
<title>Batch translation</title>
<para>To translate many messages at once, go to <ulink url="/acs-lang/admin">Administration of Localization</ulink>, click on the locale to translate, then click on a package, and then click <computeroutput>Batch edit these messages</computeroutput>.</para>
</formalpara>
<mediaobject>
<imageobject>
<imagedata fileref="images/translation-batch-edit.png" format="PNG" align="center"/>
</imageobject>
</mediaobject>
</listitem>
</itemizedlist>
<para>When creating a new locale based on an existing one, such as creating the Guatemalan version of Spanish, you can copy the existing locale's catalog files using the script <computeroutput>/packages/acs-core-docs/www/files/create-new-catalog.sh</computeroutput>.</para>
</sect1>
</chapter>
