Forum OpenACS Development: i18n of content repository content

Two ideas:

• If the store the nls_language in the cr-recisions, we use the revisions for multiple languages. To determine which is the live revision, we either create a live_p in cr_revisions or create a cr_item_live_revision_map, which shows the live revisions for each language. Obviously, the live_revision column in cr_items would be made obsolete and we have to make a unique constraint on (nls_language, cr_item_id)
• If we follow the cr_child_rel approach, the nls_language needs to go out of cr_revisions and into cr_items.

For the assessment system we look at various options, have a look at https://openacs.org/projects/openacs/packages/assessment/design/i18n. One core element is the overlaying of language as we assume that most content will not be translated. To avoid heavy joining we denormalized the localized tables, meaning all content to be localized would be stored within the table with a default_locale and only if the user does not support the default_locale we'd look up the localization table for additional text.

Dave, thanks for beginning this thread. The problem of handling i18n content is a special case of handling complex version content generally. We're grappling with that in our design for the Assessment package.

I've written up the current state of our thinking here: https://openacs.org/projects/openacs/packages/assessment/design/versioning

We want to leverage the CR in all of this, but we're not entirely clear how best to do that. This thread is an important one.

I don't like the idea of storing the different translations in one item's set of content revisions for several reasons:

• Malte's suggestion is a big change to the existing datamodel. Lots of code, including custom user code, would need to change. Lots of upgrade scripts written.

• People already claim the CR's too complex. This would make it more complex, adding fuel to their argument.

• I still want to investigate "non-versionable" content-items (item still linked to revision via live_revision and latest_revisions, but sharing the same acs_object row rather than having two objects - the default would be today's versional content item, so no existing client code would be broken). This would give us "lighter weight" content types for applications that don't need versioning, adding weight to the arguments of those of us who lobby for more usage of the CR.

• We should remember that the vast majority of sites are monolingual, not multilingual. Trade-offs, when necessary, shouldn't make the monolingual case more complex unless it is absolutely impossible to avoid because far more people are affected by this than the multilingual case.

Jun - the deletion of the original document question's a good one.  The reason Dave and I both thought about using cr_child_rel is because we both like the notion of denoting a "source document" and its translations.  In other words, we don't simply have a set of related documents but rather the original document is an authoritative one.  We felt like a translator would want to know which document is the authoritative source document, and translate from that (assuming they're fluent in the source language, no problem if you're hiring a translation service).  Translating from (say) English to Chinese and then to French seems more dodgy than simply translating from English to French, for instance ... more opportunity for error.

Of course not everyone will care about the "original" vs. "set of translations" relationship but we felt we should support it    natively in whatever datamodel/API Dave develops.

As far as relating other objects ... I see that as outside the scope of the content repository.  If an application wants to relate a set of translators to each translation, it can use acs-rels to do so easily enough or its own private methodology.  The project Dave's working on may require this and if so that code should be available as it is all to be GPL'd, so perhaps that could be turned into a generally useful API for others.

But outside the CR, I would think.  Remember the CR's used for a lot of things other than documents.  Such features could lie in some sort of common CMS API as we get further down the CMS path, perhaps?

I gave some thought to a TCL API for the CR to do what I need. I'm explicitly missing some functionality the CR gives, which does not mean I don't want them included. I just limit myself to the things needed with regards to I18N.

• Create new content (cr_item_create): This function will allow us to store new content in the CR, either as a new version, a new translation or a new revision. The API will be smart enough to detect the users wishes based on the information given. Switches:
  • -content_type: Content type of the item we want to create. This also defines what additional switches need to be given. Optional if -item_id is given.
  • -item_id: If supplied the system assumes we want to store a new revision of the cr_item, otherwise it will create a new item.
  • -language: Defines the language of the item, defaults to the system language. If item_id is given as well, check if the language stored with the item_id matches. If yes, assume it is a new revision, if not, create a new item with the parent_id "item_id" (if done via rels or any other method should not concern the API).
  • -xxx: All the switches necessary e.g. for the category system.
  • -yyyy: All the switches necessary to fill the attributes of the content_type. If a not optional value is missing, throw an error.

• Change live version: This function will define the live version. It only has to be called if the live version differentiates from the latest version (and we have versioned items in the first place).
  • -item_id: Item we want to change the live version for.
  • -revision_id: Revision of the "to be live" version.
  • -language: If given, this changes the "master" version. Usefull if you realize that the dutch version changes more often than the "master" french version and you want to display the dutch version on your page by default instead.

• Retrieve content: This function will retrieve the content of an item and put it automatically into variables in the callers context. As an additional benefit the variable cr_attributes in the callers context will contain a list of the variables set.
  • -item_id
  • -language: Language version of the item_id. If the item_id is not the master item_id for translations and the language does not match the item_id, try to find the corresponding item_id, by going up one level to the master_item and look from there for the language. (optional)
  • -revision_id: Always return the live_revision unless the revision_id is given. In this case just return this revision. (optional)
  • -attributes: List of attributes that should be set as variables in the callers context. If {all} is given, return all.

Okay, this is a very rough sketch, maybe it helps, maybe it doesn't and hopefully I just missed something and it already exists 😊.

I agree with Dave, smaller api is a bit better.  I also think there is no problem with Malte's concern.

There where times that I created a proc that called several bcms proc to make things easier.  For example... jun::create_page.  That makes a new item, a new version and a predefined content if no content is passed, it also sets the first revision to live.

Actually I take that back - the routines I wrote for GP take an explicit -new flag (pass in ad_form_new_p) and allows for passing in a pre-allocated key...

From a non-techie point of view, I think it is more
important to relate a translation to the content item
than to its versions, since translators will probably
not want to translate each version again but rather
adapt the changes incrementally (unless they do it
automatically, which guarantees poor quality, or have
it done by companies paid for line counts, which
guarantees high cost).

The distinction made above between different translation
paths ("localizaton" vs. "translation") will, IMO, be more
important to the administrators of new versions and
translations than to the reader, and the terminology
is confusing since incorrect: Localizaton is not the
end product of a translation process but may be more,
including, for instance, replacing a red cross by a red
(or green?) half-moon.

I am surprised how much effort is invested in determining
automated ways to have dotLRN present a localized page
rather than (a) let the user's browser settings negotiate
the language (does AOLserver support Apache's multiview
method at all?), and (b) let the user decide explicitely.
Especially when current quality translations are not
available, the locale setting selected for the UI need not
at all be the optimal choice for reading complicated texts.

The knowledge of which language a tranlation was translated from is not interesting except possibly to an administrator.

I disagree. It is important to the user, especially if he does not like the translation, to see what languages the content derived from. This way you can always have a link stating "view the original in language xyz". Furthermore, if the translation is plainly wrong (e.g. documentation), it might help the user to know that this documentation is a translation and *not* the original, therefore having the option to fall back on the original.

It might be my personal preference, but this is why I tend to read documents in the original language as long as I can understand it fair enough. Having read too many german translations and watched too many dubbed movies I think translations are a primary source for misunderstandings and therefore should be clearly marked as such.

I strongly disagree.

It is important in looking at any content of significance, to know if it is original source material, or a translation.

If it is a translation, the reader should:

a) know what language it is translated from (which may allow one to make sense of gramatical and cultural mistakes in the translation) and,

b) have access to the original source material if at all possible, so one may check the translation for oneself.

Obviously, with OpenACS, both of these should be possible.

Jesse

I do think so. Though this might result in a pretty long translation trail...

Ok, so it appears that exposing the tree of translations to the user is application specific. So it looks like it is generally useful to store this information.

Any bright ideas on what the relationships should be called?