Forum OpenACS Development: Lars Blogger multilanguage support

Collapse
Posted by Eduardo Santos on
Has someone worked in the possibility of having blog entries in many differente languages? I have been studying the lars-blogger datamodel in order to do so and it doesn't look like an easy task to be donne. I have to be able to store the locale for the entry, wich doesn't seem to be the worse thing, but the same entry_id must have different contents for each language.

My idea was to create a different table with the same contents from the pinds_blog_entries table, but with the locale option and another primary key, and also changing the structure of pinds_blog_entries table. I would have in the first table the entry_id (acs_object) and an auxiliary id wich would refer to the other table where the entry contents would be. So, instead of one I would have two tables for the entries.

However, I'm a little afraid to do that because it would change the whole db relationships (such as the search datasource), and I would have to find out the the "black holes" to fix.

I can't see how to do it without change the package datamodel. Has anybody worked on that and have a better idea?

Collapse
Posted by Malte Sussdorff on
Drop lars blogger entirely. Use XoWIKI Form Builder with the news form.

This is what we did and it gives you all the options of XoWIKI without having to modify lars blogger. Lars Blogger in my opinion is a dead end at the moment and we have written upgrade scripts that moves it's entries into XoWIKI.

Collapse
Posted by Dave Bauer on
Hey Malte,

Can you put your scripts in CVS, maybe in contrib, or otherwise make them available?

Collapse
Posted by Malte Sussdorff on
# Change the following variables to suit your environment
set pinds_package_id 1976
set xowiki_package_id 1153246

## Do not change anything below

db_1row template "select item_id as page_template, parent_id as xowiki_folder_id from cr_items where name = 'en:news-item' and content_type = '::xowiki::Form'"

db_foreach pinds_entry {select entry_id,creation_user, creation_ip, creation_date, entry_date as last_modified, p.title, title_url as detail_link, content, content_format, draft_p from pinds_blog_entries p, acs_objects o where p.entry_id = o.object_id and p.package_id = :pinds_package_id} {

  # Set publish status
    if {$draft_p} {
      set publish_status "production"
    } else {
      set publish_status "ready"
    }

  # Text conversion
    if {$content_format ne "text/html"} {
        set content [ad_html_text_convert -from $content_format -to "text/html" $content]
    }

    set creator [person::name -person_id $creation_user]


    set page [::xowiki::FormPage new \
        -page_template $page_template \
                  -creator $creator \
                  -instance_attributes "[list detail_link $detail_link]" \
                  -description {} \
                  -text "$content" \
                  -package_id $xowiki_package_id \
                  -name "en:pinds_$entry_id" \
                  -title $title \
                  -publish_status $publish_status \
                  -parent_id $xowiki_folder_id]
    $page save_new
    set revision_id [$page set revision_id]
    set item_id [$page set item_id]

    db_dml update_objects "update acs_objects set last_modified = :creation_date,creation_user = :creation_user, creation_date = :creation_date where object_id = :revision_id"
    db_dml update_objects "update acs_objects set creation_user = :creation_user, last_modified=:creation_date, creation_date = :creation_date where object_id = :item_id"
    db_dml update_revisions "update cr_revisions set publish_date = :creation_date where revision_id = :revision_id"
}

Collapse
Posted by Eduardo Santos on
Hi Malte,

I'm calling this topic again beacuse I'm using this script you've supplied to migrate from lars-blogger. I'm doing these steps to import:

1 - Go to the link /xowiki/news-item, wich creates de xowiki::Form for the news, if I'm not wrong

2 - Use your script in the ds/shell giving the package_id for the xowiki and the lars-blogger instance

The import goes without any errors, but the news content is not being exported. When I access the /xowiki/news page, I can only see the news title; its content just disappears. My XoWiki version is 0.60. Am I losing something that should be done in order to make it work?

Collapse
Posted by Malte Sussdorff on
I think that Gustaf made some changes so I should revisit my "move" scripts. Sadly I'm on "holidays" so I wont get to this as quickly as I'd like to. If i have an idea tomorrow morning I will post, otherwise maybe Gustaf can help.
Collapse
Posted by Eduardo Santos on
Thank you for answer this post in your "holidays" malte. I've tried to solve this by myself, but I'm not an expert in XoTCL or OO. Anyway, in my researchs, wich could be miserably wrong, I couldn't find the method new. Maybe its name changed or something?
Collapse
Posted by Gustaf Neumann on
while it is true, that i made many improvements on this front, these are only in cvs-head and not on older versions.

concerning "new": every xotcl object or class has a set of methods inherited from the most general classes. The method "new" is in xotcl a method to create a fresh, "unnamed" object in memory. The predefined xotcl methods are described on http://www.xotcl.org

Concerning the missing body of the blog entries. The ::xowiki::Form "en:news-item" defines the entry form and the display of a news item. In the version it is shipped, it uses the "description" and not the "text" for keeping the entry. Therefore, Malte's script should be changed to use

                  -description $content \
                  -text "" \
to import the news item correctly. Alternatively, one could use in the "en:news-item" definition instead of "@_description@" the name "@_text@" to display the context of the text field instead of the description field.

This is similar to what i use currently in the head-version. It has many small improvements, like e.g. rich-text syndication, many small usability improvements, better styling through CSS, etc.

Collapse
Posted by Eduardo Santos on
Hi Gustaf,

Now that I'm sutdying a little bit more of XoTCL I can see what you say about the methods. I didn't know about XoTCL predefined methods, and I can see now the advantages uf using it.

About the blog entries, I've checked the HEAD version and I could see the improvements you've made in the news-item prototype page. I have only a simple question regarding this: if I use Malte's import scripts am I going to be able to use all of them? Or I'll have to use another script so all the new stuff can work?

Collapse
Posted by Gustaf Neumann on
one of the basic oo concepts is the idea, that one performs methods on objects. The methods for all kind of objects are defined by the most-general classes, which are provided by the XOTcl implementation.

different kinds of objects might have same-named methods (e.g. save) which might need different realizations (e.g. what has to be done for saving an abject). so, specialized subclasses can overload these methods.

concerning new changes on news-item. The version provided by malte continues to work (no one has to use the new news-item form page). The basic question is where to save the body of a message, in the description (plain text) or in the content of an item. Both options are open with the 5-3 version and the head version. i tend to believe that storing the content of items in the text is better, but in some aspects, the html has disadvantages (the 5-3 version converts to a semi-good text, it is harder to truncate the text to n characters due to wellformed-aspects of html).

In the worst case, one has to use another similar script to convert from old news-item to new news-item. The biggest problem with that is the ordering by publication date, which should go most likely into a separate field, since using modification dates on the CR are maintained by the triggers, which has the consequence, that each update script alters these dates... i will look more into the details in the near future.

Collapse
Posted by Gustaf Neumann on
The updated version in cvs head supports now ordering by publish date, and allows entries (e.g. news-items) to specify and modify the publish date explicitly. also, xowiki-import preserves now the publish date.

The news-item prototype page contains now the publish date.

Collapse
Posted by Eduardo Santos on
Hi Gustaf,

That's a very good lesson about OO and XoTCL; thank you very much for that. In my weekend studies I can see a lot about what you said, and I'll be able to use this in my future developments.

About the news storage, I agree with you it should be stored in the text. I don't think the description field is intended to do that, and it looked like a strange solution to me. If we have an item in CR, its content should be the most important attribute, not the description.

I can see your changes in HEAD, but you said something that I don't know if I understood: do you have a xowiki-import script anywhere? In Malte's script above he puts the publish date in cr_revisions. In my little understanding about it this you've changed the select queries to retrieve the publish_date from cr_revisions. Am I right? If you did this, Malte's import script will work without any changes.

Collapse
Posted by Gustaf Neumann on
on xowiki/admin, there is an "export" and "import" button for selective backup and for transfering content between different xowiki packages or different oacs installations. malte's script does not use this.

it is also correct that the publish_date is the one from cr_revisions. without further action on the xowiki side, the publish_date was/is altered each time set_live_revision is issued, which happens per default in the save and save_new methods. With the changes, it is now possible to preserve the given date.

Since the "standard" publish_date is used, there are no changes required in this regards in malte's script.

concerning content vs. description: this is a matter of the point of view: when the full story is stored on a different server, one has just a small description (like an abstract) of the full story in an xowiki page (and in RSS), plus a link to the full text. Furthermore, in many syndication applications, the description is required to be plain text, so there are as well some arguments for this variant.

With xowiki it is quite easy to go both ways for representing news-items, even in one instance by e.g. using two different ::xowiki::Forms. As mentioned before, the default variant is the one using the text content.

Collapse
17: Upgrade problem (response to 16)
Posted by Eduardo Santos on
Hi Gustaf,

Afther your help in all these issues, I realized that I should upgrade my XoWIki and XoTCL versions. I was in version 0.60 of both and I'm upgrading now to the HEAD. i had a little trouble with XoTCL regarding the file xotcl-core/tcl/05-db-procs.tcl. In line 81 I had this:

if {[db_driverkey ""] eq "oracle"} {set name [string toupper $name]}

In my PostgreSQL version, this code was making the package check for the tables in upper case. So it tried to create the APM_PARAMETERS table, and I had to make an small change in order to make it work. My modified version was like this:

if {[db_driverkey ""] eq "oracle"} {
set name [string toupper $name]
} else {
set name [string tolower $name]
}

The error disapeared and I could upgrade XoTCL without any trouble. However, I'm still facing some trouble in XoWIki upgrade. The following error message appears in my log:


Error: content #
# A sample News object.
#
my initialize -parameter {
{-page_size:integer 10}
{-page_number:integer 1}
{-summary:boolean 0}
{-date ""}
{-tag ""}
{-ptag ""}
{-entries_of en:news-item}
}
(...)

set w [::xowiki::Weblog new -destroy_on_cleanup -package_id $package_id -page_size $page_size -page_number $page_numbe
r -summary $summary -date $date -category_id [ns_queryget category_id] -tag $tag -ptag $ptag -entry_label "News Article
s" -exclude_item_ids $exclude_item_ids -entry_renderer $entry_renderer -entry_flag __no_form_page_footer -entries_of $ent
ries_of ]
$w set __page $page
$w mixin add $renderer
return [$w render]
} lead to error: Transaction aborted: unknown argument '-childof' for method 'new': valid arguments {-dbn {}} -name:require
d -parent_id:required -item_id -locale {-creation_date now} -creation_user:required -context_id -creation_ip {-item_subtype c
ontent_item} {-content_type content_revision} -title -description {-mime_type text/plain} -nls_language -text -data -relation
_tag {-is_live f} {-storage_type lob} -package_id:required

Do you have any idea about how to fix it?

Collapse
18: Re: Upgrade problem (response to 17)
Posted by Gustaf Neumann on
First of all, the head version is a moving target, you should not use it in a production environment. the code contains many experimental features, which might disappear in a released version. use it with caution. Furthermore, i test the head version most times with openacs-head, and sometimes with oacs-5-3.

there is something strange in your setup. there should be no need to create the apm_parameters table, since you should have it already. what openacs version are you using? what version of xotcl do you use (did you run the aa-test on xotcl-core to check the requirements)?

Collapse
19: Re: Upgrade problem (response to 18)
Posted by Gustaf Neumann on
one more question: what version of postgres are you using?
Collapse
20: Re: Upgrade problem (response to 18)
Posted by Eduardo Santos on
Hi Gustaf,

I understand what you said about the HEAD version, and I'm trying to be cautious. Maybe that's why I'm facing these troubles.

About the apm_parameters table, it should not be necessary to create it, indeed. The problem is that the script was checking for the APM_PARAMETERS table, wich is upper case, and it was not able to find it. I have changed the code to make it check for the lower case apm_parameters table, and XoTCL could be upgraded without any other troubles.

This is my version list:
ACS-Core 5.2.3
dotLRN 2.2.1
PostgreSQL 8.1.3
XoTCL-Core 0.56.3
XoWiki 0.60

I know most of the XoWiki development is based on oacs-5-3, but as we didn't have an stable version, I've chosen to stay in oacs-5-2. I'll be looking at the upgrade scripts the next weeks. I've never had any trouble using XoWiki in this version, and in my little understanding about the package, I couldn't see any change in the ACS-Core that would afect it.

Studying a little bit more the upgrade error, I could see it breaks at the upgrade callback, when it tries to set the version 0.77. The error shows up when the XoWiki tries to import the news prototype page (line 363 in file xowiki/tcl/xowiki-callback-procs.tcl).

I appreciate your time spent with this matter, Gustaf. I know you are a very busy person, and thank you very much for your help.

Collapse
21: Re: Upgrade problem (response to 20)
Posted by Gustaf Neumann on
Eduardo,

i was trying to figure out, why in your installation it checks for the table name "APM_PARAMETERS" while on my installations, it checks for "apm_parameters". Fixing one symptom does not help much, since you will run into similar problems later on.

Since xotcl-core uses the table name from acs_object_types for the parameter class, i am wondering, what the following query returns on your system:

select object_type, table_name from acs_object_types where object_type = 'apm_parameter'; 

I've never had any trouble using XoWiki in this version, and in my little understanding about the package, I couldn't see any change in the ACS-Core that would afect it.

The versions of xotcl-core in cvs head has a much wider support of the acs-type system (see e.g. http://www.openacs.org/forums/message-view?message_id=1165841)
and is much tighter integrated with it. the according xowiki versions use these features. There are changes in this regard in acs-core, see e.g. http://www.openacs.org/forums/message-view?message_id=1201818 for some consequences.

unlike most other packages, the xo* packages tried in the past to work as good a possible with about all kind of openacs installations. Therefore, there are already many
package dependencies included. Try the following for a better understanding of the dependencies:

fgrep -R "ad_acs_version" packages/xo* 

As said before, the head development is based on 5.3 and cvs head. Providing 5.2 support is probably not hard, but currently, i have no time to work on that.

btw, you did not state, what version of XOTcl you are using (grep for XOTcl in your error log).

Collapse
22: Re: Upgrade problem (response to 21)
Posted by Eduardo Santos on
Hi Gustaf,

I'm starting to think you are right about the compatibility problem. Maybe the new version of XoTCL can not be used in ACS-Core 5.2.3. The query you've mentioned returned the APM_PARAMETERS indeed. It has something to do with this diff:

http://cvs.openacs.org/cvs/openacs-4/packages/acs-kernel/sql/postgresql/apm-create.sql?r1=1.66.2.1&r2=1.61.2.3

It seems like the data model changed a lot between the oacs-5-2 and the oacs-5-3, wich makes me think a version upgrade will be needed in order to make XoTCL and XoWiki work appropriately. I could work in some changes to provide oacs-5-2 compatibility, but I don't know if it's worth. My upgrade fear is allways related to the compatibility of my systems, assuming all the changes I've done myself, but I guess there's no other way.

At least I can say the XoTCL 0.56.3 and XoWiki 0.60 are compatible with oacs-5-2. Thank you very much for all your help.

Collapse
Posted by Malte Sussdorff on
Note that in my script I "loose" the publish_date as I'm not sure support for this exists in XoWIKI so far.
Collapse
Posted by Eduardo Santos on
Hi Malte,

I'm sure this is a good solution when you have one instance of lars-blogger wich serves the news in the website's front page. However, I don't know it's the best interface when you have blogs in .LRN. We already have the dotlrn-xowiki package, wich works very well as I can see, but it's suposed to be an Wiki. So, in my environment users access Xowiki for documentation and lars-blogger for news. If I moved the news for Xowiki, it would not be clear to the users when they are posting news or contributing with the documentation.

There's another issue. How would I deal with community creation? When I create a community and put the lars-blogger inside it, the news space is ready to post. If I moved them to Xowiki, I would have to create a weblog portlet manually and put it int hte community's front page, what can cause me a lot of extra work that I don't want to have.

Thinking about that, I can't see a solution other than change the lars-blogger datamodel.

Collapse
Posted by Malte Sussdorff on
You can just make a new "news-portal" which instantiates a second XoWIKI instance in the community, wouldn't that help? Like dotlrn-xowiki-news or something in that line.

I am just concerned you are going to put a lot of effort into something which could be solved with less effort differently.

Changing the datamodell is not the hard part, getting the multi language bit right seems to be more of an issue for me at least. And with XoWIKI you also get "Teaser" support, which allows you to display a different text on the summary page then is on the actual full page (instead of the 50 first characters or whatever you have configured).

Collapse
Posted by Eduardo Santos on
Hi everybody,

I'm facing some troubles with the lars blogger migration script, mine above or Cognovis' in the address http://www.cognovis.de/developer/en/xowiki-upgrade-lars-blogger.

I'm using the latest XoWiki and XoTcl versions (0.95 and 0.90) and it seems like the script is broken. I've tried to change the script like this:

# Change the following variables to suit your environment
set pinds_package_id 3900
set xowiki_package_id 4345
set xowiki_folder_id 4368

db_1row template "select item_id as page_template, parent_id as xowiki_folder_id from cr_items where name = 'en:news-item' and content_type = '::xowiki::Form' and parent_id = :xowiki_folder_id"

## Do not change anything below

set import ""
db_foreach pinds_entry {select creation_user, creation_ip, creation_date, entry_date as last_modified, p.title, title_url as detail_link, content, content_format, draft_p from pinds_blog_entries p, acs_objects o where p.entry_id = o.object_id and p.package_id = :pinds_package_id} {

# Set publish status
if {$draft_p} {
set publish_status "production"
} else {
set publish_status "ready"
}

# Text conversion
if {$content_format ne "text/html"} {
set content [ad_html_text_convert -from $content_format -to "text/html" $content]
}

set item_id [db_nextval acs_object_id_seq]
set creator [person::name -person_id $creation_user]

set import "::xowiki::FormPage create \
page_template $page_template \
creator $creator \
instance_attributes {detail_link $detail_link} \
description {} \
text {$content} \
package_id $xowiki_package_id \
name $item_id \
title $title \
publish_status $publish_status \
last_modified $last_modified \
parent_id $xowiki_folder_id"

}
if {[catch {namespace eval ::xo::import $import} error]} {
set msg "Error: $error"
} else {
set msg [$xowiki_package_id import -replace 0]
}

But I keep seeing this error message:

invalid command name "4345"
while executing
"$xowiki_package_id import -replace 0
"

I've tried to find out, but I don't know where it's coming from. Can you help me on this?

Collapse
Posted by Gustaf Neumann on
Eduardo, you have not created the package object (via ::xowiki::Package initialize ...) in your variant of Maltes script.

I have added an improved and more robust version for recent xowiki versions to the XoWiki documentation in:
http://alice.wu-wien.ac.at:8000/xowiki-doc/#import-export-from-other-sources

Best regards
-gustaf neumann

Collapse
Posted by Richard Hamilton on

I have been reading through the content migration scripts for general interest and, please forgive me if I am mistaken, but I think something is amiss in the third example script posted here:

http://alice.wu-wien.ac.at:8000/xowiki-doc/#import-export-from-other-sources

8.3.2 Import and Export of XoWiki Pages from other Sources

...

if {$root(package_id) == $pinds(package_id)} {
  ns_return 200 text/plain "Cannot find xowiki instance '$xowiki_url'"
  ad_script_abort
} elseif {$root(package_id) == $pinds(package_id) } {
  ns_return 200 text/plain "Cannot find lars_blogger instance '$lars_blogger_url'"
  ad_script_abort
}

...

How can the elseif clause ever be true/executed?

It looks to me as if there might once have been an array set for the xowiki url #using [site_node::get_from_url -url $xowiki_url] # and that the if/elseif has been modified but not cleaned up.

This is clearly not important, but I think it might mean that if the lars blogger instance cannot be found, the error will say that it is the xowiki instance that cannot be found.

Collapse
Posted by Gustaf Neumann on
the code testing for the existence of an xowiki instance was not correct. i've updated it to make it look more reasonable.