lang::catalog::import

lang::catalog::import_messages (private)

 lang::catalog::import_messages -file_messages_list file_messages_list \
    -package_key package_key -locale locale

Defined in packages/acs-lang/tcl/lang-catalog-procs.tcl

Import a given set of messages from a catalog file to the database for a certain package and locale. If we already have messages in the db for the given package and locale then a merge between the database messages and the file messages will be performed.

Foreach message to import, the base messages for the merge is the messages in the db from the last time db and catalog file were in sync for the corresponding message key. The first such sync point is the initial import of a message. After that, any export of messages to the filesystem will be a sync point. Also, after an upgrade, a large number of the resulting messages in the db will be identical to those in the file (the file messages take precedence on conflict) and those messages will also be sync points. A message being in sync between db and file is indicated by the lang_message.sync_time column being set to a not null value.

This proc is idempotent which means that it can be executed multiple times and after the first time it's been executed it won't have any effect on the db. See the corresponding acs-automated-testing test case called upgrade.

What follows below is a description of the logic of the proc in terms of its input, the cases considered, and the logical actions taken for each case.

There are three sets of keys, file, db, and base keys. For each key in the union of these keys there are three messages that can exist: the file message, the db message, and the base message. The base message serves as the base for the merge. We will distinguish all the different permutations of each of the three messages existing or not, and all permutations of the messages being different from each other. We don't distinguish how two messages are different, only whether they are different or not. In total that gives us 14 cases (permutations) to consider.

    *** Exactly one of messages exists (3 cases):

    1. base message (deleted in file and db). upgrade_action=none, conflict_p=f

    2. db message (added in db). upgrade_action=none, conflict_p=f

    3. file message (added in file). upgrade_action=add, conflict_p=f

    *** Exactly two of the messages exist (6 cases):

    - Base and file message (db message deleted):
      4. Differ (conflicting change). upgrade_action=resurrect, conflict_p=t
      5. No difference (no conflicting change). upgrade_action=none, conflict_p=f

    - Base and db message (file message deleted):
      6. Differ (conflicting change): upgrade_action=delete, conflict_p=t
      7. No difference (no conflicting change): upgrade_action=delete, conflict_p=f

    - File and db message (message added in both db and file):
      8. Differ (conflicting change). upgrade_action=update, conflict_p=t
      9. No difference (identical changes). upgrade_action=none, conflict_p=f

    *** All three messages exist (5 cases):

    10. All the same. upgrade_action=none, conflict_p=f

    11. File and base the same. upgrade_action=none, conflict_p=f

    12. DB and base the same. upgrade_action=update, conflict_p=f

    13. File and DB the same. upgrade_action=none, conflict_p=f

    14. All different. upgrade_action=update, conflict_p=t

Switches:

-file_messages_list: An array list with message keys as keys and the message of those keys as values, i.e. (key, value, key, value, ...)
-package_key: The package_key for the messages.
-locale: The locale of the messages.

Returns:

An array list containing the number of messages processed, number of messages added, number of messages updated, number of messages deleted by the import, and a list of errors produced. The keys of the array list are processed, added, updated, and deleted, and errors.

Authors:

Peter Marklund

Lars Pind

Partial Call Graph (max 5 caller/called nodes):

Testcases:

No testcase defined.

Source code:

    set message_count(processed) 0
    set message_count(added) 0
    set message_count(updated) 0
    set message_count(deleted) 0
    set message_count(errors) [list]

    # Form arrays for all three sets of messages
    array set file_messages $file_messages_list
    array set db_messages [lang::catalog::messages_in_db  -package_key $package_key  -locale $locale]
    array set base_messages [lang::catalog::last_sync_messages  -package_key $package_key  -locale $locale]

    foreach arrname { base_messages file_messages db_messages } {
        set dummy [list]
        foreach elm [lsort [array names $arrname]] {
            lappend dummy "$elm=[set ${arrname}($elm)]"
        }
        ns_log Debug "lang::catalog::import_messages - $arrname: $dummy"
    }

    # Remember each time we've processed a key, so we don't process it twice
    array set message_key_processed_p [list]

    # Loop over the union of import and db keys.
    foreach message_key [lsort [concat [array names db_messages] [array names file_messages] [array names base_messages]]] {
        if { [info exists message_key_processed_p($message_key)] } {
            continue
        }
        set message_key_processed_p($message_key) 1

        ###########################################
        #
        # Figure out how db and file messages have changed with regards to the base message
        #
        ###########################################

        # The variables indicate how the db and file messages have changed
        # from the base message. Valid values are: none, add, update, delete
        set db_change "none"
        set file_change "none"

        if { [info exists base_messages($message_key)] } {
            # The base message exists

            if { [info exists db_messages($message_key)] } {
                # db message exists
                if { $db_messages($message_key) ne $base_messages($message_key) } {
                    # db message and base message differ
                    set db_change "update"
                }
            } else {
                # db message does not exist
                set db_change "delete"
            }

            if { [info exists file_messages($message_key)] } {
                # file message exists
                if { $file_messages($message_key) ne $base_messages($message_key) } {
                    # file message and base message differ
                    set file_change "update"
                }
            } else {
                # file message does not exist
                set file_change "delete"
            }
        } else {
            # The base message does not exist

            if { [info exists db_messages($message_key)] } {
                # db message exists
                set db_change "add"
            }
            if { [info exists file_messages($message_key)] } {
                # file message exists
                set file_change "add"
            }
        }

        ###########################################
        #
        # Based on the change in file and db messages,
        # and based on whether file and db messages differ, decide
        # which upgrade actions to take
        #
        ###########################################

        # Default values cover the cases 2, 5, 9, 10, 11, 13
        set import_case "in 2, 5, 9, 10, 11, 13"
        set upgrade_status "no_upgrade"
        set conflict_p "f"

        switch $db_change {
            none {
                switch $file_change {
                    none {}
                    add {
                        # case 3
                        set import_case 3
                        # add message from file to db
                        set upgrade_status "added"
                    }
                    update {
                        # case 12
                        set import_case 12
                        # update db with file message
                        set upgrade_status "updated"
                    }
                    delete {
                        # case 7
                        set import_case 7
                        # mark message in db deleted
                        set upgrade_status "deleted"
                    }
                }
            }
            add {
                switch $file_change {
                    none {}
                    add {
                        if { $db_messages($message_key) ne $file_messages($message_key) } {
                            # case 8
                            set import_case 8
                            # differing additions in db and file
                            set upgrade_status "updated"
                            set conflict_p "t"
                        }
                    }
                }
            }
            update {
                switch $file_change {
                    none {}
                    update {
                        if { $db_messages($message_key) ne $file_messages($message_key) } {
                            # case 14
                            set import_case 14
                            # differing updates in file and db
                            set upgrade_status "updated"
                            set conflict_p "t"
                        }
                    }
                    delete {
                        # case 6
                        set import_case 6
                        # deletion in file but update in db
                        set upgrade_status "deleted"
                        set conflict_p "t"
                    }
                }
            }
            delete {
                switch $file_change {
                    none {}
                    update {
                        # case 4
                        set import_case 4
                        # deletion in db but update in file
                        set upgrade_status "added" ;# resurrect
                        set conflict_p "t"
                    }
                    delete {
                        # case 1
                        set import_case 1
                        # deletion in both db and file
                        # no status change, no conflict
                        # sync time should be updated below
                    }
                }
            }
        }

        ###########################################
        #
        # Execute upgrade actions
        #
        ###########################################

        # For certain messages we need to move the sync point so that we have a current base for the next upgrade.
        if { $db_change eq "none" || $file_change ne "none" } {
            # If there is no db change then any change in the file will be reflected in
            # db (file takes precedence) and file and db are identical.
            # Also, regardless of what's happened in db, if
            # there has been a change in the file then that change will take effect in
            # the db and file and db are again identical (in sync).
            set update_sync_p 1
        } else {
            set update_sync_p 0
        }

        # Store a new message in the database if we are adding or updating
        set error_p 0
        if { $upgrade_status eq "added" || $upgrade_status eq "updated" } {

            ns_log Debug "lang::catalog::import_messages - invoking lang::message::register with import_case=\"$import_case\" -update_sync=$update_sync_p $message_key $upgrade_status $conflict_p"
            ad_try {
                lang::message::register  -update_sync  -upgrade_status $upgrade_status  -conflict=$conflict_p  $locale  $package_key  $message_key  $file_messages($message_key)
            } on error {errorMsg} {
                lappend message_count(errors) $errorMsg
                set error_p 1
            }
        } elseif { $update_sync_p || $upgrade_status eq "deleted" } {
            # Set the upgrade_status, deleted_p, conflict_p, and sync_time properties of the message

            # If we are doing nothing, the only property of the message we might want to update in the db
            # is the sync_time as we might have discovered that db and file are in sync
            array unset edit_array
            if { $upgrade_status ne "no_upgrade" } {
                set edit_array(upgrade_status) $upgrade_status
                set edit_array(deleted_p) [string equal $upgrade_status "deleted"]
                set edit_array(conflict_p) $conflict_p
            }

            ns_log Debug "lang::catalog::import_messages - invoking lang::message::edit with import_case=\"$import_case\" -update_sync=$update_sync_p $message_key [array get edit_array]"
            ad_try {
                lang::message::edit  -update_sync=$update_sync_p  $package_key  $message_key  $locale  [array get edit_array]
            } on error {errorMsg} {
                lappend message_count(errors) $errorMsg
                set error_p 1
            }
        } else {
            ns_log Debug "lang::catalog::import_messages - not doing anything: import_case=\"$import_case\" $message_key $upgrade_status $conflict_p"
        }

        if { $upgrade_status in {added updated deleted} } {
            if { ! $error_p } {
                incr message_count($upgrade_status)
            }
        }
        incr message_count(processed)

    } ;# End of message key loop

    return [array get message_count]

Generic XQL file:

packages/acs-lang/tcl/lang-catalog-procs.xql

PostgreSQL XQL file:

packages/acs-lang/tcl/lang-catalog-procs-postgresql.xql

Oracle XQL file:

packages/acs-lang/tcl/lang-catalog-procs-oracle.xql

[ hide source ] | [ make this the default ]