Starting with OpenACS 5.0 and the introduction of acs-lang, we recommend retrieving date/time information from the database in ANSI format and then using lc_time_fmt to format it for display.

db_multirow -extend { mydate_pretty } {
    select to_char(mydate, 'YYYY-MM-DD HH24:MI:SS') as mydate_ansi,
          ...
    ...
} {
    set mydate_ansi [lc_time_system_to_conn $mydate_ansi]
    set mydate_pretty [lc_time_fmt $mydate_ansi "%x %X"]
}

This document describes the requirements for functionality in the OpenACS platform to support globalization of the core and optional modules. The goal is to make it possible to support delivery of applications which work properly in multiple locales with the lowest development and maintenance cost.

The Mozilla project suggests keeping two catchy phrases in mind when thinking about globalization:

Building an application often involves making a number of assumptions on the part of the developers which depend on their own culture. These include constant strings in the user interface and system error messages, names of countries, cities, order of given and family names for people, syntax of numeric and date strings and collation order of strings.

The OpenACS should be able to operate in languages and regions beyond US English. The goal of OpenACS Globalization is to provide a clean and efficient way to factor out the locale dependent functionality from our applications, in order to be able to easily swap in alternate localizations.

This in turn will reduce redundant, costly, and error prone rework when targeting the toolkit or applications built with the toolkit to another locale.

The cost of porting the OpenACS to another locale without some kind of globalization support would be large and ongoing, since without a mechanism to incorporate the locale-specific changes cleanly back into the code base, it would require making a new fork of the source code for each locale.

A globalized application will perform some or all of the following steps to handle a page request for a specific locale:

Since the internationalization APIs may potentially be used on every page in an application, the overhead for adding internationalization to a module or application must not cause a significant time delay in handling page requests.

In many cases there are facilities in Oracle to perform various localization functions, and also there are facilities in Java which we will want to move to. So the design to meet the requirements will tend to rely on these capabilities, or close approximations to them where possible, in order to make it easier to maintain Tcl and Java OpenACS versions.

Here are the cases that we need to be able to handle efficiently:

Other application servers: ATG Dyanmo, Broadvision, Vignette, ... ? Anyone know how they deal with i18n ?

Because the requirements for globalization affect many areas of the system, we will break up the requirements into phases, with a base required set of features, and then stages of increasing functionality.

10.0

A standard representation of locale will be used throughout the system. A locale refers to a language and territory, and is uniquely identified by a combination of ISO language and ISO country abbreviations.

20.0

The request processor must have a mechanism for associating a locale with each request. This locale is then used to select the appropriate template for a request, and will also be passed as the locale argument to the message catalog or locale-specific formatting functions.

30.0

A mechanism must be provided for a developer to group a set of arbitrary content resources together, keyed by a unique identifier and a locale.

For example, what approaches could be used to implement a localizable nav-bar mechanism for a site? A navigation bar might be made up of a set of text strings and graphics, where the graphics themselves are locale-specific, such as images of English or Japanese text (as on www.un.org). It should be easy to specify alternate configurations of text and graphics to lay out the page for different locales.

Design note: Alternative mechanisms to implement this functionality might include using templates, Java ResourceBundles, content-item containers in the Content Repository, or some convention assigning a common prefix to key strings in the message catalog.

40.0

A message catalog facility will provide a database of translations for constant strings for multilingual applications. It must support the following:

Character Sets

50.0 A locale will have a primary associated character set which is used to encode text in the language. When given a locale, we can query the system for the associated character set to use.

The assumption is that we are going to use Unicode in our database to hold all text data. Our current programming environments (Tcl/Oracle or Java/Oracle) operate on Unicode data internally. However, since Unicode is not yet commonly used in browsers and authoring tools, the system must be able to read and write other character sets. In particular, conversions to and from Unicode will need to be explicitly performed at the following times:

Acs-templating does the following.

When sending an email message, just as when delivering the content in web page over an HTTP connection, it is necessary to be able to specify what character set encoding to use.

Mail is not internationalized. The following issues must be addressed.

Because globalization touches many different parts of the system, we want to reduce the implementation risk by breaking the implementation into phases.

Almost all database-backed websites have users, and need to model the grouping of users. The OpenACS 4 Parties and Groups system is intended to provide the flexibility needed to model complex real-world organizational structures, particularly to support powerful subsite services; that is, where one OpenACS installation can support what appears to the user as distinct web services for different user communities.

The primary limitation of the OpenACS 3.x user group system is that it restricts the application developer to representing a "flat group" that contains only users: The user_groups table may contain the group_id of a parent group, but parent-child relationship support is limited because it only allows one kind of relationship between groups to be represented. Moreover, the Oracle database's limited support for tree-like structures makes the queries over these relationships expensive.

In addition, the Module Scoping design in OpenACS 3.0 introduced a party abstraction - a thing that is a person or a group of people - though not in the form of an explicit table. Rather, the triple of scope, user_id, and group_id columns was used to identify the party. One disadvantage of this design convention is that it increases a data model's complexity by requiring the programmer to:

...

The core of the Group Systems data model is quite simple, but it was designed in the hopes of modeling "real world" organizations which can be complex graph structures. The Groups System only considers groups that can be modeled using directed acyclic graphs, but queries over these structures are still complex enough to slow the system down. Since almost every page will have at least one membership check, a number of triggers, views, and auxiliary tables have been created in the hopes of increasing performance. To keep the triggers simple and the number of triggers small, the data model disallows updates on the membership and composition tables, only inserts and deletes are permitted.

The data model has tried to balance the need to model actual organizations without making the system too complex or too slow. The added triggers, views, and tables and will increase storage requirements and the insert and delete times in an effort to speed access time. The limited flexibility (no updates on membership) trades against the complexity of the code.

The Group System data model consists of the following tables:

New groups are created through the group.new constructor. When a specialized type of group is required, the group type can be extended by an application developer. Membership constraints can be specified at creation time by passing a parent group to the constructor.

The membership_rels and composition_rels tables indicate a group's direct members and direct components; these tables do not provide a record of the members or components that are in the group by virtue of being a member or component of one of the group's component groups. Site pages will query group membership often, but the network of component groups can become a very complex directed acyclic graph and traversing this graph for every query will quickly degrade performance. To make membership queries responsive, the data model includes triggers (described in the next paragraph) which watch for changes in membership or composition and update tables that maintain the group party mappings, i.e., group_member_index and group_component_index. One can think of these tables as a manually maintained index.

The following triggers keep the group_*_index tables up to date:

The data model provides the following views onto the group_member_index and group_component_index tables. No code outside of Groups System should modify the group_*_index tables.

The API consists of tables and views and PL/SQL functions.

function person.new (
  person_id          persons.person_id%TYPE,
  object_type        acs_objects.object_type%TYPE,
  creation_date      acs_objects.creation_date%TYPE,
  creation_user      acs_objects.creation_user%TYPE,
  creation_ip        acs_objects.creation_ip%TYPE,
  email              parties.email%TYPE,
  url                parties.url%TYPE,
  first_names        persons.first_names%TYPE,
  last_name          persons.last_name%TYPE
) return persons.person_id%TYPE;

procedure person.delete (
  person_id     persons.person_id%TYPE
);

function person.name (
  person_id     persons.person_id%TYPE
) return varchar;

function acs_user.new (
  user_id            users.user_id%TYPE,
  object_type        acs_objects.object_type%TYPE,
  creation_date      acs_objects.creation_date%TYPE,
  creation_user      acs_objects.creation_user%TYPE,
  creation_ip        acs_objects.creation_ip%TYPE,
  email              parties.email%TYPE,
  url                parties.url%TYPE,
  first_names        persons.first_names%TYPE,
  last_name          persons.last_name%TYPE
  password           users.password%TYPE,
  salt               users.salt%TYPE,
  password_question  users.password_question%TYPE,
  password_answer    users.password_answer%TYPE,
  screen_name        users.screen_name%TYPE,
  email_verified_p   users.email_verified_p%TYPE
) return users.user_id%TYPE;

procedure acs_user.delete (
  user_id       users.user_id%TYPE
);

function acs_user.receives_alerts_p (
  user_id       users.user_id%TYPE
) return varchar;

procedure acs_user.approve_email (
  user_id       users.user_id%TYPE
);

procedure acs_user.unapprove_email (
  user_id       users.user_id%TYPE
);

function acs_group.new (
  group_id           groups.group_id%TYPE,
  object_type        acs_objects.object_type%TYPE,
  creation_date      acs_objects.creation_date%TYPE,
  creation_user      acs_objects.creation_user%TYPE,
  creation_ip        acs_objects.creation_ip%TYPE,
  email              parties.email%TYPE,
  url                parties.url%TYPE,
  group_name         groups.group_name%TYPE
) return groups.group_id%TYPE;

function acs_group.name (
  group_id      groups.group_id%TYPE
) return varchar;

function acs_group.member_p (
  group_id      groups.group_id%TYPE,
  party_id      parties.party_id%TYPE,
) return char;

function membership_rel.new (
  rel_id             membership_rels.rel_id%TYPE,
  rel_type           acs_rels.rel_type%TYPE,
  object_id_one      acs_rels.object_id_one%TYPE,
  object_id_two      acs_rels.object_id_two%TYPE,
  member_state       membership_rels.member_state%TYPE,
  creation_user      acs_objects.creation_user%TYPE,
  creation_ip        acs_objects.creation_ip%TYPE,
) return membership_rels.rel_id%TYPE;

procedure membership_rel.ban (
  rel_id           membership_rels.rel_id%TYPE
);

procedure membership_rel.approve (
  rel_id           membership_rels.rel_id%TYPE
);

procedure membership_rel.reject (
  rel_id           membership_rels.rel_id%TYPE
);

procedure membership_rel.unapprove (
  rel_id           membership_rels.rel_id%TYPE
);

procedure membership_rel.deleted (
  rel_id           membership_rels.rel_id%TYPE
);

procedure membership_rel.delete (
  rel_id           membership_rels.rel_id%TYPE
);

function membership_rel.new (
  rel_id             composition_rels.rel_id%TYPE,
  rel_type           acs_rels.rel_type%TYPE,
  object_id_one      acs_rels.object_id_one%TYPE,
  object_id_two      acs_rels.object_id_two%TYPE,
  creation_user      acs_objects.creation_user%TYPE,
  creation_ip        acs_objects.creation_ip%TYPE,
) return composition_rels.rel_id%TYPE;

procedure membership_rel.delete (
  rel_id           composition_rels.rel_id%TYPE
);

Describe the admin pages.

...

...

...

This document is a guide on how to write a software package for OpenACS. OpenACS packages are installed and maintained with the OpenACS Package Manager (APM) which is part of the acs-admin package. This document presents reasons for packaging software, conventions for the file system and naming that must be followed, and step by step instructions for creating a new package for the "Notes" example package.

Here is how an OpenACS 5 server is laid out starting from the Server root (ROOT):

ROOT/
    bin/
        Various executables and scripts for server maintanence.
    content-repository-content-files/
        content repository content stored in the filesystem.
    etc/
        Installation scripts and configuration files.
    packages/
        acs-admin/
        acs-api-browser/
        ... many many more...
        workflow/
    log/
        Server error and access logs
    tcl/
        bootstrap code
    www/
        Pages not in packages (static content, customized pages)

Each package encapsulates all of its data model, library code, logic, adminstration pages and user pages in a single part of the file tree. This means developers can track down everything that is related to a particular package without hunting all over the file system. Encapsulating everything about a package in one place also makes it much easier to distribute packages independently from the OpenACS Core.

In order to make this work, we need a system that keeps track of the packages that have been installed in the server, where those packages have been installed, and a standard way to map URLs that a client sends to our server to the right page in the appropriate package. While we're at it, this tool should also automate package installation, dependency checking, upgrades, and package removal. In OpenACS 5, this tool is called the APM.

To illustrate the general structure of a package, let's see what the package for the "notes" application should look like.

ROOT/
  +-- packages/    APM Root
        |
        +-- notes/  Package Root
        |     |
        |     +-- notes.info                              Package Specification File
        |     +-- sql/
        |     |     |
        |     |     +-- oracle/
        |     |     |        |
        |     |     |        +-- notes-create.sql         Data Model Creation Script for Oracle
        |     |     |        +-- notes-drop.sql           Data Model Drop Script
        |     |     |        +-- *.sql                    Data Model Files
        |     |     |        +-- upgrade/
        |     |     |            +-- upgrade-4.1-4.5.sql  Data Model Upgrade Scripts
        |     |     +-- postgresql/
        |     |     |        |
        |     |     |        +-- notes-create.sql         Data Model Creation Script for PostgreSQL
        |     |     |        +-- notes-drop.sql           Data Model Drop Script
        |     |     |        +-- *.sql                    Data Model Files
        |     |     |        +-- upgrade/
        |     |     |            +-- upgrade-4.1-4.5.sql  Data Model Upgrade Scripts
        |     +-- tcl/
        |     |     |
        |     |     +-- notes-procs.tcl                   Tcl Library
        |     |     +-- notes-procs.xql                   SQL92 Queries for notes-procs.tcl
        |     |     +-- notes-procs-oracle.xql            Oracle-specific queries for notes-procs.tcl
        |     |     +-- notes-procs-postgresql.xql        PostgreSQL-specific Queries for notes-procs.tcl
        |     |     +-- notes-init.tcl                    Tcl Initialization
        |     |     +-- notes-init.xql                    Queries for notes-init.tcl (work in all DBs)
        |     |     +-- *.tcl                             Tcl Library Files
        |     +-- lib/
        |     |     |
        |     |     +-- *.tcl                             Includable page logic
        |     |     +-- *.adp                             Includable page templates
        |     +-- www/
        |     |     |
        |     |     +-- admin/                            Administration UI
        |     |     |     +-- tests/                      Regression Tests
        |     |     |     |     +-- index.tcl             Regression Test Index Page
        |     |     |     |     +-- ...                   Regression Tests
        |     |     |     +-- index.tcl                   Administration UI Index Page
        |     |     |     +-- ...                         Administration UI Pages
        |     |     |
        |     |     +-- doc/                              Documentation
        |     |     |     +-- index.html                  Documentation Index Page
        |     |     |     +-- ...                         Administration Pages
        |     |     +-- resources/                        Static Content
        |     |     |     +-- ...                         Static Content files
        |     |     +-- index.tcl                         UI Index Page
        |     |     +-- index.adp                         UI Index Template
        |     |     +-- index.xql                         Queries for UI Index page
        |     |     +-- *.tcl                             UI Logic Scripts
        |     |     +-- *.adp                             UI Templates
        |     |     +-- *-oracle.xql                      Oracle-specific Queries
        |     |     +-- *-postgresql.xql                  PostgreSQL-specific Queries
        +-- Other package directories.

All file locations are relative to the package root, which in this case is ROOT/packages/notes. The following table describes in detail what each of the files up in the diagram contain.

A special note on the PACKAGE-KEY/www/resources directory. Files in this directory are available at http://yourserver/resources/PACKAGE-KEY/... and are returned without any permissions checking or even checks that the package is installed or mounted. Files are returned directly, so .tcl or .adp files are not sourced in these directories. This makes it suitable for storing icons, css files, javascript, and other static content which can be treated this way.

The APM is used to create, maintain, and install packages. It takes care of copying all of the files and registering the package in the system. The APM is responsible for:

In addition for packages that are applications, the APM is responsible for keeping track of where in the site a user must go in order to use the application. To do this, the APM defines a set of objects that we call package instances. Once a package is loaded, the administrator can create as many instances of the package as she likes, and map these instances to any URL in the site that she wants. If packages are analogous to executable programs in an operating system, then package instances are analgous to multiple running copies of a single program. Each instance can be independently administered and each instance maintains its own set of application parameters and options.

The following sections will show you how to make a package for the Notes application. In addition, they will discuss some site management features in OpenACS 5 that take advantage of the APM's package instance model. The two most important of these are subsites, and the site map tool, which can be used to map applications to one or more arbitrary URLs in a running site.

We will also discuss how to organize your files and queries so they work with the OpenACS Query Dispatcher.

Here is how you make a package.

At this point, you are probably excited to see your new package in action. But, we haven't added any user visible pages yet. By convention, user visible pages go in the ROOT/packages/notes/www directory. So go there and add a file called hello.html with some text in it. Now we have to make the user pages visible in the site. Since we didn't put the pages underneath ROOT/www they will not appear on their own. What we have to do is mount the application into the site map. That is, we have to define the URL from which the application will serve its pages.

In OpenACS 5, administrators can define an arbitrary mapping between the URLs the user types and the actual file in the file system that is served. This mapping is called the site map and entries in the site map are called site nodes. Each site node maps a URL to an OpenACS object. Since package instances are objects, the site map allows us to easily map package instances to URLs. As we said before, each instance of an application has its own set of parameters and runs from its own URL within the site. What this means is that even though all the code for the notes application lives in ROOT/packages/notes, the application itself can run from any number of locations in the site. This allows developers and administrators to build sites that look to the user like a collection of many indedendent applications that actually run on a single shared code base. The request-processor document shows you how OpenACS figures out which instance of your application was requested by the user at any given time. The page development tutorial shows you how to use this information in your user interface.

In order to make the new notes application visible to users, we have to mount it in the site map. You do this by going to the Site Map page, which is by default available at /acs-admin/site-map. Use the interface here to add a new sub-folder called notes to the root of the site, then click "new application" to mount a new instance of the notes application to the site. Name the new instance notes-1.

Then type this URL into your browser: http://yourserver/notes/hello.html

Now you should see the contents of the page that you added. What has happened is that all URLs that start with /notes have been mapped in such a way as to serve content from the directory ROOT/packages/notes/www. At this point, you can experiment with the site map by mounting multiple instances of the not yet written Notes application at various places in the site. In a later document, we'll see how to write your application so that the code can detect from what URL it was invoked. This is the key to supporting subsites.

The APM performs the following tasks in an OpenACS site: