A major goal in OpenACS 4 is to unify and normalize many of the core services of the system into a coherent common data model and API. In the past, these services were provided to applications in an ad-hoc and irregular fashion. Examples of such services include:

All of these services involve relating extra information and services to application data objects, examples of which include:

In the past, developers had to use ad-hoc and inconsistent schemes to interface to the various "general" services mentioned above. Since each service used its own scheme for storing its metadata and mapping this data to application objects, we could not implement any kind of centralized management system or consistent administrative pages for all the services. Consequently, a large amount of duplicate code appeared throughout the system for dealing with these services.

Unifying and "normalizing" these interfaces, to minimize the amount of code repetition in applications, is a primary goal of OpenACS 4. Thus the Object Model (OM, also referred to later as the object system) is concerned primarily with the storage and management of metadata, on any object within a given instance of OpenACS 4. The term "metadata" refers to any extra data the OM stores on behalf of the application - outside of the application's data model - in order to enable certain generic services. The term "object" refers to any entity being represented within the OpenACS, and typically corresponds to a single row within the relational database.

The OpenACS 4 Object Model must address five high-level requirements that repeatedly exhibit themselves in the context of existing services in OpenACS 3.x, as described below.

Object Identifiers for General Services

Generic services require a single unambiguous way of identifying application objects that they manage or manipulate. In OpenACS 3.x, there are several different idioms that construct object identifiers from other data. Many modules use a (user_id, group_id, scope) triple combination for the purpose of recording ownership information on objects for access control. User/groups also uses (user_id, group_id) pairs in its user_group_map table as a way to identify data associated with a single membership relation.

Also in OpenACS 3.x, many utility modules exist that do nothing more than attach some extra attributes to existing application data. For example, general comments maintains a mapping table that maps application "page" data (static or dynamic) to one or more user comments on the page, by constructing a unique identifier for each page. This identifier is usually a combination of the table in which the data is stored, and the value of the primary key value for the particular page. This idiom is referred to as the "(on_which_table + on_what_id)" method for identifying application data. General comments stores its map from pages to comments using a "(on_which_table + on_what_id)" key, plus the id of the comment itself.

All of these composite key constructions are implicit object identifiers: they build a unique ID out of other pieces of the data model. The problem is that their definition and use is ad-hoc and inconsistent. This makes the construction of generic application-independent services difficult. Therefore, the OpenACS 4 Object Model should provide a centralized and uniform mechanism for tagging application objects with unique identifiers.

Support for Unified Access Control

Access control should be as transparent as possible to the application developer. Until the implementation of the general permissions system, every OpenACS application had to manage access control to its data separately. Later on, a notion of "scoping" was introduced into the core data model.

"Scope" is a term best explained by example. Consider some hypothetical rows in the address_book table:

The first row represents an entry in User 123's personal address book, the second row represents an entry in User Group 456's shared address book, and the third row represents an entry in the site's public address book.

In this way, the scoping columns identify the security context in which a given object belongs, where each context is either a person or a group of people or the general public (itself a group of people).

The problem with this scheme is that we are limited to using only users and groups as scopes for access control, limiting applications to a single level of hierarchy. Worse, the scoping system demanded that every page needing access to a given application had to do an explicit scope check to make sure access was allowed - if a developer was careless on just one site page, a security problem could result.

Thus the OpenACS 4 Object Model must support a more general access control system that allows access control domains to be hierarchical, and specifiable with a single piece of data, instead of the old composite keys described above.

Extensible Data Models

Another problem with previous OpenACS data models is that many of the central tables in the system became bloated as they were extended to support an increasing number of modules. The users table is the best case in point: it became full of columns that exist for various special applications (e.g. user portraits), but that aren't really related to each other in any way except that they store information on users, i.e. the table became grossly denormalized. Normalizing (breaking-down) this table into several pieces, each of which is specific to a particular application, would improve maintainability greatly. Furthermore, the ability to allow applications or users to define new extensions to existing tables, and have some central metadata facility for keeping track of what data belong to which tables, would be very useful.

Thus the motivation for providing object types and subtyping in the OpenACS 4 Object Model. The OM should allow developers to define a hierarchy of metadata object types with subtyping and inheritance. Developers can then use the framework to allow users to define custom extensions to the existing data models, and the OM does the bookkeeping necessary to make this easier, providing a generic API for object creation that automatically keeps track of the location and relationships between data.

Design Note: While this doesn't really belong in a requirements document, the fact that we are constrained to using relational databases means that certain constraints on the overall design of the object data model exist, which you can read about in Summary and Design Considerations.

Modifiable Data Models

Another recurring applications problem is how to store a modifiable data model, or how to store information that may change extensively between releases or in different client installations. Furthermore, we want to avoid changes to an application's database queries in the face of any custom extensions, since such changes are difficult or dangerous to make at runtime, and can make updating the system difficult. Some example applications in OpenACS 3.x with modifiable data models include:

Thus the Object Model must provide a general mechanism for applications and developers to modify or extend data models, without requiring changes to the SQL schema of the system. This ensures that all applications use the same base schema, resulting in a uniform and more maintainable system.

Generic Relations

Many OpenACS applications define simple relationships between application objects, and tag those relationships with extra data. In OpenACS 3.x, this was done using mapping tables. The user/groups module has the most highly developed data model for this purpose, using a single table called user_group_map that mapped users to groups. In addition, it uses the the user_group_member_fields and user_group_member_fields_map tables to allow developers to attach custom attributes to group members. In fact, these custom attributes were not really attached to the users, but to the fact that a user was a member of a particular group - a subtle but important distinction. As a historical note, in OpenACS 3.x, user/groups was the only part of the system that provided this kind of data model in a reusable way. Therefore, applications that needed this capability often hooked into user/groups for no other reason than to use this part of its data model.

The OpenACS 4 data model must support generic relations by allowing developers to define a special kind of object type called a relation type. Relation types are themselves object types that do nothing but represent relations. They can be used by applications that previously used user/groups for the same purpose, but without the extraneous, artificial dependencies.

The Object Model package is a combination of data model and a procedural API for manipulating application objects within an OpenACS instance. The OM allows developers to describe a hierarchical system of object types that store metadata on application objects. The object type system supports subtyping with inheritance, so new object types can be defined in terms of existing object types.

The OM data model forms the main part of the OpenACS 4 Kernel data model. The other parts of the Kernel data model include:

Each of these is documented elsewhere at length.

(Pending as of 8/27/00)

The data model for the object system provides support for the following kinds of schema patterns that are used by many existing OpenACS modules:

The API should let programmers accomplish the following actions:

Almost all database-backed websites have users, and need to model the grouping of users. The OpenACS 4 Parties and Groups system is intended to provide the flexibility needed to model complex real-world organizational structures, particularly to support powerful subsite services; that is, where one OpenACS installation can support what appears to the user as distinct web services for different user communities.

A powerful web service that can meet the needs of large enterprises must be able to model the the real world's very rich organizational structures and many ways of decomposing the same organization. For example, a corporation can be broken into structures (the corporation, its divisions, and their departments) or regions (the Boston office, the LA office); a person who is employed by (is a member of) a specific department is also a member of the division and the corporation, and works at (is a member of, but in a different sense) a particular office. OpenACS's Parties and Groups system will support such complex relations faithfully.

Historical Motivations

The primary limitation of the OpenACS 3.x user group system is that it restricts the application developer to representing a "flat group" that contains only users: The user_groups table may contain the group_id of a parent group, but parent-child relationship support is limited because it only allows one kind of relationship between groups to be represented. Moreover, the Oracle database's limited support for tree-like structures makes the queries over these relationships expensive.

In addition, the Module Scoping design in OpenACS 3.0 introduced a party abstraction - a thing that is a person or a group of people - though not in the form of an explicit table. Rather, the triple of scope, user_id, and group_id columns was used to identify the party. One disadvantage of this design convention is that it increases a data model's complexity by requiring the programmer to:

In sum, the goal of the Parties and Groups system is to provide OpenACS programmers and site administrators with simple tools that fully describe the complex relationships that exist among groups in the real world.

Pat Developer has a client project and wants to model the company, its offices, its divisions, and its departments as groups and the employees as users.

We start with Groups, which contain members; the member can be either a person or another group (i.e. a member is a party).

In addition to membership, the party and groups system defines a composition relationship that may exist between groups: A group can be a component of another group. The child group is called a component group; the parent group is called a composite group.

A group G_c can be a member and/or a component of another group G_p ; the difference is in the way the members of G_c are related to G_p :

Consider an example to make this less abstract: Pretend that the Sierra Club is a member of Greenpeace. The Sierra Club has chapters; each chapter is a component of the Sierra Club. If Eddie Environmentalist is a member of the Massachusetts Chapter of the Sierra Club, Eddie is automatically a member of the Sierra Club, but being a Sierra Club member does not make Eddie a member of Greenpeace.

In the OpenACS, Greenpeace, Sierra Club, and the Sierra Club chapters would be modeled as groups, and Eddie would be a user. There would be a composition relationship between each Sierra Club chapter and the Sierra Club. Membership relationships would exist between Eddie and the Massachusetts Chapter, between Eddie and the Sierra Club (due to Eddie's membership in the Massachusetts chapter), and between the Sierra Club and Greenpeace.

Membership requirements can vary from group to group. The parties and groups system must provide a base type that specifies the bare minimum necessary to join a group.

The parties and groups system must support constraints between a composite group G_P and any of its component groups, G_C . For example, the system should be able to enforce a rule like: Do not allow a party P to become a member of G_C unless P is already a member of G_P .

The data model for the parties and groups system must provide support for the following types of entities:

The data model for the parties and groups system must provide support for the following types of relationships between entities:

The API should let programmers accomplish the following tasks:

The user interface is a set of HTML pages that are used to drive the underlying API. The user interface may provide the following functions:

While many applications must deal with individuals and many applications must deal with groups, most applications must deal with individuals or groups. It is often the case with such applications that both individuals and groups are treated identically. Modelling individuals and groups as specializations of one supertype is a practical way to manage both. This concept is so mundane that there is no need to invent special terminology. This supertype is called a "party".

A classic example of the "party" supertype is evident in address books. A typical address book might contain the address of a doctor, grocery store, and friend. The first field in an entry in the address book is not labeled a person or company, but a "party".

The parties developer guide begins with an introduction to the parties data model, since OpenACS community applications likely require using it in some way.

Parties

The central table in the parties data model is the parties table itself. Every party has exactly one row in this table. Every party has an optional unique email address and an optional url. A party is an acs object, so permissions may be granted and revoked on parties and auditing information is stored in the acs objects table.

create table parties (
    party_id    not null
            constraint parties_party_id_fk references
            acs_objects (object_id)
            constraint parties_party_id_pk primary key,
    email       varchar(100)
            constraint parties_email_un unique,
    url     varchar(200)
);

The persons and groups tables extend the parties table. A row in the persons table represents the most generic form of individual modeled. An individual need not be known to the system as a user. A user is a further specialized form of an individual (discussed later). A row in the groups table represents the most generic form of group modeled, where a group is an aggregation of zero or more individuals.

Persons

If a party is an individual then there will be a row in the persons table containing first_names and last_name for that individual. The primary key of the persons table (person_id) references the primary key of the parties table (party_id), so that there is a corresponding row in the parties table when there is a row in the persons table.

create table persons (
    person_id   not null
            constraint persons_person_id_fk
            references parties (party_id)
            constraint persons_person_id_pk primary key,
    first_names varchar(100) not null,
    last_name   varchar(100) not null
);

Users

The users table is a more specialized form of persons table. A row in users table represents an individual that has login access to the system. The primary key of the users table references the primary key of the persons table. This guarantees that if there is a row in users table then there must be a corresponding row in persons and parties tables.

Decomposing all the information associated with a user into the four tables (acs_objects, parties, persons, users) has some immediate benefits. For instance, it is possible to remove access to a user from a live system by removing his entry from the users table, while leaving the rest of his information present (i.e. turning him from a user into a person).

Wherever possible the OpenACS data model references the persons or parties table, not the users table. Developers should be careful to only reference the users table in situations where it is clear that the reference is to a user for all cases and not to any other individual for any case.

create table users (
    user_id         not null
                constraint users_user_id_fk
                references persons (person_id)
                constraint users_user_id_pk primary key,
    password        varchar(100),
    salt            varchar(100),
    screen_name     varchar(100)
                constraint users_screen_name_un
                unique,
    priv_name       integer default 0 not null,
    priv_email      integer default 5 not null,
    email_verified_p    char(1) default 't'
                constraint users_email_verified_p_ck
                check (email_verified_p in ('t', 'f')),
    email_bouncing_p    char(1) default 'f' not null
                constraint users_email_bouncing_p_ck
                check (email_bouncing_p in ('t','f')),
    no_alerts_until     date,
    last_visit      date,
    second_to_last_visit    date,
    n_sessions      integer default 1 not null,
    password_question   varchar(1000),
    password_answer     varchar(1000)
);

Groups

The final piece of the parties data model is the groups data model. A group is a specialization of a party that represents an aggregation of zero or more other parties. The only extra information directly associated with a group (beyond that in the parties table) is the name of the group:

create table groups (
    group_id    not null
            constraint groups_group_id_fk
            references parties (party_id)
            constraint groups_group_id_pk primary key,
    group_name  varchar(100) not null
);

There is another piece to the groups data model that records relations between parties and groups.

Group Relations

Two types of group relations are represented in the data model: membership relations and composite relations. The full range of sophisticated group structures that exist in the real world can be modelled in OpenACS by these two relationship types.

Membership relations represent direct membership relation between parties and groups. A party may be a "member" of a group. Direct membership relations are common in administrative practices, and do not follow basic set theory rules. If A is a member of B, and B is a member of C, A is not a member of C. Membership relations are not transitive.

Composition relation represents composite relation between two groups. Composite relation is transitive. That is, it works like memberships in set theory. If A is a member of B, and B is a member of C, then A is a member of C.

For example, consider the membership relations of Greenpeace, and composite relations of a multinational corporation. Greenpeace, an organization (ie. group), can have both individuals and organizations (other groups) as members. Hence the membership relation between groups and parties. However, someone is not a member of Greenpeace just because they are a member of a group that is a member of Greenpeace. Now, consider a multinational corporation (MC) that has a U.S. division and a Eurasian division. A member of either the U.S. or Eurasian division is automatically a member of the MC. In this situation the U.S. and Eurasian divisions are "components" of the MC, i.e., membership is transitive with respect to composition. Furthermore, a member of a European (or other) office of the MC is automatically a member of the MC.

Group Membership

Group memberships can be created and manipulated using the membership_rel package. Only one membership object can be created for a given group, party pair.

It is possible in some circumstances to make someone a member of a group of which they are already a member. That is because the model distinguishes between direct membership and indirect membership (membership via some composite relationship). For example, a person might be listed in a system as both an individual (direct membership) and a member of a household (indirect membership) at a video rental store.

# sql code
create or replace package membership_rel
as

  function new (
    rel_id      in membership_rels.rel_id%TYPE default null,
    rel_type        in acs_rels.rel_type%TYPE default 'membership_rel',
    object_id_one   in acs_rels.object_id_one%TYPE,
    object_id_two   in acs_rels.object_id_two%TYPE,
    member_state    in membership_rels.member_state%TYPE default null,
    creation_user   in acs_objects.creation_user%TYPE default null,
    creation_ip     in acs_objects.creation_ip%TYPE default null
  ) return membership_rels.rel_id%TYPE;

  procedure ban (
    rel_id  in membership_rels.rel_id%TYPE
  );

  procedure approve (
    rel_id  in membership_rels.rel_id%TYPE
  );

  procedure reject (
    rel_id  in membership_rels.rel_id%TYPE
  );

  procedure unapprove (
    rel_id  in membership_rels.rel_id%TYPE
  );

  procedure deleted (
    rel_id  in membership_rels.rel_id%TYPE
  );

  procedure delete (
    rel_id  in membership_rels.rel_id%TYPE
  );

end membership_rel;
/
show errors

Group Composition

Composition relations can be created or destroyed using the composition_rel package. The only restriction on compositions is that there cannot be a reference loop, i.e., a group cannot be a component of itself either directly or indirectly. This constraint is maintained for you by the API. So users do not see some random PL/SQL error message, do not give them the option to create a composition relation that would result in a circular reference.

# sql code
create or replace package composition_rel
as

  function new (
    rel_id      in composition_rels.rel_id%TYPE default null,
    rel_type        in acs_rels.rel_type%TYPE default 'composition_rel',
    object_id_one   in acs_rels.object_id_one%TYPE,
    object_id_two   in acs_rels.object_id_two%TYPE,
    creation_user   in acs_objects.creation_user%TYPE default null,
    creation_ip     in acs_objects.creation_ip%TYPE default null
  ) return composition_rels.rel_id%TYPE;

  procedure delete (
    rel_id  in composition_rels.rel_id%TYPE
  );

end composition_rel;
/
show errors

The parties data model does a reasonable job of representing many of the situations one is likely to encounter when modeling organizational structures. We still need to be able to efficiently answer questions like "what members are in this group and all of its components?", and "of what groups is this party a member either directly or indirectly?". Composition relations allow you to describe an arbitrary Directed Acyclic Graph (DAG) between a group and its components. For these reasons the party system provides a bunch of views that take advantage of the internal representation of group relations to answer questions like these very quickly.

The group_component_map view returns all the subcomponents of a group including components of sub components and so forth. The container_id column is the group_id of the group in which component_id is directly contained. This allows you to easily distinguish whether a component is a direct component or an indirect component. If a component is a direct component then group_id will be equal to container_id. You can think of this view as having a primary key of group_id, component_id, and container_id. The rel_id column points to the row in acs_rels table that contains the relation object that relates component_id to container_id. The rel_id might be useful for retrieving or updating standard auditing info for the relation.

create or replace view group_component_map
as select group_id, component_id, container_id, rel_id
...

The group_member_map view is similar to group_component_map except for membership relations. This view returns all membership relations regardless of membership state.

create or replace view group_member_map
as select group_id, member_id, container_id, rel_id
...

The group_approved_member_map view is the same as group_member_map except it only returns entries that relate to approved members.

create or replace view group_approved_member_map
as select group_id, member_id, container_id, rel_id
...

The group_distinct_member_map view is a useful view if you do not care about the distinction between direct membership and indirect membership. It returns all members of a group including members of components --the transitive closure.

create or replace view group_distinct_member_map
as select group_id, member_id
...

The party_member_map view is the same as group_distinct_member_map, except it includes the identity mapping. It maps from a party to the fully expanded list of parties represented by that party including the party itself. So if a party is an individual, this view will have exactly one mapping that is from that party to itself. If a view is a group containing three individuals, this view will have four rows for that party, one for each member, and one from the party to itself.

create or replace view party_member_map
as select party_id, member_id
...

The party_approved_member_map view is the same as party_member_map except that when it expands groups, it only pays attention to approved members.

create or replace view party_approved_member_map
as select party_id, member_id
...

The parties data model can represent some fairly sophisticated real world situations. Still, it would be foolish to assume that this data model is sufficiently efficient for every application. This section describes some of the more common ways to extend the parties data model.

Specializing Users

Some applications will want to collect more detailed information for people using the system. If there can be only one such piece of information per user, then it might make sense to create another type of individual that is a further specialization of a user. For example a Chess Club community web site might want to record the most recent score for each user. In a situation like this it would be appropriate to create a subtype of users, say chess_club_users. This child table of the users table would have a primary key that references the users table, thereby guaranteeing that each row in the chess_club_users table has a corresponding row in each of the users, persons, parties, and acs_objects tables. This child table could then store any extra information relevant to the Chess Club community.

Specializing Groups

If one were to build an intranet application on top of the party system, it is likely that one would want to take advantage of the systems efficient representation of sophisticated organizational structures, but there would be much more specialized information associated with each group. In this case it would make sense to specialize the group party type into a company party type in the same manner as Specializing Users.

Specializing Membership Relations

The final portion of the parties data model that is designed to be extended is the membership relationship. Consider the intranet example again. It is likely that a membership in a company would have more information associated with it than a membership in an ordinary group. An obvious example of this would be a salary. It is exactly this need to be able to extend membership relations with mutable pieces of state that drove us to include a single integer primary key in what could be thought of as a pure relation. Because a membership relation is an ordinary acs object with object identity, it is as easy to extend the membership relation to store extra information as it is to extend the users table or the groups table.

OpenACS has a form manager called ad_form. Ad_form has an adaptable UI. Error handling includes inline error reporting, and is customizable. However, ad_form can be tricky to use. In addition to this document, the ad_form api documentation is helpful.

Some elements have more than one choice, or can submit more than one value.

A situation you may run into often is where you want to pull in form items from a sub-category when the first category is selected. Ad_form makes this fairly easy to do. In the definition of your form element, include an html section

    {pm_task_id:integer(select),optional
         {label "Subject"}
         {options {$task_options}}
        {html {onChange "document.form_name.__refreshing_p.value='1';submit()"}}
        {value $pm_task_id}
    }
    

What this will do is set the value for pm_task_id and all the other form elements, and resubmit the form. If you then include a block that extends the form, you'll have the opportunity to add in subcategories:

    if {[exists_and_not_null pm_task_id]} {
    db_1row get_task_values { }
    ad_form -extend -name form_name -form { ... }
    

Note that you will get strange results when you try to set the values for the form. You'll need to set them explicitly in an -on_refresh section of your ad_form. In that section, you'll get the values from the database, and set the values as so:

    db_1row get_task_values { }
    template::element set_value form_name estimated_hours_work $estimated_hours_work
    

A good way to troubleshoot when you're using ad_form is to add the following code at the top of the .tcl page (thanks Jerry Asher):

ns_log notice it's my page!
set mypage [ns_getform]
if {[string equal "" $mypage]} {
    ns_log notice no form was submitted on my page
} else {
    ns_log notice the following form was submitted on my page
    ns_set print $mypage
}
    

Here are some tips for dealing with some of the form widgets:

Current widget

Here are some common errors and what to do when you encounter them:

The first task is to create an appropriate environment for finding out what is going on inside Oracle. Oracle provides Statspack, a package to monitor and save the state of the v$ performance views. These reports help finding severe problems by exposing summary data about the Oracle wait interface, executed queries. You'll find the installation instructions in $ORACLE_HOME/rdbms/admin/spdoc.txt. Follow the instructions carefully and take periodic snapshots, this way you'll be able to look at historical performance data.

Also turn on the timed_statistics in your init.ora file, so that Statspack reports (and all other Oracle reports) are timed, which makes them a lot more meaningful. The overhead of timing data is about 1% per Oracle Support information.

To be able to get a overview of how Oracle executes a particular query, install "autotrace". I usually follow the instructions here http://asktom.oracle.com/~tkyte/article1/autotrace.html.