0.00%
Search · Index

Weblog Page

Showing 41 - 50 of 230 Postings (summary)

Package Manager Requirements

Created by Gustaf Neumann, last modified by Gustaf Neumann 13 Feb 2009, at 09:20 AM

By Bryan Quinn and Todd Nightingale

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

The following is a requirements document for the OpenACS Package Manager (APM), version 4.0 (APM4). APM4 offers a superset of APM v3.3 functionality with the following specific enhancements:

  • A public procedural API. (v 3.3 only has web-based UI)

  • Support for dependency checking.

  • Support for compound packages (to support installation chaining).

  • Support for on-line parameter setting.

  • Support for sub-site level configuration (requires revised parameter and /admin pages at sub-site level; deprecation of site-wide parameter file).

To differentiate these new requirements from the requirements of version 3.3, all requirements new in v4 are prefaced with the number 4.

We gratefully acknowledge the authors of APM 3 for their original design documentation which suggested these features, as well as the influence of the design and open-source implementation of the Red Hat Package manager, the Debian packaging system, and PERL's CPAN in the development of the ideas behind this document.

A typical website will tend to offer its users a number of web-based services or applications, e.g. a bulletin board, calendaring, classified ads, etc. A website may also have underlying subsystems, such as a permissions system, content management system, etc. For such applications and subsystem components, modularity - or the degree to which a component can be encapsulated and decoupled from the rest of the system - is of great value. Thus the OpenACS Package Manager (APM) was created to allow website components, or packages, to be added, removed, and upgraded easily, with minimum disturbance to the rest of the system. This allows site owners to steadily offer users new and improved services, and also allows programmers to quickly and easily distribute their OpenACS components in a standardized manner to other OpenACS sites.

In general terms, a package is a unit of software that serves a single well-defined purpose. The OpenACS Package Manager (APM) provides a mechanism for packaging, installing, and configuring OpenACS software in a consistent, user-friendly, and subsite-aware manner.

The OpenACS Package Manager (APM) consists of:

  • A standard format for APM packages including:

    • Version numbering, independent of any other package and the OpenACS as a whole

    • Specification of the package interface

    • Specification of dependencies on other packages (if any)

    • Attribution (who wrote it) and ownership (who maintains it)

  • Web-based tools for package management:

    • Obtaining packages from a remote distribution point

    • Installing packages, if and only if:

      1. All prerequisite packages are installed

      2. No conflicts will be created by the installation

    • Configuring packages (obsoleting the monolithic OpenACS configuration file)

    • Upgrading packages, without clobbering local modifications

    • Uninstalling unwanted packages

  • A registry of installed packages, database-backed and integrated with file system-based version control

  • Web-based tools for package development:

    • Creating new packages locally

    • Releasing new versions of locally-created packages

    • Uploading packages to a global package repository on the web

    • Use of these tools should be safe, i.e. installing or removing a package should never break an OpenACS installation

  • Web-based tools for package configuration:

    • The ability to change package parameter values on-line through a simple web interface.

    • A new ad_parameter which does not require a monolithic site-wide parameter's file or server restarts for changes to take effect.

    • The ability to manage multiple package instances at the sub-site level.

The APM is intended for the following classes of users, which may or may not overlap:

  1. Developers (referred to as 'the developer') use the APM to create a software package for distribution and use the procedural API for direct control of the APM system.

  2. Site-wide administrators (referred to as 'the administrator') use the APM to install packages for their OpenACS instance, and optionally make them available to sub-sites.

  3. Sub-site administrators (referred to as 'the sub-admin') use an administration interface to configure and enable packages for their sub-site.

Initial Package Development

David Developer writes a piece of software used to do knowledge management (km) for the OpenACS. He distributes his data model, procedure code, UI pages, and his documentation according to the APM specification. He splits the documentation and the code into sub-packages, and creates a KM installation-chain to install both with the APM developer UI. Noting that his software was built with Patricia Programmer's Super Widget toolkit, he specifies that as a dependency. Moreover, since this package is capable of being used at the sub-site level, David configures this option in the package. When the package development is complete, David uses the APM developer UI to construct a distribution file. He assigns it a version number, 1.0, and makes the package available for download at the OpenACS package repository.

Initial Package Installation

Annie Admin learns of David's KM system by browsing the OpenACS package repository. Annie Admin uses the APM administrator UI on her system. She selects to install a package from a URL and types the URL displayed on the system. The APM automatically downloads the package. The dependency checker notices that Patricia's Super Widget toolkit is required, so it warns Annie of this. Annie selects an option to find a package that satisfies the dependency at the OpenACS APM repository. The APM informs Annie that it can download and install Jim's Super Widget toolkit. Annie confirms this option. After successfully installing Jim's toolkit, Annie proceeds to install David's KM system. The data model is loaded and all of the files necessary for the software are installed. Because installation was successful, the package is available for use.

Since the package is available for use, its initialization routines are set to run automatically on server startup. Annie is warned that since there are initialization routines, she must restart the server for the package to be ready for use. Annie restarts the server.

Initial Subsite Use of Package

Annie Admin decides to make the KM module available only to a particular sub-site type on her OpenACS system, and not others. She specifies this option using the Sub-site type UI (not part of APM).

Annie Admin notifies Sally SubAdmin by e-mail that a new package is now available for use. Sally goes to her sub-site /admin page and sees that a new entry, KM, is available. Sally clicks on it and finds links to the installed KM documentation and to the web based configuration utility. Then, Sally configures the package using an automatically generated web interface and enables KM for use on her sub-site. After some initial use of the package, Sally decides to change some parameters using the SubAdmin UI. These changes take effect immediately, without any server restarts.

Upgrade Process

Sally SubAdmin finds a bug in the KM system and sends a report to David Developer. David reads the bug report and verifies that the bugs are present in the current version. Because the bugs are present in the shared procedure file, David assigns a watch to the file. David makes the necessary modifications to the source code and saves the file. Because a watch was assigned to the file, the APM automatically reloads the updated code. David tests the program and confirms that the bug is fixed. He increments the minor version number and makes km v 1.1 available for download at the repository.

Sally SubAdmin asks Annie Administrator to upgrade the package using the APM UI. This upgrade supersedes the old version of KM at the site-wide level. Once Annie upgrades the package, the new version starts working immediately in Sally's sub-site.

Procedural API

Danielle Developer wants her software to perform different actions depending on what version of another package is installed. She uses the APM procedural API to check if KM version 1.0 is installed or version 1.1. Based on the results of this procedural call, the software exhibits different behavior.

  • 4.500.0 Package Identification (All of these items are entered by the developer using the developer UI.)

    4.500.1 A human readable package key that is guaranteed to be unique to the local OpenACS site must be maintained by the APM. For example, "apm."

    4.500.5 A package id (primary key) that is guaranteed to be unique to the local site must be maintained by the APM. For example, "25."

    4.500.10 A package URL that is guaranteed to be unique across all sites must be maintained by the APM. The package URL should point to a server that allows download of the latest version of the package. For example, "http://openacs.org/software."

  • 4.505.0 Version Identification (All of these items are entered by the developer using the developer UI.)

    4.505.1 A version id (primary key) that is guaranteed to be unique to the local site must be maintained by the APM.

    4.505.5 A version URL that is guaranteed to be unique across all sites must be maintained by the APM. The version URL should point to a server that allows download of a specific version of the package.

The API for APM v3 is explicitly a private API. However, it would be useful to obtain information from the APM through a procedural API. Implementing the API specified below is quite easy given that there are pages that already do all of the below in raw SQL.

  • 4.400.0 Packages Status Predicates

    4.400.1 Given defining information such as a package URL, the APM API can return the status of the package on the local OpenACS instance.

  • 4.405.0 Package Information Procedures

    4.405.1 The APM API can return information for any locally installed packages, including the version number, paths and files, and package key.

  • 4.410.0 Sub-site Procedures

    4.410.1 After a package has been installed at the site-wide level, the system API will provide means to check for package presence, creation, enabling, disabling, and destruction on a subsite.

  • 4.415.0 Parameter Values (replaces ad_parameter)

    4.415.1 The system API shall allow subsite parameters for an installed package to be set by either site-wide administrators or sub-site admins. The subsite parameter can be set to be non-persistent (but default is to survive server restarts). The subsite parameter can also be set to only take effect after a server restart (default is immediate).

    4.415.5 Parameters for a given subsite and package can be returned by the system API.

Provisions will be made to assure that packages are securely identified.

  • 4.600.1 Each package will have a PGP signature and there will be MD5 time stamps for each file within the package.

  • 4.600.5 The APM will provide a facility to validate both the PGP signature and MD5 stamps information before a package install.

The user interface is a set of HTML pages that are used to drive the underlying API. It is restricted to site-wide administrators because the actions taken here can dramatically affect the state of the running OpenACS.

The intent of the developer's interface is to enable the developer to construct and maintain APM packages. It will be possible to disable the developer's interface for production sites to help reduce the chance of site failure; much of the functionality here can have cascading effects throughout the OpenACS and should not be used on a production site.

  • 10.0 Define a package.

    The developer must be able to create a new package by specifying some identifying information for the package. This includes a package name, a package key, version information, owner information, and a canonical URL.

    10.1 The APM must maintain the state of all locally generated packages.

    10.50 If the developer fails to provide the required information, the package cannot be created.

    10.55 All of the package information should be editable after creation, except for the package key.

    4.10.60 The package creator must specify whether the package is capable of being used in sub-sites, or if only a single, global instance of the package is permitted.

    4.10.65 If the developer fails to provide unique information for unique fields specified in the data model requirements, the package cannot be created.

  • 20.0 Add files to a package

    20.1 The developer must be able to add files to the package. This is done by copying the files into the package directory in the host OS's file system. Files can be added at any point after package creation.

    20.3 Once a package has been versioned and distributed, no new files should be added to the package without incrementing the version number.

    20.5 The APM's UI should facilitate the process of adding new files, by scanning the file system for new files automatically, and allowing the developer to confirm adding them.

    20.10 The developer cannot add files to a given package via the UI that do not exist in the file system already.

    20.15 Package file structure must follow a specified convention. Please see the design document for what we do currently.

  • 30.0 Remove files from a package

    The developer must be able to remove files from a package. This can be done in two ways.

    • 30.1 Access the APM UI, browse the file list, and remove files.

      30.1.1If a file is removed from the package list, but not from the file system, an error should be generated at package load time.

    • 30.5 Remove the file from file system.

      30.5.1 The APM UI should take note of the fact that the file is gone and offer the developer an option to confirm the file's deletion.

  • 40.0 Modify files in a package.

    40.1 The developer should be able to modify files in the file system. The APM UI should not interfere with this.

    40.5 However, if the developer modifies files containing procedural definitions, APM UI should allow a means to watch those files and automatically reload them if changed. See requirement 50.0 for more detail.

    40.10 Also, although a change in files implies that the package distribution file is out of date, it is the developer's responsibility to update it.

  • 4.45.0 Manage Package Dependency Information.

    4.45.1 The developer should be able to specify which interfaces the package requires.

    4.45.5 The developer should be able to specify which interfaces the package provides.

    4.45.10 Circular dependencies are not allowed.

  • 50.0 Watch a file

    4.50.1 The developer should be able to assign a watch to any Tcl procedure file, whether in /packages or /tcl.

    50.5 If a watched file is locally modified, then it will be automatically reloaded, thus allowing for any changes made to take affect immediately.

    4.50.10 The setting of a watch should be persistent across server restarts.

  • 60.0 Display an XML package specification

    60.1 The developer should be able to view the XML package specification that encodes all package information.

  • 70.0 Write an XML package specification to the file system

    70.1 The developer should be able to write an up-to-date XML specification to disk.

    70.5 The developer should be able to request the current XML specification for all installed, locally generated packages.

  • 130.0 Distribution file generation

    130.1 The developer should be able to generate a .APM distribution file for the package with just one click.

    130.5 Generating a distribution file implies doing an "up-to-date" check on all of the files. If any of the files have changed since package installation, then a new version of the package is created.

  • 140.0 Access CVS information

    140.1 The developer should be able to determine the CVS status of a package, or all packages, with a single click.

  • 4.400.0 Compound Package Construction

    4.400.1 The developer can include .APM packages (sub-packages) within a package (the compound package) like any other file.

    4.400.5 The recommended usage for this feature is to allow for separation of optional and required components from the installation as well as better organization of files once installed. For example, all documentation for the community-core can be packages as community-core-doc.apm. It is legal to include sub-packages with dependencies that are not satisfied by the packages in the compound package, but this is discouraged. In such a case, the sub-package should really be a separate package that is required by the compound package.

    4.400.10 If a sub-package is required for the installation of the compound package, the compound package should have a registered dependency on the sub-package.

The requirement of the administrator's interface is to enable the administrator to install, enable, upgrade, disable, deinstall, and delete packages.

  • 80.0 Package Enable/Disable

    4.80.1 The administrator should be able mark an installed package as enabled. This means that the package is activated and its functionality is delivered through the Request Processor. As of OpenACS 4, this is done through the sub-site system.

    4.80.5 Moreover, the administrator must be able to disable a package, thereby removing the functionality provided to a sub-site. As of OpenACS 4, this is done through the sub-site system.

  • 90.0 Package Install

    90.1 The administrator must be able to install new packages either from locally maintained .APM files or from URLs.

    90.5 In the case of an URL, the APM transparently downloads the APM file off the web, proceeds with a file based installation, and then optionally removes the .APM file just downloaded.

    90.10.1 If .APM files are present in a package, then it is considered a compound package (use 4.410.0).

    90.15.0 Installation requires these steps:

    1. 90.15.1The package dependencies are scanned. If some dependencies are not present, the system warns the administrator that installation cannot proceed until those packages are installed.

    2. 90.15.2 Assuming all dependencies are present, APM extracts the contents of the APM file into the /packages directory.

    3. 90.15.3 The administrator is offered the option of importing directly into CVS.

    4. 90.15.4 The administrator is given a list of data model scripts found in the package and can select which ones to be executed.

    5. 90.15.5 If no errors are recorded during this process, the package is enabled.

  • 4.410.0 Compound package Install

    4.410.1 If .APM files are present in a package, then it is considered a compound package.

    4.410.5.0 Installation of a compound package proceeds according to the following sequence:

    1. 4.410.5.1 Identify the set of all sub-packages within the compound package by scanning for all files with .APM.

    2. 4.410.5.2 Identify which sub-packages are required by checking the dependencies of the compound package. If there dependencies not satisfied by the current system or the packages included with the compound package, halt installation and inform user to install these packages first.

    3. 4.410.5.3 Present Administrator with the ability to choose which sub-packages to install. Required sub-packages must be installed.

    4. 4.410.5.4 Proceed with the installation of each sub-package, starting with required packages. If the sub-package is already installed, then do nothing. Else, If the sub-package is a normal package, proceed according to 90.15.0, otherwise if it is a compound package, proceed according to 4.410.5.0.

    5. 4.410.5.5 If all required sub-packages are installed, proceed to install non-required sub-packages. If there was a failure during the installation of a required sub-package, then the installation of the compound package is also a failure.

    6. 4.410.5.6 Any attempt to install a compound package in the future involves a choice presented to the admin of installing any uninstalled sub-packages.

  • 4.420.0 Recovering from failed package installation

    4.420.1 If any error is generated during package installation, the package is not considered installed. To recover from this failure, the package should be selected for installation again.

  • 100.0 Version Upgrade

    100.1 The administrator can upgrade to a new version of a package. This entails

    1. 100.1.1 Running any necessary and included upgrade scripts.

    2. 100.1.5 Replacing any old files with new versions.

    3. 100.1.10 Marking the old version of the package as 'superseded' and disabling it.

    4. 100.1.15 Assuming no errors from above, the new package is enabled.

  • 110.0 Package Deinstall

    110.1 The administrator must be able to deinstall a package that has already been installed. Deinstallation entails:

    1. 110.1.1 Running any data model scripts necessary to drop the package.

    2. 110.1.5 Moving all of the files into a separate location in the file system from the installed packages.

    3. 4.110.1.10 If the package is a compound package, then the administrator must confirm removing all sub-packages. Optionally, some sub-packages can be kept.

    110.5 Deinstalled packages can be re-installed at a later date.

    4.110.10 If deinstalling a package or any of its sub-packages breaks a dependency, then deinstallation cannot proceed until the package registering the dependency is removed.

  • 120.0 Package Deletion

    120.1 The administrator should be able to completely erase all records of the package. This involves removing all instances of the package, all related database tables and content.

    120.5 This option can only be used if all package instances are deleted or marked as disabled. This is purposefully cumbersome because deleting all instances of a package can have far-sweeping consequences throughout a site and should almost never be done.

  • 150.0 Scan for new or modified packages

    150.1 The administrator should be able to scan the file system for any changes made in any of the installed package files.

    150.5 The administrator should be able to scan the file system for any newly installed packages.

If the developer is in charge of creating packages and the administrator for installing them, then the sub-site administrator is responsible for configuring and enabling packages. In order for a package to be available for a sub-site it must be associated with the sub-site's type specification. This interface is part of the sub-site /admin interface.

  • 4.300 Creating a package instance.

    4.300.1 From the sub-site /admin interface, there should be an option to view all packages available in the system as well as an option to add a package to the subsite.

    4.300.5 From the "add" option, the sub-admin can select from a list of packages registered as available in the sub-site type to which the sub-site belongs.

    4.300.19 Once a package instance is added, it is available on the list of the subsite's available packages.

  • 4.305 Configuring a package instance.

    4.305.1 An automatic web interface that lists all parameters with current values must be available.

    4.305.5 Changing the values for the parameters is accomplished simply by submitting an HTML form.

  • 4.310 Enabling a package instance.

    4.310.1 The sub-admin should be able to enable a package with a single click. Enabling a package means that the OpenACS will serve its URLs properly.

  • 4.315 Disabling a package instance.

    4.315.1 The sub-admin should be able to disable a package with a single click. Disabling a package means that the OpenACS will no longer serve those URLs.

  • 4.320 Deleting a package instance.

    4.320.1 Deleting a package instance involves deleting not only the package instance, but any and all content associated with it. It is questionable whether this option should even be available due to its drastic consequences. Reviewer comments appreciated.

Despite the fact that requirements are meant to be design/implementation neutral, the following thoughts were in our head when specifying these requirements. You must be familiar with the new object design for this to be comprehensible.

When a package is installed system-wide, a corresponding acs_object_type is created for it. All parameters registered for the package are registered for that acs_object_type.

When a package instance is created, it is an acs_object. Its parameters are set using the acs_attribute_values table. The automatic web interface for setting package parameters should be one and the same with the interface for setting acs object attribute values. Consequently, the implementation of these features should be quite straightforward.

Document Revision # Action Taken, Notes When? By Whom?
0.1 Creation 8/10/2000 Bryan Quinn, Todd Nightingale

Reviewed 8/11/2000 John Prevost, Mark Thomas, and Pete Su
0.2 Revised and updated 8/12/2000 Bryan Quinn
0.3 Reviewed, revised, and updated - conforms to requirements template. 8/18/2000 Kai Wu
0.4 Minor edits before ACS 4 Beta. 9/30/2000 Kai Wu

Prerequisite Software

Created by Gustaf Neumann, last modified by Gustaf Neumann 09 Jan 2009, at 12:53 AM

by Joel Aufrecht

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

OpenACS requires, at a minimum, an operating system, database, and webserver to work. Many additional programs, such as a build environment, Mail Transport Agent, and source control system, are also needed for a fully effective installation.

Table 2.2. Version Compatibility Matrix

OpenACS Version3.2.5 4.5 4.6 4.6.14.6.24.6.35.05.1 5.2 (core)
5.3 (core)
5.4 (core)
5.5 (core)
AOLserver3Yes No
3.3+ad13MaybeYes No
3.3oacs1MaybeYes No
3.4.4No
3.4.4oacs1MaybeYes No
3.5.5MaybeYes No
4.0Maybe Yes
4.5 No Yes
Tcl8.4Yes
8.5.4 - Maybe Yes
PostgreSQL 7.0Yes No
7.2MaybeYes No
7.3.2 - 7.3.xNoYes No
7.4NoYes No
8.0NoMaybe Yes
8.1 NoYes
8.2 No tar: no, CVS: Yes Yes
8.3 No Yes
Oracle8.1.6MaybeYes Maybe
8.1.7MaybeYes Maybe
9iNo Yes
10gNo Yes

The OpenACS installation instructions assume the operating system and build environment are installed. The instructions explain installation of TCL, tDOM, tclwebtest, a Web Server, a Database, a Process Controller, and Source Control software. The following external links are for reference only.

  • OpenACS 5.2.3rc1.The OpenACS tarball comprises the core packages and many useful additional packages. This includes a full set of documentation. The tarball works with both PostgreSQL and Oracle. Some scripts require bash shell.

  • Operating System.OpenACS is designed for a Unix-like system. It is developed primarily in Linux. It can be run on Mac OS X, and in Windows within VMWare.

    • GNU/Linux.The installation assumes a linux kernel of 2.2.22 or newer, or 2.4.14 or newer.

    • FreeBSD.FreeBSD guide. The OpenACS Reference Platform uses shell scripts written for bash, which is the standard Linux shell. If you are using a different shell, you will need to substitute your shell's conventions for setting environment variables when appropriate, and install bash to work with the scripts. Substitute fetch when the instructions suggest you use wget to download software.

    • Mac OS X.the section called “OpenACS Installation Guide for Mac OS X”

    • Windows/VMWare.the section called “OpenACS Installation Guide for Windows2000” The only way to run OpenACS on Windows is through the VMWare emulator. (Please let me know if you have OpenACS running directly in Windows.)

  • Build Environment.The Reference Platform installation compiles most programs from source code.

    • glibc 2.2 or newer, REQUIRED.You need recent versions of these libraries for Oracle to work properly. For Unicode support, you need glibc 2.2 or newer. This should be included in your operating system distribution.

    • GNU Make 3.76.1 or newer, REQUIRED.PostgreSQL and AOLserver require gmake to compile. Note that on most linux distributions, GNU Make is simply named make and there is no gmake, whereas on BSD distributions, make and gmake are different --use gmake.

  • TCL 8.4.x.

    • TCL 8.4.x, REQUIRED.OpenACS is written in TCL, an interpreted language. A threaded version of the TCL interpreter must be installed for OpenACS to work. The TCL interpreter that is included in most standard distributions may not be thread safe.

    • TCL 8.4.x development headers and libraries, OPTIONAL. The site-wide-search service, OpenFTS, requires these to compile. (Debian users: apt-get install tcl8.4-dev). You need this to install OpenFTS.

  • tDOM, REQUIRED.OpenACS 5.2.3rc1 stores queries in XML files, so we use an AOLserver module called tDOM to parse these files. (This replaces libxml2, which was used prior to 4.6.4.)

  • tclwebtest, OPTIONAL.tclwebtest is a tool for testing web interfaces via tcl scripts.

  • Web Server.The web server handles incoming HTTP requests, provides a runtime environment for OpenACS's tcl code, connects to the database, sends out HTTP responses, and logs requests and errors. OpenACS uses AOLserver; some people have had success running Apache with mod_nsd.

    • AOLserver 4.x, REQUIRED.Provides the base HTTP server

    Mat Kovach is graciously maintaining an AOLserver distribution that includes all the patches and modules needed to run OpenACS 5.2.3rc1. These instructions will describe how to install using his source distribution. He also has binaries for SuSE 7.3 and OpenBSD 2.8 (and perhaps more to come), currently located at uptime.openacs.org.

    It's also possible to download all the pieces and patches yourself:

    • AOLserver is available at aolserver.com

    • The OpenACS PostgreSQL driver (nspostgres.so) is available from SourceForge. If you do decide to use nspostgres.so, you have to remember to change the AOLserver config file to point to nspostgres.so instead of postgres.so. This guide uses Mat Kovach's distro (i.e. postgres.so)

    • The patch that makes exec work on BSD is available at sourceforge.net

    • The patch for aolserver 3.x that makes ns_uuencode work for binary files is available at sourceforge.net

    • The patch that makes AOLserver 3.x respect the -g flag is available at sourceforge.net

  • nsopenssl, OPTIONAL.Provides SSL capabilities for AOLserver. It requires OpenSSL. You need this if you want users to make secure (https) connections to your webserver. aolserver3.x requires nsopenssl 2.1a. aolserver4.x requires nsopenssl3; see aolserver.com for latest release. (home page)

  • ns_pam 0.1 or newer, OPTIONAL.Provides PAM capabilities for AOLserver. You need this if you want OpenACS users to authenticate through a PAM module (such as RADIUS).

  • pam_radius 1.3.16, OPTIONAL.Provides RADIUS capabilities for PAM. You need this if you want to use RADIUS authentication via PAM in OpenACS.

  • ns_ldap 0.r8, OPTIONAL.Provides LDAP capabilities for AOLserver. You need this if you want to use LDAP authentication in OpenACS.

  • OpenFTS TCL 0.3.2, OPTIONAL.Adds full-text-search to PostgreSQL and includes a driver for AOLserver. You need this if you want users to be able to search for any text on your site. For postgres 7.4.x and higher, full text search is also available via tsearch2.

  • Analog 5.32 or newer, OPTIONAL.This program examines web server request logs, looks up DNS values, and produces a report. You need this if you want to see how much traffic your site is getting.

  • Balance 3.11 or newer, OPTIONAL."Balance is a simple but powerful generic tcp proxy with round robin load balancing and failover mechanisms." You need this or something equivalent if you are running a high-availability production site and do not have an external load balancing system.

  • Database.The data on your site (for example, user names and passwords, calender entries, and notes) is stored in the database. OpenACS separates the database with an abstraction layer, which means that several different databases all function identically. While you can run the core OpenACS on any supported database, not all contributed packages support all databases.

    • Oracle 8.1.7 (Either this or PostgreSQL is REQUIRED).You can register and download Oracle from Oracle TechNet. You need this if you want to use an Oracle database.

    • PostgreSQL 7.4.x (Either this or Oracle is REQUIRED).You need this if you want to use a PostgreSQL database.

  • Process Controller.This is software that initiates other software, and restarts that software if it fails. On Linux, we recommend using Daemontools to control AOLserver and qmail.

    • Daemontools 0.76, OPTIONAL.You need this if you want AOLserver and qmail to run "supervised," meaning that they are monitored and automatically restarted if they fail. An alternative would be to run the services from inittab.

  • Mail Transport Agent.A Mail Transport Agent is a program that handles all incoming and outgoing mail. The Reference Platform uses Qmail; any MTA that provides a sendmail wrapper (that is, that can be invoked by calling the sendmail program with the same variables that sendmail expects) can be used.

    • Netqmail 1.04, OPTIONAL.You need this (or a different Mail Transport Agent) if you want your webserver to send and receive email.

    • ucspi-tcp 0.88, OPTIONAL.This program listens for incoming TCP connections and hands them to a program. We use it instead of inetd, which is insecure. You need this if you are running qmail.

  • DocBook, OPTIONAL.(docbook-xml v4.4, docbook-xsl v1.56, libxslt 1.0.21, xsltproc 1.0.21). You need this to write or edit documentation.

  • Source Control.A Source Control system keeps track of all of the old versions of your files. It lets you recover old files, compare versions of file, and identify specific versions of files. You can use any source control system; the Reference Platform and the OpenACS.org repository (where you can get patched and development code in between releases) use cvs.

    • cvs 1.11.18, OPTIONAL.cvs is included in most unix distributions. You need this if you want to track old versions of your files, do controlled deployment of code from development to production, or get or contribute development code from openacs.org.

Contributing code back to OpenACS

Created by Gustaf Neumann, last modified by Gustaf Neumann 21 Nov 2008, at 12:03 PM

There are three main ways to contribute code to OpenACS:

  1. To contribute a small fix, if you do not have a developer account, submit a patch.

  2. If you are making many changes, or would like to become a direct contributor, send mail to the Core Team asking for commit rights. You can then commit code directly to the repository:

    1. Use one of the checkout methods described above to get files to your system. This takes the place of steps 1 and 2 in Section, "Installation Option 2: Install from tarball". Continue setting up the site as described there.

    2. Fix bugs and add features.

    3. Commit that file (or files):

      cvs commit -m "what I did and why" filename

      Because this occurs in your personal checkout and not an anonymous one, this commit automagically moves back upstream to the Mother Ship repository at cvs.openacs.org. The names of the changed files, and your comments, are sent to a mailing list for OpenACS developers. A Core Team developer may review or roll back your changes if necessary.

    4. Confirm via the OpenACS CVS browser that your changes are where you intended them to be.

  3. Add a new package. Contact the Core Team to get approval and to get a module alias created.

    1. Check out acs-core on the HEAD branch. (Weird things happen if you add files to a branch but not to HEAD):

      cd /tmp
      cvs -d:ext:cvs.openacs.org:/cvsroot checkout acs-core

      Copy your package directory from your working directory to this directory. Make sure not to copy any CVS directories.

      cp -r /var/lib/aolserver/service0/packages/newpackage /tmp/openacs-4/packages

      Import the package into the cvs.openacs.org cvs repository:

      cd /tmp/openacs-4/packages/newpackage
      cvs import -m "Initial import of newpackage" openacs-4/packages/newpackage mynamenewpackage-0-1d
      
    2. Add the new package to the modules file. (An administrator has to do this step.) On any machine, in a temporary directory:

      cvs -d :ext:cvs.openacs.org:/cvsroot co CVSROOT
      cd CVSROOT
      emacs modules

      Add a line of the form:

      photo-album-portlet openacs-4/packages/photo-album-portlet
      

      Commit the change:

      cvs commit -m "added alias for package newpackage" modules

      This should print something like:

      cvscommit:Examining.
      ****Accessallowed:PersonalKarmaexceedsEnvironmentalKarma.
      Checkinginmodules;
      /cvsroot/CVSROOT/modules,v<--modules
      newrevision:1.94;previousrevision:1.93
      done
      cvscommit:Rebuildingadministrativefiledatabase

    3. Although you should add your package on HEAD, you should do package development on the latest release branch that your code is compatible with. So, after completing the import, you may want to branch your package:

      cd /var/lib/aolserver/service0/packages/newpackage
      cvs tag -b oacs-5-1
      
    4. See Section, "How to package and release an OpenACS Package"

    Note

    Some packages are already in cvs at openacs-4/contrib/packages. Starting with OpenACS 5.1, we have a Maturity mechanism in the APM which makes the contrib directory un-necessary. If you are working on a contrib package, you should move it to /packages. This must be done by an OpenACS administrator. On cvs.openacs.org:

    1. cp -r /cvsroot/openacs-4/contrib/packages/package0 /cvsroot/openacs-4/packages
    2. Update the modules file as described above.

    3. Remove the directory from cvs in the old location using cvs rm. One approach for file in `find | grep -v CVS`; do rm $file; cvs remove $file; done

CVS commit procedures are governed by TIP (Technical Improvement Proposal) #61: Guidelines for CVS committers

  1. Which branch?

    1. For core packages, new features should always be committed on HEAD, not to release branches.

    2. For core packages, bug fixes should be committed on the current release branch whenever applicable.

    3. For non-core packages, developers should work on a checkout of the release branch of the lastest release. For example, if OpenACS 5.1.0 is released, developers should work on the oacs-5-1 branch. When oacs-5-2 is branched, developers should continue working on oacs-5-1 until OpenACS 5.2.0 is actually released.

    4. The current release branch is merged back to HEAD after each dot release.

  2. New packages should be created in the /packages directory and the maturity flag in the .info file should be zero. This is a change from previous policy, where new packages went to /contrib/packages)

  3. Code

    1. Only GPL code and material should be committed to the OpenACS CVS repository (cvs.openacs.org)

    2. Code should only be reformatted when functionality is changed, e.g. when you change control flow and reindent to reflect it.

    3. Database upgrade scripts should only span one release increment, and should follow Naming Database Upgrade Scripts .

    4. Database upgrade scripts should never go to the release version, e.g., should always have a letter suffix such as d1 or b1.

    5. CVS commit messages should be intelligible in the context of Changelogs. They should not refer to the files or versions.

    6. CVS commit messages and code comments should refer to bug, tip, or patch number if appropriate, in the format "resolves bug 11", "resolves bugs 11, resolves bug 22". "implements tip 42", "implements tip 42, implements tip 50", "applies patch 456 by User Name", "applies patch 456 by User Name, applies patch 523 by ...".

  4. When to TIP

    1. A TIP is a Techical Improvement Proposal ( more information ). A proposed change must be approved by TIP if:

      1. It changes the core data model, or

      2. It will change the behavior of any core package in a way that affects existing code (typically, by changing public API), or

      3. It is a non-backwards-compatible change to any core or standard package.

    2. A proposed change need not be TIPped if:

      1. it adds a new function to a core package in a way that:

        1. does not change the backwards-compatibility of public API functions.

        2. does not change the data model

        3. has no negative impact on performance

      2. it changes private API, or

      3. it is a change to a non-core, non-standard package

  5. Tags

    1. When a package is released in final form, the developer shall tag it "packagename-x-y-z-final" and "openacs-x-y-compat". x-y should correspond to the current branch. If the package is compatible with several different core versions, several compat tags should be applied.

    2. When OpenACS core is released, the openacs-x-y-z-final tag shall be applied to all compat packages.

    For example, adding a new API function wouldn't require a TIP. Changing an existing API function by adding an optional new flag which defaults to no-effect wouldn't require a TIP. Added a new mandatory flag to an existing function would require a TIP.

We don't currently have clear standards for committing code.

Rule 1.3: First, this ensures that developers are working against stable core code. Second, it ensures that new package releases are available to OpenACS users immediately.

Rule 3.4: If an upgrade script ends with the final release number, then if a problem is found in a release candidate it cannot be addressed with another upgrade script. E.g., the last planned upgrade script for a package previously in dev 1 would be upgrade-2.0.0d1-2.0.0b1.sql, not upgrade-2.0.0d1-2.0.0.sql. Note that using rc1 instead of b1 would be nice, because that's the convention with release codes in cvs, but the package manager doesn't support rc tags.

Rule 5.1: Reason 1: The packagename tag is a permanent, static tag that allows for future comparison. The compat tag is a floating tag which is used by the repository generator to determine the most recent released version of each package for each core version. This allows package developers to publish their releases to all users of automatic upgrade without any intervention from the OpenACS release team.Reason 2: The compat tags allows CVS users to identify packages which have been released since the last core release.Reason 3: The compat tag or something similar is required to make Rule 6 possible.

Rule 5.2: This allows OpenACS developers who are creating extensively customized sites to branch from a tag which is stable, corresponds to released code instead of development code, and applies to all packages. This tag can be used to fork packages as needed, and provides a common ancestor between the fork and the OpenACS code so that patches can be generated.

Informal guidelines which may be obsolete in places and should be reviewed:

  • Before committing to cvs you must submit a bug report and patch to the OpenACS bug tracker . The only exceptions to this rule are for package maintainers committing in a package they are maintaining and for members of the core team.

  • If you are committing a bug fix you need to coordinate with the package maintainer. If you are a maintainer then coordinate with any fellow maintainers.

  • If you are to commit a new feature, an architecture change, or a refactoring, you must coordinate with the OpenACS core team first. Also, such changes should have a discussion in the forums to allow for feedback from the whole community.

  • If you are changing the data model you *must* provide an upgrade script and bump up the version number of the package.

  • Consider any upgradability ramifications of your change. Avoid changing the contract and behaviour of Tcl procedures. If you want to build a new and clean API consider deprecating the old proc and making it invoke the new one.

  • Never rush to commit something. Before committing double check with cvs diff what exactly you are committing.

  • Always accompany a commit with a brief but informative comment. If your commit is related to bug number N and/or patch number P, indicate this in the commit comment by including "bug N" and/or "patch P". This allows us to link bugs and patches in the Bug Tracker with changes to the source code. For example suppose you are committing a patch that closes a missing HTML tag, then an appropriate comment could be "Fixing bug 321 by applying patch 134. Added missing h3 HTML close tag".

  • Commit one cohesive bug fix or feature change at a time. Don't put a bunch of unrelated changes into one commit.

  • Before you throw out or change a piece of code that you don't fully understand, use cvs annotate and cvs log on the file to see who wrote the code and why. Consider contacting the author.

  • Test your change before committing. Use the OpenACS package acs-automated-testing to test Tcl procedures and the tool Tclwebtest to test pages

  • Keep code simple, adhere to conventions, and use comments liberally.

  • In general, treat the code with respect, at the same time, never stop questioning what you see. The code can always be improved, just make sure you change the code in a careful and systematic fashion.

CVS Guidelines

Created by Gustaf Neumann, last modified by Gustaf Neumann 21 Nov 2008, at 12:02 PM

By Joel Aufrecht with input from Jeff Davis, Branimir Dolicki, and Jade Rubick.

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

All OpenACS code is available anonymously. To get code anonymously, use the parameter -d:pserver:anonymous@cvs.openacs.org:/cvsroot immediately after cvs in a cvs command to check out or export code.

If you are an OpenACS developer, you should check out code so that you or any other developer can commit it. To do this, use the parameter -d:ext:cvs.openacs.org:/cvsroot immediately after cvs in checkout commands. This will create a local checkout directory that uses cvs.openacs.org but does not specify the user. By default, it will use your local account name as the user, so if you are logged in as "foobar" it will try to check out and commit as if you had specified :ext:foobar@cvs.openacs.org:/cvsroot. The advantage of not specifying a user in the checkout command is that other users can work in the directory using their own accounts.

OpenACS.org supports non-anonymous cvs access only over ssh, so you must have CVS_RSH=ssh in your environment. (Typically this is accomplished by putting export CVS_RSH=ssh into ~/.bash_profile.). If your local account name does not match your cvs.openacs.org account name, create a file ~/.ssh/config with an entry like:

Host cvs.openacs.org
User joel

With this setup, you will be asked for your password with each cvs command. To avoid this, set up ssh certificate authentication for your openacs account. (More information)

You may want to set some more default actions for CVS usage. To do so, create the file ~/.cvsrc with the contents:

cvs -z6
cvs -q

-z6 speeds up cvs access over the network quite a bit by enabling compressed connection by default. -q suppresses some verbose output from commands. For example, it makes the output of cvs up much easier to read.

If you are actively developing a non-core package, you should work from the latest core release branch. Currently this is oacs-5-4. This ensures that you are working on top of a stable OpenACS core, but still allows you to commit feature changes to non-core packages. To check out all packages,

cvs -d :ext:cvs.openacs.org:/cvsroot checkout -r oacs-5-4 openacs-4

If you work in the directories created with this command, all of your cvs updates and commits will be confined to the oacs-5-4 branch. Your work will be merged back to HEAD for you with each release.

Because the entire openacs-4 directory is large, you may want to use only acs-core plus some specific modules. To do this, check out core first:

cvs -d:ext:cvs.openacs.org:/cvsroot checkout -r oacs-5-4  acs-core

Then add modules as needed:

cd /var/lib/aolserver/service0/packages
cvs up -d packagename

... where packagename is the name of the package you want. Visit the Package Inventory and Package maintainers and status for a list of available packages and their current state.

If you are actively developing packages in the OpenACS Core, work from the HEAD branch. HEAD is used for active development of the next version of core OpenACS. It may be very buggy; it may not even install correctly. Do not use this branch for development of non-core features unless your work depends on some of the HEAD core work. To check out HEAD, omit the -r tag.

To check out HEAD for development, which requires an OpenACS developer account:

cvs -d:ext:cvs.openacs.org:/cvsroot checkout acs-core

To check out HEAD anonymously:

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout acs-core

.LRN consists of a given version openacs core, plus a set of packages. These are collectively packages together to form a distrubution of .LRN. F .LRN 2.0.0 sits on top of OpenACS 5.0.0. .LRN also uses an OpenACS install.xml file during installation; this file is distributed within the dotlrn package and must be moved. To get a development checkout of .LRN in the subdirectory dotlrn:

cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-4 acs-core
mv openacs-4 dotlrn
cd dotlrn/packages
cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-4 dotlrn-all
mv dotlrn/install.xml ..

Once you have a checkout you can use some commands to track what has changed since you checked out your copy. cvs -n update does not change any files, but reports which changes have been updated or locally modified, or are not present in CVS.

To update your files, use cvs update. This will merge changes from the repository with your local files. It has no effect on the cvs.openacs.org repository.

All OpenACS code resides within a single CVS module, openacs-4. (The openacs-4 directory contains code for all versions of OpenACS 4 and later, and .LRN 1 and later.) Checking out this module retrieves all openacs code of any type. For convenience, subsets of openacs-4 are repackaged as smaller modules.

acs-core contains only critical common packages. It does not have any user applications, such as forums, bug-tracker, calendar, or ecommerce. These can be added at any time.

The complete list of core packages is:

acs-admin
acs-api-browser
acs-authentication
acs-automated-testing
acs-bootstrap-installer
acs-content-repository
acs-core-docs
acs-kernel
acs-lang
acs-mail
acs-messaging
acs-reference
acs-service-contract
acs-subsite
acs-tcl
acs-templating
ref-timezones search

dotlrn-all contains the packages required, in combination with acs-core, to run the .LRN system.

project-manager-all contains the packages required, in combination with acs-core, to run the project-manager package.

Each OpenACS package (i.e., directory in openacs-4/packages/) is also aliased as a module of the same name.

Tags and Branches look similar in commands, but behave differently. A tag is a fixed point on a branch. Check out a tag to get a specific version of OpenACS. Check out a branch to get the most current code for that major-minor version (e.g., 5.0.x or 5.1.x). You can only commit to a branch, not a tag, so check out a branch if you will be working on the code.

  • openacs-x-y-z-final tags mark final releases of OpenACS. This tag is applied to the acs-core files for an OpenACS core release, and to the latest released versions of all other packages at the time of release. Example: openacs-5-0-4-final.

  • dotlrn-x-y-z-final tags mark final releases of .LRN. These tags apply only to .LRN packages. Example: dotlrn-2-0-1-final

  • packagename-x-y-z-final tags apply to releases of individual packages. For example, calendar-2-0-0-final is a tag that will retrieve only the files in the calendar 2.0.0 release. It applies only to the calendar package. All non-core, non-dotlrn packages should have a tag of this style, based on the package name. Many packages have not been re-released since the new naming convention was adopted and so don't have a tag of this type.

  • openacs-x-y-compat tags point to the most recent released version of OpenACS X.Y. It is similar to openacs-x-y-z-compat, except that it will always get the most recent dot-release of Core and the most recent compatible, released version of all other packages. All of the other tag styles should be static, but -compat tags may change over time. If you want version 5.0.4 exactly, use the openacs-5-0-4-final tag. If you want the best newest released code in the 5.0.x release series and you want to upgrade within 5.0.x later, use the compat tag.

    For example, if you check out the entire tree with -r openacs-5-0-compat, you might get version 5.0.4 of each OpenACS core package, version 2.0.1 of calendar, version 2.0.3 of each .LRN package, etc. If you update the checkout two months later, you might get version 5.0.5 of all OpenACS core packages and version 2.1 of calendar.

  • oacs-x-y is a branch, , not a tag. All core packages in the 5.0 release series (5.0.0, 5.0.1, 5.0.2, etc) are also on the oacs-5-0 branch. Similarly, OpenACS core packages for 5.1.0 are on the oacs-5-1 branch.

    These branches are used for two purposes. OpenACS Core packages on these branches are being tidied up for release. Only bug fixes, not new features, should be added to core packages on release branches. For all other packages, release branches are the recommended location for development. For example, if you are working on calendar, which is compatible with openacs 5.0 but not 5.1, work on the oacs-5-0 branch.

  • HEAD is a branch used for development of core packages.

There are three main ways to contribute code to OpenACS:

  1. To contribute a small fix, if you do not have a developer account, submit a patch.

  2. If you are making many changes, or would like to become a direct contributor, send mail to the Core Team asking for commit rights. You can then commit code directly to the repository:

    1. Use one of the checkout methods described above to get files to your system. This takes the place of steps 1 and 2 in the section called "Installation Option 2: Install from tarball". Continue setting up the site as described there.

    2. Fix bugs and add features.

    3. Commit that file (or files):

      cvs commit -m "what I did and why" filename

      Because this occurs in your personal checkout and not an anonymous one, this commit automagically moves back upstream to the Mother Ship repository at cvs.openacs.org. The names of the changed files, and your comments, are sent to a mailing list for OpenACS developers. A Core Team developer may review or roll back your changes if necessary.

    4. Confirm via the OpenACS CVS browser that your changes are where you intended them to be.

  3. Add a new package. Contact the Core Team to get approval and to get a module alias created.

    1. Check out acs-core on the HEAD branch. (Weird things happen if you add files to a branch but not to HEAD):

      cd /tmp
      cvs -d:ext:cvs.openacs.org:/cvsroot checkout acs-core

      Copy your package directory from your working directory to this directory. Make sure not to copy any CVS directories.

      cp -r /var/lib/aolserver/service0/packages/newpackage /tmp/openacs-4/packages

      Import the package into the cvs.openacs.org cvs repository:

      cd /tmp/openacs-4/packages/newpackage
      cvs import -m "Initial import of newpackage" openacs-4/packages/newpackage mynamenewpackage-0-1d
      
    2. Add the new package to the modules file. (An administrator has to do this step.) On any machine, in a temporary directory:

      cvs -d :ext:cvs.openacs.org:/cvsroot co CVSROOT
      cd CVSROOT
      emacs modules

      Add a line of the form:

      photo-album-portlet openacs-4/packages/photo-album-portlet
      

      Commit the change:

      cvs commit -m "added alias for package newpackage" modules

      This should print something like:

      cvscommit:Examining.
      ****Accessallowed:PersonalKarmaexceedsEnvironmentalKarma.
      Checkinginmodules;
      /cvsroot/CVSROOT/modules,v<--modules
      newrevision:1.94;previousrevision:1.93
      done
      cvscommit:Rebuildingadministrativefiledatabase

    3. Although you should add your package on HEAD, you should do package development on the latest release branch that your code is compatible with. So, after completing the import, you may want to branch your package:

      cd /var/lib/aolserver/service0/packages/newpackage
      cvs tag -b oacs-5-1
      
    4. See the section called "How to package and release an OpenACS Package"

    Note

    Some packages are already in cvs at openacs-4/contrib/packages. Starting with OpenACS 5.1, we have a Maturity mechanism in the APM which makes the contrib directory un-necessary. If you are working on a contrib package, you should move it to /packages. This must be done by an OpenACS administrator. On cvs.openacs.org:

    1. cp -r /cvsroot/openacs-4/contrib/packages/package0 /cvsroot/openacs-4/packages
    2. Update the modules file as described above.

    3. Remove the directory from cvs in the old location using cvs rm. One approach for file in `find | grep -v CVS`; do rm $file; cvs remove $file; done

CVS commit procedures are governed by TIP (Technical Improvement Proposal) #61: Guidelines for CVS committers

  1. Which branch?

    1. For core packages, new features should always be committed on HEAD, not to release branches.

    2. For core packages, bug fixes should be committed on the current release branch whenever applicable.

    3. For non-core packages, developers should work on a checkout of the release branch of the lastest release. For example, if OpenACS 5.1.0 is released, developers should work on the oacs-5-1 branch. When oacs-5-2 is branched, developers should continue working on oacs-5-1 until OpenACS 5.2.0 is actually released.

      Reason: First, this ensures that developers are working against stable core code. Second, it ensures that new package releases are available to OpenACS users immediately.

    4. The current release branch is merged back to HEAD after each dot release.

  2. New packages should be created in the /packages directory and the maturity flag in the .info file should be zero. This is a change from previous policy, where new packages went to /contrib/packages)

  3. Code

    1. Only GPL code and material should be committed to the OpenACS CVS repository (cvs.openacs.org)

    2. Do not mix formatting changes with code changes. Instead, make a formatting-only change which does not affect the logic, and say so in the commit comment. Then, make the logic change in a separate commit. Reason: This makes auditing and merging code much easier.

    3. Database upgrade scripts should only span one release increment, and should follow Naming Database Upgrade Scripts .

      Reason: If an upgrade script ends with the final release number, then if a problem is found in a release candidate it cannot be addressed with another upgrade script. E.g., the last planned upgrade script for a package previously in dev 1 would be upgrade-2.0.0d1-2.0.0b1.sql, not upgrade-2.0.0d1-2.0.0.sql. Note that using rc1 instead of b1 would be nice, because that's the convention with release codes in cvs, but the package manager doesn't support rc tags.

    4. Database upgrade scripts should never go to the release version, e.g., should always have a letter suffix such as d1 or b1.

    5. CVS commit messages should be intelligible in the context of Changelogs. They should not refer to the files or versions.

    6. CVS commit messages and code comments should refer to bug, tip, or patch number if appropriate, in the format "resolves bug 11", "resolves bugs 11, resolves bug 22". "implements tip 42", "implements tip 42, implements tip 50", "applies patch 456 by User Name", "applies patch 456 by User Name, applies patch 523 by ...".

  4. When to TIP

    1. A TIP is a Techical Improvement Proposal ( more information ). A proposed change must be approved by TIP if:

      1. It changes the core data model, or

      2. It will change the behavior of any core package in a way that affects existing code (typically, by changing public API), or

      3. It is a non-backwards-compatible change to any core or standard package.

    2. A proposed change need not be TIPped if:

      1. it adds a new function to a core package in a way that:

        1. does not change the backwards-compatibility of public API functions.

        2. does not change the data model

        3. has no negative impact on performance

      2. it changes private API, or

      3. it is a change to a non-core, non-standard package

  5. Tags

    1. When a package is released in final form, the developer shall tag it "packagename-x-y-z-final" and "openacs-x-y-compat". x-y should correspond to the current branch. If the package is compatible with several different core versions, several compat tags should be applied.

      Reason 1: The packagename tag is a permanent, static tag that allows for future comparison. The compat tag is a floating tag which is used by the repository generator to determine the most recent released version of each package for each core version. This allows package developers to publish their releases to all users of automatic upgrade without any intervention from the OpenACS release team.Reason 2: The compat tags allows CVS users to identify packages which have been released since the last core release.Reason 3: The compat tag or something similar is required to make Rule 6 possible.

    2. When OpenACS core is released, the openacs-x-y-z-final tag shall be applied to all compat packages.

      Reason: This allows OpenACS developers who are creating extensively customized sites to branch from a tag which is stable, corresponds to released code instead of development code, and applies to all packages. This tag can be used to fork packages as needed, and provides a common ancestor between the fork and the OpenACS code so that patches can be generated.

    For example, adding a new API function wouldn't require a TIP. Changing an existing API function by adding an optional new flag which defaults to no-effect wouldn't require a TIP. Added a new mandatory flag to an existing function would require a TIP.

Informal guidelines which may be obsolete in places and should be reviewed:

  • Before committing to cvs you must submit a bug report and patch to the OpenACS bug tracker . The only exceptions to this rule are for package maintainers committing in a package they are maintaining and for members of the core team.

  • If you are committing a bug fix you need to coordinate with the package maintainer. If you are a maintainer then coordinate with any fellow maintainers.

  • If you are to commit a new feature, an architecture change, or a refactoring, you must coordinate with the OpenACS core team first. Also, such changes should have a discussion in the forums to allow for feedback from the whole community.

  • If you are changing the data model you *must* provide an upgrade script and bump up the version number of the package.

  • Consider any upgradability ramifications of your change. Avoid changing the contract and behaviour of Tcl procedures. If you want to build a new and clean API consider deprecating the old proc and making it invoke the new one.

  • Never rush to commit something. Before committing double check with cvs diff what exactly you are committing.

  • Always accompany a commit with a brief but informative comment. If your commit is related to bug number N and/or patch number P, indicate this in the commit comment by including "bug N" and/or "patch P". This allows us to link bugs and patches in the Bug Tracker with changes to the source code. For example suppose you are committing a patch that closes a missing HTML tag, then an appropriate comment could be "Fixing bug 321 by applying patch 134. Added missing h3 HTML close tag".

  • Commit one cohesive bug fix or feature change at a time. Don't put a bunch of unrelated changes into one commit.

  • Before you throw out or change a piece of code that you don't fully understand, use cvs annotate and cvs log on the file to see who wrote the code and why. Consider contacting the author.

  • Test your change before committing. Use the OpenACS package acs-automated-testing to test Tcl procedures and the tool Tclwebtest to test pages

  • Keep code simple, adhere to conventions, and use comments liberally.

  • In general, treat the code with respect, at the same time, never stop questioning what you see. The code can always be improved, just make sure you change the code in a careful and systematic fashion.

Using CVS with OpenACS

Created by Gustaf Neumann, last modified by Gustaf Neumann 20 Jul 2008, at 05:24 PM

All OpenACS code is available anonymously. To get code anonymously, use the parameter -d:pserver:anonymous@cvs.openacs.org:/cvsroot immediately after cvs in a cvs command to check out or export code.

If you are an OpenACS developer, you should check out code so that you or any other developer can commit it. To do this, use the parameter -d:ext:cvs.openacs.org:/cvsroot immediately after cvs in checkout commands. This will create a local checkout directory that uses cvs.openacs.org but does not specify the user. By default, it will use your local account name as the user, so if you are logged in as "foobar" it will try to check out and commit as if you had specified :ext:foobar@cvs.openacs.org:/cvsroot. The advantage of not specifying a user in the checkout command is that other users can work in the directory using their own accounts.

OpenACS.org supports non-anonymous cvs access only over ssh, so you must have CVS_RSH=ssh in your environment. (Typically this is accomplished by putting export CVS_RSH=ssh into ~/.bash_profile.). If your local account name does not match your cvs.openacs.org account name, create a file ~/.ssh/config with an entry like:

Host cvs.openacs.org
User joel

With this setup, you will be asked for your password with each cvs command. To avoid this, set up ssh certificate authentication for your openacs account. (More information)

You may want to set some more default actions for CVS usage. To do so, create the file ~/.cvsrc with the contents:

cvs -z6
cvs -q

-z6 speeds up cvs access over the network quite a bit by enabling compressed connection by default. -q suppresses some verbose output from commands. For example, it makes the output of cvs up much easier to read.

If you are actively developing a non-core package, you should work from the latest core release branch. Currently this is oacs-5-4. This ensures that you are working on top of a stable OpenACS core, but still allows you to commit feature changes to non-core packages. To check out all packages,

cvs -d :ext:cvs.openacs.org:/cvsroot co -r oacs-5-4 openacs-4

If you work in the directories created with this command, all of your cvs updates and commits will be confined to the oacs-5-4 branch. Your work will be merged back to HEAD for you with each release.

Because the entire openacs-4 directory is large, you may want to use only acs-core plus some specific modules. To do this, check out core first:

cvs -d:ext:cvs.openacs.org:/cvsroot -r oacs-5-4 checkout acs-core

Then add modules as needed:

cd /var/lib/aolserver/service0/packages
cvs up -d packagename

... where packagename is the name of the package you want. Visit the Package Inventory and Package maintainers and status for a list of available packages and their current state.

If you are actively developing packages in the OpenACS Core, work from the HEAD branch. HEAD is used for active development of the next version of core OpenACS. It may be very buggy; it may not even install correctly. Do not use this branch for development of non-core features unless your work depends on some of the HEAD core work. To check out HEAD, omit the -r tag.

To check out HEAD for development, which requires an OpenACS developer account:

cvs -d:ext:cvs.openacs.org:/cvsroot checkout acs-core

To check out HEAD anonymously:

cvs -d:pserver:anonymous@cvs.openacs.org:/cvsroot checkout acs-core

.LRN consists of a given version openacs core, plus a set of packages. These are collectively packages together to form a distrubution of .LRN. F .LRN 2.0.0 sits on top of OpenACS 5.0.0. .LRN also uses an OpenACS install.xml file during installation; this file is distributed within the dotlrn package and must be moved. To get a development checkout of .LRN in the subdirectory dotlrn:

cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-4 acs-core
mv openacs-4 dotlrn
cd dotlrn/packages
cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r oacs-5-4 dotlrn-all
mv dotlrn/install.xml ..

Once you have a checkout you can use some commands to track what has changed since you checked out your copy. cvs -n update does not change any files, but reports which changes have been updated or locally modified, or are not present in CVS.

To update your files, use cvs update. This will merge changes from the repository with your local files. It has no effect on the cvs.openacs.org repository.

Automated Backup

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

The recommended backup strategy for a production sit is to use an automated script which first backs up the database to a file in /var/lib/aolserver/$OPENACS_SERVICE_NAME/database-backup and then backs up all of /var/lib/aolserver/$OPENACS_SERVICE_NAME to a single zip file, and then copies that zip file to another computer.

  1. Make sure that the manual backup process described above works.

  2. Customize the default backup script. Edit /var/lib/aolserver/$OPENACS_SERVICE_NAME/etc/backup.sh with your specific parameters.

  3. Make sure the file is executable:

    chmod +x backup.sh
  4. Set this file to run automatically by adding a line to root's crontab. (Typically, with export EDITOR=emacs; crontab -e.) This example runs the backup script at 1:30 am every day.

    30 1 * * *        sh /var/lib/aolserver/$OPENACS_SERVICE_NAME/etc/backup.sh

External uptime validation

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

The OpenACS uptime site can monitor your site and send you an email whenever your site fails to respond. If you test the url http://yourserver.test/SYSTEM/dbtest.tcl, you should get back the string success.

OpenACS Style Guide

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By Jeff Davis

Why have coding standards for OpenACS? And if the code works why change it to adhere to some arbitrary rules?

Well, first lets consider the OpenACS code base (all this as of December 2003 and including dotLRN). There are about 390,000 lines of tcl code, about 460,000 lines of sql (in datamodel scripts and .xql files), about 80,000 lines of markup in .adp files, and about 100,000 lines of documentation. All told, just about a million lines of "stuff". In terms of logical units there are about 160 packages, 800 tables, 2,000 stored procedures, about 2,000 functional pages, and about 3,200 tcl procedures.

When confronted by this much complexity it's important to be able to make sense of it without having to wade through it all. Things should be coherent, things should be named predictably and behave like you would expect, and your guess about what something is called or where it is should be right more often than not because the code follows the rules.

Unfortunately, like any large software project written over a long period by a lot of different people, OpenACS sometimes lacks this basic guessability and in the interest of bringing it into line we have advanced these guidelines.

Here is a short list of the basic rules code contributed to OpenACS should follow...

  1. Follow the file naming and the package structure rules. Some of the file naming rules are requirements for things to function correctly (for example data model creation scripts and tcl library files must be named properly to be used), while some are suggestions (the object-verb naming convention) which if ignored won't break anything, but if you follow the rules people will be able to understand your package much more easily.

  2. Be literate in your programming. Use ad_proc, ad_library, and ad_page_contract to provide documentation for your code, use comments on your datamodel, explain what things mean and how they should work.

  3. Test. Write test cases for your API and data model; test negative cases as well as positive; document your tests. Provide tests for bugs which are not yet fixed. Test, Test, Test.

  4. Use namespaces. For new packages choose a namespace and place all procedures in it and in oracle create packages.

  5. Follow the constraint naming and the PL/SQL and PL/pgSQL rules. Naming constraints is important for upgradability and for consistency. Also, named constraints can be immensely helpful in developing good error handling. Following the PL/SQL and PL/pgSQL rules ensure that the procedures created can be handled similarly across both Oracle and PostgreSQL databases.

  6. Follow the code formatting guidelines. The code base is very large and if things are formatted consistently it is easier to read. Also, if it conforms to the standard it won't be reformatted (which can mask the change history and making tracking down bugs much harder). Using spaces rather than tabs makes patches easier to read and manage and does not force other programmers to decipher what tab settings you had in place in your editor.

  7. Use the standard APIs. Don't reinvent the wheel. Prefer extending an existing core API to creating your own. If something in the core does not meet your particular needs it probably won't meet others as well and fleshing out the core API's makes the toolkit more useful for everyone and more easily extended.

  8. Make sure your datamodel create/drop scripts work. Break the table creation out from the package/stored procedure creation and use create or replace where possible so that scripts can be sourced more than once. Make sure your drop script works if data has been inserted (and permissioned and notifications have been attached etc).

  9. Practice CVS/Bug Tracker Hygiene. Commit your work. commit with sensible messages and include patch and bug numbers in your commit messages.

    Create bug tracker tickets for things you are going to work on yourself (just in case you don't get to it and to act as a pointer for others who might encounter the same problem).

  10. Solicit code reviews. Ask others to look over your code and provide feedback and do the same for others.

Document Revision # Action Taken, Notes When? By Whom?
0.1 Creation 12/2003 Jeff Davis

Object Model Design

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By Pete Su, Michael Yoon, Richard Li, Rafael Schloming

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

Before OpenACS 4, software developers writing OpenACS applications or modules would develop each data model separately. However, many applications built on OpenACS share certain characteristics or require certain common services. Examples of such services include:

  • User comments

  • Storage of user-defined or extensible sets of attributes

  • Access control

  • General auditing and bookkeeping (e.g. creation date, IP addresses, and so forth)

  • Presentation tools (e.g. how to display a field in a form or on a page)

All of these services involve relating additional service-related information to application data objects. Examples of application objects include:

  • forum messages

  • A user home page

  • A ticket in the ticket tracker

In the past, developers had to use ad-hoc and inconsistent schemes to interface to various "general" services. OpenACS 4 defines a central data model that keeps track of the application objects that we wish to manage, and serves as a primary store of metadata. By metadata, we mean data stored on behalf of an application outside of the application's data model in order to enable certain central services. The OpenACS 4 Object Model (or object system) manages several different kinds of data and metadata to allow us to provide general services to applications:

  • Object Identification

    Every application object is given a unique identifier in the system. This identifier can be used to find all data related to a particular object.

  • Object Context and Access Control

    Every object is created in a particular security context, so the system can provide centralized access control.

  • Object Types and Attributes

    Objects are instances of developer-defined object types. Object types allow developers to customize the data that is stored with each object.

  • Relation Types

    Relation types provide a general mechanism for mapping instances of one object type (e.g. users) to instances of another object type (e.g. groups).

The next section will explore these facilities in the context of the the particular programming idioms that we wish to generalize.

Related Links

This design document should be read along with the design documents for the new groups system, subsites and the permissions system

The motivation for most of the facilities in the OpenACS 4 Object Model can be understood in the context of the 3.x code base and the kinds of programming idioms that evolved there. These are listed and discussed below.

Object identification is a central mechanism in OpenACS 4. Every application object in OpenACS 4 has a unique ID which is mapped to a row in a central table called acs_objects. Developers that wish to use OpenACS 4 services need only take a few simple steps to make sure that their application objects appear in this table. The fact that every object has a known unique identifier means that the core can deal with all objects in a generic way. In other words, we use object identifiers to enable centralized services in a global and uniform manner.

Implicit Object Identifiers in OpenACS 3.x

The motivation for implementing general object identifiers comes from several observations of data models in OpenACS 3.x. Many modules use a (user_id, group_id, scope) column-triple for the purpose of recording ownership information on objects, for access control. User/groups also uses (user_id, group_id) pairs in its user_group_map table as a way to identify data associated with a single membership relation.

Also, in OpenACS 3.x many utility modules exist that do nothing more than attach some extra attributes to existing application data. For example, general comments maintains a table that maps application "page" data (static or dynamic pages on the website) to one or more user comments on that page. It does so by constructing a unique identifier for each page, usually a combination of the table in which the data is stored, and the value of the primary key value for the particular page. This idiom is referred to as the "(on_which_table + on_what_id)" method for identifying application data. In particular, general comments stores its map from pages to comments using a "(on_which_table + on_what_id)" key plus the ID of the comment itself.

All of these composite key constructions are implicit object identifiers - they build a unique ID out of other pieces of the data model. The problem is that their definition and use is ad-hoc and inconsistent, making the construction of generic application-independent services unnecessarily difficult.

Object Identifiers in OpenACS 4

The OpenACS 4 Object Model defines a single mechanism that applications use to attach unique identifiers to application data. This identifier is the primary key of the acs_objects table. This table forms the core of what we need to provide generic services like access control, general attribute storage, general presentation and forms tools, and generalized administrative interfaces. In addition, the object system provides an API that makes it easy to create new objects when creating application data. All an application must do to take advantage of general services in OpenACS 4 is to use the new API to make sure every object the system is to manage is associated with a row in acs_objects. More importantly, if they do this, new services like general comments can be created without requiring existing applications to "hook into" them via new metadata.

Note: Object identifiers are a good example of metadata in the new system. Each row in acs_objects stores information about the application object, but not the application object itself. This becomes more clear if you skip ahead and look at the SQL schema code that defines this table.

Until the implementation of the general permissions system, every OpenACS application had to manage access control to its data separately. Later on, a notion of "scoping" was introduced into the core data model.

"Scope" is a term best explained by example. Consider some hypothetical rows in the address_book table:

... scope user_id group_id ...
... user 123 ...
... group 456 ...
... public ...

The first row represents an entry in User 123's personal address book, the second row represents an entry in User Group 456's shared address book, and the third row represents an entry in the site's public address book.

In this way, the scoping columns identify the security context in which a given object belongs, where each context is either a person or a group of people or the general public (itself a group of people).

In OpenACS 4, rather than breaking the world into a limited set of scopes, every object lives in a single context. A context is just an abstract name for the default security domain to which the object belongs. Each context has a unique identifier, and all the contexts in a system form a tree. Often this tree will reflect an observed hierarchy in a site, e.g. a forum message would probably list a forum topic as its context, and a forum topic might list a subsite as its context. Thus, contexts make it easier to break the site up into security domains according to its natural structure. An object's context is stored in the context_id column of the acs_objects table.

We use an object's context to provide a default answer to questions regarding access control. Whenever we ask a question of the form "can user X perform action Y on object Z", the OpenACS security model will defer to an object's context if there is no information about user X's permission to perform action Y on object Z.

The context system forms the basis for the rest of the OpenACS access control system, which is described in in two separate documents: one for the permissions system and another for the party groups system. The context system is also used to implement subsites.

As mentioned above, many OpenACS modules provide extensible data models, and need to use application specific mechanisms to keep track of user defined attributes and to map application data to these attributes. In the past, modules either used user/groups or their own ad hoc data model to provide this functionality.

User/Groups in OpenACS 3.x

The user/group system allowed developers to define group types along with attributes to be stored with each instance of a group type. Each group type could define a helper table that stored attributes on each instance of the group type. This table was called the "_info" table because the name was generated by appending _info to the name of the group type.

The user/groups data model also provided the user_group_type_member_fields and user_group_member_fields tables to define attributes for members of groups of a specific type and for members of a specific group, respectively. The user_group_member_field_map table stored values for both categories of attributes in its field_value column. These tables allowed developers and users to define custom sets of attributes to store on groups and group members without changing the data model at the code level.

Many applications in OpenACS 3.x and earlier used the group type mechanism in ways that were only tangentially related to groups of users, just to obtain access to this group types mechanism. Thus the motivation for generalizing the group types mechanism in OpenACS 4.

Object Types and Subtypes

In OpenACS 4 object types generalize the OpenACS 3.x notion of group types. Each object type can define one or more attributes to be attached to instances of the type. This allows developers to define new types without being artificially tied to a particular module (i.e. user/groups).

In addition, the OpenACS 4 object model provides mechanism for defining subtypes of existing types. A subtype of a parent type inherits all the attributes defined in the parent type, and can define some of its own. The motivation for subtypes comes from the need for OpenACS to be more extensible. In OpenACS 3.x, many applications extended the core data models by directly adding more columns, in order to provide convenient access to new information. This resulted in core data tables that were too "fat", containing a hodge podge of unrelated information that should have been normalized away. The canonical example of this is the explosion of the users table in OpenACS 3.x. In addition to being sloppy technically, these fat tables have a couple of other problems:

  • They degrade performance.

  • Denormalization can make it hard to maintain consistency constraints on the data.

Object subtypes provide a way to factor the data model while still keeping track of the fact that each member of a subtype (i.e. for each row in the subtype's table), is also a member of the parent type (i.e. there is a corresponding row in the parent type table). Therefore, applications an use this mechanism without worrying about this bookkeeping themselves, and we avoid having applications pollute the core data model with their specific information.

As we described above, the OpenACS 3.x user/groups system stored object attributes in two ways. The first was to use columns in the helper table. The second consisted of two tables, one describing attributes and one storing values, to provide a flexible means for attaching attributes to metadata objects. This style of attribute storage is used in several other parts of OpenACS 3.x, and we will refer to it as "skinny tables". For example:

  • In the Ecommerce data model, the ec_custom_product_fields table defines attributes for catalog products, and the ec_custom_product_field_values table stores values for those attributes.

  • In the Photo DB data model, the ph_custom_photo_fields table defines attributes for the photographs owned by a specific user, and tables named according to the convention "ph_user_<user_id>_custom_info" are used to store values for those attributes.

In addition, there are some instances where we are not using this model but should, e.g. the users_preferences table, which stores preferences for registered users in columns such as prefer_text_only_p and dont_spam_me_p. The "standard" way for an OpenACS 3.x-based application to add to the list of user preferences is to add a column to the users_preferences table (exactly the kind of data model change that has historically complicated the process of upgrading to a more recent OpenACS version).

The Objet Model generalizes the scheme used in the old OpenACS 3.x user/groups system. It defines a table called acs_attributes that record what attributes belong to which object types, and how the attributes are stored. As before, attributes can either be stored in helper tables, or in a single central skinny table. The developer makes this choice on a case by case basis. For the most part, attribute data is stored in helper tables so that they can take full advantage of relational data modeling and because they will generally be more efficient. Occasionally, a data model will use skinny tables because doing so allows developers and users to dynamically update the set of attributes stored on an object without updating the data model at the code level. The bottom line: Helper tables are more functional and more efficient, skinny tables are more flexible but limited.

Many OpenACS 3.x modules use mapping tables to model relationships between application objects. Again, the 3.x user/groups system provides the canonical example of this design style. In that system, there was a single table called user_group_map that kept track of which users belonged to what groups. In addition, as we discussed in the previous section, the system used the user_group_member_fields and user_group_member_fields_map tables to allow developers to attach custom attributes to group members. In fact, these attributes were not really attached to the users, but to the fact that a user was a member of a particular group - a subtle but important distinction.

In OpenACS 4, relation types generalize this mechanism. Relation types allow developers to define general mappings from objects of a given type T, to other objects of a given type R. Each relation type is a subtype of acs_object, extended with extra attributes that store constraints on the relation, and the types of objects the relation actually maps. In turn, each instance of a relation type is an object that represents a single fact of the form "the object t of type T is related to the object r of type R." That is, each instance of a relation type is essentially just a pair of objects.

Relation types generalize mapping tables. For example, the 3.x user/groups data model can be largely duplicated using a single relation type describing the "group membership" relation. Group types would then be subtypes of this membership relation type. Group type attributes would be attached to the relation type itself. Group member attributes would be attached to instances of the membership relation. Finally, the mapping table would be replaced by a central skinny table that the relation type system defines.

Relation types should be used when you want to be able to attach data to the "fact" that object X and object Y are related to each other. On the face of it, they seem like a redundant mechanism however, since one could easily create a mapping table to do the same thing. The advantage of registering this table as a relation type is that in principle the OpenACS 4 object system could use the meta data in the types table to do useful things in a generic way on all relation types. But this mechanism doesn't really exist yet.

Relation types are a somewhat abstract idea. To get a better feel for them, you should just skip to the data model.

The OpenACS 4 Object Model is designed to generalize and unify the following mechanisms that are repeatedly implemented in OpenACS-based systems to manage generic and application specific metadata:

The presence of a framework for subtyping and inheritance always brings up the question of why we don't just use an object database. The main reason is that all of the major object database vendors ship products that are effectively tied to some set of object oriented programming languages. Their idea is to provide tight language-level integration to lower the "impedance mismatch" between the database and the language. Therefore, database objects and types are generally directly modeled on language level objects and types. Of course, this makes it nearly impossible to interact with the database from a language that does not have this tight coupling, and it limits the data models that we can write to ideas that are expressible in the host language. In particular, we lose many of the best features of the relational database model. This is a disaster from an ease of use standpoint.

The "Object relational" systems provide an interesting alternative. Here, some notion of subtyping is embedded into an existing SQL or SQL-like database engine. Examples of systems like this include the new Informix, PostgreSQL 7, and Oracle has something like this too. The main problem with these systems: each one implements their own non-portable extensions to SQL to implement subtyping. Thus, making OpenACS data models portable would become even more difficult. In addition, each of these object systems have strange limitations that make using inheritance difficult in practice. Finally, object databases are not as widely used as traditional relational systems. They have not been tested as extensively and their scalability to very large databases is not proven (though some will disagree with this statement).

The conclusion: the best design is to add a limited notion of subtyping to our existing relational data model. By doing this, we retain all the power of the relational data model while gaining the object oriented features we need most.

In the context of OpenACS 4, this means using the object model to make our data models more flexible, so that new modules can easily gain access to generic features. However, while the API itself doesn't enforce the idea that applications only use the object model for metadata, it is also the case that the data model is not designed to scale to large type hierarchies. In the more limited domain of the metadata model, this is acceptable since the type hierarchy is fairly small. But the object system data model is not designed to support, for example, a huge type tree like the Java runtime libraries might define.

This last point cannot be over-stressed: the object model is not meant to be used for large scale application data storage. It is meant to represent and store metadata, not application data.

Like most data models, the OpenACS Core data model has two levels:

  1. The knowledge level (i.e. the metadata model)

  2. The operational level (i.e. the concrete data model)

You can browse the data models themselves from here:

(Note that we have subdivided the operational level into the latter two files.)

The operational level depends on the knowledge level, so we discuss the knowledge level first. In the text below, we include abbreviated versions of the SQL definitions of many tables. Generally, these match the actual definitions in the existing data model but they are meant to reflect design information, not implementation. Some less relevant columns may be left out, and things like constraint names are not included.

The knowledge level data model for OpenACS objects centers around three tables that keep track of object types, attributes, and relation types. The first table is acs_object_types, shown here in an abbreviated form:

create table acs_object_types (
        object_type          varchar(100) not null primary key,
        supertype            references acs_object_types (object_type),
        abstract_p           char(1) default 'f' not null
        pretty_name          varchar(100) not null unique,
        pretty_plural        varchar(100) not null unique,
        table_name           varchar(30) not null unique,
        id_column            varchar(30) not null,
        name_method          varchar(30),
        type_extension_table varchar(30)
);

This table contains one row for every object type in the system. The key things to note about this table are:

  • For every type, we store metadata for how to display this type in certain contexts (pretty_name and pretty_plural).

  • If the type is a subtype, then its parent type is stored in the column supertype.

  • We support a notion of "abstract" types that contain no instances (as of 9/2000 this is not actually used). These types exist only to be subtyped. An example might be a type representing "shapes" that contains common characteristics of all shapes, but which is only used to create subtypes that represent real, concrete shapes like circles, squares, and so on.

  • Every type defines a table in which one can find one row for every instance of this type (table_name, id_column).

  • type_extension_table is for naming a table that stores extra generic attributes.

The second table we use to describe types is acs_attributes. Each row in this table represents a single attribute on a specific object type (e.g. the "password" attribute of the "user" type). Again, here is an abbreviated version of what this table looks like. The actual table used in the implementation is somewhat different and is discussed in a separate document.

create table acs_attributes (
        attribute_id    integer not null primary key
        object_type     not null references acs_object_types (object_type),
        attribute_name  varchar(100) not null,
        pretty_name     varchar(100) not null,
        pretty_plural   varchar(100),
        sort_order      integer not null,
        datatype        not null,
        default_value   varchar(4000),
        storage         varchar(13) default 'type_specific'
                        check (storage in ('type_specific',
                                           'generic')),
        min_n_values    integer default 1 not null,
        max_n_values    integer default 1 not null,
        static_p        varchar(1)
);

The following points are important about this table:

  • Every attribute has a unique identifier.

  • Every attribute is associated with an object type.

  • We store various things about each attribute for presentation (pretty_name, sort_order).

  • The data_type column stores type information on this attribute. This is not the SQL type of the attribute; it is just a human readable name for the type of data we think the attribute holds (e.g. "String", or "Money"). This might be used later to generate a user interface.

  • The sort_order column stores information about how to sort the attribute values.

  • Attributes can either be stored explicitly in a table ("type specific storage") or in a skinny table ("generic storage"). In most cases, an attribute maps directly to a column in the table identified by the table_name of the corresponding object type, although, as mentioned above, we sometimes store attribute values as key-value pairs in a "skinny" table. However, when you ask the question "What are the attributes of this type of object?", you don't really care about how the values for each attribute are stored (in a column or as key-value pairs); you expect to receive the complete list of all attributes.

  • The max_n_values and min_n_values columns encode information about the number of values an attribute may hold. Attributes can be defined to hold 0 or more total values.

  • The static_p flag indicates whether this attribute value is shard by all instances of a type, as with static member fields in C++. Static attribute are like group level attributes in OpenACS 3.x.

The final part of the knowledge level model keeps track of relationship types. We said above that object relationships are used to generalize the 3.x notion of group member fields. These were fields that a developer could store on each member of a group, but which were contextualized to the membership relation. That is, they were really "attached" to the fact that a user was a member of a particular group, and not really attached to the user. This is a subtle but important distinction, because it allowed the 3.x system to store multiple sets of attributes on a given user, one set for each group membership relation in which they participated.

In OpenACS 4, this sort of data can be stored as a relationship type, in acs_rel_types. The key parts of this table look like this:

create table acs_rel_types (
        rel_type        varchar(100) not null
                        references acs_object_types(object_type),
        object_type_one not null
                        references acs_object_types (object_type),
        role_one        references acs_rel_roles (role),
        object_type_two not null
                        references acs_object_types (object_type),
        role_two        references acs_rel_roles (role)
        min_n_rels_one  integer default 0 not null,
        max_n_rels_one  integer,
        min_n_rels_two  integer default 0 not null,
        max_n_rels_two  integer
);

Things to note about this table:

  • The main part of this table records the fact that the relation is between instances of object_type_one and instances of object_type_two. Therefore, each instance of this relation type will be a pair of objects of the appropriate types.

  • The role columns store human readable names for the roles played by each object in the relation (e.g. "employee" and "employer"). Each role must appear in the acs_rel_roles.

  • The min_n_rels_one column, and its three friends allow the programmer to specify constraints on how many objects any given object can be related to on either side of the relation.

This table is easier to understand if you also know how the acs_rels table works.

To summarize, the acs_object_types and acs_attributes tables store metadata that describes every object type and attribute in the system. These tables generalize the group types data model in OpenACS 3.x. The acs_rel_types table stores information about relation types.

This part of the data model is somewhat analogous to the data dictionary in Oracle. The information stored here is primarily metadata that describes the data stored in the operational level of the data model, which is discussed next.

The operational level data model centers around the acs_objects table. This table contains a single row for every instance of the type acs_object. The table contains the object's unique identifier, a reference to its type, security information, and generic auditing information. Here is what the table looks like:

create table acs_objects (
        object_id               integer not null,
        object_type             not null
                                references acs_object_types (object_type),
        context_id              references acs_objects(object_id),
        security_inherit_p      char(1) default 't' not null,
                                check (security_inherit_p in ('t', 'f')),
        creation_user           integer,
        creation_date           date default sysdate not null,
        creation_ip             varchar(50),
        last_modified           date default sysdate not null,
        modifying_user          integer,
        modifying_ip            varchar(50)
);

As we said in Section III, security contexts are hierarchical and also modeled as objects. There is another table called acs_object_context_index that stores the context hierarchy.

Other tables in the core data model store additional information related to objects. The table acs_attribute_values and acs_static_attr_values are used to store attribute values that are not stored in a helper table associated with the object's type. The former is used for instance attributes while the latter is used for class-wide "static" values. These tables have the same basic form, so we'll only show the first:

create table acs_attribute_values (
        object_id       not null
                        references acs_objects (object_id) on delete cascade,
        attribute_id    not null
                        references acs_attributes (attribute_id),
        attr_value      varchar(4000),
        primary key     (object_id, attribute_id)
);

Finally, the table acs_relsis used to store object pairs that are instances of a relation type.

create table acs_rels (
        rel_id          not null
                        references acs_objects (object_id)
                        primary key
        rel_type        not null
                        references acs_rel_types (rel_type),
        object_id_one   not null
                        references acs_objects (object_id),
        object_id_two   not null
                        references acs_objects (object_id),
        unique (rel_type, object_id_one, object_id_two)
);

This table is somewhat subtle:

  • rel_id is the ID of an instance of some relation type. We do this so we can store all the mapping tables in this one table.

  • rel_type is the ID of the relation type to which this object belongs.

  • The next two object IDs are the IDs of the objects being mapped.

All this table does is store one row for every pair of objects that we'd like to attach with a relation. Any additional attributes that we'd like to attach to this pair of objects is specified in the attributes of the relation type, and could be stored in any number of places. As in the 3.x user/groups system, these places include helper tables or generic skinny tables.

This table, along with acs_attributes and acs_attribute_values generalize the old user/group tables user_group_map, user_group_member_fields_map and user_group_member_fields.

The core tables in the OpenACS 4 data model store information about instances of object types and relation types. The acs_object table provides the central location that contains a single row for every object in the system. Services can use this table along with the metadata in stored in the knowledge level data model to create, manage, query and manipulate objects in a uniform manner. The acs_rels table has an analogous role in storing information on relations.

These are all the tables that we'll discuss in this document. The rest of the Kernel data model is described in the documents for subsites, the permissions system and for the groups system.

Some examples of how these tables are used in the system can be found in the discussion of the API, which comes next.

Now we'll examine each piece of the API in detail. Bear in mind that the Object Model API is defined primarily through PL/SQL packages.

The object system provides an API for creating new object types and then attaching attributes to them. The procedures create_type and drop_type are used to create and delete type definitions.

The two calls show up in the package acs_object_type.

  procedure create_type (
    object_type         in acs_object_types.object_type%TYPE,
    pretty_name         in acs_object_types.pretty_name%TYPE,
    pretty_plural       in acs_object_types.pretty_plural%TYPE,
    supertype           in acs_object_types.supertype%TYPE
                           default 'acs_object',
    table_name          in acs_object_types.table_name%TYPE default null,
    id_column           in acs_object_types.id_column%TYPE default 'XXX',
    abstract_p          in acs_object_types.abstract_p%TYPE default 'f',
    type_extension_table in acs_object_types.type_extension_table%TYPE
                            default null,
    name_method         in acs_object_types.name_method%TYPE default null
  );

  -- delete an object type definition
  procedure drop_type (
    object_type         in acs_object_types.object_type%TYPE,
    cascade_p           in char default 'f'
  );

Here the cascade_p argument indicates whether dropping a type should also remove all its subtypes from the system.

We define a similar interface for defining attributes in the package acs_attribute:

  function create_attribute (
    object_type         in acs_attributes.object_type%TYPE,
    attribute_name      in acs_attributes.attribute_name%TYPE,
    datatype            in acs_attributes.datatype%TYPE,
    pretty_name         in acs_attributes.pretty_name%TYPE,
    pretty_plural       in acs_attributes.pretty_plural%TYPE default null,
    table_name          in acs_attributes.table_name%TYPE default null,
    column_name         in acs_attributes.column_name%TYPE default null,
    default_value       in acs_attributes.default_value%TYPE default null,
    min_n_values        in acs_attributes.min_n_values%TYPE default 1,
    max_n_values        in acs_attributes.max_n_values%TYPE default 1,
    sort_order          in acs_attributes.sort_order%TYPE default null,
    storage             in acs_attributes.storage%TYPE default 'type_specific',
    static_p            in acs_attributes.static_p%TYPE default 'f'
  ) return acs_attributes.attribute_id%TYPE;

  procedure drop_attribute (
    object_type in varchar,
    attribute_name in varchar
  );

In addition, the following two calls are available for attaching extra annotations onto attributes:

  procedure add_description (
    object_type         in acs_attribute_descriptions.object_type%TYPE,
    attribute_name      in acs_attribute_descriptions.attribute_name%TYPE,
    description_key     in acs_attribute_descriptions.description_key%TYPE,
    description         in acs_attribute_descriptions.description%TYPE
  );

  procedure drop_description (
    object_type         in acs_attribute_descriptions.object_type%TYPE,
    attribute_name      in acs_attribute_descriptions.attribute_name%TYPE,
    description_key     in acs_attribute_descriptions.description_key%TYPE
  );

At this point, what you must do to hook into the object system from your own data model becomes clear:

  • Create a table that will store the instances of the new type.

  • Call acs_object_type.create_type() to fill in the metadata table on this new type. If you want your objects to appear in the acs_objects table, then your new type must be a subtype of acs_object.

  • Call acs_attribute.create_attribute() to fill in information on the attributes that this type defines.

So, suppose we are writing a new version of the ticket tracker for 4.0. We probably define a table to store tickets in, and each ticket might have an ID and a description. If we want each ticket to be an object, then ticket_id must reference the object_id column in acs_objects:

create table tickets (
    ticket_id references acs_objects (object_id),
    description varchar(512),
    ...
) ;

In addition to defining the table, we need this extra PL/SQL code to hook into the object type tables:

declare
 attr_id acs_attributes.attribute_id%TYPE;
begin
 acs_object_type.create_type (
   supertype => 'acs_object',
   object_type => 'ticket',
   pretty_name => 'Ticket',
   pretty_plural => 'Tickets',
   table_name => 'tickets',
   id_column => 'ticket_id',
   name_method => 'acs_object.default_name'
 );

 attr_id := acs_attribute.create_attribute (
   object_type => 'ticket',
   attribute_name => 'description',
   datatype => 'string',
   pretty_name => 'Description',
   pretty_plural => 'Descriptions'
 );

 ... more attributes ...

commit;
end;

Thus, with a small amount of extra code, the new ticket tracker will now automatically be hooked into every generic object service that exists. Better still, this code need not be changed as new services are added. As an aside, the most important service that requires you to subtype acs_object is permissions.

The next important piece of the API is defined in the acs_object package, and is concerned with creating and managing objects. This part of the API is designed to take care of the mundane bookkeeping needed to create objects and query their attributes. Realistically however, limitations in PL/SQL and Oracle will make it hard to build generic procedures for doing large scale queries in the object system, so developers who need to do this will probably have to be fairly familiar with the data model at a lower level.

The function acs_object.new() makes a new object for you. The function acs_object.del() deletes an object. As before, this is an abbreviated interface with all the long type specs removed. See the data model or developer's guide for the full interface.

 function new (
  object_id     in acs_objects.object_id%TYPE default null,
  object_type   in acs_objects.object_type%TYPE
                           default 'acs_object',
  creation_date in acs_objects.creation_date%TYPE
                           default sysdate,
  creation_user in acs_objects.creation_user%TYPE
                           default null,
  creation_ip   in acs_objects.creation_ip%TYPE default null,
  context_id    in acs_objects.context_id%TYPE default null
 ) return acs_objects.object_id%TYPE;

 procedure delete (
  object_id     in acs_objects.object_id%TYPE
 );

Next, we define some generic functions to manipulate attributes. Again, these interfaces are useful to an extent, but for large scale queries, it's likely that developers would have to query the data model directly, and then encapsulate their queries in procedures.

For names, the default_name function is used if you don't want to define your own name function.

 function name (
  object_id     in acs_objects.object_id%TYPE
 ) return varchar;

 function default_name (
  object_id     in acs_objects.object_id%TYPE
 ) return varchar;

The following functions tell you where attributes are stored, and fetch single attributes for you.

 procedure get_attribute_storage (
   object_id_in      in  acs_objects.object_id%TYPE,
   attribute_name_in in  acs_attributes.attribute_name%TYPE,
   v_column          out varchar2,
   v_table_name      out varchar2,
   v_key_sql         out varchar2
 );

 function get_attribute (
   object_id_in      in  acs_objects.object_id%TYPE,
   attribute_name_in in  acs_attributes.attribute_name%TYPE
 ) return varchar2;

 procedure set_attribute (
   object_id_in      in  acs_objects.object_id%TYPE,
   attribute_name_in in  acs_attributes.attribute_name%TYPE,
   value_in          in  varchar2
 );

The main use of the acs_object package is to create application objects and make them available for services via the acs_objects table. To do this, you just have to make sure you call acs_object.new() on objects that you wish to appear in the acs_objects table. In addition, all such objects must be instances of some subtype of acs_object.

Continuing the ticket example, we might define the following sort of procedure for creating a new ticket:

 function new_ticket (
  package_id        in tickets.ticket_id%TYPE
            default null,
  description       in tickets.description%TYPE default '',
     ...
  ) return tickets.ticket_id%TYPE
  is
   v_ticket_id tickets
  begin
   v_ticket_id := acs_object.new(
      object_id => ticket_id,
      object_type => 'ticket',
        ...
     );
    insert into tickets
    (ticket_id, description)
    values
    (v_ticket_id, description);
    return v_ticket_id;
  end new_ticket;

This function will typically be defined in the context of a PL/SQL package, but we've left it stand-alone here for simplicity.

To summarize: in order to take advantage of OpenACS 4 services, a new application need only do three things:

  • Define a data model to describe application objects. This can just be a normal SQL table.

  • Create an object type, using code like in the example from the previous section.

  • Make sure application objects are created using acs_object.new() in addition to whatever SQL code is needed to insert a new row into the application data model.

One of the design goals of OpenACS 4 was to provide a straightforward and consistent mechanism to provide applications with general services. What we have seen here is that three simple steps and minimal changes in the application data model are sufficient to make sure that application objects are represented in the acs_objects table. Subsequently, all of the general services in OpenACS 4 (i.e. permissions, general comments, and so on) are written to work with any object that appears in acs_objects. Therefore, in general these three steps are sufficient to make OpenACS 4 services available to your application.

The relations system defines two packages: acs_rel_type for creating and managing relation types, and acs_rel for relating objects.

These two procedures just insert and remove roles from the acs_rel_roles table. This table stores the legal relationship "roles" that can be used when creating relation types. Examples of roles are, say, "member", or "employer".

 procedure create_role (
    role        in acs_rel_roles.role%TYPE
  );

  procedure drop_role (
    role        in acs_rel_roles.role%TYPE
  );

The main functions in the acs_rel_type package are used to create and drop relation types.

  procedure create_type (
    rel_type            in acs_rel_types.rel_type%TYPE,
    pretty_name         in acs_object_types.pretty_name%TYPE,
    pretty_plural       in acs_object_types.pretty_plural%TYPE,
    supertype           in acs_object_types.supertype%TYPE
                           default 'relationship',
    table_name          in acs_object_types.table_name%TYPE,
    id_column           in acs_object_types.id_column%TYPE,
    abstract_p          in acs_object_types.abstract_p%TYPE default 'f',
    type_extension_table in acs_object_types.type_extension_table%TYPE
                            default null,
    name_method         in acs_object_types.name_method%TYPE default null,
    object_type_one     in acs_rel_types.object_type_one%TYPE,
    role_one            in acs_rel_types.role_one%TYPE default null,
    min_n_rels_one      in acs_rel_types.min_n_rels_one%TYPE,
    max_n_rels_one      in acs_rel_types.max_n_rels_one%TYPE,
    object_type_two     in acs_rel_types.object_type_two%TYPE,
    role_two            in acs_rel_types.role_two%TYPE default null,
    min_n_rels_two      in acs_rel_types.min_n_rels_two%TYPE,
    max_n_rels_two      in acs_rel_types.max_n_rels_two%TYPE
  );

  procedure drop_type (
    rel_type            in acs_rel_types.rel_type%TYPE,
    cascade_p           in char default 'f'
  );

Finally, the acs_rel package provides an API that you use to create and destroy instances of a relation type:

  function new (
    rel_id              in acs_rels.rel_id%TYPE default null,
    rel_type            in acs_rels.rel_type%TYPE default 'relationship',
    object_id_one       in acs_rels.object_id_one%TYPE,
    object_id_two       in acs_rels.object_id_two%TYPE,
    context_id          in acs_objects.context_id%TYPE default null,
    creation_user       in acs_objects.creation_user%TYPE default null,
    creation_ip         in acs_objects.creation_ip%TYPE default null
  ) return acs_rels.rel_id%TYPE;

  procedure delete (
    rel_id      in acs_rels.rel_id%TYPE
  );

A good example of how to use relation types appears in the OpenACS 4 data model for groups. As in 3.x, group membership is modeled using a mapping table, but now we create this mapping using relation types instead of explicitly creating a table. First, we create a helper table to store state on each membership fact:

create table membership_rels (
        rel_id          constraint membership_rel_rel_id_fk
                        references acs_rels (rel_id)
                        constraint membership_rel_rel_id_pk
                        primary key,
        -- null means waiting for admin approval
        member_state    varchar(20) constraint membership_rel_mem_ck
                        check (member_state in ('approved', 'banned',
                                                'rejected', 'deleted'))
);

Then, we create a new object type to describe groups.

 acs_object_type.create_type (
   object_type => 'group',
   pretty_name => 'Group',
   pretty_plural => 'Groups',
   table_name => 'groups',
   id_column => 'group_id',
   type_extension_table => 'group_types',
   name_method => 'acs_group.name'
 );

In this example, we've made groups a subtype of acs_object to make the code simpler. The actual data model is somewhat different. Also, we've assumed that there is a helper table called groups to store information on groups, and that there is a helper table called group_types that has been defined to store extra attributes on groups.

Now, assuming we have another object type called person to represent objects that can be group members, we define the following relationship type for group membership:

 acs_rel_type.create_role ('member');

 acs_rel_type.create_type (
   rel_type => 'membership_rel',
   pretty_name => 'Membership Relation',
   pretty_plural => 'Membership Relationships',
   table_name => 'membership_rels',
   id_column => 'rel_id',
   object_type_one => 'group',
   min_n_rels_one => 0, max_n_rels_one => null,
   object_type_two => 'person', role_two => 'member',
   min_n_rels_two => 0, max_n_rels_two => null
 );

Now we can define the following procedure to add a new member to a group. All this function does is create a new instance of the membership relation type and then insert the membership state into the helper table that we define above. In the actual implementation, this function is implemented in the membership_rel package. Here we just define an independent function:

function member_add (
    rel_id              in membership_rels.rel_id%TYPE default null,
    rel_type            in acs_rels.rel_type%TYPE default 'membership_rel',
    group               in acs_rels.object_id_one%TYPE,
    member              in acs_rels.object_id_two%TYPE,
    member_state        in membership_rels.member_state%TYPE default null,
    creation_user       in acs_objects.creation_user%TYPE default null,
    creation_ip         in acs_objects.creation_ip%TYPE default null
  ) return membership_rels.rel_id%TYPE
  is
    v_rel_id integer;
  begin
      v_rel_id := acs_rel.new (
      rel_id => rel_id,
      rel_type => rel_type,
      object_id_one => group,
      object_id_two => person,
      context_id => object_id_one,
      creation_user => creation_user,
      creation_ip => creation_ip
    );

    insert into membership_rels
     (rel_id, member_state)
    value
     (v_rel_id, new.member_state);
  end;

Another simple function can be defined to remove a member from a group:

  procedure member_delete (
    rel_id  in membership_rels.rel_id%TYPE
  )
  is
  begin
    delete from membership_rels
    where rel_id = membership_rel.delete.rel_id;

    acs_rel.del(rel_id);
  end;

The Object Model's API and data model provides a small set of simple procedures that allow applications to create object types, object instances, and object relations. Most of the data model is straightforward; the relation type mechanism is a bit more complex, but in return it provides functionality on par with the old user/groups system in a more general way.

Nothing here yet.

Pete Su generated this document from material culled from other documents by Michael Yoon, Richard Li and Rafael Schloming. But, any remaining lies are his and his alone.

Document Revision # Action Taken, Notes When? By Whom?
0.1 Creation 9/09/2000 Pete Su
0.2 Edited for ACS 4 Beta 9/30/2000 Kai Wu
0.3 Edited for ACS 4.0.1, fixed some mistakes, removed use of term "OM" 11/07/2000 Pete Su

Detailed Design Documentation Template

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By You

NOTE: Some of the sections of this template may not apply to your package, e.g. there may be no user-visible UI elements for a component of the OpenACS Core. Furthermore, it may be easier in some circumstances to join certain sections together, e.g. it may make sense to discuss the data model and transactions API together instead of putting them in separate sections. And on occasion, you may find it easier to structure the design discussion by the structure used in the requirements document. As this template is just a starting point, use your own judgment, consult with peers when possible, and adapt intelligently.

Also, bear in mind the audience for detailed design: fellow programmers who want to maintain/extend the software, AND parties interested in evaluating software quality.

When applicable, each of the following items should receive its own link:

  • User directory

  • OpenACS administrator directory

  • Subsite administrator directory

  • Tcl script directory (link to the API browser page for the package)

  • PL/SQL file (link to the API browser page for the package)

  • Data model

  • Requirements document

  • ER diagram

  • Transaction flow diagram

This section should provide an overview of the package and address at least the following issues:

  • What this package is intended to allow the user (or different classes of users) to accomplish.

  • Within reasonable bounds, what this package is not intended to allow users to accomplish.

  • The application domains where this package is most likely to be of use.

  • A high-level overview of how the package meets its requirements (which should have been documented elsewhere). This is to include relevant material from the "features" section of the cover sheet (the cover sheet is a wrapper doc with links to all other package docs).

Also worthy of treatment in this section:

  • When applicable, a careful demarcation between the functionality of this package and others which - at least superficially - appear to address the same requirements.

Note: it's entirely possible that a discussion of what a package is not intended to do differs from a discussion of future improvements for the package.

For a given set of requirements, typically many possible implementations and solutions exist. Although eventually only one solution is implemented, a discussion of the alternative solutions canvassed - noting why they were rejected - proves helpful to both current and future developers. All readers would be reminded as to why and how the particular solution developed over time, avoiding re-analysis of problems already solved.

Although currently only a few package documentation pages contain a discussion of competing software, (e.g. chat, portals), this section should be present whenever such competition exists.

  • If your package exhibits features missing from competing software, this fact should be underscored.

  • If your package lacks features which are present in competing software, the reasons for this should be discussed here; our sales team needs to be ready for inquiries regarding features our software lacks.

Note that such a discussion may differ from a discussion of a package's potential future improvements.

No single design solution can optimize every desirable software attribute. For example, an increase in the security of a system will likely entail a decrease in its ease-of-use, and an increase in the flexibility/generality of a system typically entails a decrease in the simplicity and efficiency of that system. Thus a developer must decide to put a higher value on some attributes over others: this section should include a discussion of the tradeoffs involved with the design chosen, and the reasons for your choices. Some areas of importance to keep in mind are:

Areas of interest to users:

  • Performance: availability and efficiency

  • Flexibility

  • Interoperability

  • Reliability and robustness

  • Usability

Areas of interest to developers:

  • Maintainability

  • Portability

  • Reusability

  • Testability

Here's where you discuss the abstractions used by your package, such as the procedures encapsulating the legal transactions on the data model. Explain the organization of procedures and their particulars (detail above and beyond what is documented in the code), including:

  • Problem-domain components: key algorithms, e.g. a specialized statistics package would implement specific mathematical procedures.

  • User-interface components: e.g. HTML widgets that the package may need.

  • Data management components: procedures that provide a stable interface to database objects and legal transactions - the latter often correspond to tasks.

Remember that the correctness, completeness, and stability of the API and interface are what experienced members of our audience are looking for. This is a cultural shift for us at aD (as of mid-year 2000), in that we've previously always looked at the data models as key, and seldom spent much effort on the API (e.g. putting raw SQL in pages to handle transactions, instead of encapsulating them via procedures). Experience has taught us that we need to focus on the API for maintainability of our systems in the face of constant change.

The data model discussion should do more than merely display the SQL code, since this information is already be available via a link in the "essentials" section above. Instead, there should be a high-level discussion of how your data model meets your solution requirements: why the database entities were defined as they are, and what transactions you expect to occur. (There may be some overlap with the API section.) Here are some starting points:

  • The data model discussion should address the intended usage of each entity (table, trigger, view, procedure, etc.) when this information is not obvious from an inspection of the data model itself.

  • If a core service or other subsystem is being used (e.g., the new parties and groups, permissions, etc.) this should also be mentioned.

  • Any default permissions should be identified herein.

  • Discuss any data model extensions which tie into other packages.

  • Transactions

    Discuss modifications which the database may undergo from your package. Consider grouping legal transactions according to the invoking user class, i.e. transactions by an OpenACS-admin, by subsite-admin, by a user, by a developer, etc.

In this section, discuss user interface issues and pages to be built; you can organize by the expected classes of users. These may include:

  • Developers

  • OpenACS administrators (previously known as site-wide administrators)

  • Subsite administrators

  • End users

You may want to include page mockups, site-maps, or other visual aids. Ideally this section is informed by some prototyping you've done, to establish the package's usability with the client and other interested parties.

Note: In order that developer documentation be uniform across different system documents, these users should herein be designated as "the developer," "the OpenACS-admin," "the sub-admin," and "the user," respectively.

Finally, note that as our templating system becomes more entrenched within the OpenACS, this section's details are likely to shift from UI specifics to template interface specifics.

Under OpenACS 5.2.3rc1, parameters are set at two levels: at the global level by the OpenACS-admin, and at the subsite level by a sub-admin. In this section, list and discuss both levels of parameters.

If the system presently lacks useful/desirable features, note details here. You could also comment on non-functional improvements to the package, such as usability.

Note that a careful treatment of the earlier "competitive analysis" section can greatly facilitate the documenting of this section.

Although a system's data model file often contains this information, this isn't always the case. Furthermore, data model files often undergo substantial revision, making it difficult to track down the system creator. An additional complication: package documentation may be authored by people not directly involved in coding. Thus to avoid unnecessary confusion, include email links to the following roles as they may apply:

  • System creator

  • System owner

  • Documentation author

The revision history table below is for this template - modify it as needed for your actual design document.

Document Revision # Action Taken, Notes When? By Whom?
0.3 Edited further, incorporated feedback from Michael Yoon 9/05/2000 Kai Wu
0.2 Edited 8/22/2000 Kai Wu
0.1 Creation 8/21/2000 Josh Finkler, Audrey McLoghlin

Next Page