0.00%
Search · Index

Weblog Page

Showing 111 - 120 of 230 Postings (summary)

Deleting a tablespace

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Skip down for instructions on Deleting a PostgreSQL tablespace.

Should it become necessary to rebuild a tablespace from scratch, you can use the drop user command in SVRMGRL with the cascade option. This command will drop the user and every database object the user owns.

SVRMGR> drop user $OPENACS_SERVICE_NAME cascade;

If this does not work because svrmgrl "cannot drop a user that is currently connected", make sure to kill the AOLserver using this user. If it still does not work, do:

SVRMGR> select username, sid, serial# from v$session where lower(username)='$OPENACS_SERVICE_NAME';

and then

SVRMGR> alter system kill session 'sid, serial#';

where sid and serial# are replaced with the corresponding values for the open session.

Use with caution!

If you feel the need to delete everything related to the service, you can also issue the following:

SVRMGR> drop tablespace $OPENACS_SERVICE_NAME including contents cascade constraints;

Dropping a PostgreSQL tablespace is easy. You have to stop any AOLserver instances that are using the database that you wish to drop. If you're using daemontools, this is simple, just use the 'down' flag (-d). If you're using inittab, you have to comment out your server in /etc/inittab, reread the inittab with /sbin/init q, and then restart-aolserver $OPENACS_SERVICE_NAME .

Then, to drop the db, just do:

[$OPENACS_SERVICE_NAME ~]$ dropdb $OPENACS_SERVICE_NAME

DROP DATABASE

OpenACS Core and .LRN

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

  1. Update Translations.the section called “How to Update the translations”

  2. Rebuild the Changelog.Rebuild the Changelog. I use a tool called cvs2cl. Run this command from the package root to automatically generate a Changelog file in the same dir. We generate two changelogs, one for the minor branch and one for the most recent release. The example below is for OpenACS 5.0.2:

    cd /var/lib/aolserver/$OPENACS_SERVICE_NAME
    cvs2cl -F oacs-5-0 --delta openacs-5-0-0-final:oacs-5-0 -f ChangeLog
    cvs2cl -F oacs-5-0 --delta openacs-5-0-1-final:oacs-5-0 -f ChangeLog-recent
  3. Update Version Numbers.The version numbers in the documentation and in the packages must be updated. This should only happen after a release candidate is approved.

    .LRN: this must be repeated for .LRN modules (dotlrn-core in the dotlrn cvs tree) and for any modified modules in the .LRN prerequisites (dotlrn-prereq in openacs cvs tree). My current working model is that I bulk-update .LRN and OpenACS core but that I don't touch dotlrn-prereq modules - I just use the most recent release and it's up to individual package developers to tag and release those packages when they change. This model is already broken because following it means that dotlrn-prereqs don't get new translations.

    1. Update /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/acs-core-docs/www/xml/variables.ent with the new version number.

    2. Add new section in /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/acs-core-docs/www/xml/for-everyone/release-notes.xml

    3. Regenerate all HTML docs

      cd /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/acs-core-docs/www/xml
      make
    4. Update /var/lib/aolserver/$OPENACS_SERVICE_NAME/readme.txt with the new version number

    5. Update version number and release date in all of the core packages. Use /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages/acs-core-docs/www/files/update-info.sh with the new version number and the release date as arguments. Run it from /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages:

      cd /var/lib/aolserver/$OPENACS_SERVICE_NAME/packages
             ./acs-core-docs/www/files/update-info 5.2.12006-01-16
      
    6. Install a new site using the modified code and verify that the automated tests pass.

    7. Commit changes to CVS

  4. Tag the files in CVS.The steps to this point should have ensured that the head of the current branch contains the full set of code to release. Now we need to tag it as the code to be released.

    1. Check out OpenACS Core. The files must be checked out through a cvs account with write access and should be a checkout from the release branch. In this example, we are assuming this is being done as a local user on openacs.org (which make the checkout and tagging operations much faster).

      cd /var/tmp
      cvs -d /cvsroot checkout -r oacs-5-0 acs-core

      If doing .LRN, repeat with the dotlrn cvs tree.

      cd /var/tmp
      mkdir dotlrn-packages
      cd dotlrn-packages
      cvs -d /dotlrn-cvsroot checkout -r dotlrn-2-0 dotlrn-all
      
    2. Tag the tree. If it's a final release of core, move or create the appropriate openacs-major-minor-compat tag. (Ie, if releasing 5.0.3 final, move the openacs-5-0-compat flag.)

      cd /var/tmp/openacs-4
      cvs tag -F openacs-5-0-0a1
      cvs tag -F openacs-5-0-compat
      

      Branching

      When we feature-freeze on HEAD as part of the release process, we are blocking new development. To avoid this, we branch the code at this point, so that new work can continue on HEAD while the branch is stabilized for release. However, branching means that bug fixes have to be synchronized between HEAD and the branch, and bug fixes tend to be more frequent right at this time. Therefore, our actual branch point is as late as possible - essentially, we do not branch until and unless new feature work is actively blocked by the feature freeze. Branching is almost the same as tagging, except for the flag and slightly different tag nomenclature. To see the list of old branches, cvs status -v somefile.

      cvs tag -b oacs-5-0
      

      If doing .LRN: Since the .LRN packages aren't all in one module, we iterate through all of the modules. Log in first (cvs login) so that you don't have to log in for each module.

      cd /var/tmp/dotlrn-packages
      for dir in *; do ( cd $dir && cvs tag dotlrn-2-0-2-final ); done
      for dir in *; do ( cd $dir && cvs tag -F openacs-5-0-compat ); done
      

      Note that for the compat tag we use the -F flag which will force the tag to the new version (just in case someone has created the tag already on another version). Excercise care when doing this since you don't want to inadvertently move a prior release tag. Also if the tagging goes horribly wrong for some reason you can delete the tag via "cvs tag -d <symbolic_tag>".

    3. Apply the final tag across the tree. First, check out the entire OpenACS tree, getting the most recent stable version of each package. This is most simply done on openacs.org:

      cd /var/tmp
      cvs -d /cvsroot checkout -r openacs-5-1-compat openacs-4
      cd openacs-4
      cvs tag openacs-5-1-2-final
      
  5. Make the tarball(s).

    • openacs-core.

      1. Go to a new working space and export the tagged files.

        mkdir /var/tmp/tarball
        cd /var/tmp/tarball
        cvs -d /cvsroot export -r openacs-5-0-0a1 acs-core
      2. Generate the tarball.

        cd /var/tmp/tarball
        mv openacs-4 openacs-5.0.0a1
        tar cz -f openacs-5.0.0a1.tar.gz openacs-5.0.0a1
        
    • dotlrn.

      1. Go to a new working space and export the tagged files. (was getting errors here trying to use -d, so gave up and just moved things from openacs-4 to openacs at the end)

        mkdir /var/tmp/dotlrn-tarball
        cd /var/tmp/dotlrn-tarball
        cvs -d /cvsroot export -r openacs-5-0-0a1 acs-core
        cd /var/tmp/dotlrn-tarball/openacs-4/packages
        cvs -d /cvsroot export -r openacs-5-0-0a1 dotlrn-prereq
        cvs -d /dotlrn-cvsroot export -r dotlrn-2-0-0a1 dotlrn-core
        
      2. Copy the dotlrn install.xml file, which controls which packages are installed on setup, to the root location:

        cp /var/tmp/dotlrn-tarball/openacs-4/packages/dotlrn/install.xml  /var/tmp/dotlrn-tarball/openacs-4
        
      3. Generate the tarball

        cd /var/tmp/dotlrn-tarball
        mv openacs-4 dotlrn-2.0.0a1
        tar cz -f dotlrn-2.0.0a1.tar.gz dotlrn-2.0.0a1
        
  6. Test the new tarball(s).Download the tarballs just created and install them and make sure everything looks okay and that automated tests pass.

  7. Update Web site.Update the different places on OpenACS.org where we track status.

    • Release Status for the current version - something like http://openacs.org/projects/openacs/5.0/milestones

    • Home page of openacs.org

    • Post a new news item

  8. Clean Up.Clean up after yourself.

    cd /var/tmp
    rm -rf tarball dotlrn-tarball dotlrn-packages openacs-5.0.0a1
    rm -rf /var/tmp/openacs-4

Here is a shell script that automates packaging the tarball (it's a bit out of date with the new steps - I've been doing everything manually or with little throwaway scripts as detailed above until the process is stabilized).

#!/bin/bash

# if TAG=1 create the cvs tags otherwise assume they exist.
TAG=1

# What release version are we building; version format should be
# dashes rather than dots eg. OACS_VERSION=5-0-0b4

OACS_VERSION=5-0-0b4
DOTLRN_VERSION=2-0-0b4

OACS_BRANCH=oacs-5-0
DOTLRN_BRANCH=dotlrn-2-0

DOTLRN_CVSROOT=/dotlrn-cvsroot
OACS_CVSROOT=/cvsroot

#
# Nothing below here should need to change...
#
BASE=/var/tmp/release-$OACS_VERSION
mkdir $BASE
if [ ! -d $BASE ]; then
    echo "Failed creating base dir $BASE"
    exit 1
fi

cd $BASE

if [ $TAG -eq 1 ]; then

    # Checkout and tag the release
    cvs -d $OACS_CVSROOT checkout -r $OACS_BRANCH openacs-4
    cd openacs-4
    cvs tag -F openacs-$OACS_VERSION
    cd ../


    # Checkout and tag the dotlrn release
    mkdir dotlrn-packages
    cd dotlrn-packages
    cvs -d $DOTLRN_CVSROOT checkout -r $DOTLRN_BRANCH dotlrn-all
    for dir in *; do ( cd $dir && cvs tag -F dotlrn-$DOTLRN_VERSION ); done
    cd ../

    #
    # Should check for .sql .xql .adp .tcl .html .xml executable files and squak if found.
    #

fi



# Generate tarballs...
#

# openacs
#
mkdir tarball
cd tarball
cvs -d $OACS_CVSROOT export -r openacs-$OACS_VERSION acs-core
mv opeancs-4 openacs-${OACS_VERSION//-/.}
tar -czf ../openacs-${OACS_VERSION//-/.}.tar.gz openacs-${OACS_VERSION//-/.}
cd ..

# dotlrn
#
mkdir dotlrn-tarball
cd dotlrn-tarball
cvs -d $OACS_CVSROOT export -r openacs-$OACS_VERSION acs-core
cd  openacs-4/packages
cvs -d $OACS_CVSROOT export -r openacs-$OACS_VERSION dotlrn-prereq
cvs -d $DOTLRN_CVSROOT export -r dotlrn-$DOTLRN_VERSION dotlrn-core
cd ../..
cp -f openacs-4/packages/dotlrn/install.xml openacs-4
mv openacs-4 dotlrn-${DOTLRN_VERSION//-/.}
tar -czf ../dotlrn-${DOTLRN_VERSION//-/.}.tar.gz dotlrn-${DOTLRN_VERSION//-/.}


# Clean up after ourselves...
cd $BASE && rm -rf dotlrn-tarball tarball openacs-4 dotlrn-packages

Install PHP for use in AOLserver

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By Malte Sussdorff

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

To be able to use PHP software with AOLserver (and OpenACS), you have to install PHP with AOLserver support. Get the latest version from www.php.net. For convenience we get version 4.3.4 from a mirror

[root root]# cd /usr/local/src
[root src]# wget http://de3.php.net/distributions/php-4.3.4.tar.gz
[root src]# tar xfz php-4.3.4.tar.gz
[root src]# cd php-4.3.4
[root php-4.3.4]# cd php-4.3.4
[root php-4.3.4]#  ./configure --with-aolserver=/usr/local/aolserver/ --with-pgsql=/usr/local/pgsql --without-mysql
[root php-4.3.4]# make install

Once installed you can enable this by configuring your config file. Make sure your config file supports php (it should have a php section with it). Furthermore add index.php as the last element to your directoryfile directive.

Automated Testing

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

By Jeff Davis

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

Best practices in writing OpenACS automated tests

  • Special characters in Tcl. Try strings starting with a -Bad and strings containing [BAD], {, \077, and $Bad. For user input, [BAD] should never be evaluated, \077 should not be turned into a ? and $Bad should not be interpolated. The string -Bad [BAD] \077 { $Bad should be valid user input, should pass through the system unaltered, and if it isn't that's a bug.

  • Quoting issues.Put some html in plain text fields and make sure the result is properly quoted anywhere it shows up (I use "<b>bold</b>" usually). Look out especially for quoting errors in the context bar and in round trips via an edit form. For fields that disallow html tags you can use &amp; to check that the field is quoted properly. If it is not displayed as &amp; then the quoting for the field is incorrect. (It's not clear whether this should be considered an error but given that data for text fields can come from various sources if it's text it should be properly quoted and we should not rely on input validation to prevent XSS security holes.)

  • Whitespace input.Check that whitespace is not considered valid input for a field if it does not make sense. For example, the subject of a forum post is used to construct a link and if it is " " it will have a link of <a href="..."> </a> which would not be clickable if whitespace was allowed as a valid input.

  • Doubleclick. Make sure that if you submit a form, use the back button, and submit again that the behavior is reasonable (correct behavior depends on what the form is for, but a server error is not reasonable).

  • Duplicate names. Make sure that if a duplicate name is entered that there is a reasonable error rather than a server error. Check for insert, move, copy, and rename.

Add PSGML commands to emacs init file (OPTIONAL)

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

If you plan to write or edit any documentation with emacs, install a customized emacs configuration file with DocBook commands in the skeleton directory, so it will be used for all new users. The file also fixes the backspace -> help mis-mapping that often occurs in terminals.

[root tmp]# cp /tmp/openacs-5.2.3rc1/packages/acs-core-docs/www/files/emacs.txt /etc/skel/.emacs
cp: overwrite `/etc/skel/.emacs'? y
[root tmp]# 

Debian users:

apt-get install psgml

Note: The new nxml mode for emacs, when used in combination with psgml, provides a pretty good set of functionality that makes DocBook editing much less painless. In particular, nxml does syntax testing in real-time so that you can see syntax errors immediately instead of in the output of the xsltproc hours or days later. For debian, apt-get install nxml.

OpenACS Permissions Tediously Explained

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

by Vadim Nasardinov. Modified and converted to Docbook XML by Roberto Mello

The code has been modified since this document was written so it is now out of date. See this forum thread.

Who (grantee_id) can do what (privilege) on which object (object_id).

The general permissions system has a flexible (and relatively complex) data model in OpenACS. Developers who have not had the time to learn the internals of the data model may end up writing seemingly correct code that crashes their system in weird ways. This writeup is the result of my running into such a piece of code and trying to understand exactly what went wrong. It is geared towards developers who understand the general permissions system to the extent that is described in the Groups, Context, Permissions documentation, but who have not had the opportunity to take a long, careful look at the system internals.

In OpenACS, most of the interesting tables are expected to extend (subtype) the acs_objects table, i.e. they are expected to have an integer primary key column that references the object_id column of acs_objects.

create table acs_objects (
      object_id             integer
          not null
          constraint acs_objects_pk primary key,
      object_type
          not null
          constraint acs_objects_object_type_fk references acs_object_types (object_type),
      context_id
          constraint acs_objects_context_id_fk references acs_objects(object_id),
      security_inherit_p	  char(1) default 't'
          not null,
      constraint acs_objects_sec_inherit_p_ck
          check (security_inherit_p in ('t', 'f')),
      creation_user         integer,
      creation_date         date default sysdate not null,
      creation_ip           varchar2(50),
      last_modified         date default sysdate not null,
      modifying_user        integer,
      modifying_ip          varchar2(50),
      constraint acs_objects_context_object_un
          unique (context_id, object_id) disable
);
    

This means that items that want to use the features of the OpenACS object system needs to have an entry in the acs_objects. This allows developers to define relationships between any two entities A and B by defining a relationship between their corresponding entries in the acs_objects table. One of the applications of this powerful capability is the general permissions system.

At the heart of the permission system are two tables: acs_privileges and acs_permissions.

  create table acs_privileges (
      privilege           varchar2(100) not null
          constraint acs_privileges_pk primary key,
      pretty_name         varchar2(100),
      pretty_plural       varchar2(100)
  );
    
  create table acs_permissions (
      object_id
          not null
          constraint acs_permissions_on_what_id_fk references acs_objects (object_id),
      grantee_id
          not null
          constraint acs_permissions_grantee_id_fk references parties (party_id),
      privilege
          not null
          constraint acs_permissions_priv_fk references acs_privileges (privilege),
      constraint acs_permissions_pk
          primary key (object_id, grantee_id, privilege)
  );
    

The acs_privileges table stores named privileges like read, write, delete, create, and admin. The acs_permissions table stores assertions of the form:

Who (grantee_id) can do what (privilege) on which object (object_id).

The micromanaging approach to system security would be to require application developers to store permission information explicitly about every object, i.e. if the system has 100,000 and 1,000 users who have the read privilege on all objects, then we would need to store 100,000,000 entries of the form:

object_id grantee_id privilege
object_id_1 user_id_1 'read'
object_id_1 user_id_2 'read'
...
object_id_1 user_id_n 'read'
object_id_2 user_id_1 'read'
object_id_2 user_id_2 'read'
...
object_id_2 user_id_n 'read'
...
...
object_id_m user_id_1 'read'
object_id_m user_id_2 'read'
...
object_id_m user_id_n 'read'

Although quite feasible, this approach fails to take advantage of the fact that objects in the system are commonly organized hierarchally, and permissions usually follow the hierarchical structure, so that if user X has the read privilege on object A, she typically also has the read privilege on all objects attached under A.

The general permission system takes advantage of the hierarchical organization of objects to unburden developers of the necessity to explicitly maintain security information for every single object. There are three kinds of hierarchies involved. These are discussed in the following sections.

Suppose objects A, B, ..., and F form the following hierarchy.

Table11.2.Context Hierarchy Example

A

object_id=10

B

object_id=20

C

object_id=30

D

object_id=40

E

object_id=50

F

object_id=60


This can be represented in the acs_objects table by the following entries:

Table11.3.acs_objects example data

object_id context_id
20 10
30 10
40 20
50 20
60 30

The first entry tells us that object 20 is the descendant of object 10, and the third entry shows that object 40 is the descendant of object 20. By running a CONNECT BY query, we can compute that object 40 is the second-generation descendant of object 10. With this in mind, if we want to record the fact that user Joe has the read privilege on objects A, ..., F, we only need to record one entry in the acs_permissions table.

object grantee privilege
A Joe read

The fact that Joe can also read B, C, ..., and F can be derived by ascertaining that these objects are children of A by traversing the context hierarchy. As it turns out, hierarchical queries are expensive. As Rafael Schloming put it so aptly, Oracle can't deal with hierarchies for shit.

One way to solve this problem is to cache a flattened view of the context tree like so:

object ancestor n_generations
A A 0
B B 0
B A 1
C C 0
C A 1
D D 0
D B 1
D A 2
E E 0
E B 1
E A 2
F F 0
F C 1
F A 2

Note that the number of entries in the flattened view grows exponentially with respect to the depth of the context tree. For instance, if you have a fully populated binary tree with a depth of n, then the number of entries in its flattened view is

1 + 2*2 + 3*4 + 4*8 + 5*16 + ... + (n+1)*2n = n*2n+1 + 1

Despite its potentially great storage costs, maintaining a flattened representation of the context tree is exactly what OpenACS does. The flattened context tree is stored in the acs_object_context_index table.

  create table acs_object_context_index (
      object_id
          not null
          constraint acs_obj_context_idx_obj_id_fk references acs_objects (object_id),
      ancestor_id
          not null
          constraint acs_obj_context_idx_anc_id_fk references acs_objects (object_id),
      n_generations	    integer
          not null
          constraint acs_obj_context_idx_n_gen_ck check (n_generations >= 0),
      constraint acs_object_context_index_pk
          primary key (object_id, ancestor_id)
  ) organization index;
    

A few things to note about this table are these. Number one, it is an index-organized table, which means it is substantially optimized for access by primary key. Number two, as the above computations suggest, the size of the table grows polynomially with respect to the average number of descendants that an object has, and exponentially with respect to the depth of the context tree.

The acs_object_context_index is kept in sync with the acs_objects table by triggers like this:

create or replace trigger acs_objects_context_id_in_tr
after insert on acs_objects
for each row
begin
    insert into acs_object_context_index
     (object_id, ancestor_id, n_generations)
    values
     (:new.object_id, :new.object_id, 0);

    if :new.context_id is not null and :new.security_inherit_p = 't' then
      insert into acs_object_context_index
       (object_id, ancestor_id,
        n_generations)
      select
       :new.object_id as object_id, ancestor_id,
       n_generations + 1 as n_generations
      from acs_object_context_index
      where object_id = :new.context_id;
    elsif :new.object_id != 0 then
      -- 0 is the id of the security context root object
      insert into acs_object_context_index
       (object_id, ancestor_id, n_generations)
      values
       (:new.object_id, 0, 1);
    end if;
end;

One final note about acs_objects. By setting an object's security_inherit_p column to 'f', you can stop permissions from cascading down the context tree. In the following example, Joe does not have the read permissions on C and F.


A
object_id=10
readablebyJoe


B
object_id=20
readablebyJoe


C
object_id=30
security_inherit_p='f'
notreadablebyJoe


D
object_id=40


E
object_id=50


F
object_id=60
security_inherit_p='f'
notreadablebyJoe

Privileges are also organized hierarchically. In addition to the five main system privileges defined in the ACS Kernel data model, application developers may define their own. Note, however, that this is no longer recommended practice.

By defining parent-child relationship between privileges, the OpenACS data model makes it easier for developers to manage permissions. Instead of granting a user explicit read, write, delete, and create privileges on an object, it is sufficient to grant the user the admin privilege to which the first four privileges are tied. Privileges are structured as follows.

admin
create delete read write

Note that admin privileges are greater than read, write, create and delete privileges combined. Issuing someone read, write, create and delete privileges will not result in the person getting admin privileges.

The parent-child relationship between privileges is represented in the acs_privilege_hierarchy table:

  create table acs_privilege_hierarchy (
      privilege
          not null
          constraint acs_priv_hier_priv_fk references acs_privileges (privilege),
      child_privilege
          not null
          constraint acs_priv_hier_child_priv_fk references acs_privileges (privilege),
      constraint acs_privilege_hierarchy_pk
          primary key (privilege, child_privilege)
  );
    

As in the case of the context hierarchy, it is convenient to have a flattened representation of this hierarchal structure. This is accomplished by defining the following view.

  create or replace view acs_privilege_descendant_map
  as
  select
    p1.privilege,
    p2.privilege as descendant
  from
    acs_privileges p1,
    acs_privileges p2
  where
    p2.privilege in
      (select
         child_privilege
       from
         acs_privilege_hierarchy
       start with
         privilege = p1.privilege
       connect by
         prior child_privilege = privilege
      )
    or p2.privilege = p1.privilege;
    

As the number of different privileges in the system is expected to be reasonably small, there is no pressing need to cache the flattened ansector-descendant view of the privilege hierarchy in a specially maintained table like it is done in the case of the context hierarchy.

Now for the third hierarchy playing a promiment role in the permission system. The party data model is set up as follows.

  create table parties (
      party_id
          not null
          constraint parties_party_id_fk references acs_objects (object_id)
          constraint parties_party_id_pk primary key,
      email               varchar2(100)
          constraint parties_email_un unique,
      url                 varchar2(200)
  );
    
  create table persons (
      person_id
          not null
          constraint persons_person_id_fk references parties (party_id)
          constraint persons_person_id_pk primary key,
      first_names          varchar2(100)
          not null,
      last_name            varchar2(100)
          not null
  );
    
  create table users (
      user_id
          not null
          constraint users_user_id_fk references persons (person_id)
          constraint users_user_id_pk primary key,
      password        char(40),
      -- other attributes
  );
    
  create table groups (
      group_id
          not null
          constraint groups_group_id_fk references parties (party_id)
          constraint groups_group_id_pk primary key,
      group_name           varchar2(100) not null
  );
    

Recall that the grantee_id column of the acs_permissions table references parties.party_id. This means that you can grant a privilege on an object to a party, person, user, or group. Groups represent aggregations of parties. The most common scenario that you are likely to encounter is a group that is a collection of users, although you could also have collections of persons, groups, parties, or any mix thereof.

Given that the most common use of groups is to partition users, how do you build groups? One way is to grant membership explicitly. If you have a group named Pranksters, you can assign membership to Pete, Poly, and Penelope. The fact that these users are members of the Pranksters group will be recorded in the membership_rels and acs_rels tables:

  create table acs_rels (
      rel_id
          not null
          constraint acs_rels_rel_id_fk references acs_objects (object_id)
          constraint acs_rels_pk primary key,
      rel_type
          not null
          constraint acs_rels_rel_type_fk references acs_rel_types (rel_type),
      object_id_one
          not null
          constraint acs_object_rels_one_fk references acs_objects (object_id),
      object_id_two
          not null
          constraint acs_object_rels_two_fk references acs_objects (object_id),
      constraint acs_object_rels_un
          unique (rel_type, object_id_one, object_id_two)
  );
    
  create table membership_rels (
      rel_id
          constraint membership_rel_rel_id_fk references acs_rels (rel_id)
          constraint membership_rel_rel_id_pk primary key,
      -- null means waiting for admin approval
      member_state         varchar2(20)
          constraint membership_rel_mem_ck
           check (member_state in ('approved', 'banned', 'rejected', 'deleted'))
  );
    

The acs_rels table entries would look like so:

rel_type object_one object_two
membership_rel Pranksters Pete
membership_rel Pranksters Poly
membership_rel Pranksters Penelope

Read acs_rels: right-side is a subset of left-side, ie object2 is a part of object1.

Another way of building up groups is by adding subgroups. Suppose we define Merry Pranksters and Sad Pranksters as subgroups of Pranksters. We say that the Pranksters group is composed of groups Merry Pranksters and Sad Pranksters. This information is stored in the acs_rels and composition_rels tables.

create table composition_rels (
    rel_id
        constraint composition_rel_rel_id_fk references acs_rels (rel_id)
        constraint composition_rel_rel_id_pk primary key
);
    

The relevant entries in the acs_rels look like so.

rel_type object_one object_two
composition_rel Pranksters Merry Pranksters
composition_rel Pranksters Sad Pranksters

The composition relationship means that if I add Matt, Mel, and Mary to the Merry Pranksters, they should also automatically become members of the Pranksters group. The situation we are facing in trying to determine whether or not a user is member of a group is similar to the one discussed above in the case of the context hierarchy. Groups can form hierarchies with respect to the composition relationship. The compositon relationship is transitive. If G1 is a subgroup of G2, and G2 is a subgroup of G3, then G1 is a subgroup of G3; that is, any member of G1 is also a member of G3.

Traversing the group composition hierarchy requires running hierarchical queries, which are expensive in Oracle. As we saw in the Context Hierarchy section, one way of reducing the performance hit incurred by hierarchical queries is to cache query results in a table maintained by triggers. The OpenACS data model defines two such tables:

 create table group_component_index (
          group_id        not null
                          constraint group_comp_index_group_id_fk
                          references groups (group_id),
          component_id    not null
                          constraint group_comp_index_comp_id_fk
                          references groups (group_id),
          rel_id          not null
                          constraint group_comp_index_rel_id_fk
                          references composition_rels (rel_id),
          container_id    not null
                          constraint group_comp_index_cont_id_ck
                          references groups (group_id),
          constraint group_component_index_ck
          check (group_id != component_id),
          constraint group_component_index_pk
          primary key (group_id, component_id, rel_id)
  ) organization index;
    
  create table group_member_index (
      group_id
          not null
          constraint group_member_index_grp_id_fk references groups (group_id),
      member_id
          not null
          constraint group_member_index_mem_id_fk references parties (party_id),
      rel_id
          not null
          constraint group_member_index_rel_id_fk references membership_rels (rel_id),
      container_id
          not null
          constraint group_member_index_cont_id_fk references groups (group_id),
      constraint group_member_index_pk
          primary key (member_id, group_id, rel_id)
  ) organization index;
    

The group_component_index table stores a flattened representation of the group composition hierarchy that is maintained in sync with the acs_rels and composition_rels tables through triggers.

additional comments

As far as the group_member_index table goes, I am not sure I understand its purpose. It maintains group-member relationships that are resolved with respect to group composition. Note that information stored in group_member_index can be trivially derived by joining membership_rels, acs_rels, and group_component_index. Here is a view that does it. (This view is not part of the OpenACS Kernel data model.)

create or replace view group_member_view
as
select
  gci.group_id, r.object_id_two as member_id
from
  (
   select
     group_id, group_id as component_id
   from
     groups
   union
   select
     group_id, component_id
   from
     group_component_index
  ) gci,
  membership_rels mr,
  acs_rels r
where
  mr.rel_id = r.rel_id
  and r.object_id_one = gci.component_id;
    

A heuristic way to verify that group_member_view is essentially identical to group_member_index is to compute the symmetric difference between the two:

select
  group_id, member_id
from
  (
   select group_id, member_id from group_member_view
   minus
   select group_id, member_id from group_member_index
  )
union
select
  group_id, member_id
from
  (
   select group_id, member_id from group_member_index
   minus
   select group_id, member_id from group_member_view
  )
    

The query returns no rows. The important point is, if we have a flattened view of the composition hierarchy -- like one provided by the group_component_index table -- membership relationship resolution can be computed trivially with no hierarchical queries involved. There is no need to keep the view in a denormalized table, unless doing so results in substantial performance gains.

Security information is queried by calling the acs_permission.permission_p function in OpenACS. This is accessible from Tcl via the permission::permission_p procedure.

  create or replace package body acs_permission
  as
    -- some stuff removed for the sake of brevity

    function permission_p (
      object_id	 acs_objects.object_id%TYPE,
      party_id	 parties.party_id%TYPE,
      privilege	 acs_privileges.privilege%TYPE
    ) return char
    as
      exists_p char(1);
    begin
      -- XXX This must be fixed: -1 shouldn't be hardcoded (it is the public)
      select decode(count(*),0,'f','t') into exists_p
        from acs_object_party_privilege_map
       where object_id = permission_p.object_id
         and party_id in (permission_p.party_id, -1)
         and privilege = permission_p.privilege;
      return exists_p;
    end;

  end acs_permission;
    

problem avoidance

The function queries acs_object_party_privilege_map, which is a humongous view that joins three flattened hierarchies: the context tree, the privilege hierarchy, the party composition (and membership) hierarchy. It contains an extremely large number of rows. About the only kind of query you can run against it is the one performed by the acs_permission.permission_p function. Anything other than that would take forever to finish or would ultimately result in a query error.

For example, do not try to do things like

select count(*)
  from acs_object_party_privilege_map;
    

To give another example of things to avoid, I have seen code like this:

  declare
      cursor cur is
        select
           object_id, party_id
        from
           acs_object_party_privilege_map
        where
           privilege = 'foo_create';
  begin
      -- revoke all 'foo_create' permissions
      for rec in cur
      loop
          acs_permission.revoke_permission (
              object_id  => rec.object_id,
              grantee_id => rec.party_id,
              privilege  => 'foo_create'
          );
      end loop;

      acs_privilege.remove_child('admin','foo_create');
      acs_privilege.drop_privilege('foo');

  end;
  /
    

The acs_permission.revoke_permission function merely runs a delete statement like so:

  delete from
     acs_permissions
  where
     object_id = revoke_permission.object_id
     and grantee_id = revoke_permission.grantee_id
     and privilege = revoke_permission.privilege;
    

Note that in the above example, acs_permissions had only one entry that needed to be deleted:

object_id grantee_id privilege
default_context registered_users foo_create

The above script would never get around to deleting this entry because it had to loop through a gazillion rows in the humongous acs_object_party_privilege_map view.

create or replace view acs_object_party_privilege_map
as
select
  ogpm.object_id,
  gmm.member_id as party_id,
  ogpm.privilege
from
  acs_object_grantee_priv_map ogpm,
  group_member_map gmm
where
  ogpm.grantee_id = gmm.group_id
union
select
  object_id,
  grantee_id as party_id,
  privilege
from
  acs_object_grantee_priv_map;
    
create or replace view acs_object_grantee_priv_map
as
select
  a.object_id,
  a.grantee_id,
  m.descendant as privilege
from
  acs_permission_all a,
  acs_privilege_descendant_map m
where
  a.privilege = m.privilege;
    
create or replace view acs_permissions_all
as
select
  op.object_id,
  p.grantee_id,
  p.privilege
from
  acs_object_paths op,
  acs_permissions p
where
  op.ancestor_id = p.object_id;
    
create or replace view acs_object_paths
as
select
  object_id,
  ancestor_id,
  n_generations
from
  acs_object_context_index;
    

create or replace view group_member_map
as
select
  group_id,
  member_id,
  rel_id,
  container_id
from
  group_member_index;
    

Enabling WYSIWYG

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

by Nima Mazloumi

OpenACS docs are written by the named authors, and may be edited by OpenACS documentation staff.

Most of the forms in OpenACS are created using the form builder, see the section called “Using Form Builder: building html forms dynamically”. For detailed information on the API take a look here.

The following section shows how you can modify your form to allow WYSIWYG functionalities.

Convert your page to use ad_form (some changes but worth it)

Here an examples. From:

	template::form create my_form
	template::element create my_form my_form_id -label "The ID" -datatype integer -widget hidden
	template::element create my_form my_input_field_1 -html { size 30 } -label "Label 1" -datatype text -optional
	template::element create my_form my_input_field_2 -label "Label 2" -datatype text -help_text "Some Help" -after_html {<a name="#">Anchor</a>}
	

To:

	ad_form -name my_form -form {
		my_form_id:key(acs_object_id_seq)
 		{my_input_field_1:text,optional
               {label "Label 1"}
               {html {size 30}}}
      	{my_input_field_2:text
               {label "Label 2"}
               {help_text "Some Help"}
	       	   {after_html
               {<a name="#">Anchor</a>}}}
	} ...
	

Warning

You must not give your your form the same name that your page has. Otherwise HTMLArea won't load.

Convert your textarea widget to a richtext widget and enable htmlarea.

The htmlarea_p-flag can be used to prevent WYSIWYG functionality. Defaults to true if left away.

From:

	{my_input_field_2:text
	

To:

	{my_input_field_2:richtext(richtext)
			{htmlarea_p "t"}
	

The richtext widget presents a list with two elements: text and content type. To learn more on existing content types search in Google for "MIME-TYPES" or take a look at the cr_mime_types table.

Make sure that both values are passed as a list to your ad_form or you will have problems displaying the content or handling the data manipulation correctly.

Depending on the data model of your package you either support a content format or don't. If you don't you can assume "text/html" or "text/richtext" or "text/enhanced".

The relevant parts in your ad_form definition are the switches -new_data, -edit_data, -on_request and -on_submit.

To allow your data to display correctly you need to add an -on_request block. If you have the format stored in the database pass this as well else use "text/html":

	set my_input_field_2 [template::util::richtext::create $my_input_field_2 "text/html"]
	

Now make sure that your SQL queries that do the data manipulation retrieve the correct value. If you simply use my_input_field_2 you will store a list. Thus you need to add an -on_submit block:

	set my_input_field_2 [ template::util::richtext::get_property contents $my_input_field_2]
	set format [ template::util::richtext::get_property format $my_input_field_2] #This is optional
	

Now the correct values for my_input_field_2 and format are passed to the -new_data and -edit_data blocks which don't need to get touched.

To make HTMLArea optional per package instance define a string parameter UseWysiwygP which defaults 0 for your package using the APM.

In your edit page make the following changes

	# Is WYSIWYG enabled?
	set use_wysiwyg_p [parameter::get -parameter "UseWysiwygP" -default "f"]
	
	...
	
	{htmlarea_p $use_wysiwyg_p}
	

The -on_request switch should set this value for your form.

	set htmlarea_p $use_wysiwyg_p
	

All you need now is a configuration page where the user can change this setting. Create a configure.tcl file:

	ad_page_contract {

    	This page allows a faq admin to change the UseWysiwygP setting

	} {
    	{return_url ""}
	}

	set title "Should we support WYSIWYG?"
	set context [list $title]

	set use_wysiwyg_p

	ad_form -name categories_mode -form {
    	{enabled_p:text(radio)
        	{label "Enable WYSIWYG"}
        	{options {{Yes t} {No f}}}
        	{value $use_wysiwyg_p}
    	}
    	{return_url:text(hidden) {value $return_url}}
    	{submit:text(submit) {label "Change"}}
	} -on_submit {
    	parameter::set_value  -parameter "UseWysiwygP" -value $enabled_p
    	if {![empty_string_p $return_url]} {
        	ns_returnredirect $return_url
    	}
	}
	

In the corresponding ADP file write

	<master>
	<property name="title">@title@</property>
	<property name="context">@context@</property>

	<formtemplate id="categories_mode"></formtemplate>
	

And finally reference this page from your admin page

	#TCL:
	set return_url [ad_conn url]

	#ADP:
	<a href=configure?<%=[export_url_vars return_url]%>>Configure</a>
	

Initialize CVS (OPTIONAL)

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

CVS is a source control system. Create and initialize a directory for a local cvs repository.

[root tmp]# mkdir /cvsroot
[root tmp]# cvs -d /cvsroot init
[root tmp]#
mkdir /cvsroot
cvs -d /cvsroot init

Running multiple services on one machine

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

Services on different ports.To run a different service on another port but the same ip, simply repeat Install OpenACS 5.2.3rc1 replacing $OPENACS_SERVICE_NAME, and change the

set httpport              8000
set httpsport             8443 

to different values.

Services on different host names.For example, suppose you want to support http://service0.com and http://bar.com on the same machine. The easiest way is to assign each one a different ip address. Then you can install two services as above, but with different values for

set hostname               [ns_info hostname]
set address                127.0.0.1 

If you want to install two services with different host names sharing the same ip, you'll need nsvhr to redirect requests based on the contents of the tcp headers. See AOLserver Virtual Hosting with TCP by markd.

Upgrading the OpenACS files

Created by Gustaf Neumann, last modified by Gustaf Neumann 17 Feb 2008, at 07:08 AM

OpenACS is distributed in many different ways:

  • as a collection of files

  • as one big tarball

  • via CVS

  • via automatic download from within the APM (package manager)

Upgrades work by first changing the file system (via any of the previous methods), and then using the APM to scan the file system, find upgrade scripts, and execute them. Starting with OpenACS 5.0, the last method was added, which automatically changes the file system for you. If you are using the last method, you can skip this page. This page describes whether or not you need to be upgrading using this page or not: the section called “Upgrading an OpenACS 5.0.0 or greater installation”

  • Upgrading files for a site which is not in a CVS repository.Unpack the tarball into a new directory and copy its contents on top of your working directory. Or just 'install software', select remote repository, and upgrade your files from there.

    [root root]# su - $OPENACS_SERVICE_NAME
    
    [$OPENACS_SERVICE_NAME aolserver]$ cd /var/lib/aolserver
    [$OPENACS_SERVICE_NAME web]$ tar xzf /var/tmp/openacs-5-1.tar.gz
    [$OPENACS_SERVICE_NAME web]$ cp -r openacs-5-1/* openacs-4
    [$OPENACS_SERVICE_NAME openacs-upgrade]$ exit
    [root root]#
    su - $OPENACS_SERVICE_NAME
    cd /var/lib/aolserver
    tar xzf /var/tmp/openacs-5-1.tgz
    cp -r openacs-5-1/* openacs-4
    exit
    
  • Upgrading files for a site in a private CVS repository

    Many OpenACS site developers operate their own CVS repository to keep track of local customizations. In this section, we describe how to upgrade your local CVS repository with the latest OpenACS version, without overriding your own local customizations.

    This diagram explains the basic idea. However, the labels are incorrect. Step 1(a) has been removed, and Step 1(b) should be labelled Step 1.

    Figure5.2.Upgrading a local CVS repository

    Upgrading a local CVS repository

    • Step 0: Set up a working CVS checkout.To get your OpenACS code into your local CVS repository, you will set up a working CVS checkout of OpenACS. When you want to update your site, you'll update the working CVS checkout, import those changes into your local CVS checkout, create a temporary CVS checkout to merge your local changes, fix any conflicts, commit your changes, and then update your site. It sounds complicated, but it's not too bad, and it is the best way to work around CVS's limitations.

      This part describes how to set up your working CVS checkout. Once it is set up, you'll be able to update any packages using the existing working CVS checkout. We use one dedicated directory for each branch of OpenACS - if you are using OpenACS 5.1,x, you will need a 5.1 checkout. That will be good for 5.1, 5.11, 5.12, and so on. But when you want to upgrade to OpenACS 5.2, you'll need to check out another branch.

      The openacs-5-1-compat tag identifies the latest released version of OpenACS 5.1 (ie, 5.1.3 or 5.1.4) and the latest compatible version of each package. Each minor release of OpenACS since 5.0 has this tagging structure. For example, OpenACS 5.1.x has openacs-5-1-compat.

      You will want to separately check out all the packages you are using.

      [root root]# su - $OPENACS_SERVICE_NAME
      
      [$OPENACS_SERVICE_NAME aolserver]$ cd /var/lib/aolserver
      [$OPENACS_SERVICE_NAME aolserver]$ cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r openacs-5-1-compat acs-core
      [$OPENACS_SERVICE_NAME aolserver]$ cd openacs-4/packages
      [$OPENACS_SERVICE_NAME aolserver]$ cvs -d :pserver:anonymous@cvs.openacs.org:/cvsroot checkout -r openacs-5-1-compatpackagename packagename2...
      
      [$OPENACS_SERVICE_NAME aolserver]$ cd ../..
      [$OPENACS_SERVICE_NAME aolserver]$ mv openacs-4 openacs-5-1
      
      

      Make sure your working CVS checkout doesn't have the entire CVS tree from OpenACS. A good way to check this is if it has a contrib directory. If it does, you probably checked out the entire tree. You might want to start over, remove your working CVS checkout, and try again.

    • Step 1: Import new OpenACS code.

      • Update CVS.Update your local CVS working checkout (unless you just set it up).

        [root root]# su - $OPENACS_SERVICE_NAME
        
        [$OPENACS_SERVICE_NAME aolserver]$ cd /var/lib/aolserver/openacs-5-1
        
        [$OPENACS_SERVICE_NAME aolserver]$ cvs up -Pd ChangeLog *.txt bin etc tcl www packages/*
        
      • Update a single package via cvs working checkout.You can add or upgrade a single package at a time, if you already have a cvs working directory.

        [root root]# su - $OPENACS_SERVICE_NAME
        
        [$OPENACS_SERVICE_NAME aolserver]$ cd /var/lib/aolserver/packages/openacs-5-1
        
        [$OPENACS_SERVICE_NAME openacs-5-1]$ cvs up -Pd packagename
        
        

        In the next section, the import must be tailored to just this package.

    • Step 2: Merge New OpenACS code.Now that you have a local copy of the new OpenACS code, you need to import it into your local CVS repository and resolve any conflicts that occur.

      Import the new files into your cvs repository; where they match existing files, they will become the new version of the file.

      [$OPENACS_SERVICE_NAME openacs-5-1]$  cd /var/lib/aolserver/openacs-5-1
      
      [$OPENACS_SERVICE_NAME openacs-5-1]$  cvs -d /var/lib/cvs import -m "upgrade to OpenACS 5.1" $OPENACS_SERVICE_NAME OpenACS openacs-5-1
      
      

      Tip

      If adding or upgrading a single package, run the cvs import from within the base directory of that package, and adjust the cvs command accordingly. In this example, we are adding the myfirstpackage package.

      [root root]# su - $OPENACS_SERVICE_NAME
      
      [$OPENACS_SERVICE_NAME aolserver]$ cd /var/lib/aolserver/openacs-5-0/package/myfirstpackage
      
      [$OPENACS_SERVICE_NAME myfirstpackage]$ cvs -d /var/lib/cvs/ import -m "importing package" $OPENACS_SERVICE_NAME/packages/myfirstpackage OpenACS openacs-5-1
      

      Create a new directory as temporary working space to reconcile conflicts between the new files and your current work. The example uses the cvs keyword yesterday, making the assumption that you haven't checked in new code to your local tree in the last day. This section should be improved to use tags instead of the keyword yesterday!

      [$OPENACS_SERVICE_NAME openacs-5.1]$  cd /var/lib/aolserver
      [$OPENACS_SERVICE_NAME tmp]$ rm -rf $OPENACS_SERVICE_NAME-upgrade
      [$OPENACS_SERVICE_NAME tmp]$ mkdir $OPENACS_SERVICE_NAME-upgrade
      [$OPENACS_SERVICE_NAME tmp]$ cvs checkout -d $OPENACS_SERVICE_NAME-upgrade -jOpenACS:yesterday -jOpenACS -kk $OPENACS_SERVICE_NAME > cvs.txt 2>&1
      (CVS feedback here)

      The file /var/tmp/openacs-upgrade/cvs.txt contains the results of the upgrade. If you changed files that are part of the OpenACS tarball and those changes conflict, you'll have to manually reconcile them. Use the emacs command M-x sort-lines (you may have to click Ctrl-space at the beginning of the file, and go to the end, and then try M-x sort-lines) and then, for each line that starts with a C, open that file and manually resolve the conflict by deleting the excess lines. When you're finished, or if there aren't any conflicts, save and exit.

      Once you've fixed any conflicts, commit the new code to your local tree.

      [$OPENACS_SERVICE_NAME tmp]$ cd $OPENACS_SERVICE_NAME-upgrade
      [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME-upgrade]$ cvs commit -m "Upgraded to 5.1"
      
    • Step 3: Upgrade your local staging site.Update your working tree with the new files. The CVS flags ensure that new directories are created and pruned directories destroyed.

      [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME-upgrade]$ cd /var/lib/aolserver/$OPENACS_SERVICE_NAME
      
      [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ cvs up -Pd
      (CVS feedback)
      [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ exit
      [root root]# 

Upgrading files for a site using the OpenACS CVS repository (cvs.openacs.org)

  1. [$OPENACS_SERVICE_NAME ~]$ cd /var/lib/aolserver/$OPENACS_SERVICE_NAME
    
    [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$ cvs up -Pd
    (CVS feedback)
    [$OPENACS_SERVICE_NAME $OPENACS_SERVICE_NAME]$

If you are upgrading a production OpenACS site which is on a private CVS tree, this process lets you do the upgrade without risking extended downtime or an unusable site:

  1. Declare a freeze on new cvs updates - ie, you cannot run cvs update on the production site

  2. Make a manual backup of the production site in addition to the automated backups

  3. Import the new code (for example, OpenACS 5.0.4, openacs-5-0-compat versions of ETP, blogger, and other applications) into a "vendor branch" of the $OPENACS_SERVICE_NAME CVS tree, as described in "Upgrading a local CVS repository", step 1, above. As soon as we do this, any cvs update command on production might bring new code onto the production site, which would be bad.

    Do step 2 above (merging conflicts in a $OPENACS_SERVICE_NAME-upgrade working tree).

  4. Manually resolve any conflicts in the working upgrade tree

  5. Use the upgrade script and a recent backup of the production database, to ake a new upgraded database called $OPENACS_SERVICE_NAME-upgrade. Now we have a new website called $OPENACS_SERVICE_NAME-upgrade.

  6. Test the $OPENACS_SERVICE_NAME-upgrade site

  7. If $OPENACS_SERVICE_NAME-upgrade is fully functional, do the real upgrade.

    1. Take down the $OPENACS_SERVICE_NAME site and put up a "down for maintenance" page.

    2. Repeat the upgrade with the most recent database

    3. Test the that the new site is functional. If so, change the upgraded site to respond to yourserver.net requests. If not, bring the original production site back up and return to the merge.

Next Page