Forum OpenACS Development: Major revision of the XOTcl interface to the Content Repository and to ACS objects and ACS Object Types in general

Request notifications

Dear all,

CVS head contains now a major rewrite of the oo interface 
to content repository entries and a new oo interface to 
acs-objects and acs-object-types based on XOTcl slots. 
In short, a slot is a generalization of XOTcl parameters 
used to configure objects. XOTcl slots allow to define 
different kind of slots (e.g. classical parameters or 
database slots) which can hold meta-information about XOTcl 
instance attributes and SQL attributes. For example, the 
database slots contain among other things the attributes of 
acs_attributes like pretty_name, etc. but also the sql_specs 
for e.g. creating tables.

The interface is a generalization of the XOTcl interface for 
content repository items in use since about two years: 
http://openacs.org/forums/message-view?message_id=346699

In addition to the old functionality, one can not now only 
work on content repository items via the oo interface, but 
as well on arbitrary acs objects. In short, one can 
create/query/destroy acs-objects and acs-object-types, 
create for example new object types, or turn existing 
acs-object-type definitions into XOTcl classes by fetching 
the necessary information from the database.

Below is a session script which gives a short introduction 
how to use the XOTcl interface to ACS Object classes and
Content Repository classes.

In detail, it demonstrates the following tasks:

 1) Create new ACS Objects, destroy it in memory, 
    load it again from the database, delete it in 
    the database.

 2) Create new ACS Object Types, ACS Attributes and 
    SQL Tables from XOTcl Classes with slot definitions.

 3) Create XOTcl classes from existing ACS Object Types
    and ACS Attributes based on the definitions in the
    database

 4) Use the generic Content Repository interface (using 
    just cr_items and cr_revisions)
 
 5) Create new application classes by sub-typing the 
    Content Repository, adding additional attributes

Before showing the tasks 1 - 5, please note that this 
change is not only an extension of the functionality, 
but is contains as well a refactoring of some parts of 
the interface. The old ::Generic namespace is not used 
for the interface any more, all content is now in the 
::xo::db namespace. Many internal methods are renamed and 
moved e.g. from ::Generic::CrItem into ::xo::db::Class or
::xo::db::CrClass to reflect the generalization.

Check the short summary of the interface below. All listed 
methods are documented via the OpenACS API browser.

Note, that this is a major change and it requires XOTcl 
1.5.0 or newer (1.5.0 was released about two years ago, the 
current version of XOTcl is 1.5.4). Most of the changes 
are in xotcl-core, but certainly, applications based on
xotcl-core (such as xowiki) need some updates as well.

The stable and conservative version of xotcl-core and 
xowiki is in the oacs-5-3 branch.




#
# The interface:
#
# All generic lookup queries or instantiate methods
# just receiving object-ids or names, returning arbitrary
# object are now under one of the following classes 
#
#    ::xo::db::Class
#    ::xo::db::CrClass
#
# depending on function on ordinary ACS Objects (::xo::db::Class) or on
# content repository objects (::xo::db::CrClass)
#
# Generic operations on ACS Object IDs
#
#    ::xo::db::Class get_object_type -id
#    ::xo::db::Class get_instance_from_db -id
#    ::xo::db::Class exists_in_db -id
#    ::xo::db::Class delete -id
#
# Generic operations on ACS Object Types
#
#    ::xo::db::Class get_table_name -object_type
#    ::xo::db::Class object_type_exists_in_db -object_type
#    ::xo::db::Class get_class_from_db -object_type
#    ::xo::db::Class drop_type -object_type -drop_table -cascade_p
#    ::xo::db::Class delete_all_acs_objects -object_type
#
# Generic operations on items of the content repository
# 
#    ::xo::db::CrClass get_object_type -item_id -revision_id
#    ::xo::db::CrClass get_instance_from_db -item_id -revision_id
#    ::xo::db::CrClass lookup -item_id -revision_id
#    ::xo::db::CrClass delete -item_id
#
# Methods on classes for ACS Object Types
#
#    <classname> new_persistent_object -package_id -creation_user -creation_ip ...
#    <classname> object_types -subtypes_first
#    <classname> create_object_type
#    <classname> drop_object_type -cascade
#    <classname> instance_select_query ...
#    <classname> get_instances_from_db ...
#
# Methods on content repository classes 
#
#    <classname> new_persistent_object -package_id -creation_user -creation_ip ...
#    <classname> create_object_type
#    <classname> drop_object_type
#    <classname> get_instance_from_db -item_id -revision_id
#    <classname> instance_select_query ...
#    <classname> get_instances_from_db ...
#    <classname> folder_type_unregister_all -include_subtypes -folderid register|unregister
#    <classname> folder_type_unregister_all -include_subtypes
#
# Methods on instances of ordinary classes
#
#    <object> save -package_id -modifying_user
#    <object> save_new -package_id -creation_user -creation_ip
#    <object> delete
#
# Methods on instances of content repository classes
#
#    <object> save -modifying_user -live_p
#    <object> save_new -package_id -creation_user -creation_ip -live_p
#    <object> set_live_revision -revision_id -publish_status
#    <object> delete
#


############################################################
#
# 1) Create new ACS Objects, destroy it in memory, 
#    load it from the database, delete it in the database.
#
.. Create a plain new ACS object just for demo purposes.
.. The ACS object is created with a new object id.

>> set o [::xo::db::Object new_persistent_object]
=  ::7845

.. Show the contents of object ::7845 by serializing it:

>> ::7845 serialize
=  ::xo::db::Object create ::7845 -noinit \
    -set object_title {Object 7845} \
    -set object_id 7845 


.. In the next steps, we (a) get the object_id of the newly
.. created ACS object, (b) destroy the XOTcl object (the ACS
.. object is still in the database, (c) we recreate the
.. XOTcl object from the database, and (d) delete it in the 
.. database.

.. Step (a)
>> set o_id [::7845 object_id]
=  7845

#
# Delete object from memory: <object> destroy 
# Check, if an XOTcl object exists: ::xotcl::Object isobject <obj>
#
>> ::xotcl::Object isobject ::7845
=  1

.. Step (b)
>> ::7845 destroy

>> ::xotcl::Object isobject ::7845
=  0

#
# Load an object from the database: ::xo::db::Class get_instance_from_db -id <id>
#
.. Step (c)
>> set o [::xo::db::Class get_instance_from_db -id 7845]
=  ::7845

>> ::xotcl::Object isobject ::7845
=  1

.. Now, we have recreated the same object as before:

>> ::7845 serialize
=  ::xo::db::Object create ::7845 -noinit \
    -set object_title {Object 7845} \
    -set object_id 7845 


#
# Check, if an ACS object exists in the database: ::xo::db::Class exists_in_db -id <id>
# Delete object from memory and database: <object> delete 
#
>> ::xo::db::Class exists_in_db -id 7845
=  1

.. Step (d)
>> ::7845 delete

.. Now, we have deleted the ACS Object and the XOTcl object:

>> ::xo::db::Class exists_in_db -id 7845
=  0

>> ::xotcl::Object isobject ::7845
=  0


############################################################
#
# 2) Create new ACS Object Types, ACS Attributes and 
#    SQL Tables from XOTcl Classes with slot definitions.
#
.. We want to create a new ACS Object type and 
.. an XOTcl class named ::demo::Person.

.. Does the ACS Object type ::demo::Person exist in the database?
>> ::xo::db::Class object_type_exists_in_db -object_type ::demo::Person
=  0

.. The ACS Object Type should not exist (statement should return 0)

.. We create a new XOTcl Class '::demo::Person'.
.. By defining this class, the database layer takes care
.. of creating the ACS Object Type and the necessary table via SQL.

.. The persistent attributes (stored in the database) are defined
.. as slots of type ::xo::db::Attribute.

>> 
::xo::db::Class create ::demo::Person  \
   -superclass ::xo::db::Object  \
   -slots {
      ::xo::db::Attribute create name -column_name pname
      ::xo::db::Attribute create age -default 0 -datatype integer
      ::xo::db::Attribute create projects -default {} -multivalued true
    }

=  ::demo::Person

.. If the ACS Object Type and the ACS Attributes would be
.. already defined in the database, the class definition above 
.. would be a no-op operation.

.. Now, the ACS Object Type exists in the database

>> ::xo::db::Class object_type_exists_in_db -object_type ::demo::Person
=  1

.. The XOTcl class definition created automatically the 
.. following table for storing instances:

CREATE TABLE demo_person (
    age  integer DEFAULT '0' ,
    pname  text ,
    projects  text DEFAULT '' ,
    person_id  integer REFERENCES acs_objects(object_id) ON DELETE CASCADE
    CONSTRAINT demo_person_person_id_pk PRIMARY KEY 
)

.. SQL attribute slot names:
>> ::demo::Person array names db_slot
=  name age projects person_id

.. The XOTcl class definition created as well a 'save' and
.. an 'insert' method:

.. Created 'save' method:

::demo::Person instproc save {}  {
      db_transaction {
    next
    my instvar object_id name age projects
    db_dml dbqd..update_demo_person {update demo_person
      set pname = :name,age = :age,projects = :projects
          where person_id = :object_id
    }
      }
    }

.. Created 'insert' method:

::demo::Person instproc insert {}  {
      set __table_name [[self class] table_name]
      set __id [[self class] id_column]
      my set $__id [my set object_id]
      my log "ID insert in $__table_name, id = $__id = [my set $__id]"
      next
      foreach {__slot_name __slot} [[self class] array get db_slot] {
    my instvar $__slot_name
    if {[info exists $__slot_name]} { 
      lappend __vars $__slot_name
      lappend __atts [$__slot column_name]
    }
      }
      db_dml dbqd..insert_$__table_name "insert into $__table_name
        ([join $__atts ,]) values (:[join $__vars ,:])"
    }

.. The 'insert' method is typically called via
.. '<classname> new', which takes care about db_transaction (see below)

#
# Create a new instance of ::demo::Person with name 'Gustaf'
#
# The method 'new_persistent_object' of a database class (instance of ::xo::db::Class)
# creates an ACS Object with a fresh id in the database and 
# creates as well an XOTcl object in memory

>> set p [::demo::Person new_persistent_object -name Gustaf -age 105]
=  ::7846

::7846 created

.. check, if object ::7846 exists in memory
>> ::xotcl::Object isobject ::7846
=  1

.. check, if object ::7846 exists in the database
>> ::xo::db::Class exists_in_db -id 7846
=  1

.. Every persistent XOTcl object has an object_id.
>> ::7846 exists object_id
=  1

.. Show the contents of object ::7846 (using serialize)

>> ::7846 serialize
=  ::demo::Person create ::7846 -noinit \
    -set object_title {Person 7846} \
    -set name Gustaf \
    -set age 105 \
    -set projects {} \
    -set person_id 7846 \
    -set object_id 7846 


.. modify some attributes of the XOTcl object
>> ::7846 incr age
=  106

.. show modified contents of ::7846

::demo::Person create ::7846 -noinit \
    -set object_title {Person 7846} \
    -set name Gustaf \
    -set age 106 \
    -set projects {} \
    -set person_id 7846 \
    -set object_id 7846 

.. save the modified object data in the database
>> ::7846 save

>> set person_id [$p person_id]
=  7846

.. deleting xotcl object ::7846 in memory
>> $p destroy

.. check, if object ::7846 exists in memory
>> ::xotcl::Object isobject ::7846
=  0

.. check, if object ::7846 exists in the database
>> ::xo::db::Class exists_in_db -id 7846
=  1

.. fetch person again from database:

>> set p [::xo::db::Class get_instance_from_db -id 7846]
=  ::7846

.. check, if object ::7846 exists in memory
>> ::xotcl::Object isobject ::7846
=  1

::demo::Person create ::7846 -noinit \
    -set object_title {Person 7846} \
    -set name Gustaf \
    -set age 106 \
    -set projects {} \
    -set object_id 7846 \
    -set person_id 7846 


.. The object ::7846 was fetched from the database using the
.. automatically created query:

SELECT demo_person.pname AS name,demo_person.age,
   demo_person.projects,demo_person.person_id,
   acs_objects.title AS object_title,acs_objects.object_id
FROM demo_person,acs_objects
WHERE person_id = 7846 and object_id = person_id

.. In order to delete an object from the database, 
.. the method 'delete' can be used. 'delete' deletes the 
.. ACS object in the database as well the XOTcl object in memory.

>> set p_id [::7846 object_id]
=  7846

>> ::xo::db::Class exists_in_db -id 7846
=  1

>> ::7846 delete

>> ::xo::db::Class exists_in_db -id 7846
=  0

.. Instead of using 'new_persistent_object' to create
.. new acs_objects, one can use the method 'new' in 
.. combination with 'save_new'. While the method 'new' creates
.. an XOTcl object in memory, the method 'save_new' can be used
.. to create a new ACS object (with a new object_id) an to 
.. save this object persistently in the database.

>> set p [::demo::Person new -name Gustaf -age 105]
=  ::xotcl::__#A

>> ::xotcl::__#A exists object_id
=  0

>> ::xotcl::__#A save_new
=  7847

>> ::xotcl::__#A exists object_id
=  1


.. Now, we create a subclass of ::demo::Person called ::demo::Employee
.. which has a few more attributes. Again, we define an XOTcl class
.. ::demo::Employee which creates the ACS Object Type, the ACS
.. attributes and the table, if necessary.

>> 
::xo::db::Class create ::demo::Employee  \
   -superclass ::demo::Person  \
   -table_name demo_employee \
   -id_column employee_id  \
   -slots {
      ::xo::db::Attribute create salary -datatype integer
      ::xo::db::Attribute create dept_nr -datatype integer -default "0"
    }

=  ::demo::Employee


.. The XOTcl class definition created automatically the 
.. following table for storing instances:

CREATE TABLE demo_employee (
    dept_nr  integer DEFAULT '0' ,
    salary  integer ,
    employee_id  integer REFERENCES demo_person(person_id) ON DELETE CASCADE
    CONSTRAINT demo_employee_employee_id_pk PRIMARY KEY 
)

.. Create a first new employee:
>> set p1 [::demo::Employee new_persistent_object -name Neophytos -age 25]
=  ::7848

.. show contents of ::7848

::demo::Employee create ::7848 -noinit \
    -set employee_id 7848 \
    -set object_title {Employee 7848} \
    -set dept_nr 0 \
    -set name Neophytos \
    -set age 25 \
    -set projects {} \
    -set person_id 7848 \
    -set object_id 7848 

.. Create a second new employee:
>> set p2 [::demo::Employee new_persistent_object -name Lore -age 35 -salary 100000]
=  ::7849

.. show contents of ::7849

::demo::Employee create ::7849 -noinit \
    -set employee_id 7849 \
    -set object_title {Employee 7849} \
    -set salary 100000 \
    -set dept_nr 0 \
    -set name Lore \
    -set age 35 \
    -set projects {} \
    -set person_id 7849 \
    -set object_id 7849 

.. Create a third new employee:
>> set p3 [::demo::Employee new_persistent_object -name Nora -age 7 -salary 100]
=  ::7850

.. show contents of ::7850

::demo::Employee create ::7850 -noinit \
    -set employee_id 7850 \
    -set object_title {Employee 7850} \
    -set salary 100 \
    -set dept_nr 0 \
    -set name Nora \
    -set age 7 \
    -set projects {} \
    -set person_id 7850 \
    -set object_id 7850 

#
# Delete object from memory: <object> destroy 
#
>> ::xotcl::Object isobject ::7848
=  1

>> set p1_id [$p1 object_id]
=  7848

>> ::xo::db::Class exists_in_db -id 7848
=  1

>> $p1 destroy

>> ::xotcl::Object isobject ::7848
=  0

>> ::xo::db::Class exists_in_db -id 7848
=  1

#
# Delete object from memory and database: <object> delete 
#
>> ::xotcl::Object isobject ::7849
=  1

>> set p2_id [$p2 object_id]
=  7849

>> ::xo::db::Class exists_in_db -id 7849
=  1

>> $p2 delete

>> ::xotcl::Object isobject ::7849
=  0

>> ::xo::db::Class exists_in_db -id 7849
=  0

.. Fetch employee with id 7848 again from the database:

>> set p [::xo::db::Class get_instance_from_db -id 7848]
=  ::7848

.. show contents of ::7848

::demo::Employee create ::7848 -noinit \
    -set object_title {Employee 7848} \
    -set salary {} \
    -set employee_id 7848 \
    -set dept_nr 0 \
    -set name Neophytos \
    -set age 25 \
    -set projects {} \
    -set object_id 7848 \
    -set person_id 7848 

.. The object ::7848 was fetched from the database using the
.. automatically created query:

SELECT demo_employee.employee_id,demo_employee.salary,
   demo_employee.dept_nr, 
   demo_person.pname AS name,demo_person.age,
   demo_person.projects,demo_person.person_id,
   acs_objects.title AS object_title,acs_objects.object_id
FROM demo_employee,demo_person,acs_objects
WHERE employee_id = 7848 and person_id = employee_id 
   and object_id = employee_id

>> ::7848 destroy



.. Query the instances from the database

.. Instances are queried by default based on the following,
.. automatically generated SQL Query:

>> ::demo::Employee instance_select_query
=  SELECT demo_employee.employee_id,demo_employee.salary,
     demo_employee.dept_nr,
     demo_person.pname AS name,demo_person.age,
     demo_person.projects, demo_person.person_id,
     acs_objects.title AS object_title,acs_objects.object_id 
   FROM demo_employee,demo_person,acs_objects  
   WHERE person_id = employee_id and object_id = employee_id   

.. Query the object using this query into a set of objects.
>> set s [::demo::Employee get_instances_from_db]
=  ::xotcl::__#H

.. The result is an OrderedComposite. Children can be accessed
.. via the 'children' method:

>> llength [$s children]
=  2

.. serialize all children
::xotcl::Object create ::xotcl::__#I -noinit \
    -set __parent ::xotcl::__#H \
    -set object_title {Employee 7848} \
    -set dept_nr 0 \
    -set salary {} \
    -set employee_id 7848 \
    -set age 25 \
    -set name Neophytos \
    -set projects {} \
    -set object_id 7848 \
    -set person_id 7848 

::xotcl::Object create ::xotcl::__#J -noinit \
    -set __parent ::xotcl::__#H \
    -set object_title {Employee 7850} \
    -set dept_nr 0 \
    -set salary 100 \
    -set employee_id 7850 \
    -set age 7 \
    -set name Nora \
    -set projects {} \
    -set object_id 7850 \
    -set person_id 7850 

.. The instance_select_query can be configured in many ways.
.. Here we use the '-count true' option to return just the number
.. of solutions:

>> ::demo::Employee instance_select_query -count true
=  SELECT count(*) 
   FROM demo_employee,demo_person,acs_objects  
   WHERE person_id = employee_id and object_id = employee_id   

>> db_string cnt [::demo::Employee instance_select_query -count true]
=  2

.. The same queries for ::demo::Person instead of ::demo::Employee.
.. Note, that we get the person data of employees as well, since
.. employees are as well persons (i.e. a specialization of person):

>> ::demo::Person instance_select_query
=  SELECT demo_person.pname AS name,demo_person.age,
     demo_person.projects,demo_person.person_id,
     acs_objects.title AS object_title,acs_objects.object_id 
   FROM demo_person,acs_objects  
   WHERE object_id = person_id   

>> set s [::demo::Person get_instances_from_db]
=  ::xotcl::__#M

>> llength [$s children]
=  3

::xotcl::Object create ::xotcl::__#N -noinit \
    -set __parent ::xotcl::__#M \
    -set object_title {Person 7847} \
    -set age 105 \
    -set name Gustaf \
    -set projects {} \
    -set object_id 7847 \
    -set person_id 7847 

::xotcl::Object create ::xotcl::__#O -noinit \
    -set __parent ::xotcl::__#M \
    -set object_title {Employee 7848} \
    -set age 25 \
    -set name Neophytos \
    -set projects {} \
    -set object_id 7848 \
    -set person_id 7848 

::xotcl::Object create ::xotcl::__#P -noinit \
    -set __parent ::xotcl::__#M \
    -set object_title {Employee 7850} \
    -set age 7 \
    -set name Nora \
    -set projects {} \
    -set object_id 7850 \
    -set person_id 7850 

TODO: explain more options for instance_select_query and get_instances_from_db

############################################################
#
# 3) Create XOTcl classes from existing ACS Object Types
#    and ACS Attributes based on the definitions in the
#    database

>> set c [::xo::db::Class get_class_from_db -object_type party]
=  ::xo::db::party

.. XOTcl class ::xo::db::party created (superclass ::xo::db::Object)
.. SQL attributes:
>> ::xo::db::party array names db_slot
=  email party_id url

>> set c [::xo::db::Class get_class_from_db -object_type person]
=  ::xo::db::person

.. XOTcl class ::xo::db::person created (superclass ::xo::db::party)
.. SQL attributes:
>> ::xo::db::person array names db_slot
=  last_name first_names person_id

>> set c [::xo::db::Class get_class_from_db -object_type user]
=  ::xo::db::user

.. XOTcl class ::xo::db::user created (superclass ::xo::db::person)
.. SQL attributes:
>> ::xo::db::user array names db_slot
=  user_id

>> set c [::xo::db::Class get_class_from_db -object_type group]
=  ::xo::db::group

.. XOTcl class ::xo::db::group created (superclass ::xo::db::party)
.. SQL attributes:
>> ::xo::db::group array names db_slot
=  group_name group_id

.. Create XOTcl object for user_id = 0

.. please log in
>> set u [::xo::db::Class get_instance_from_db -id $myuser]
=  ::0

.. ::0 created, class ::xo::db::user
.. class hierarchy for ::0 ::xo::db::user ::xo::db::person ::xo::db::party ::xo::db::Object ::xotcl::Object
.. show contents of ::0

::xo::db::user create ::0 -noinit \
    -set object_title Unregistered Visitor \
    -set party_id 0 \
    -set email {} \
    -set last_name Visitor \
    -set object_id 0 \
    -set url {} \
    -set person_id 0 \
    -set first_names Unregistered \
    -set user_id 0 

.. The object ::0 was fetched from the database using the
.. automatically created query:

SELECT users.user_id,persons.last_name,persons.first_names,
  persons.person_id,parties.email,
  parties.party_id,parties.url,
  acs_objects.title AS object_title,acs_objects.object_id
FROM users,persons,parties,acs_objects
WHERE user_id = 0 and person_id = user_id and 
  party_id = user_id and object_id = user_id

.. Now we modify the user. To be on the safe side, we
.. a) save firstly the original URL of the current user, then
.. b) we set it to a new value and save the user information
..    to the database
.. c) delete the user object in memory
.. d) load it again from the database to see the modified data
.. e) reset it to the original value and save it again to restore
..    the original state

.. Step a:
>> set url [::0 url]

.. Step b:
>> ::0 url http://nowhere.land/http://nowhere.land/

>> ::0 save

.. Step c:
>> ::0 destroy

.. Step d:
>> set u [::xo::db::Class get_instance_from_db -id $myuser]
=  ::0

.. show contents of ::0

::xo::db::user create ::0 -noinit \
    -set object_title Unregistered Visitor \
    -set party_id 0 \
    -set email {} \
    -set last_name Visitor \
    -set object_id 0 \
    -set url http://nowhere.land/ \
    -set person_id 0 \
    -set first_names Unregistered \
    -set user_id 0 

.. Step e:
>> ::0 url $url

>> ::0 save

>> ::0 destroy

>> set u [::xo::db::Class get_instance_from_db -id $myuser]
=  ::0

.. show contents of ::0

::xo::db::user create ::0 -noinit \
    -set object_title Unregistered Visitor \
    -set party_id 0 \
    -set email {} \
    -set last_name Visitor \
    -set object_id 0 \
    -set url {} \
    -set person_id 0 \
    -set first_names Unregistered \
    -set user_id 0 



############################################################
#
# 4) Use the generic Content Repository interface (using 
#    just cr_items and cr_revisions)
#

.. check, if cr_item with name 'testing' exists
>> set item_id [::xo::db::CrClass lookup -name testing]
=  7841

.. yes, it exists; delete it without instantiating
>> ::xo::db::CrItem delete -item_id 7841

.. create item
>> set i [::xo::db::CrItem new  -name "testing"  -title "Title of Hello World"  -text "Hello World"  ]
=  ::xotcl::__#a

>> $i save_new
=  7852

.. show contents of item

::xo::db::CrItem create ::xotcl::__#a -noinit \
    -set creation_date {2007-09-03 23:28:26.727511+02} \
    -set creation_user 0 \
    -set object_id 7852 \
    -set description {} \
    -set text {Hello World} \
    -set nls_language en_US \
    -set mime_type text/plain \
    -set name testing \
    -set publish_status ready \
    -set title {Title of Hello World} \
    -set revision_id 7851 \
    -set item_id 7852 \
    -set last_modified {2007-09-03 23:28:26.727511+02} \
    -set parent_id -100 

.. update content

>> ::xotcl::__#a append text {... more text.. more text.. more text.}
=  Hello World... more text.. more text.. more text.

>> ::xotcl::__#a append title { - v2}
=  Title of Hello World - v2

.. show modified contents of item

::xo::db::CrItem create ::xotcl::__#a -noinit \
    -set creation_date {2007-09-03 23:28:26.727511+02} \
    -set creation_user 0 \
    -set object_id 7852 \
    -set description {} \
    -set text {Hello World... more text.. more text.. more text.} \
    -set nls_language en_US \
    -set mime_type text/plain \
    -set name testing \
    -set publish_status ready \
    -set title {Title of Hello World - v2} \
    -set revision_id 7851 \
    -set item_id 7852 \
    -set last_modified {2007-09-03 23:28:26.727511+02} \
    -set parent_id -100 

>> ::xotcl::__#a save
=  7852

.. method 'save' creates a new revision; see changed revision_id below

::xo::db::CrItem create ::xotcl::__#a -noinit \
    -set creation_date {2007-09-03 23:28:26.727511+02} \
    -set creation_user 0 \
    -set object_id 7852 \
    -set description {} \
    -set text {Hello World... more text.. more text.. more text.} \
    -set nls_language en_US \
    -set mime_type text/plain \
    -set name testing \
    -set publish_status ready \
    -set title {Title of Hello World - v2} \
    -set revision_id 7853 \
    -set item_id 7852 \
    -set last_modified {2007-09-03 23:28:26.727511+02} \
    -set parent_id -100 

>> set item_id 7852
=  7852

>> ::xotcl::__#a destroy

>> set i [::xo::db::CrClass get_instance_from_db -item_id 7852]
=  ::7852

::xo::db::CrItem create ::7852 -noinit \
    -set object_type content_revision \
    -set creation_date {2007-09-03 23:28:26.727511+02} \
    -set object_id 7853 \
    -set creation_user 0 \
    -set description {} \
    -set text {Hello World... more text.. more text.. more text.} \
    -set nls_language en_US \
    -set package_id 492 \
    -set name testing \
    -set mime_type text/plain \
    -set publish_status ready \
    -set title {Title of Hello World - v2} \
    -set item_id 7852 \
    -set revision_id 7853 \
    -set last_modified {2007-09-03 23:28:26.750059+02} \
    -set parent_id -100 

############################################################
#
# 5) Create new application classes by sub-typing the 
#    Content Repository, adding additional attributes
#

.. We create a subclass of ::xo::db::CrItem called ::demo::Page
.. which has a few more attributes. Actually, this class is very
.. similar to ::xowiki::Page. Again, we define an XOTcl class
.. ::demo::Page which creates the ACS Object Type, the ACS
.. attributes and the table, if necessary.

>> 
  # content class very similar to ::xowiki::Page
  ::xo::db::CrClass create Page \
    -superclass ::xo::db::CrItem  \
    -pretty_name "demo Page"  \
    -mime_type text/html  \
    -slots {
        if {[::xo::db::has_ltree]} {
          ::xo::db::CrAttribute create page_order \
            -sqltype ltree -validator page_order \
            -default ""
        }
        ::xo::db::CrAttribute create creator \
            -column_name creator_string
      }

=  ::demo::Page

.. The sql_attribute_names for content items contain
.. the attributes of all supertypes:

>> ::demo::Page array names db_slot
=  creator page_order page_id object_id description text object_title nls_language mime_type name title item_id revision_id

>> ::demo::Page info slots
=  ::demo::Page::slot::creator ::demo::Page::slot::page_order ::demo::Page::slot::mime_type ::demo::Page::slot::page_id

>> ::xo::slotobjects ::demo::Page
=  ::demo::Page::slot::creator ::demo::Page::slot::page_order ::demo::Page::slot::mime_type ::demo::Page::slot::page_id ::xo::db::CrItem::slot::package_id ::xo::db::CrItem::slot::nls_language ::xo::db::CrItem::slot::description ::xo::db::CrItem::slot::name ::xo::db::CrItem::slot::text ::xo::db::CrItem::slot::publish_status ::xo::db::CrItem::slot::parent_id ::xo::db::CrItem::slot::title ::xo::db::CrItem::slot::revision_id ::xo::db::CrItem::slot::item_id ::xo::db::Object::slot::object_title ::xo::db::Object::slot::object_id

.. create page
>> set i [Page new  \
     -name "page0"  \
     -title "Joke of the Month" \
     -creator "GN" \
     -text "Three cannibals meet in a NYC subway station..."  ]
=  ::xotcl::__#j

>> $i save_new
=  7855

.. show contents of page object ::xotcl::__#j (including creator and page_order,
.. when ltree is available)

::demo::Page create ::xotcl::__#j -noinit \
    -set page_order {} \
    -set creator GN \
    -set page_id {} \
    -set creation_date {2007-09-03 23:28:29.149591+02} \
    -set object_id 7855 \
    -set creation_user 0 \
    -set description {} \
    -set text {Three cannibals meet in a NYC subway station...} \
    -set nls_language en_US \
    -set mime_type text/html \
    -set name page0 \
    -set publish_status ready \
    -set title {Joke of the Month} \
    -set revision_id 7854 \
    -set item_id 7855 \
    -set last_modified {2007-09-03 23:28:29.149591+02} \
    -set parent_id -100 

>> set item_id [::xotcl::__#j item_id]
=  7855

>> ::xotcl::__#j destroy

>> ::demo::Page get_instance_from_db -item_id 7855
=  ::7855

::demo::Page create ::7855 -noinit \
    -set page_order {} \
    -set object_type ::demo::Page \
    -set page_id 7854 \
    -set creation_date {2007-09-03 23:28:29.149591+02} \
    -set object_id 7854 \
    -set creation_user 0 \
    -set description {} \
    -set text {Three cannibals meet in a NYC subway station...} \
    -set nls_language en_US \
    -set package_id 492 \
    -set name page0 \
    -set mime_type text/html \
    -set title {Joke of the Month} \
    -set publish_status ready \
    -set revision_id 7854 \
    -set item_id 7855 \
    -set last_modified {2007-09-03 23:28:29.149591+02} \
    -set parent_id -100 \
    -set creator_string GN
Hi, Gustaf!

I got your code from head and I get the next error in the log:

[04/Sep/2007:09:40:44][5111.18446744071932798112][-main-] Error: Error sourcing .../packages/xotcl-core/tcl/05-db-procs.tcl:
invalid command name "::xo::db::sql::acs_object_type"
while executing
"::xo::db::sql::acs_object_type create_type -object_type $object_type -supertype $supertype -pretty_name $pretty_name -pretty_plural $pretty_plural..."
(procedure "create_object_type" line 15)

Any pointer?

Agustin

xotcl-core has now stronger dependencies on correct (and fairly complete) setting of define_function_args(), which are used to manage the acs_function_args table in OpenACS. I have completed and fixed some define_function_args() ages ago on my instances and forgot to mention this.

Run the following script against your database to get rid the error above.

-- acs-kernel: acs-metadata-create.sql
select define_function_args('acs_object_type__create_type','object_type,pretty_name,pretty_plural,supertype,table_name;null,id_column;null,package_name;null,abstract_p;f,type_extension_table;null,name_method;null');
select define_function_args('acs_object_type__drop_type','object_type,cascade_p;f');
select define_function_args('acs_attribute__create_attribute','object_type,attribute_name,datatype,pretty_name,pretty_plural;null,table_name;null,column_name;null,default_value;null,min_n_values;1,max_n_values;1,sort_order;null,storage;type_specific,static_p;f');
select define_function_args('acs_attribute__add_description','object_type,attribute_name,description_key,description');
select define_function_args('acs_attribute__drop_description','object_type,attribute_name,description_key');
-- acs-kernel: acs-objects-create.sql
select define_function_args('acs_object__new','object_id;null,object_type;acs_object,creation_date;now(),creation_user;null,creation_ip;null,context_id;null,security_inherit_p;t,title;null,package_id;null');
select define_function_args('acs_object__delete','object_id');
-- acs-content-respository: content-type.sql ... incorrectly defined twice, fix redefinition
select define_function_args('content_type__drop_type','content_type,drop_children_p;f,drop_table_p;f,drop_objects_p;f');

These function definitions should be fixed in the acs kernel packages in CVS head as well (if nobody objects). Otherwise i would need to come up with some redundant definition in xotcl-core.

I did some more checks on fresh installs and fixed a few more things in xotcl-core and xowiki. There is as well a new version of the xotcl-request-monitor (for the version in head and oacs-5-3) and a version of the s5 package (depending on the changes in cvs head) in CVS.

I think you are safe to fix them and please provide an upgrade as well. Do you know if this problem exists in Oracle as well?
define_function_args does not exist in oracle at all, since it is implemented to mimic the oracle meta data for stored procedures. It is a postgres only problem.

I am pretty sure, some more interface descriptions are still missing or incorrect. My suggestion is to fix the above set of definitions soon in cvs head and do a second round, checking attribute names and defaults of the existing definitions against the actual stored procedures, and then compare the interface definitions with Oracle and complete where necessary.

Any idea why you cannot redefine the function_args in postgresql dependent on how many arguments the function has? We do have two content_type__drop_type functions, one with three, one with four arguments, so why can't we define the function args for both?
Sorry, the last attempt was fired before even getting started. I'd say the simple reason is the data model and constraints currently in place, e.g.:

"acs_function_args_un" UNIQUE, btree ("function", arg_name)

this does not leave anymore options.

from the perspective of xotcl-core's db layer, we would be constraint due to XOTcl currently not providing for overloading/ multi-methods, i.e. defining equally named methods on objects which are only distinct with respect to their signature (types of their arguments) and are resolved according to some specific order of precedence by the dispatcher.

So the recommended "solution" would be to only define the function_args for the largest function, as we usually (always?) only add more parameters and not have two equally named functions where one had other attributes than the other?

Oracle's plsql interface is very nice and friendly when you want to create an object layer like these. It handles multiple functions of the same name, different defaults for the arguments, and the argument order is not important. That means that usually the developer doesn't have to go back and look at the signature, Oracle will decide which proc to use. How does it do this? The function calls include the names of the attributes along with the value. In postgres it is like a Tcl call. So the only thing you need to mimic the behavior for postgres is the name of the arguments, their position in each function (the function signature), and any defaults for optional attributes, plus a way of choosing which function to call given the information.

By a strange fluke, code exists in OpenACS which does exactly this. A tiny subset of query-writer is in the cronjob package, just a few procedures, with examples for setup and use. It doesn't use the database. I wrote this code to specifically aid porting from Oracle to Postgres. Here is an example of the original oracle, and the postgres:


begin
  cronjob.set_attrs(
    cronjob_id => :cronjob_id,
    description => :description,
    approved_p => :approved_p,
    disabled_p => :disabled_p,
    minute => :minute,
    hr => :hr,
    mon => :mon,
    day => :day,
    dayofweek => :dayofweek,
    run_sql => :run_sql,
    run_tcl => :run_tcl,
    email => :email);
end;
select [qd_write_query_select cronjob__set_attrs {
  cronjob_id => :cronjob_id
  description => :description
  approved_p => :approved_p
  disabled_p => :disabled_p
  minute => :minute
  hr => :hr
  mon => :mon
  day => :day
  dayofweek => :dayofweek
  run_sql => :run_sql
  run_tcl => :run_tcl
  email => :email } ]

The current version of the code in cronjob is here:

http://rmadilo.com/files/cronjob/cronjob/tcl/

Usage is in the www/admin directory. Mostly current code is also in CVS.

It is important to point out that you can submit a subset of the arguments as long as you provide all the params which don't have defaults, for instance, when creating a new object, the object_id has a default of null, meaning that the function will create a new acs_object first, so you don't have to look at the function signatures before writing a query. This is exactly similar to Oracle. Also, each function can have different parameter defaults, different from the same attribute in other functions of the same name.

As Stefan says, this is restriction is not xotcl related (same e.g. in package_exec_plsql). A call like define_function_args(FUNCNAME, ...) tries fist to delete all entries for FUNCNAME before it defines the args new.

since define_function_args() is an oacs-function, it would be possible to extend it, but note that one has to care about oracle as well

So the recommended "solution" would be to only define the function_args for the largest function ...

yes

fixed the function args above in cvs head, including upgrade scripts.
Hi,

I'm trying to install the 0.79 version of xotcl-core and xowiki on 5.3.2 system and running into problems, which seem to be around the default on sort_order of null in ::xo::db::sql::acs_attribute create_attribute.

[04/Dec/2007:16:26:01][21694.3073522576][-sched:10-] Error: Ns_PgExec: result status: 7 message: ERROR: invalid in"ut syntax for integer: "null

[04/Dec/2007:16:26:01][21694.3073522576][-sched:10-] Error: Tcl exception:
Database operation "0or1row" failed
"exception ERROR, "ERROR: invalid input syntax for integer: "null
")

while executing
"ns_pg_bind 0or1row $db $sql"
invoked from within
"ns_set value [ns_pg_bind 0or1row $db $sql] 0 "
invoked from within
"return [ ns_set value [ns_pg_bind 0or1row $db $sql] 0 ]"
("uplevel" body line 3)
invoked from within
"uplevel 1 $code_block "
invoked from within
"db_with_handle -dbn $dbn db {
#ns_log notice "--sql=$sql"
return [ ns_set value [ns_pg_bind 0or1row $db $sql] 0 ]
}"
(procedure "create_attribute" line 17)
::xo::db::sql::acs_attribute->create_attribute
invoked from within
"::xo::db::sql::acs_attribute create_attribute -object_type $object_type -attribute_name $column_name -datatype $datatype -pretty_name $pretty_name..."
(procedure "create_attribute" line 14)
::xo::db::Object::slot::object_id ::xo::db::Attribute->create_attribute
invoked from within
"my create_attribute"
(procedure "init" line 8)
invoked from within
"::xotcl::next"
(procedure "init" line 2)
::xo::db::Object::slot::object_id ::xotcl::Slot::Optimizer->init
::xo::db::Attribute ::xotcl::Class->create
invoked from within
"::xo::db::Attribute create ::xo::db::Object::slot::object_id \
-set min_n_values 1 \
-set max_n_values 1 \
-set manager ::xo::db::Object::slot::obj..."
invoked from within
"ns_ictl update"

Any ideas?

Cheers
Matthew

Hmm,

the error message you provided does not contain the information, what acs_attribute was trying to be created with which arguments. You should have the SQL statement, triggering the error a few lines before the snippet you have provided. If this does for some reason not contain meaningful information, add the following command to the method create_attribute of ::xo::db::Attribute in 05-db-procs.tcl

ns_log notice "CREATE ATT ::xo::db::sql::acs_attribute create_attribute [list \
-object_type $object_type \
-attribute_name $column_name \
-datatype $datatype \
-pretty_name $pretty_name \
-min_n_values $min_n_values \
-max_n_values $max_n_values ]"
before the call of "::xo::db::sql::acs_attribute create_attribute ..." and report back what it says.

Did this happen on a fresh installation of 5.3.2? The combination 5.3.2 + xotcl-core + xowiki from oacs-head is supposed to work (see e.g. http://alice.wu-wien.ac.at:8000/).

Hi Gustaf,

System is a fresh install of 5.3.2 on a pg 8.1.10 (originally tried on 8.0)

Actually printing the log message I get;
Notice: CREATE ATT ::xo::db::sql::acs_attribute create_attribute
-object_type acs_object
-attribute_name object_id
-datatype integer
-pretty_name {Object ID}
-min_n_values 1
-max_n_values 1
[05/Dec/2007:17:41:28][27003.3072904080][-sched:10-] Error: Ns_PgExec: result st"tus: 7 message: ERROR: invalid input syntax for integer: "null

[05/Dec/2007:17:41:28][27003.3072904080][-sched:10-] Error: Tcl exception:
Database operation "0or1row" failed
"exception ERROR, "ERROR: invalid input syntax for integer: "null
")

while executing
"ns_pg_bind 0or1row $db $sql"
invoked from within
"ns_set value [ns_pg_bind 0or1row $db $sql] 0 "
invoked from within
"return [ ns_set value [ns_pg_bind 0or1row $db $sql] 0 ]"
("uplevel" body line 3)
invoked from within
"uplevel 1 $code_block "
invoked from within
"db_with_handle -dbn $dbn db {
#ns_log notice "--sql=$sql"
return [ ns_set value [ns_pg_bind 0or1row $db $sql] 0 ]
}"
(procedure "create_attribute" line 17)
::xo::db::sql::acs_attribute->create_attribute
invoked from within
"::xo::db::sql::acs_attribute create_attribute -object_type $object_type -attribute_name $column_name -datatype $datatype -pretty_name $pretty_name..."
(procedure "create_attribute" line 16)
::xo::db::Object::slot::object_id ::xo::db::Attribute->create_attribute
invoked from within
"my create_attribute"
(procedure "init" line 8)
invoked from within
"::xotcl::next"
(procedure "init" line 2)
::xo::db::Object::slot::object_id ::xotcl::Slot::Optimizer->init
::xo::db::Attribute ::xotcl::Class->create
invoked from within
"::xo::db::Attribute create ::xo::db::Object::slot::object_id \
-set min_n_values 1 \
-set max_n_values 1 \
-set manager ::xo::db::Object::slot::obj..."
invoked from within
"ns_ictl update"

also errors at this point;
[05/Dec/2007:17:40:08][27003.3086079680][-main-] Notice: CREATE ATT ::xo::db::sql::acs_attribute create_attribute -object_type apm_package -attribute_name instance_name -datatype string -pretty_name MESSAGE KEY MISSING: 'xo.Package-instance_name' -min_n_values 1 -max_n_values 1
[05/Dec/2007:17:40:08][27003.3086079680][-main-] Error: Ns_PgExec: result status: 7 message: ERROR: invalid input syntax for integer: "null
"

[05/Dec/2007:17:40:08][27003.3086079680][-main-] Error: Error sourcing /usr/local/acs/oacs_test/packages/xotcl-core/tcl/06-package-procs.tcl:
Database operation "0or1row" failed
(exception ERROR, "ERROR: invalid input syntax for integer: "null
"
")
during '::xo::Package slots'
::xo::Package ::xotcl::Object->configure
::xo::PackageMgr ::xotcl::Class->create
invoked from within
"PackageMgr create ::xo::Package -superclass ::xo::db::Object -table_name apm_packages -id_column package_id -object_type apm_package -package_key a..."
(in namespace eval "::xo" script line 93)
invoked from within
"namespace eval ::xo {
#
# Meta-Class for Application Package Classes
#

Class create ::xo::PackageMgr \
-superclass ::xo::db::Class \
..."
(file "/usr/local/acs/oacs_test/packages/xotcl-core/tcl/06-package-procs.tcl" line 9)

From the table below I'm assuming the problem is with sort_order not being set as sort_order defaults to null from the info in the xotcl-core api-docs for ::xo::db::sql::acs_attribute proc create_attribute

oacs_532_81=# \d acs_attributes
Table "public.acs_attributes"
Column | Type | Modifiers
----------------+------------------------+--------------------------------------------
attribute_id | integer | not null
object_type | character varying(100) | not null
table_name | character varying(30) |
attribute_name | character varying(100) | not null
pretty_name | character varying(100) | not null
pretty_plural | character varying(100) |
sort_order | integer | not null
datatype | character varying(50) | not null
default_value | text |
min_n_values | integer | not null default 1
max_n_values | integer | not null default 1
storage | character varying(13) | default 'type_specific'::character varying
static_p | boolean | default false
column_name | character varying(30) |
Indexes:
"acs_attributes_attribute_id_pk" PRIMARY KEY, btree (attribute_id)
"acs_attributes_attr_name_un" UNIQUE, btree (attribute_name, object_type)
"acs_attributes_pretty_name_un" UNIQUE, btree (pretty_name, object_type)
"acs_attributes_sort_order_un" UNIQUE, btree (attribute_id, sort_order)
"acs_attrs_datatype_idx" btree (datatype)
"acs_attrs_obj_type_idx" btree (object_type)
"acs_attrs_tbl_name_idx" btree (table_name)
Check constraints:
"acs_attributes_max_n_values_ck" CHECK (max_n_values >= 0)
"acs_attributes_min_n_values_ck" CHECK (min_n_values >= 0)
"acs_attributes_n_values_ck" CHECK (min_n_values <= max_n_values)
"acs_attributes_storage_ck" CHECK ("storage"::text = 'type_specific'::text OR "storage"::text = 'generic'::text)
Foreign-key constraints:
"acs_attributes_datatype_fk" FOREIGN KEY (datatype) REFERENCES acs_datatypes(datatype)
"acs_attributes_object_type_fk" FOREIGN KEY (object_type) REFERENCES acs_object_types(object_type)
"acs_attrs_obj_type_tbl_name_fk" FOREIGN KEY (object_type, table_name) REFERENCES acs_object_type_tables(object_type, table_name)

Other packages seem to have installed fine.
5.3.2 versions of xotcl-code and xowiki have installed fine on a parallel system with the same oacs core.

cheers
Matthew

i have just now set up a fresh version of openacs 5.3.2 (from the tar file) and installed the head version of xotcl-core and xowiki, it installs nicely and runs the regression test. i have tried with 8.2.0, since i have no 8.1 version lying around anymore.

What is different on your "parallel systems", where you installed successfully? my strongest guess is that there is something special with your configuration.

some observations from your reports:
a) what is the strange quote in 'Ns_PgExec: result st"tus: 7 message: '??
b) in the "ERROR: invalid input syntax for integer:" there is a line break between null and the double quote
c) you did not report back the SQL statement causing the error, as i requested. It should look like:

CONTEXT:  SQL statement "INSERT INTO acs_attributes (attribute_id, object_type, table_name, column_name, attribute_name, pretty_name, pretty_plural, sort_order, datatype, default_value, min_n_values, max_n_values, storage, static_p) values ( $1 ,  $2 ,  $3 ,  $4 ,  $5 ,  $6 ,  $7 ,  $8 ,  $9 ,  $10 ,  $11 ,  $12 ,  $13 ,  $14 )"
PL/pgSQL function "acs_attribute__create_attribute" line 30 at SQL statement
): '
select acs_attribute__create_attribute('acs_object', 'object_id', 'integer', 'Object ID', null, null, null, null, '1', '1', null, 'type_specific', 'f'
)

I have the suspicion that there might be line-breaks in the passed arguments of the statement as well.

Can it be, that you are working on windows, an that some crlf line-terminators causing the problem?

What do you see, if you run

::xo::db::sql::acs_attribute get_function_args acs_attribute create_attribute
from ds/shell?

Hi Gustaf,

Thanks for the reply. You are correct that I'm using windows. I've been de-installing and re-installed the vanilla system and getting some weirdness on just a straight import of xowiki export to 60.0.2, so something else must be going on.

I'm going to re-set up my VMware installation and try again. I suspect that will give more consistent results.

Thanks for your help.

Matthew

Hi Matthew,

since you did not answer my above questions, I assume that my suspicion with line-breaks in the SQL statement was right, and that the provided command returns indeed line breaks for the default values. I have altered the function to strip cr and lf from the default values. Please, get a new version of xotcl-core from cvs head and check if you experience some differences.

Just in case: the version of xowiki in openacs-5-3 works with the version of xotcl-core from that branch, and that the version of xowiki in cvs head works with the version from xotcl-core from cvs head. don't mix these.

-gustaf neumann

Gustaf,

This is interesting. Are the defaults maybe provided (by the user) as a list using curly braces? I ran into a bug in some code of my own where I provided a nice looking list, like a list of proc args. It was essentially a name/value list. But when I tried to do a match, it didn't work. Obviously Tcl includes following newlines in some cases. The fix was to not provide such a nice list, but it looks like your idea is to strip off additional whitespace.

i doubt, that this is the same problem. xotcl-core (in cvs head) parses the default values for stored procedures from the source code of the stored procedures obtained from the PostgreSQL system catalogs. In a pure unix environment, this works fine; i would also assume it works ok in a pure windows environment. My guess was that in a mixed environment (running unix, but sourcing the crlf files) there might be a problem (null is often used as a default value, specified at the end of a line). Note that this effects only default values for stored procedures, not other default values (e.g. for non-positional arguments).

When reading crlf line terminated files in a unix environment, problems are not unlikely. In Tcl in general, input translation with "auto" translates all incoming cr, lf or crlf into a singe newline character. This should be fine. However, the SQL files are not read by Tcl, and might be a problem. But this was just a guess, no idea, if it was right.

Hi Gustaf,

Thanks for the correction uploads. I was going to try the pure unix system (away from windows) before answering those questions as your guess looked very reasonable with the extra "'s in the error message. In trying to reset my system I've crashed my laptop so am delayed by re-installation hassles and adding a linux partition (and finding a distribution that recognises my graphics card!)

Thanks for the uploads - hope to try them later today.

Cheers
Matthew