Forum OpenACS Q&A: Some tips about XoTcl development best practices

Hello everyone,

in a project of mine there is some logic which would be better handled by usign objects.

I have looked around into available xotcl code to put down a worflow which could be easy to understand and mantain and also that could save me some time.

From other platforms, as Django[1] (which I know only in its basics), there exist a nice feature that allows you to define the data model in python, then let the software build the sql statements accordingly, even when upgrading/deleting. Some trace of this kind of feature I have found in the ::xo::db::sql::acs_attribute api, but I lack some expertise and examples to understand it.

I wanted to know:
- what is the best practice in xotcl for using objects that should be persisted into the db?
- is there some feature in OpenAcs that could avoid me to use plain sql all the time for tipical Create, Read, Update, and Destroy use cases?

I also have seen that we have an ::xo::dc interface that shoud be the xotcl (and preferrable) interface to the db, if I am not mistaken. Why can't I see its available methods in the api browser? Is it the right class to call queries in xotcl?

With some pointers for this, I could offer myself for writing a tutorial.

[1] https://www.djangoproject.com

Collapse
Posted by Gustaf Neumann on
Hi Antonio,

have you seen the tutorial slide-sets for xotcl-core http://alice.wu-wien.ac.at:8000/s5-xotcl-core-tutorial/slides
and xowiki http://alice.wu-wien.ac.at:8000/s5-xowiki-tutorial/slides

The xotcl-core tutorial shows, how to autocreate the OpenACS data-model based on class definitions.

-gn

Collapse
Posted by Antonio Pisano on
The tutorial is great, thanks! Would you mind linking this pages in http://www.openacs.org/xowiki/ ? I don't have permissions for that...

What about the xotcl equivalents for db_1row, db_list etc? My guess is ::xo::db::Driver, is that right?

Collapse
Posted by Antonio Pisano on
Another thing I use a lot for big-size tables is pagination by template::list construct in procedural api (defining page_query, rows_per_page etc).

Is there an equivalent in xotcl for paginated results?

Collapse
Posted by Antonio Pisano on
In the meantime I am going through the tutorial and I am facing an issue:

when pasting this code from tutorial into a file in the tcl folder of my package, I am getting an error

::xo::db::Class create ::demo::Person ­-superclass ::xo::db::Object ­-slots {
::xo::db::Attribute create name ­column_name pname
::xo::db::Attribute create age ­default 0 ­datatype integer
::xo::db::Attribute create projects ­default {} ­multivalued true
}

the error is

wrong # args: should be "init"
::demo::Person ::xo::db::Class->init
::demo::Person ::xotcl::Object->residualargs
::xo::db::Class ::xotcl::Class->create

What could that be?

Collapse
Posted by Antonio Pisano on
Ok, about the last issue it was something weird happening because of copy/paste, forget about it.

Related to my real application, I have a column which should be a postGIS GEOGRAPHY data type (a gps location). Can I define such column from xotcl? This code

::xo::db::Class create ::cic::Location -superclass ::xo::db::Object -slots {
foreach slot {name street number zone region country} {
::xo::db::Attribute create $slot
}
::xo::db::Attribute create location ­datatype GEOGRAPHY(POINT,4326)
}

complains about "datatype GEOGRAPHY(POINT,4326)" not being defined.

Can I define new datatypes?

Collapse
Posted by Antonio Pisano on
Another thing I am interested into is the right workflow when upgrading the data model: I have tried defining a class like this

::xo::db::Class create ::cic::Location -superclass ::xo::db::Object -slots {
::xo::db::Attribute create name
::xo::db::Attribute create street
::xo::db::Attribute create number
::xo::db::Attribute create zone
::xo::db::Attribute create region
}

Then I have modified the definition so it could include other columns like this

::xo::db::Class create ::cic::Location -superclass ::xo::db::Object -slots {
::xo::db::Attribute create name
::xo::db::Attribute create street
::xo::db::Attribute create number
::xo::db::Attribute create zone
::xo::db::Attribute create region
::xo::db::Attribute create country
::xo::db::Attribute create category -datatype integer -references categories(category_id)
}

The definition seems correct, and issuing the command in the shell returns the previously created object type, but the new fields are not added to the data model. What is the right way to do that?

Collapse
Posted by Gustaf Neumann on
Dear Antonio, many questions... here is the list of questions and answers:
... I also have seen that we have an ::xo::dc interface...
The ::xo::dc interface is quite new and allows one script to talk ot different backends of potentially different drivers. "dc" stands for "database context" (similar to the connection context ::xo::cc) and supports the use different drivers (such as e.g. nsdb or nsdbi under naviserver) as well as different sql dialects on the database backends (e.g. postgres via oracle). See e.g. the function ::xo::db::select_driver

... extending database definitions on the fly (added new fields, but these are not visible)
the xo::db interface supports creation and deletion of the datamodel, but does not permform any kind of migration (it does not add or delete attributes on the fly). In case, you have no data in the table, drop the table manually (it will be recreated on next startup) or add the attributes manually to the database. xo::db adds the acs-meta-data on the fly i f possible.

... how to extend the datatypes (e.g. from PostGIS: GEOGRAPHY(POINT,4326))
the OpenACS meta-data model complains here: OpenACS has a type-name abstraction managed via acs_datatypes (see e.g. select * from acs_datatypes). acs_attributes has a foreign key to thes table, therefore one has to make an entry to this table as well. See some discussions about acs_datatypes in [1] or [2].

... pagination in xo*
various of the functions in the xo framework provide pagination support (e.g. get_instances_from_db, instance_select_query, various includelets), one can certainly as well mix xo*-functionality with template::list, although there is no deep integration. the closest to template::list is xo::Table, which allows in combination with the instance_select_query pagination.

... link tutorial to wiki ... no permissions
Are you sure? it is the purpose of the wiki offer liberal write access. anyhow, i've added a link to the tutorial to the xotcl-core page.
Hope this helps!
all the best
-gustaf neumann

[1] https://openacs.org/forums/message-view?message_id=81377
[2] https://openacs.org/forums/message-view?message_id=34850

Collapse
Posted by Antonio Pisano on
Many thanks for your answers! Now from them come some others, hope you have patience 😊:

Why is it sometimes impossible to view documentation of certain xotcl classes/methods? Two examples:
- ::xo::db::select_driver I cannot find either in http://www.openacs.org/api-doc/ and the xotcl doc http://www.openacs.org/xotcl/
- ::xo::db::Attribute, same thing. The class is of course working, and I can find documentation by googling on openacs.org (https://openacs.org/xotcl/show-object?object=%3A%3Axo%3A%3Adb%3A%3AAttribute&show_source=1&show_methods=2&show_variables=1), but trying to find it through the api-doc or by navigating xotcl doc manually is impossible from what I can see...

Also, some object oriented api is visible in the api-doc page and some is not. From what does this depend?

Wouldn't it be super convenient to let the system automatically change tables in the sql when new attributes are added in the code?
I dare thinking that at least for added columns this should be enough safe... Deletion could be a different matter, but we could at least intercept the most common use case and avoid people a lot of sql mantaining.

About the wiki: I swear, the main page is restricted for editing, every other page is allowed.

Collapse
Posted by Gustaf Neumann on
what do you mean by "view documentation"? on your local installation? on OpenACS.org? One can certainly view ::xo::db::select_driver in the api-browser [1]. can it be that you made a typo?

::xo::db::Attribute was another issue (related to xotcl2): the class is not from the xotcl class hierarchy, but from the nx hierarchy). I've adjusted the code the the newest version of nx, and installed this version on OpenACS.org.
Now it is visible (that should be a transitional issue).

Concerning changing tables: i tend to agree that adding columns is sage, but altering or deleting is not. Also when adding columns to existing tables, you might have some problems with existing tuples. also doing this portable (PostgreSQL and oracle) is some more work since one has to query table properties and compare these with then intended changes. During development, it is usually not a big problem to drop the fresh tables frequently. Changing the tables for existing applications happens seldom enough and requires usually an update script.

yes, it is true, the index page is protected on OpenACS.org, but its content is mostly generated. It is sufficient to edit the page containing the xotcl info, here xotcl-core.

all the best
-g

[1] https://openacs.org/api-doc/proc-view?proc=xo%3a%3adb%3a%3aselect_driver&source_p=1

Collapse
Posted by Antonio Pisano on
Ok, I've found it, didn't look for xo::db::select_driver without trailing semicolons, sorry.

I've just refreshed my installation to freshest cvs (upgrading everything from apm) and now /xotcl page gives me a complain:

unable to dispatch sub-method "superclasses" of ::nx::BootStrapVariableSlot info; valid are: info children, info class, info configure, info configure defaultmethod, info configure parameters, info configure syntax, info filter guard, info filter methods, info has mixin, info has namespace, info has type, info heritage, info info, info instances, info lookup configure parameters, info lookup configure syntax, info lookup filter, info lookup method, info lookup methods, info lookup slots, info lookup variables, info method args, info method body, info method definition, info method definitionhandle, info method exists, info method handle, info method origin, info method parameters, info method postcondition, info method precondition, info method registrationhandle, info method returns, info method submethods, info method syntax, info method type, info methods, info mixin classes, info mixin guard, info mixinof, info name, info object filter guard, info object filter methods, info object method args, info object method body, info object method definition, info object method definitionhandle, info object method exists, info object method handle, info object method origin, info object method parameters, info object method postcondition, info object method precondition, info object method registrationhandle, info object method returns, info object method submethods, info object method syntax, info object method type, info object methods, info object mixin classes, info object mixin guard, info object slots, info object variables, info parameter default, info parameter name, info parameter syntax, info parameter type, info parent, info precedence, info slots, info subclass, info superclass, info variable definition, info variable name, info variable parameter, info variables, info vars
while executing
"error "unable to dispatch sub-method $ref; valid are: [join [lsort $valid] {, }]""
(procedure "unknown" line 11)
::nx::Class::slot::__info ::nx::EnsembleObject->unknown
::nx::BootStrapVariableSlot ::nx::Class->info
invoked from within
"$o info superclasses"
(procedure "::xo::getObjectProperty" line 45)
invoked from within
"::xo::getObjectProperty $cl $key"
(procedure "info_classes" line 4)
invoked from within
"info_classes $cl superclass"
("foreach" body line 6)
invoked from within
"foreach cl [lsort $classes] {
if {!$all_classes && [string match "::xotcl::*" $cl]} continue

append output "

  • [::xotcl::api..."
    ("uplevel" body line 60)
    invoked from within
    "uplevel {
    ad_page_contract {
    Show classes defined in the connection threads
  • Seems like somehing in nx framework is missing, if I am not wrong. ::xo::db::Attribute was working, so some nx must be there already, but maybe there is something else I need to enable...
    Collapse
    Posted by Gustaf Neumann on
    Antonio, please get the newest version of nsf/nx from git and recompile+install it to get the xotcl class browser with the newest version of xotcl-core working on your local instance (e.g. use the HEAD option for xotcl in install-ns.sh). Most of the other stuff is robust against the latest changes, but the detailed introspection is not).

    The head version of nsf/nx/xotcl2 will be released quite soon.

    Collapse
    Posted by Antonio Pisano on
    Ok, fine with that.

    There is something else that I still can't figure:
    In apy doc, if i look at ::xo::db::sql::acs_attribute I get in the source code something that seems it has been auto-generated.

    api-doc states that such method is defined in packages/xotcl-core/tcl/05-db-procs.tcl, but I can't grasp how: the code I can find there calls such proc, but the real definition seems to come from somewere else.

    Collapse
    Posted by Gustaf Neumann on
    xotcl-core generates at startup automatically interfaces to all stored procedures in the database. For every package (in the oracle terminology) an object in the ::xo::db::sql namespace is generated. By calling e.g.
    ::xo::db::sql::acs_attribute create_attribute
    one is calling actually in an postgres installation the stored procedure
    acs_attribute__create_attribute
    This interface is much faster than the usual OpenACS interface and allows to handle defaults for attribtues, give error messages about expected attributes, etc.

    btw, i've updated xotcl-core once more, it should be now more backwards compatible (not tested), so you might not have to update nsf/nx.

    all the best
    -gn