Forum OpenACS Development: RDF

Collapse
1: RDF
Posted by John Cusack on
I need some information for my project. Does OpenACS supports for RDF storage and querying?. if the answer is yes. then can someone tell me how it is going to be?
Collapse
2: Re: RDF Support in OpenACS (response to 1)
Posted by Jeff Davis on
Not specifically no, we do use tDOM for our XML needs and you could parse an RDF XML serialization with it but there is no direct support on OpenACS for RDF things.

You could also look at the XOTcl RDF parser (which should work in AOLServer4) which I think is quite good.

Collapse
3: Re: RDF (response to 1)
Posted by Nick Carroll on
What is the latest with RDF in OpenACS?

I know of two XOTcl and Redland RDF Application Framework. Redland RDF has bindings for TCL. I haven't used either, but would like to know of anybody's experiences with RDF before committing to a solution.

Collapse
4: Re: RDF (response to 1)
Posted by Nick Carroll on
I've discovered this thread on XOTcl. I apologise for not following XOTcl integration before. Could I please have an update on this work? Will it be integrated into OpenACS? If so, when will it be formally adopted by OpenACS and in which version? I need this information to plan ahead for a dotfolio application.

Thanks,
Nick.

Collapse
5: Re: RDF (response to 1)
Posted by Malte Sussdorff on
There is no specific need so far within OpenACS to use XOTcl, but if someone gives a good reason (in a TIP) sure we would adapt it (as we adapted TCLLIB from 5.3 onward).
Collapse
6: Re: RDF (response to 1)
Posted by Nima Mazloumi on
I think Gustaf wouldn't mind XoTCL being integrated. I asked him during the conference in Madrid. I think we should integrate that with OpenACS and let the developer choose wether to use an object oriented or procedural concept for developing. It would definetly help Vienna to return stuff back to cvs.
Collapse
7: Re: RDF (response to 1)
Posted by Dave Bauer on
Is XoTcl available for installation from most linux distributions? I just would hesitate to add additional complexity to the OpenACS install.
Collapse
8: Re: RDF (response to 1)
Posted by Nima Mazloumi on
XoTcl was tested on major distributions and the source should compile on linux without any problem.
Collapse
9: Re: RDF (response to 1)
Posted by Dave Bauer on
Nima, right, but the fact that is must be compiled, is an additional step, right?
Collapse
10: Re: RDF (response to 1)
Posted by Nima Mazloumi on
All you need are the xotcl tgz and the tcl thread extension from there for:

1. XoTcl you do

./configure --prefix=${AOLSERVER_INST_DIR} --enable-threads --without-tk --without-gdbm
--with-tcl=${AOLSERVER_INST_DIR}/lib
make
make install-aol

2. For thread you do in unix sub folder:

./configure --enable-threads --with-aolserver=$AOLSERVER --prefix=$AOLSERVER --exec-prefix=$AOLSERVER

make
cp libthread2.6.1.so ${AOLSERVER_INST_DIR}/bin

3. Finally you add this to the config file:
ns_param thread ${bindir}/libthread.so

That's it.

Thanks to Patrick Rebscher for figuring that out.

Collapse
11: Re: RDF (response to 1)
Posted by Dave Bauer on
Nima,

I am not saying its "difficult" or I don't know how to compile XoTcl, but that it should not be necessary. Unless there is a compelling reason to add complexity to the install, I don't think we should add an additional requirement.

(note: this opinion is subject to change, I just haven't see anything yet to change it)

Collapse
12: Re: RDF (response to 1)
Posted by Nima Mazloumi on
the main reason I see is
- that it would allow developer to decide how they want to code. I personally prefer object-orientation.
- several great stuff developed in Vienna would find their way to HEAD like
- throttle check for limiting access for IP addresses when users use download managers
- server benchmark to compare different installations of OpenACS and dotLRN
- and many more
Collapse
13: Re: RDF (response to 1)
Posted by Dave Bauer on
There isn;t any reason an optional package can't take advantage of XoTcl right now! I am just hesitant to _require_ XoTcl for a base install of OpenACS. The two options can coexist.

I hope the folks from Vienna will share their code. If that happens then it would be one incentive to investigate adopting XoTcl for OpenACS core. Without any existing code though, I don't see adding a new requirement as very compelling.

Also there is a HUGE body of existing code that does not require XoTcl and it seems to be working reasonably well :) I think just taking the ideas of encapsulation and well defined procedures that do what you expect from object oriented programming woudl go a long way. That is, good design is what will make OpenACS a good platform. The programming style is secondary at best.

Also, now that I am on a roll, I think if someone wanted to make a big contribution to OpenACS, writing automated tests would be of a much greater benefit than trying to inject a new programming style into OpenACS. Making sure the code we already have is the best we can do is very important.

Collapse
14: xotcl support for oacs (response to 1)
Posted by Gustaf Neumann on
Malte: There is no specific need so far within OpenACS to use XOTcl
certainly, oacs works fine without xotcl; so all needs for xotcl are by definition soft needs or installation needs. i certainly believe that the oacs community can benefit from an oo language like xotcl, since it (a) can make your code more readable and shorter and (b) can improve the performance in many places significantly. xotcl has various functionalities implemented in C, which are implemented in many places in oacs in tcl. For example, the invocation overhead of an ad_proc can be reduced by a factor of six by replacing an ad_proc by an xotcl method.

Dave: Is XoTcl available for installation from most linux distributions?
xotcl is part of the active state core distribution (which is the most significant tcl distribution), it is part of tcl aqua BI and part of the Apple developer tools for Mac OS X tiger.

Dave: I just would hesitate to add additional complexity to the OpenACS install.
xotcl is as easy to install as all other c modules for the aol-server. The best thing for a source distribution of open acs would be certainly a single tar file including a version of the aolserver and the set of c modules (including xotcl and libthread) and a toplevel configure-file, where one can do a single (configure; make; make install) cycle.

Nima: All you need are the xotcl tgz and the tcl thread extension..
yes and no. the thead library is used currently only for the request monitor and throttle packages, which is the most probably the best monitor for oacs around (it does not use ns_server and therefore it does not crash under high load). but strictly speaking, xotcl does not need the tcl thread extension.

For real work, you will like some more support: the xotcl objects, classes and methods should show up in the api-browser, etc.; most probably you will like to have the following things:

Both patch files do not harm when xotcl is not installed. The README file for the installation is here http://media.wu-wien.ac.at/download/README.xotcl-oacs

Dave: Also there is a HUGE body of existing code that does not require XoTcl and it seems to be working reasonably well :) I think just taking the ideas of encapsulation and well defined procedures that do what you expect from object oriented programming would go a long way
OACS is implemented already in various places in some latent OO style, which is implemented by hand, not always consistently. Doing this with some programming language support can certainly help. from some postings in the forums i get the impression that there is a certain oo-phobia, maybe from the days of the split btwn. ACS and OpenACS. doing the right level of encalsulation between object and the database is under performance considerations not an easy task, doing it in the oacs-style seems a good compromise. therefore i am not suggesting any radical changes in the oacs architecture. xotcl can help to realize the current architecture in a better and faster way.

Certainly, we can load xotcl fine into oacs, as we did it since four years. For other installations, the use of our xotcl modules is more hassle; many e.g. dotlrn admins are not the best system guys and are somewhat reluctant to go to the installation guides sketched above. Easing these steps would make it easier for us to share code. For the performance changes, certainly the core functionality used all over the place benefits most. It is not so easy to provide a module, that achives this.

Over the last days, i have implemented some improvements for the calendar package (based on the code in dotlrn 2.1.3b) in xotcl. One can now dowload one or many calendars in the ical format (e.g. to mozilla calendar or iCal on Mac OS X), or upload ical calendars to openacs, or share calendars via webdav (mozilla calendar/sunbird/XUL-files for Firefox or Thunderbird only). It works up to some architetural limits quite well. I am not completely through with it, but i can package it quite soon...

Should we maintain two versions of calendar, one with ical based on xotcl, the other one without xotcl? this does not sound very reasonable. xotcl is an ordinary tcl extension. packaging it with openacs is as complicated as nscache or nssha1 or whatever. If it is installed together with the other c-components, it is the least hassle.

Collapse
Posted by Andrew Piskorski on
Heck, even if there was no other reason for doing so at all, OpenACS should be made to integrate easily with XOTcl out-of-the-box just to encourage further contributions from Gustaf to OpenACS. :)
Collapse
15: Make XOTCL standard (response to 1)
Posted by Malte Sussdorff on
Gustaf, taking your posting, please make a TIP so we adopt xotcl along with the already approved TCLLIB in 5.3 and the 5.3 documentation. I think you have provided enough arguments for the OCT to make a decision on whether to adopt it (and make use of your calendar).

One major issue though is the additional learning curve that comes with xoTCL enabled code. I personally don't mind and I think this might even make OpenACS yet more attractive to other developers, but nevertheless, it is more to learn and OpenACS is already difficult to pick up in the first place.

When you write the TIP, maybe you can share some more of your enhancements or ideas for enhancement where it would make sense to switch to xoTCL instead or in addition to the functionality we use within OpenACS (after all, we are somewhat oo-ish in our code anyway).

Collapse
17: Re: RDF (response to 1)
Posted by Nima Mazloumi on
Gustaf,

I am happy you took over the discussion. Since we have only started working with XoTCL I am not the right person to praise the advantages of XoTCL. Something I can definetly confirm is the much more increased readable code.

I think what we would need is our note tutorial implemented with XoTCL to show where the differences take place. Would it be possible to write such a tutorial?

Collapse
18: Re: RDF (response to 1)
Posted by Dave Bauer on
Gustaf,

Thank you for this post! I learned quite a bit. I think you are correct that XoTcl could improve code in OpenACS, I just worry about a transition, and if there is a committment to actually rewrite the code in a new style.

That said if we have a good technical proposal to adopt XoTcl I won't object to it. We just need good documentation for installation.

Collapse
19: Re: RDF (response to 1)
Posted by Gustaf Neumann on
yep, i will check whats needed for a TIP.

The notes tutorial is fairly simple, so by taking a
conservative approach, the difference is very little.
For an xotcl version, see:
http://media.wu-wien.ac.at/download/myfirstpackage-0.2.apm

Only the tcl/*-procs file, www/note-*.tcl and tcl/test/*tcl
changed relative to the origianl. The expamples does
not use classes, but simply an object similar to a
namespace. on larger and more complex cases, one can
certainly benefit more from oo.

concerning RDF: our RDF-implementation is quite good
for small to medium applications. we did some benchmarks
some days ago, the performance was better than most other
RDF-packages. For larger apps, redland has it advantages,
since it constists of well desigend c code. Uwe developed
an xotcl linkage to Redland (based on some older versions).
it should be ready in the not to far future....

all the best
-gustaf

Collapse
20: Re: RDF (response to 1)
Posted by Nima Mazloumi on
Thank you Gustaf. Is there also a package (simple) that fully benefits from XoTCL's functionalities available somewhere?
Collapse
21: Re: RDF (response to 1)
Posted by Andrew Grumet on
Nima, I think your xotcl installation instructions have a bug. The final make should read

make install-aol

If I do a regular "make install", it doesn't work. If this is true, could you edit your post, so the next person to try doesn't have the same problem?

Thanks!

Collapse
22: Re: RDF (response to 1)
Posted by Andrew Grumet on
I've installed the various packages, read the tutorial and reviewed the code in the new request monitor. I'm far from grok-level understanding but am fairly enthusiastic about what I'm seeing.

I believe there are definitely places in standard oacs that could benefit from this style of programming. There are also potential pitfalls like object-relational mapping which most OO roads trend towards.

I agree pretty strongly with a lot of the statements above, particularly Andy P's comment and Dave B's point that a strong proposal would identify where to apply it. The ideal candidates would be places that could really be improved, not just re-coded. For example site node resolution.

I will comment on the TIP after I've had a bit more time to think and tweak.

Collapse
23: Re: RDF (response to 1)
Posted by Andrew Grumet on
The ideal candidates would be places that could really be
improved, not just re-coded. For example site node resolution.

I nearly forgot: The new request monitor is itself a significant contribution, particularly in light of the fact that the old monitoring.tcl page which you really can't use on medium and high traffic servers. Right now this is the only real option.

Collapse
24: Re: RDF (response to 1)
Posted by Nima Mazloumi on
I cannot edit my posts.
Collapse
25: Re: RDF (response to 1)
Posted by Andrew Grumet on
Okay, I've edited it.
Hi Folks,

here is an improved the xotcl packages and added functionality for a generic handling of cr-items with classes and subclasses. Essentially, one can generate the data model (object-type, views, tables, queries) by defining an instance of CrClass. The following two statements (from xotcl-note/tcl/note-data-model-procs.tcl) generate a CrNote and a subclass of CrNote named CrSubNote (which has one attribute more) with its relevant tables during startup, if these do not exist already.

::Generic::CrClass create CrNote \
    -pretty_name "Note" -pretty_plural "Notes" \
    -table_name "generic_cr_note" -id_column "note_id" 
  
::Generic::CrClass create CrSubNote -superclass CrNote \
    -pretty_name "SubNote" -pretty_plural "SubNotes" \
    -table_name "generic_cr_subnote" -id_column "subnote_id" \
    -sql_attributes {{number integer}}
The www/index.tcl file contains in essence the following fragment, through which a listing of the existing instances of CrNote and CrSubNote and edit buttons can be obtained.
::Generic::List index \
    -object_type CrNote \
    -with_subtypes true \
    -object_types {CrNote CrSubNote} \
    -fields {
      EDIT {}
      title {label "Title"}
      object_type {label "Object Type"}
      DELETE {}
    }

index generate -order_by title
The editing magic in www/edit.tcl using ad_form is hidden by:
::Generic::Form form1 \
    -object_type CrNote \
    -fields {
      {item_id:key}
      {title:text {label Title}}
      {description:text,optional {label Description}}
      {number:text {label Number}}
      {text:richtext(richtext),optional {label Content}}
    }

form1 generate
Some admin functionality is provided in the admin/ directory.

Alltogether, these simple classes can be used to

  • create and delete the schema
  • add, edit, delete instances
  • provide a polymorphic listing (containing content repository items of different types) where the appropriate editing forms are choosen based on the object type)
i haved placed the ::Generic:: classes into xotcl-core (in tcl/generic-procs.tcl), since these have to be loaded before the application package (here: xotcl-note).

This package serves mostly as programming example but might be useful for simple kind of applications already. i have developed and tested it with dotlrn 2.1.3b. It can be dowloaded from: http://media.wu-wien.ac.at/download/xotcl-note-0.2.apm

Please, obtain as well the updated version of xotcl-core for this. If you have not installed xotcl yet, but you want to try this out, please read: http://media.wu-wien.ac.at/download/README-xotcl-oacs. Below is as well an improved version of the request monitor.

Collapse
27: Re: RDF (response to 1)
Posted by Jim Lynch on
One thing that requires some care, is the performance associated with reading/writing from the database... look back in the history of arsdigita community system to see why they adopted their "halfway-oop" style.

I don't remember all the details, but what I do recall is that if an object does things like automatically write itself back to the database as a side effect of, say, doing a set_property on it, it's possible to end up with more actual database hits than you want, something that could seriously degrade performance.

Having said that, Welcome Gustaf :) I had asked at an OCT meeting some years back to deprecate versions of aolserver where "package require ..." did not work properly (would be versions before the first aolserver4), and begun work with you to clean up the build process on tcl (which Jeff Hobbs eventually streamlined and took over). I've been waiting for this collaboration of openacs and xotcl for quite awhile; glad to see it's starting!

Collapse
28: Re: RDF (response to 1)
Posted by Jim Lynch on
Gustaf,

P.S: You should start your work on openacs version 5.2, since it is that version that completes the deprecation of old, unsupported aolservers that don't have working "package require ..." in it.

Collapse
29: Re: RDF (response to 1)
Posted by Gustaf Neumann on
Hi Jim,

i am aware of the performance issues when oo meets sql.
the particular hard problem is, where one implements
persistent objects via sql and does joins on the object
level. This would be certainly be no good idea when one
wants to use the oacs framework. There are already many
places in oacs, where a lot of small queries
(returning one tuple) originating from the tcl-api
are issued, where certainly a few "larger queries"
would be much more efficient.

The simple package does try to implement a persisten
object layer, but implements rather a class mapping.
The classes knows how to fetch, delete, update
a single item, or to retrieve a set of instances via
a single query. Currently, the retrieved values are
kept on the class level, there are no instances created
(this might change in the future, there are already to
many global variables for the communication with the
templates around).

Concerning package require: i am not talking about the
tcl package require, but about controling the order
in which oacs packages are sourced (say, an apm
package require). assume, that xotcl-core is needed
for e.g. mypackage. I have solved this so far through
the patch mentioned above, that causes to load xotcl-core
early in the bootstrap process. The same situation arises
as well, when i define a package xotcl-generic-class,
which will be needed for "someotherpackage".

I am not aware, that something has
changed in this respect in 5.2.

Concerning development in 5.2: It seems to me that
the core people are not happy about non-core
development in 5.2 before it is actually released:
https://openacs.org/forums/message-view?message_id=316194
This is a petty, since there a nice packages like the
assessment only available in 5.2; others are newer in 5.1:
https://openacs.org/forums/message-view?message_id=315991
This version-confusion is not doing any good.

Btw., i have just updated the packages above, such they
work with 5.2 now (the title attribute became ambiguous
due to the oacs data model change). I have left the version
numbers unchanged, such that the links in my note above
still point to the actual version.