Forum OpenACS Development: RFC: PostgreSQL-specific packages

As mentioned in the tDOM thread I've done some work to enable ODP/DMOZ catalogs in an OpenACS instance. ODP/DMOZ data is processed by tDOM into an appropriate form for loading into postgresql database. However, there is an issue with the package in the sense that it exploits specific pgsql constructs, in particular the gist-based ltree module by Oleg Bartunov and Teodor Sigaev.

The ltree data structure is central in handling a text based hierarchy like the ODP/DMOZ. So, the question is what should be the policy for PostgreSQL-specific packages (not packages that are not ported to oracle but pgsql-specific in the sense that there are no "similar" oracle constructs). Should they reside in the contrib directory? What do you think?

Collapse
Posted by Don Baccus on
In contrib, yes.
Collapse
Posted by Neophytos Demetriou on
Thank you Don. Before my trip to Central Europe I had commit privileges to the whole tree. I do not know if I still do but one way or another I will submit the package to the project.

I have another package for maintaining RSS feeds. I know that there has been work on that direction but I'll post the details anyway when I'm back home. I've checked the other package a bit and we had solve the problem with different approaches -- I am using XSLT transformations in the package which is one more reason that tDOM is needed. One good thing about the package is that it adapts refresh intervals based how often feeds are updated.

Given this opportunity, I want to announce that I have worked on a new bookmarks package (much like Yahoo) and have experimented with a friend on a persistence layer. There are a couple of things that will need to be addressed for the last two packages, e.g. the bookmarks package is using xotcl and a different templating system. I do not want to start the discussion about these things until I'm back and settled but I'm just bringing everybody up to date. Other stuff for the curious that want to know, I had the chance to experiment with a friend is RBAC support. I have already mentioned the summarization code for OpenFTS in another thread.

Collapse
Posted by Don Baccus on
Why do we need a different templating system and packages written in a different language?

This would greatly increase the learning curve for newcomers and add to our maintenance load.

I'm not even sure we really want such things in our contrib section, to be honest.  I realize that you're exploring such things out due to personal interests but when it comes to a software suite the size of OpenACS 4, I don't see that anarchistic programming practices are a good idea.  You may have to be content to put up your own repository of custom packages if you prefer using tools that aren't part of the toolkit.

I've been working hard to *minimize* the variety of techniques and tools used in various packages in order to make things more understandable and simpler.

I'm curious as to what others may think.

Collapse
Posted by Neophytos Demetriou on
Just to make it clear: I did not post about these things here in order to suggest that OpenACS should include them in its list of packages. Only that the packages exist and that someone may take some useful ideas from these packages and incorporated them to the toolkit. But, let us have the discussion about these when I'm back to Cyprus if you do not mind.

As far as xotcl goes, xotcl is much more tcl than any other package that we are using. It is an *enhanced* tcl that empowers the procedural tcl. Again, I did not post to provoke the discussion now. Only to bring people up to date about my work. I would appreciate if we could have the discussion when I'm back home and settled.

Collapse
Posted by Neophytos Demetriou on
Don, the first two packages, i.e. dmoz and rss, are normal openacs packages.

<blockquote>>>>
</blockquote>
I don't see that anarchistic programming practices are a good idea.
<<<<

I would be very interested to hear your definition of "anarchistic programming practices". The software engineering community has well adopted design patterns as *the* concept for best practices in the field and xotcl provides tcl constructs for implementing design patterns.

As I already said, I am not advocating that OpenACS should follow this path but I would say quite the opposite, i.e. it is OpenACS that has way to many hacks and thus increasing the learning curve.

You are right when you say that the packages I mention in the end of my message, i.e. bookmarks and the persistence layer, were just some experiments of mine but as I clarified in my previous message, I were only bringing people up to date with my work.

<blockquote>>>>
</blockquote>
You may have to be content to put up your own repository of custom packages if you prefer using tools that aren't part of the toolkit.
<<<<

That could be an option that I have not considered yet. However, I would rather spend some time around here and when the time comes for such a repository, myself will do so in such a way that it won't conflict with my OpenACS developer attribute -- I will ask for suggestions from the community in what way that could be accomplished so that there would be no differences (but that's for *when/if* the time comes for such a repository).

I hope this is a more complete reply to your message. I would gladly clarify any mood points. Comments?

Collapse
Posted by Don Baccus on
The software engineering community has well adopted design patterns as *the* concept for best practices in the field and xotcl provides tcl constructs for implementing design patterns.
This is not true. Certain segments of the software engineering community have adopted design patterns as *the* concept for best practices, large segments have not.

Unless, of course, you don't consider those working on tools like the Linux kernel, PostgreSQL, Apache, and AOLserver software engineers.

Design patterns may turn out to be yet another passing fad among a fashionable set of software engineers and academics.

Regardless, we've adopted an abandoned piece of software that was not written with this paradigm in mind. Is this a great design or a design that couldn't be improved? In my opinion, no it's not and yes it could be. However it's a servicable design, much more so than the supposedly more pure and holy and object-oriented design of its successor, ACS Java implemented by True Believers in the design patterns paradigm.

You are right that OpenACS 4 already includes too many hacks, different code approaches which solve the same problem, no standard way of using the object-oriented datamodel, etc. All things we've inherited. If you read my posts to the forums - and I've posted a time or two, as I'm sure you're aware - you'll see a couple of common threads that winds through all my thinking.

The first is that, despite its flaws, I think the current design is servicable and I'm not personally interested in leading or being involved with a project to rewrite it. I've encouraged those who might - you, for instance - to consider starting up an alternative project and have suggested that we might even want to host it here at openacs.org as a research project that may or may not bear fruit in the future.

A second thread is that I constantly argue that we work to rationalize the use of the code base and datamodel and that we put a stop to various packages implementing ad hoc solutions that duplicate those that already exist and are servicable. A recent example is that I tossed out Ben's reimplementation of non-versionable objects that are used to provided URL content in file-storage. The content repository already had a perfectly servicable external link type that wasn't completely implemented by aD before ACS Tcl was dropped. I completed the implementation and got rid of Ben's parallel code. Another example is the growing consensus in the community behind the use of the ATS form builder to provide a consistent look and feel for forms, and a consistent coding paradigm to implement form handling pages. We've got a long ways to go, but surely adopting one standard way to build and handle forms is better than the situtation we inherited from aD, where ad hoc form handling along with the form builder appeared in various packages.

I could go on and on but certainly one of my major themes, perhaps the major theme of whatever level of direction I provide for the project, has been that we should scrap the use of redundant and usually inadequate approaches to various coding problems and adopt a single approach for use throughout the toolkit.

I describe your approach as "anarchist" because, rather than following that trend, you've written a package that uses a custom templating system rather than the perfectly servicable (though clearly imperfect) existing templating system. This automatically makes your new package uninteresting in the context of the OpenACS 4 toolkit.

As a research project, or a demonstration of how a rewrite or reimplementation of the toolkit might be an improvement over the existing system, it may very well be interesting indeed, but code written in a different programming language using a parallel templating system rather that the existing templating system we've adopted as an OpenACS 4 standard will not become part of the OpenACS 4 toolkit. Not as long as I'm in a leadership position, that is. On the other hand, a clean break with the past wouldn't bother me at all, as was done with ACS 4 vs. ACS 3, if there are folks that would like to do that rather than flog OpenACS 4 into shape.

BTW, xotcl used properly as an object-oriented extension of Tcl leads to code different enough than the Tcl code folks are used to reading that for all practical purposes it is a different language. In the same sense that C++ is a different language than C even though it's just an extension to C. I realize that quantitatively the difference between xotcl and tcl proper is much less than C++ vs. C - I am a language implementor by trade, after all - but qualitatively xotcl was developed to support a particular programming paradigm.

Collapse
Posted by Peter Marklund on
Don,
I have to concur with you that I am not too interested either in re-architecting OpenACS. I'm not sure that's what Neophytos suggests but I just wanted to stress that. The top priorities for OpenACS that come to my mind right now are:

1) Develop loads of out-of-the-box nice and useful applications on top of it. I'm looking at Bug Tracker as an ideal here. The classic UI and usability of the old school aD applications just don't cut it anymore.

2) Make OpenACS easier to get started with (documentation and installation etc.) and spread the word. Spreading the word involves going to conferences like the O'Reilley Open Source conference, sell the platform to your local Linux user group, arrange boot camps etc.

3) Give OpenACS the reputation of being a rock solid robust web toolkit with reliable releases so that more people will make the leap to it. With other words more testing (for example load testing) and bug fixing.

Collapse
Posted by Don Baccus on
I mentioned rewriting OpenACS because at one point Neophytos had talked about perhaps wanting to do so, keeping parts of the datamodel but having a persistence layer designed in from the beginning, using xotcl, etc etc.

I just want to make sure that he understands that if he (or anyone else) is interested in starting such a project, we might host it here, there might be others interested, etc.  But ... it would be a *separate* project altogether.

Collapse
Posted by Neophytos Demetriou on
A short reply and the rest when I'm back home:

There are several places that I do not agree with the xotcl case. However, I understand why Don is sceptical from the perspective of the code maintainer of a large software project. No suggestions of changing OpenACS though.

<blockquote>>>>
</blockquote>
I mentioned rewriting OpenACS because at one point Neophytos had talked about perhaps wanting to do so, keeping parts of the datamodel but having a persistence layer designed in from the beginning, using xotcl, etc etc.

I just want to make sure that he understands that if he (or anyone else) is interested in starting such a project, we might host it here, there might be others interested, etc.  But ... it would be a *separate* project altogether.
<<<<

Yes, Don and I, discussed this during last summer. I'm surely (still) interested in starting such a project. Yes, it could be hosted here (that was the idea back then) as a *separate* project if there are no serious objections.

However, I would like to avoid any conflicts with my OpenACS developer attribute (comments?) -- I'll continue developing for OpenACS as I have been doing for the past few months.

There's nothing solid in the sense of a whole product but some pieces are ready.

What do you think?