Forum OpenACS Q&A: Guidelines for new modules

Posted by Edmund Lian on 02/07/01 07:31 PM

I've been going through the OpenACS 3.x code to understand how I
should be structuring a module that I'm about to write. I have
several questions regarding how I should be dealing with PostgreSQL
and ACS...

1. I see that all the OpenACS data models are straight ports of the
Oracle versions, with minimal changes. For example, rather than using
the PostgreSQL boolean type, char(1) is still used. Doesnt' this
affect efficiency and remove some of the benefits of not using
Oracle? I thought I should redesign my data models to take advantage
of PostgreSQL...

2. Given that OpenACS is about to shift to the ACS v4.x base it would
be really nice to write for v4.x rather than the soon to be obsoleted
v3.x. What's everyone's advice regarding how I should proceed on
this? The stuff that I'd really like to use are the DB APIs, the
request processor, Karl's template system, and a few other bits.

3. Could somebody point me to archived discussion (if any) regarding
why we should not be optimizing for PostgreSQL rather than slavishly
remaining tied to Oracle's oddities like the lack of booleans, "" ==
NULL, etc.? The only two decent Open Source databases are PostgreSQL
and Interbase, and both are rather more SQL-92 compliant to begin
with. As an Open Source community, why do we care about remaining
compatible with Oracle? Is this not a case for a fork in the code?

2: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/07/01 09:59 PM

You've pretty much answered the question yourself when you mentioned forking. We decided we didn't want to fork at the time, though that was certainly discussed as a possibility. With the toolkit changing so rapidly at the time, we decided it was advantageous to be able to quickly import new aD code.

char(1) vs. boolean isn't really much of an issue, as PG stores bools as 'f' or 't' anyway (or at least outputs them in this form, it accepts 'false' or 'true' for input, too). Sticking with char(1) was easier. The queries work unchanged with either datatype.

That's also why we added things like sysdate and the dual view. More queries ported without change this way.

With ACS 4x I intend to use bools in the datamodel. We will rewrite queries into SQL'92 form when possible, for the very reason you mention (compatibility with other dbs). When that results in a query that won't work with Oracle, we'll use the query dispatcher to choose which query to run (i.e. the Oracle one or the PG one). When it results in a query that does work with Oracle, we'll just change the query directly. coalesce vs. nvl, that kind of thing.

As far as writing for 3x vs. 4x, timing should play a factor in your thinking. If you need to roll out a site in the near term, you'll be using OpenACS 3.2. If you don't have a pressing need for the module, consider downloading Oracle and write it for 4.1, doing your best to avoid Oracle-specific features. Then rewrite the necessary queries for Postgres. We're interested in providing modules that will work with both Postgres and Oracle (and later IB, etc) when possible. That won't be a requirement, obviously, and some folks could care less about Oracle.

But ... if a module writer can support both easily, why not do so?

3: Response to Guidelines for new modules (response to 1)

Posted by Carl Coryell-Martin on 02/07/01 11:10 PM

Just as a note, I have had success (with some tweaks) Running the Arsdigita Templating System with openACS 3.2.4.

It's pretty confusing, there is a distribution from cvs.arsdigita.com and there is anotherone from cms.arsdigita.org. I think I used the latter.

-ccm

4: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/08/01 03:35 AM

That's cool, can your changes be bundled and made available? I bet there are others who'd like to use it.

5: Response to Guidelines for new modules (response to 1)

Posted by Edmund Lian on 02/08/01 04:02 AM

<blockquote>>char(1) vs. boolean isn't really much of an issue, as PG stores bools as 'f' or 't' anyway (or at least outputs them in this form, it accepts 'false' or 'true' for input, too). Sticking with char(1) was easier. The queries work unchanged with either datatype.<<
</blockquote>

Oh I didn't know this, OK... still, using a boolean is much nicer from a readability perspective.

<blockquote>>With ACS 4x I intend to use bools in the datamodel.<<
</blockquote>

Will you use any other Postgres specific features, like the text type instead of limiting certain columns to Oracle's varchar limit? Just trying to follow your example...

<blockquote>>As far as writing for 3x vs. 4x, timing should play a factor in your thinking. If you need to roll out a site in the near term, you'll be using OpenACS 3.2. If you don't have a pressing need for the module, consider downloading Oracle and write it for 4.1, doing your best to avoid Oracle-specific features.<<
</blockquote>

I do have/use Oracle, but was switching to Postgres since it has evolved so fast--I think 90% of RDBMS users could probably do with Postgres and avoid the complexity and admin overhead of Oracle. Kudos to the Postgres folks. It's a very nice piece of work indeed.

<blockquote>>But ... if a module writer can support both easily, why not do so?<<
</blockquote>

Hmmm... true, but I guess I was just thinking of taking advantage of all the nice things about Postgres...

Are you, Ben or the others any closer to pinning down a timeline for OpenACS 4x?

6: Response to Guidelines for new modules (response to 1)

Posted by Roberto Mello on 02/08/01 04:26 AM

I was looking into using the ATS with OpenACS myself. Are your changes available anywhere? I'd love to take a look at it.

7: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/08/01 05:25 AM

Let's see ... varchars of limited size. When it makes sense to limit them, I actually prefer using them to text, for documentation reasons.

We do intend to use some postgresisms when it makes sense, yes. As to how much, that's kind of up in the air a bit. Ben and I spent a full day a few weeks ago talking about this, and I've been doing more thinking since. Keep in mind that this will be a big job when you include all the modules, the more postgresims in the data model, the more the number of queries that have to be re-written, tested, etc.

8: Response to Guidelines for new modules (response to 1)

Posted by Ryan Campbell on 02/08/01 06:32 AM

I've also gotten the ATS running with OpenACS 3.2.4. The biggest problem I had was knowing which version was the latest offical release. I downloaded the latest non-apm version of CMS from cms.arsdigita.com that I could find. Really, I think that you could download the apm from the ACS repository and just grab the ATS out of it after untar | ungzipping the apm, though I personally haven't tried this.

I put the ATS directory in my serveroot directory and symlinked it's subdirectories (tcl, demo, doc) to the appropriate directories. I was really slowed down because I didn't read the part of the installation guide where it tells you what settings you need to have in your nsd.tcl. They are very important!

I still haven't gotten ATS Forms to work correctly (running the form demo pages causes Aolserver to crash), though I suspect that it's because I'm running an older, buggier version. I'll probably upgrade to the latest APM once I have a need for the form generating functionality.

P.S. One of the nice things about the templating system is that you can source templates programmatically, since the internal API is fairly well documented. For instance, I have a file called header.adp (and it's corresponding data file, header.tcl) which contains the header I want to use across the site. Unfortunately, since none of the vanilla ACS pages use the master template, I was going to have to put my HTML both in the header.adp template as well as ad_header.

Being the lazy hacker that I am, I wasn't interested in maintaining the HTML in two different places. So I came up with this for ad_header_with_extra_stuff:

    set root_path [ns_info pageroot]

    set url_stub /header

    template::filter exec url_stub root_path

    set file_stub $root_path/$url_stub

    return [template::adp_parse $file_stub title $page_title]

So now, whenever I change header.adp, all the pages on my site are updated, not just the templates. I'm not sure that I need to apply the filter to this, but I thought it best until I understood filters better; it might be unnecessary overhead.

9: Response to Guidelines for new modules (response to 1)

Posted by Vlad Seryakov on 02/08/01 08:28 PM

Aolserver is crashing with ATS system in demos because of PostgreSQL driver. Some time ago i posted the fix to aolserver site and somewhere else, i don't remember.
I've got ATS working with PG7.1 and Sybase including forms and queries. If someone interested i can bundli it with my small extensions.

10: Response to Guidelines for new modules (response to 1)

Posted by Roberto Mello on 02/08/01 09:00 PM

Vlad,
I am very interested in getting ATS to work with OpenACS. If you could mail me what you got that'd be great.

You said you had to make any changes to PG driver?f you posted a patch here, Don and Dan could take a quick look.

Thanks.

11: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/08/01 09:06 PM

Yes, please, I'm interested in seeing the driver fixes and a sample query that was crashing it.

Can we get the ATS tested and loaded onto the site for distribution, with whatever changes were necessary to get it working? Roberto, you interested in doing that? A lot of folks will be working with legacy OpenACS 3.2 sites even when OpenACS 4.1 becomes available, so this isn't dead-end work.

12: Response to Guidelines for new modules (response to 1)

Posted by Vlad Seryakov on 02/09/01 08:57 PM

When i was testing ATS with PostgreSQL, almost all demos caused aolserver to crash because that SQL statements were for Oracle, and postgresql driver were complaining about it an crashed the server itself.

The problem is:
in function Ns_PgExec line 546 should be

Ns_Log(Error, "nspostgres: result status '%d', message '%s'", PQresultStatus(nsConn->res),PQerrorMessage(nsConn->conn));

PQresultStatus returns integer values, but sprintf symbol was '%s' instead.

Also where i can upload my copy of ATS, basically i made it completely
independent of any other tools, it can be installed as a Tcl module.

13: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/09/01 09:16 PM

Our driver - at least the one in the CVS tree but I assume the downloadable one, too - does not have that error in it.

You may've discovered why the driver distributed by AOLserver.com doesn't work while our driver does, though.

Strange, it's been right in our driver for a very long time.

14: Response to Guidelines for new modules (response to 1)

Posted by Vlad Seryakov on 02/09/01 10:29 PM

I just downloaded postgresql driver from aolserver.com, it has %s.

Anyway, here is the working ATS demo, i use postgresql 7.1
http://63.89.89.78/demo

15: Response to Guidelines for new modules (response to 1)

Posted by Roberto Mello on 02/09/01 11:27 PM

Vlad, as Don stated, the driver at aolserver.com has been broken, but the one distributed at openacs.org is not and has had the fix you pointed for quite some time.

It's great that you put the demo up, but could you make a tarball of the working ATS somewhere so I can download and take a good look at it? If everything goes well and the OpenACS community so desires, we could even include it in the 3.2.x tree for others to use.

Thanks.

16: Response to Guidelines for new modules (response to 1)

Posted by Vlad Seryakov on 02/10/01 05:12 AM

I put tarball at ftp://63.89.89.77/pub/vlad/ats.tgz

put ats subdir under /usr/aolserver/modules/tcl,
demo under $pageroot and add entries from etc/nsd.tcl into
your server's nsd.tcl ini file.

There is file named acs-procs.tcl, it includes all ACS related functions. I extracted them in order to run ATS as a standalone package.

17: Response to Guidelines for new modules (response to 1)

Posted by Don Baccus on 02/10/01 07:09 AM

Vlad - please re-read my earlier statement closely:

"You may've discovered why the driver distributed by AOLserver.com doesn't work while our driver does, though."

Then you go on to say you download the latest version from aolserver.com and it still has the problem.

Yes. We know the aolserver.com driver doesn't work. We tell people this.

At the risk of coming off as being rude, we've told people many, many times to use our driver (openacs.org, not aolserver.com) because we know ours works, and the one that got into their tree doesn't.

This isn't news to us. And we're not aolserver.com. Feel free to submit your patch to them, they need it ... this is not the place to report aolserver.com bugs. We don't need your patch, because our driver works.

Not that I don't want patches - just check your patches against our sources before you ask us to incorporate them. aolserver.com nspostgres.c != openacs.org postgres.c

18: Response to Guidelines for new modules (response to 1)

Posted by C. R. Oldham on 05/15/01 07:14 AM

Greetings,

Is there a working port of the ATS somewhere? I can't seem to access Vlad's copy.

Thanks.
--cro

19: Response to Guidelines for new modules (ftp not working?) (response to 1)

Posted by Hans Gaasenbeek on 06/28/01 11:41 AM

Maybe it is an idea not to use the ftp://-thing. This may be obvious but sometimes little things make a difference. So: 63.89.89.77/pub/vlad/ats.tgz