Forum OpenACS Q&A: PostgreSQL driver

Collapse
Posted by Roberto Mello on
Hi all,

Lately there have been a flurry of AOLserver modules committed to the
AOLserver CVS: ns_imapc (imap client), ns_snmp, ns_cache got adopted,
ns_xml 1.5 released, ns_encrypt (strong encryption, horray!), ns_image
(for identifying image).

This is all great news, but even better news is that Scott Goodwin
seems to have become the (un)official community liason wrt the
AOLserver tree. He is importing these modules into the AOLserver CVS,
and giving commit rights to their maintainers.

He is explicitely asking for us (OpenACS) to move the latest
PostgreSQL driver to the tree, and he will give the maintainer commit
right to that module.

"The biggest problem is: will the OpenACS crew let me take the OpenACS

postgres module, refactor it and then use that refactored copy,

maintaining it at SourceForge? I realize OpenACS has a terrific SDM,

but I think the AOLserver C modules should be maintained and

distributed from a central location, and that would be SourceForge. If

the OpenACS folks continue with their version of postgres.so, that's

fine, but I hate to see two separate modules whose only real

differences may be cosmetic."

I for one, would prefer if we could just tell people "Go to
AOLserver.com and you can grab everything you need AOLserver-wise", as
long as we have commit privs to the PostgreSQL driver.

What do others think? Who would be the maintainer (one with commit privs)?

Collapse
Posted by Scott Goodwin on
<p>If there are multiple maintainers, we can add them all so they can commit directly to the postgres module.

<p>I'd also like to rename the module to nspostgres for consistency. Call me anal, but, well, I am an analyst.

Collapse
Posted by Don Baccus on
Renaming it nspostgres.so is fine, of course.  Personally sourceforge drives me nuts, that's why I moved the driver from sourceforge to our site in the first place (this was before AOLserver was hosted there).

But it makes sense to have the driver and other modules hosted in one place, and if AOLserver is to be hosted on Sourceforge, so be it.

Who is maintaining the Oracle driver?

Scott ... thanks for your hard work and it's good to see these useful modules such as ns_imapc get into the official tree.

Collapse
Posted by Don Baccus on
We'll still be left with at leats one issue that makes Roberto's goal of sending everyone over to AOLserver.com to get all the AOLserver pieces they need.

One of our community members has resurrected windows support and quite a few members of our community are interested in seeing that packaged and released once AOLserver 4.0 is released.  At the moment I can't imagine that being hosted at AOLserver.com.

Also ... are all the necessary aolserver3.3+ad13 fixes included in AOLserver 4.0?  In particular there have been patches made in the area of globalization which a large proportion of our community will need (they were made by Henry Minsky when he was living in Japan, building Japanese [Open]ACS sites).

Collapse
Posted by Jeff Davis on
I can pick up the Oracle driver if no one else wants to deal with
it.  I have fiddled with it a fair bit in the past.  I would say
it is in serious need of a rewrite though.

And, credit where credit is due, most of the ad patches were
from Rob Mayoff.

Collapse
Posted by Scott Goodwin on
Postgres driver: great! Let me know who needs access via their SF username and I'll add them to the committer's list. Then I'll work with whoever to import the latest OpenACS-hosted copy.

Oracle driver: lot's of discussion about this driver about a month or two ago. I don't have any way to test the driver, but I will research the previous discussions and see if I can identify a good "baseline" nsora and import it, then update it with patches from changes that people have been making to it. I may need some help here -- once I have the baseline checked in, if someone could take the lead on getting the changes that are out there integrated, let me know and we can work together on it. My role would be to make sure the nsora CVS code is tagged after each patch is applied to make it easier to get the older revisions if necessary, and to generate the release files.

As for Windows and the aD version support in AOLserver, I do believe that SF can be the place to do this. Could we import or create AOLserver Windows version and AOLserver ArsDigita version into CVS as modules? Both versions could track the core server. At least they would be easily available from and developed at one central point, which would make it easier for AOL's dev team to see and potentially integrate some of it.

I agree that SF tools aren't the best, but getting the code centrally located and managed from one CVS tree, cleaning up the code base, standardizing how the modules work, documenting the development process and coding style etc. will go a long way to improving what I see as the best application server out there.

Collapse
Posted by Scott Goodwin on
Looks like I'll be working with Jeff Davis w/respect to nsora.

:)

Collapse
Posted by Don Baccus on
Welcome back, Jeff ... our copy of nsora is a good place to start. I know I fixed one bug in regard to the handling of LONG datatypes before we released OpenACS 4.5. LONG is deprecated (in favor of BLOB) but is still used in a lot of Oracle's internal tables, such as those interrogated by the OpenACS schema browser. Somewhere along the path from Oracle 7.x to Oracle 8.1.6 the handling of LONG by the Oracle client library got munged and a driver dependence on documented behavior caused segtraps ...

Put me down as a committer for nspostgres.so ("dhogaza").

As far as whether or not the Windows stuff can be a bolt-on to the AOLserver tree ... you'll need to talk to the guy who's doing it, Jamie Rasmussen.

Collapse
9: SF Foundry (response to 1)
Posted by Wojciech Kocjan on
Hello.

I'm finishing up some of my modules as well, so I'm kind of interested in this subject.

I've seen how Tcl handled this. It is quite logical. So why can't we just set up an AOLserver foundry? Then I would create a project and just have it added to the foundry. The same with the rest. After all, there may be more and more modules/scripts/stuff with time, so things could get a little messy with just mailing Scottg on setting up a branch in the AOLserver project.

Collapse
Posted by Dossy Shiobara on

I just wanted to respond to a snippet Don Bacchus wrote (that I originally sent to the AOLserver mailing list):

> [...]
> One of our community members has resurrected
> windows support and quite a few members of our
> community are interested in seeing that packaged
> and released once AOLserver 4.0 is released. At
> the moment I can't imagine that being hosted at
> AOLserver.com.
> [...]
> -- Don Baccus, September 29, 2002

I would love to see Win32 support continue for AOLserver. I don't see why Don says "I can't imagine that being hosted at AOLserver.com" -- at least the AOL folks have said that "we simply can't maintain Win32 support any longer" ... they never said "we no longer want Win32 support for AOLserver."

If the community will work on Win32 support and do it in such a way that it doesn't negatively impact the AOLserver core, I am convinced that the AOL core team will accept the necessary changes to make it happen. I'd volunteer to be the coordinator of the patching effort if that would help.

Collapse
Posted by Jeff Davis on
Well, given that I implemented the LONG stuff it was my bad for
trusting oracle's docs :)

I think there was also a fix from scorecard (getactive now I guess)
which should be pulled in.  I am not sure who to ping there though.

Collapse
Posted by Scott Goodwin on
Don, you're now on the committer's list. I've checked out nspostgres from AOLserver, made a note in the ChangeLog, then tagged the nspostgres tree as "pre-don".

I highly doubt you will, but let me know if you need any help with the CVS issues at SF.

Collapse
13: Re: SF Foundry (response to 1)
Posted by Scott Goodwin on

Good idea, but not sure we need to do that yet.

I wasn't suggesting we make a branch on the real AOLserver core tree, but instead import an entirely new copy of aolserver-aD and aolserver-win32 as totally separate modules within the top-level AOLserver SF tree.

That way if anyone messes up with CVS, the damage will be limited to the separate tree and not affect the core.

Also, I'd like to manage user perms and such from one SF project admin interface so the rest of you can focus on working on code.

Collapse
Posted by Dan Wickstrom on
I would also like to be added as a commiter for the postgres driver.  My sf id is "danw".
Collapse
Posted by Jeremy Collins on
I started cleaning up and adding stuff to the Oracle driver a few months back.  I added support for multiple OUT variables (in PL/SQL) and REF CURSOR support.  Like I said I also started cleaning up the code which basically means stupid formatting changes.  Anyway I labeled it 2.6 (which is incorrect because 2.6 was bundled with the AOLserver-aD distribution) and put it the following site:

http://phpsource.net/aolserver/

Please make sure you check this driver out when you start importing stuff into CVS.  I don't care if you use it but just want to make it available.  We've been using it in production for a while now and it seems stable.

Collapse
Posted by Scott Goodwin on
Dan, I've added you to the commit list at AOLserver SF.
Collapse
Posted by Scott Goodwin on

In response to Jeremy's posting: the things you said are the reasons why I want to get the modules centralized at SF, particularly your comment about making formatting changes. I want to help refactor and make the modules consistent inside and out. It will be easier to accomplish that if there is one place the code is maintained.

I'll work with you and Jeff Davis to get the initial import of nsora in. You two can discuss what order you'll patch the module to make it current.

I'd like to see each commit tagged in the following way:

I import the initial code and then tag the code with 'start'.

Every release is tagged as:

  v1_0
  v1_1

and so on.

Right after a new release, the major/minor numbers are incremented as needed, and "beta" tacked on the end, like this:

  v1_1       (the release)
  v1_2beta1  (the first beta point)
  v1_2beta2  (the second beta point)
  v1_2beta3  (the third beta point)
  v1_2       (the next release)
  v2_0beta1  (...and you get the picture...)
  v2_0beta2
  v2_0

This keeps the tags short and easier to read on the cvs browse web pages. I realize it's rather arbitrary, but I'd like to standardize on this scheme for all modules.

Collapse
Posted by Jamie Rasmussen on
I'm very glad to see the AOLserver community becoming more organized. (Special thanks to Scott of course.)  I'd also like to encourage everyone to come to the AOLserver chats we have every Thursday - we regularly discuss these issues.

I think non-Win32 OpenACS ought to run on a stock AOLserver distribution.  Other than possible internationalization issues, I don't think there are any changes Arsdigita made that would affect performance.  Most of the changes were bugfixes or new modules that should be fine in 4.0.  Which reminds me - I think we should ask Scott to import the nssha1 module too.

Regarding the Windows version of 4.0, I'm happy to keep the code separate for now.  Once 4.0 is released, I can make a full set of patches and we can properly discuss whether the Windows code should be folded into the core.  (I might actually lean towards "no" myself.)  I *would* like to see Windows changes to the modules merged back in, as very few changes are usually needed.  I'd respect the desires of each module's admin on that issue however.  I'm willing to work with Win32 core changes as a "module" on SourceForge if that is what the community prefers.

I'm also hoping to revive Yon's AOLserver test suite framework, which is currently Windows only.  It would be especially useful for modules which I don't personally use but would like to compile and include in my distribution.

Collapse
Posted by Scott Goodwin on
I have aolserver 3.3ad13 with nssha1; I'll import it shortly. Ok, it's now imported and there's a release in the Files area.

Would it be possible to create an automated process to take a release of AOLserver core and apply the patches to make it work on Windows? If so, then I'd say get the Win32 port working with AOLserver 3.5.0 / 4.x and package the patches into a distribution that has a makefile to apply them. Every time a new AOLserver release was ready, Win32 maintainers could go and update the patch distribution.

As for the modules, there's no reason a patch file can't be maintained in the module dist and a makefile target added to apply the patch for Windows users. Can we do that for now?

Where can I go look at Yon's test suite framework? I'm already working on one. I hear Jerry Asher has a test framework too that I'm going to look at.

Collapse
Posted by Andrew Piskorski on
Don, where is your Oracle driver LONG fix?  The Oracle driver isn't in
OpenACS CVS, is it?
Collapse
Posted by Jamie Rasmussen on
Checkout nstest from acs-misc CVS.  There is a document at
http://acs-misc.sourceforge.net/nstest-manual.html you can read too.  There are currently only tests for nsxml and some request processing stuff.  I was trying to add some for nsimage, which has relatively easy functionality to test.  (At the moment, I'm not sure if nstest is broken or my port of nsimage is, but jpeg size calculation isn't working.)

Patches for modules would probably be fine.  I wouldn't be optimistic about an automatic system working with the core; I expect some manual merging will be necessary.

Collapse
Posted by Jamie Rasmussen on
Just a quick note to say that Yon wrote the nsxml and core tests that are in nstest, not me.  I just started learning how to use his system two days ago.  Full credit where it is due - sorry if that wasn't clear in my post above.
Collapse
Posted by Jeff Davis on
I posted the following on the AOLServer list.
I have oracle 2.3, 2.4, 2.5 and 2.6 all as released from aD. I checked and the copy of 2.6 at Eve's site and the OpenACS tarball both match.

I also have the version from Jeremy Collins which he called 2.6 and was based on the aD v2.5 but there are a lot of cosmetic changes and it is hard to figure out what should be folded in to the next version. The diff from aD 2.5 -> aD 2.6 is very small and easy to add to Jeremy's version but I am not sure how well his version has been tested.

There is also a later version than the 2.6 release which Rob Mayoff had (which hopefully Rob or Andrew will send to me shortly which was ora8.c v 1.60 in the aD CVS).

Also, there was a change that I never put into the released version since it was never clear to me what the right answer was (cf http://ccm.redhat.com/bboard-archive/webdb/000d9f.html )

I think what we should do is put up the 2.3 -> 2.6 sequence, then try to pull in anything else that might have been fixed elsewhere and make that 2.7beta.

Collapse
Posted by mark dalrymple on
Ya want crusty old oracle drivers? I managed to dig up a 2.2 and a 2.0.1, which I put at http://badgertronics.com/hacks/oracle-history.tgz. I also found my 1.0.4b1 version that was hacked up for AOLserver 3.0 beta 2, which replaced a handful of commands that AOLserver 3 let drop by the wayside.
Collapse
Posted by Lamar Owen on
These are great developments.  Don, Dan, it is great you both now
have commit on AOLserver CVS for the driver.  This makes it much
more of a community effort.  And Scott, congrats on the 'promotion'
to admin -- I'm sure it wasn't an accident, as you've done some
great work.
Collapse
Posted by Lamar Owen on
Incidentally, Marc's Oracle modifications way back when were the
model for the original AS 3 port of the postgres driver I did way
back when.  Good historical information.  I might even have a
postgres driver lying around from a beta of AOLserver 2.2.1 if
anyone is interested in history.