Forum OpenACS Development: Big honkin' commit now available ...

OK, I've made my first round of commits that implement a multi-db
aware APM and includes a merge with Ben's initial Query dispatcher work.

It's avaiable for testing.  Some quick notes:

1. I've not actually ported the APM to PostgreSQL at this point, and
of course only a bit of datamodel porting has been done for other
acs-kernel submodules.

2. We still require Oracle EE but for simple poking around and testing
the APM changes SE will work fine if you don't mind ignoring the
errors due to creating bitmap indices and other EE features.  A
freshly installed system's mostly empty and the missing indices don't
hurt.

3. If you want to install this under Oracle (and I'd appreciate
someone doing so to double-check that I got everything in) check out
the APM.  Some core packages - acs-kernel, for instance - are now
labelled as supporting both PG and Oracle, while others - acs-mail,
for instance - are labelled as Oracle only.  Yet others say "none
specified" (not all of the core modules use the database).

4. Install under PG makes it past acs-kernel and tries to load other
core modules.  The APM doesn't yet give an error when you try to load
an Oracle-only module in a PG environment, but it will soon - no need
to report this to me!

5. The APM still has other rough edges, I'll be working on it next
week.  We're OK for the moment now, I think, the needed clean-ups
aren't critical-path issues.

I'll post information about what's up next separately so folks can
comment if they want.

The source can be gotten from cvs at openacs.org:/cvsroot.  Copy (co)
openacs-4.  Anonymous pserv is supposd to work...

The installation for Oracle is no different than for ACS Classic 4.x.
I'd appreciate your trying it out under EE to make sure there are no
errors (I'll grab a copy of EE at some point to install I guess).

For PostgreSQL, you need to use version 7.1, which is in release
candidate form at the moment.  You can get it from ftp.postgresql.org
in pub/dev - the biggest files are the complete system for each
release (RC2 is the latest there).

You do need to install PL/pgSQL.  It can be done easily after you do
your createdb, i.e.:

createdb pgtest;
createlang plpgsql pgtest

and you're set.

Then you need to tell OpenACS where you've installed the binaries so
it can find PSQL (I'll make it smart enough to do this from
environment variables if they're set later on):

ns_section "ns/db/driver/postgres"
ns_param pg_bin "/home/pgtest/install/bin"

works for me as I installed PG7.1 as my pgtest user using
/home/pgtest/install rather than /usr/local/pgsql.

After you've done this just fire things up like you would for Oracle
4.x and off you go.  The only differences will be in your AOLserver
config file - you need three PG pools rather than three Oracle pools.
Use the same pageroot in both cases.

Only the core is included in our source tree at the moment.  If you
want to load other modules you'll have to go to the aD repository
(unless you have them yourself, of course).

Oh - you'll get lots and lots of "notice" lines in your AOLserver log,
from Ben's very preliminary version of the query processor.  They'll
go away soon.

I ran into a problem when loading this under PG (7.1rc1). The datamodel loads, but acs-kernel fails :

Failed to install ACS Kernel, version 4.2. The following error was generated: 

     Database operation "0or1row" failed (exception NSDB, "Query was not a statement returning rows.")

Package enabled. 

The error log shows:

[05/Apr/2001:18:45:12][8255.25605][-conn4-] Error: dbinit: error(localhost::openacs-4,ERROR:  ExecAppend: rejected due to CHECK constraint apm_package_vers_desc_for_ck
): '
select apm_package_version__new(
       '',
       'acs-kernel',
       '4.2',
       'http://www.arsdigita.com/acs-repository/download/apm/acs-kernel-4.2.apm',
       'Routines and data models providing the foundation for ACS-based Web services.',
       '',
       '',
       '2001-03-06',
       'ArsDigita Corporation',
       'http://www.arsdigita.com/',
       't',
       't');
'

There's a check constraint on apm_package_versions that the description_format should be either "text/html" or "text/plain", but we're trying to set it to '' (the fifth argument). I think this is the old '' vs NULL issue, because if I enter the above command with NULL's instead of empty strings, it works. If I fix this issue, the same issue comes up later in apm_package_version__add_file (can't enter '' for db_type, because of foreign key violation). Am I missing something?

I then tried to load via Oracle and everything works beautifully!!! (8.1.6 EE). Really, REALLY cool!

My installation into Oracle 8.1.6EE went fine with a few exceptions.

Here is my mini-list of the steps involved.

• Follow http://developer.arsdigita.com/doc/acs.html to create a tablespace
• Refer to cvs instructions on SourceForge at http://sourceforge.net/docman/display_doc.php?docid=763&group_id=1 , but the exact commands I used were:

     $ cvs -d:pserver:anonymous@openacs.org:/cvsroot login
  
      (just press enter at the password prompt, which returns you to $)
  
     $ cvs -z8 -d:pserver:anonymous@openacs.org:/cvsroot co openacs-4
  
  

• Installation required the nsxml module. I got the module from: http://aolserver.com/archive/pub/contrib
• The nsxml module requires a working libxml2. I didn't have one so I downloaded version 2.3.5 from www.xmlsoft.org: ftp://xmlsoft.org/libxml2-2.3.5.tar.gz
• I had to modify the Makefile slightly after building libxml2. I had to change -lxml to -lxml2 and then the make worked.
• libxml2.so.2.3.5 gets installed in /usr/local/lib, which my aolserver didn't figure out, so I put in another sym link:

     $ ln -s /usr/local/lib/libxml2.so.2.3.5 /lib/libxml2.so.2  
  

This seemed to allow AOLserver to startup and stay running. I connected to the pageroot and was able to install OpenACS4 into Oracle 8.1.6EE.

Yeah, that sounds like the right place.  Ideally you'd get an error earlier in the process saying "hey! acs-whatever only supports Oracle!" but that error check's not there yet.

Instead if it can't find a datamodel it just keeps chuggin' along at the moment.

The pgbin param shouldn't be so picky, I'll try to look at that and also the default to use PG environment variables by default if they're  available next week...

Database operation "0or1row" failed (exception NSDB, "Query was not a statement returning rows.")
         while executing
     "ns_pg_bind 0or1row nsdb0 {
         select case when count(*) = 0 then 0 else 1 end from apm_package_db_types}"

Well, this is weird. Are you sure you have the latest driver?

On the surface, the message is particulary weird because of course it returned no rows - and it is a 0or1row query so shouldn't fail!

Any more information you can provide, the better. I hate to say this, but my first response when this kind of thing happens on my machine is to seek a virgin environment, even if it means deleting a lot of stuff...

Your datamodel didn't get loaded, that's the problem.  The code doing the select thinks that an earlier install attempt got further along than it really did.  It thinks the acs-kernel datamodel installed without error and is checking to see if any packages are enabled in the APM.

Could be a weak point in the bootstrap/installer's checks to see if the kernel datamodel's been installed.  aD wrote in some sanity checking but not a lot (nor should it need a lot).

The acs-kernel datamodel will load on my laptop ... let me know what happens when you blow away the database and try again.

Are you sure you've got the right path to PSQL specified in your
.tcl init file that's read by OpenNSD on startup?  Are you sure that the command will execute if you run it as the same user you're running OpenNSD as?

I ask because at one point I had silent non-loading of the datamodel due to filtering out all but "ERROR" lines from running PSQL in db_source_sql_file (or whatever it is called).  Turns out that was not  a very good idea given the fact that PSQL will also return FATAL error lines :)

I filter these out so you don't get a bazillian NOTICE and messages along with other PSQL stuff dumped  on your screen as error messages ...

Come to think of it, I'm not certain I've committed that change as I've been saving some APM changes to submit as one fell swoop.  I'll look into this later today ... for now you can look at 10-database-procs-postgresql.tcl and strip out the filtering of PSQL output and you'll see everything PSQL emits.

You need to do a "createlang plpgsql database_foo" after you do "createdb".  It's never added by default ...

That's it.