Forum OpenACS Q&A: Just the database API
Has anyone ever documented the various database API's available? By various I mean
AOLserver API (which is the most raw)
ACS TCL API rev1
ACS TCL API rev2 (I think around 3.0)
OpenACS TCL API
I just ask because there have been a few times where I have installed an entire ACS instance just to be able to use the database API which is sort of overkill, because then you have a gazillion tables and stored procs etc that you can't avoid because of all the mutual dependencies.
So having said that, what is the minimum install of OpenACS that allows you to use a nicer abstracted interface to the database (db_foreach for example) without the whole kit and kaboodle of an OpenACS install?
I suppose a parallel question exists for form handling, for page contracts and many other 'core' features, but I think focusing the answer on the database layer will probably answer this for a larger set of subsystems.
The template system can be ripped out standalone, that includes the form builder. I know at least one person did this to make it available for OpenACS 3.x. You might have to rip out some hooks but if so they'd be very minimal.
If you wanted to bundle these up as standalone, tested tarballs we could probably host them here. They certainly are useful for AOLserver users who don't want to develop an OpenACS site per se.
Sort of leads to another issue that always tickles (pun intended) the back of my brain - what layer should features be included in? To whit - OpenACS is an open-source project built on AOLserver - which is open-source project built on/in/around TCL which is an open-source project (which is written largely in C in which case you should use SWIG and make the modules available in ALL scripting languages, but I digress)..
To my thinking any feature should be pushed as far DOWN the stack as possible (or practical) to allow it to have the widest reach/appeal and 'bazaar' qualities that will lead to its continual improvement.
So should the 'improved' database API make its way into AOLserver as an optional package someday? Is the distinction that the AOLserver modules are all written in C and OpenACS is a layer of TCL native code on top? That's what appears to be the case to my eye.
I could send you an old tarball of that code if you want. However, it's old ACS 4.2 stuff, and somewhat hackish. I have not yet tried doing the same with recent OpenACS code, but based on my past experience I'm sure it could be made to work. (On snafu might be dealing with the query dispatcher, however.)
At any rate, I too would like to see the current OpenACS Tcl code be refactored for easy, drop-in use in non-OpenACS AOLserver environments, but so far I haven't done anything about it. It's rather far down on my personal list of priorities, currently. But definitely worth doing.
I think the AOLserver people are still discussing what to do about hosting non C code modules and applications.
But regardless, the OpenACS db API should always live and be maintained in the OpenACS CVS, since OpenACS is almost certainly going to stay the most active user and developer of the code for a long, long time. What would be very nice is for it and other OpenACS Tcl utility code to be very easily packaged up and used in other Tcl environments. (Not necessarily just AOLserver either, although that's the first and most biggest step.) Which is a much different issue from whose CVS server the code lives in.
I suppose memory plays tricks - there was a point - perhaps on the 4.0 boundary where the database kit in ACS got 'nicer' maybe it was that I finally got it because I wasn't in a rush to get projects out the door and read the docs more thoroughly. I do recall it getting easier to nest queries and deal with database handles (er, pools) at some point.
As far as helper routines - other than [set_variables_after_query] there wasn't too much to write home about .. haha.
I'd love to get a tarball of what you ripped out of 4.6 - I just built two virgin machines for a project I'm gonna be implementing for my own fun/edification (and a dotLRN site for a friend's school) so now's the time to experiment before I have too much other code in the way.
(Any idea how to get files > 2GB on a Linux 2.4 kernel?)
Small unmarked procs in a brown paper bag if you please ;) Leave it behind the statue in the town square at midnight.
Actually, I too remember there being a db_* API in ACS 3.4.x and the docs mention it too (https://openacs.org/doc/openacs-4/db-api.html). Jonathan Ellis ported the ACS 4.2 db api to openacs 3.x and created a patch.
There might be some useful stuff in the (not so) new-file-storage area of the old site (http://sdm.openacs.org/new-file-storage). I think some of Andrew's work is there.
The patch looks basically like a standalone solution to what I was asking about - I think that side by side with the 10-database-procs.tcl and database-init.tcl might answer entirely what I was looking for.
I think based on what Andrew said earlier that there might be a few niggling dependencies, but it doesn't seem as odious now that I see the code in question.
Would there be a speed gain ?
Would it make sense, as changes will be harder to implement?
Is the DB-API a moving target at the moment ?
There's no real advantage to having it as a module, a set of Tcl procs that are sourced at start-up are just fine.
What I'm looking for is a Goldilocks release - when AOLserver is too raw, OpenACS is too cooked, but a nice database API and forms abstraction would be 'just right'...
I think with the leads I got from this thread I can start putting something together in that regard.
it, let me know and I'll see about sticking it in file storage or
Personally I'd be surprised if the string overhead was significant enough to justify the hassle of using some other than the standard ns_db API. But, anyone curious should probably ask Jeff...
It's not that common a use case though.
The other thing was that tcl's handling of floats and ints etc is problematic. This turned up when doing these financial calculations since you would have round numbers going into a formula and get the wrong result like this:
% set x 1 1 % set y 3 3 % expr $x / $y 0I also thought about making it so that floats were returned with .0 tacked on to avoid this sort of problem.
It needs a customized AOLserver that Tomasz Kosiak announced to the mailing list a few days ago - see the "AOLserver Improvement Proposal" thread. (Mostly vserver-related stuff.)
I haven't used it and Tomasz has told me that they don't have the resources to support it.
I'm guessing that's what "does not enforce particular SQL data model like ACS does" means, anyway.
and I'm not really seeing the benefit of hardcoding the db user and other config parameters into nsd...