Forum OpenACS Improvement Proposals (TIPs): Re: TIP #37 (Proposed): Queries in XQL file conventions

Collapse
22: Re: TIP #37 (Proposed): Queries in XQL file conventions (response to 1)
Posted by Don Baccus on 12/05/03 10:08 PM
Well, as far as the original history goes I didn't care much one way or the other, i.e. using query files or conditionalized db* code.

But some people did care so the decision was made to use queryfiles.  The drive in this direction originally was made by aD and Ben when it was decided to put querynames in the db_* API (if the conditionalized inline query approach is taken there's no reason for queries to have names).

But I will repeat that no one argued for a mixed approach, at the time of the discussion the value of choosing one and only one approach was accepted by all.  Adopting a mixed approach seems to me much worse than taking one direction or the other.

Collapse
23: Re: TIP #37 (Proposed): Queries in XQL file conventions (response to 22)
Posted by John Sequeira on 12/05/03 11:02 PM
I have a slightly different approach to this problem that I think I mentioned before.  What if someone wrote a preprocessor that parsed the XQL and the TCL files and emitted a set of TCL files with the SQL (or just the db-agnostic SQL) inline?

It's use would be optional ... all the "KEEP SQL AND TCL SEPARATE" folks could ignore it,  make their cvs check-in and -outs as normal,  and be happy.  All the "IT'S EASIER TO UNDERSTAND WHEN THE LOGIC IS ON ONE PLACE" folks could do a cvs checkout,  run the preprocessor,  and have the code in a much easier to understand format.  As a side benefit,  you could start off your AOLServer/OpenACS instance without the humungous XML parse overhead,  have a slightly lower memory footprint,  lose a huge number of useless XQL files from your version control tree,  and avoid a large chunk of query processing code (as well as the tDOM requirement -- making for a simpler install).

There is a small problem if they wanted to check code back into the project,  but doing query extraction by hand or in combination with a tool that did most of it wouldn't be all that hard.

Speaking of which, when I brought this up before,  someone pointed out that the existing query extractor (TCL+SQL -> TCL+XQL) tool (written in python) didn't really work very well.  And,  as another data point,  I tried writing the aforementioned tool in perl without doing a formal grammar,  and I think I got it to 90%+ working.  Still not good enough.  Despite that,  I think that a more robust implementation using tclparse (which exposes the actual tcl parser) could work.  From my experience,  there were only a handful of dynamically assembled sql calls that an automatic approach might have trouble with.

I see XQL files as leaning way to heavily towards code organization (OpenACS maintainers) rather than ease of understanding for new folks or people trying to do work with OpenACS.  That's understandable considering who came up with it.  But it would be nice to come up with a solution that kept both camps happy.

This isn't the kind of thing you could do without good unit tests,  but since we're not too far off from having those I thought I'd bring it  up.

Collapse
27: Re: TIP #37 (Proposed): Queries in XQL file conventions (response to 22)
Posted by Andrew Piskorski on 12/06/03 06:20 PM
Oh, I think using "queryfiles" for most SQL rather than conditionalizing all the Tcl code was probably a good choice, I just used to wonder why the heck use XML for defining those queryfiles, rather than Tcl (sort of like the AOLserver config.tcl file). I've asked this before in other threads though, so I know the answer now: There were other reasons at the time that made XML seem more attractive, like the fact that people were making loud noises about wanting to port all the Tcl code to a Python version of OpenACS.

Of course that never happened, and probably never will. But if it ever does happen, those language-neutral XML query files will (as it stands now anyway) be there waiting for them. (Who knows, maybe those SMLserver guys will want to guys will want to write OpenACS modules in Standard ML. Maybe in some alternate universe.)

But while all that is useful to think about for whatever lessons it may teach us for future projects, it's all water long since under the bridge as far as OpenACS is concerned now.

Regardless, Lars's original proposal is much simpler and more focussed than most of this discussion. And I don't see any serious problem regardless of which way the decision goes on that proposal.