Forum OpenACS Improvement Proposals (TIPs): Re: TIP #37 (Proposed): Queries in XQL file conventions

I have a slightly different approach to this problem that I think I mentioned before.  What if someone wrote a preprocessor that parsed the XQL and the TCL files and emitted a set of TCL files with the SQL (or just the db-agnostic SQL) inline?

It's use would be optional ... all the "KEEP SQL AND TCL SEPARATE" folks could ignore it,  make their cvs check-in and -outs as normal,  and be happy.  All the "IT'S EASIER TO UNDERSTAND WHEN THE LOGIC IS ON ONE PLACE" folks could do a cvs checkout,  run the preprocessor,  and have the code in a much easier to understand format.  As a side benefit,  you could start off your AOLServer/OpenACS instance without the humungous XML parse overhead,  have a slightly lower memory footprint,  lose a huge number of useless XQL files from your version control tree,  and avoid a large chunk of query processing code (as well as the tDOM requirement -- making for a simpler install).

There is a small problem if they wanted to check code back into the project,  but doing query extraction by hand or in combination with a tool that did most of it wouldn't be all that hard.

Speaking of which, when I brought this up before,  someone pointed out that the existing query extractor (TCL+SQL -> TCL+XQL) tool (written in python) didn't really work very well.  And,  as another data point,  I tried writing the aforementioned tool in perl without doing a formal grammar,  and I think I got it to 90%+ working.  Still not good enough.  Despite that,  I think that a more robust implementation using tclparse (which exposes the actual tcl parser) could work.  From my experience,  there were only a handful of dynamically assembled sql calls that an automatic approach might have trouble with.

I see XQL files as leaning way to heavily towards code organization (OpenACS maintainers) rather than ease of understanding for new folks or people trying to do work with OpenACS.  That's understandable considering who came up with it.  But it would be nice to come up with a solution that kept both camps happy.

This isn't the kind of thing you could do without good unit tests,  but since we're not too far off from having those I thought I'd bring it  up.