Forum OpenACS Development: Query Extractor available in file-storage

Collapse
1: Query Extractor available in file-storage
Posted by Don Baccus on 04/22/01 08:43 PM
I've hacked Kapil's original work on query extraction tools in order
to make it immediately useful for our porting effort.  You'll find
this tool in file-storage.  It's written in Python, and works fine
with the version that is distributed with RH 6.2 Linux.

I'll take this and CVS tree work off the "critical path" portion of
the status report.  Ben has one tricky problem to fix with the query
dispatcher and that will be off the list, too (I actually won't bother
editing the status report until all three are complete since we're
dead in the water until they are).  Ben expects to have cracked his
bug by tonight or tomorrow.

As soon as these pieces are completed I'll port the APM completely to
PG, so folks who don't have Oracle on their box can manipulate
packages (necessary for polishing ported packages for distribution,
not necessary for the actual porting work).

Collapse
2: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/22/01 08:46 PM
Oh, I forgot to say "Be sure to read the 'doc/README_PORTING' file then the other README files" if you download this tool.
Collapse
3: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/23/01 07:51 PM
And I've updated a slightly improved version this AM, that splits some  of the repaired or flagged Oracle-isms into the generic .xql file rather than the PG .xql file.

This will be particularly useful for outer joins.  Once we port them to SQL92 form for PG they'll work for InterBase and other RDBMS's that  support the standard syntax, and since they'll be in the generic .xql  files they'll be found automatically.

COOL!

Collapse
4: Query Extractor available in file-storage, Where I can find it ? (response to 1)
Posted by Domingo Alvarez duarte on 04/23/01 11:24 PM

Sorry but whow can I access and download the query extractor.

Collapse
5: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/24/01 12:00 AM
As said in my first post, "You'll find this tool in file-storage"

In other words https://openacs.org/file-storage.  It's named "query extractor: porter's version".

Ben's working on a fix for the util_memoize [list db_foo...] problem, BTW.  He's got a quick hack for this particular case, and we've discussed the possibility of just walking the call stack looking for a [proc.queryname] match in the query dispatcher cache.  He seems to think that will work, solving the problem in a more general way.

I haven't tested the query extractor on this construct, I'm sure it will fail.  I'm not too worried, I've checked the entire 4.x release (all of it, not just the core) and there are only NINE examples of queries like this.  We can pull these nine out by hand easily enough.  If there were more I'd look into automating the process but it just ain't worth it for such a small number.

Collapse
6: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/24/01 04:29 AM
The query parser doesn't put files in the right place, the files are hard-wired to go into "www" and "tcl" subdirectories, not much use for  packages that have nested directory structures (which frequently have  multiple occurances of "index.tcl").

I didn't know it was written this way (not my code!) and my testing on acs-tcl didn't reveal it (as it doesn't have a nested subdirectory structure).  I just ran across it this afternoon when I started to port acs-admin/apm.

I'll fix this tomorrow AM and update.  I also have fine-tuned the query destination selector (*-oracle.xpg? pg? both? generic?) some more.

Grrr...sorry 'bout this further delay.

Collapse
7: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/24/01 07:56 PM
OK ... the new and improved Query Extractor is up in file-storage (https://openacs.org/file-storage).

It now correctly mirrors the directory structure of the package it is processing.  Check it out, and let me know if you have nay problems.

Collapse
8: Response to Query Extractor available in file-storage (response to 1)
Posted by Kapil Thangavelu on 04/26/01 03:44 PM
hi, i'm back from my rsi imposed vacation. i've taken a look at some
of the changes don has made to the query extractor and will be
integrating the self described hacks into next version. there is one
change which i'm a bit curious about though, its the removal of
encoding on < and > which means that the query xml files aren't
valid xml. any comments?, can the query-dispatcher handle these if
they are properly encoded?
Collapse
9: Response to Query Extractor available in file-storage (response to 1)
Posted by Don Baccus on 04/26/01 04:44 PM
Yes, the query dispatcher is handling "<" and ">" by ripping the queries through ad_quotehtml before passing the munged file to ns_xml.
We want to keep the query files readable and don't want to require those who might write them by hand to have to convert their less-than and great-than operators manually.