Forum OpenACS Development: Search indexer question

Posted by Gilbert Wong on 03/08/02 07:42 PM

Is there an easy way to reindex my searchable tables or should I just run the search_observer__enqueue sql function with DELETE and then INSERT? The site I am working on has a problem with the search indexer, which I am still trying to track down. Some of the searchable objects aren't being indexed. I should note that I haven't done a CVS update on that site since late December or early January (or whenever Don changed the tree_sortkey to varbit) so I might just have a buggy version of the search indexer procs. Any thoughts?

2: Response to Search indexer question (response to 1)

Posted by Neophytos Demetriou on 03/08/02 07:55 PM

Gilbert, you can reindex all of your content by running search_observer__enqueue sql function with UPDATE. The "problem with the search indexer" seems to be related with content stored as a file or lob, right (therefore, you would need db_blob_get)? I clearly remember testing it for all types of storage, so it's a bit of a surprise that you see this problem.

I don't expect that the tree_sortkey change affected the search package in any way. It's probably what Don described in his reply to your message. I can't test this right now, so I would appreciate if you could try to add a db_transaction around db_blob_get and let us know of the outcome.

3: Response to Search indexer question (response to 1)

Posted by Gilbert Wong on 03/08/02 08:12 PM

Neophytos,

Is there a way to see which objects were indexed? What is the queue table name? I'm not convinced that all of my searchable objects were inserted into the queue. Maybe they are in the queue but the indexer can't read it. So I'd like to see if I can reduce some of the variables to identify the real problem.

I did what Don suggested (db_transaction around db_blob_get) and it didn't work. However, I didn't spend much time playing around with it and I could have added it in the wrong place. First I'll update the search code to see if there were any major changes since my last cvs update. Then I'll play around with the db_transaction again.

Thanks.

4: Response to Search indexer question (response to 1)

Posted by Neophytos Demetriou on 03/08/02 08:41 PM

Gilbert, afaics, the search code hasn't change since it was released (September). All indexed items are available in the 'txt' table (if you haven't changed the default). All unindexed searchable items are stored in the search_observer_queue table. Let me know if that helps.

5: Response to Search indexer question (response to 1)

Posted by Don Baccus on 03/08/02 09:03 PM

Gilbert - it did get rid of the "can only use get blob in transaction" error, though, didn't it???

6: Response to Search indexer question (response to 1)

Posted by Gilbert Wong on 03/12/02 11:14 PM

Yes it did work. I'm a moron. 😉 I was editing the wrong function. I played around with ns_db instead of db_blob_get. Anyway, here is the patch. It fixed the problem on my site. But I'm still confused why the problem didn't crop up on my development machine. hmmm...

7: Response to Search indexer question (response to 1)

Posted by Don Baccus on 03/13/02 02:53 AM

Actually, it is the *caller* that's meant to wrap the blob calls in a db_transaction, not the calls inside the db API. This is mostly for consistency's sake - you might want to insert a lob in a transaction with other inserts, and having the db API wrap the lob call in a transaction won't do what you think ...

I've submitted a patch to the search package that should work.

I couldn't help but notice that content stored in files don't work ...