Forum OpenACS Q&A: Re: OpenFTS problem with non US_ASCII characters

Collapse
3: Re: OpenFTS problem with non US_ASCII characters (response to 2)
Posted by Claudio Pasolini on 11/08/04 04:27 PM
Thank you very much, Gary: now it works!

Actually I was already quite close to the solution, but I converted the lexem to unicode instead of utf-8, without success.

Furthermore I had to shut down and restart aolserver to get things work.

Collapse
4: Re: OpenFTS problem with non US_ASCII characters (response to 3)
Posted by Joel Aufrecht on 11/08/04 08:55 PM
I have two related problems. I applied the fix above and restarted but it didn't have any effect, which is unsurprising because my error messages are slightly different: 
[08/Nov/2004:13:47:18][16217.163850][-sched:25-] Error: Ns_PgExec: result status: 7 message: ERROR:  Invalid UNICODE character sequence found (0xc200)

transaction error
[08/Nov/2004:13:47:18][16217.163850][-sched:25-] Error: Aborting transaction due to error:
Database operation "dml" failed (exception ERROR, "ERROR:  Invalid UNICODE character sequence found (0xc200)
")

ERROR:  Invalid UNICODE character sequence found (0xc200)

SQL:
                    insert into index8
                        (lexem,tid,pos)
                         values
                        ('0rÂ,16537,
                        '{1005}')
and 
[08/Nov/2004:13:47:54][16217.163850][-sched:25-] Error: Ns_PgExec: result status: 7 message: ERROR:  Cannot insert a duplicate key into unique index in
dex10_key

transaction error
[08/Nov/2004:13:47:54][16217.163850][-sched:25-] Error: Aborting transaction due to error:
Database operation "dml" failed (exception ERROR, "ERROR:  Cannot insert a duplicate key into unique index index10_key
")

ERROR:  Cannot insert a duplicate key into unique index index10_key

SQL:
                    insert into index10
                        (lexem,tid,pos)
                         values
                        ('2004',18789,
                        '{20}')
Collapse
5: Re: OpenFTS problem with non US_ASCII characters (response to 4)
Posted by Claudio Pasolini on 11/09/04 08:26 AM
Joel,

regarding the UNICODE error something has gone wrong, because you are again trying to insert an unconverted (or badly converted) string: perhaps you could try your patch in a sample tcl script and verify if it actually does the conversion.

I also got the duplicate key problem, but I ignored it for the moment, because it is caused by the double insertion into the search_observer_queue when you create a new content (I observed this creating a news): in this case the content will be processed correctly.