Forum OpenACS Q&A: OpenFTS 0.3.1 Problems

can anyone (I'm desparate) help out with this one.

I've got pretty much all the way with installing the newer version of the OpenFTS but when i initialise it (via the admin screen) it bombs.

I kindof expected that as I wasn't able to perform one of the steps in the instalation instructions, namely

psql nsprint < /web/nsprint/packages/openfts-driver/sql/postgresql/load.sql

as it failed complaining that

ERROR: stat failed on file $libdir/_int : no such file or directory

Any ideas how I can get that to work?

Everything else has gone fine, but the problem is that the installation instructions are out of date and still refer to version 0.2, and my suspicion is there's a step to change/miss/add.

any advice very, very gratefully received (muggins has a delivery date on Friday ... yuk)

Installation:

1. Install aolserver.
2. Build and install postgresql tsearch option from contrib directory.
3. Build aolserver module:


        tar xvzf openfts-tcl-0.3.tar.gz
        cd openfts-tcl-0.3
        ./configure --with-aolserver-src=DIR 
        cd aolserver
        make
        cp fts.so [aolserver bin directory]
        cd ..
        cp fts_*.tcl [aolserver tcl lib directory] (don't do this if installing with openacs)
        (if not installing with openacs4 use: cp aolserver/test/10-database-procs.tcl [aolserver tcl lib dir])
        cd PGSQL_SRC_HOME/contrib/tsearch
        make
        make install
        psql template1 < tsearch.sql
        cp -r pgsql_contrib_openfts PGSQL_SRC_HOME/contrib      
        cd PGSQL_SRC_HOME/contrib/pgsql_contrib_openfts
        make 
        make install
        psql template1 < openfts.sql (installing in template1 makes openfts available to all new dbs that are created - if you don't want openfts in all of your newly created dbs, then only load this file into the db you want to use openfts with)

4. Edit nsd.tcl file and add entry for fts module:

ns_section "ns/server/${server}/modules" 
        ns_param   nssock          ${bindir}/nssock.so 
        ns_param   nslog           ${bindir}/nslog.so 
  ...
        ns_param   nsfts           ${bindir}/nsfts.so

juts want to check I'm getting everything right from the 4.6 branch.

the INSTALL file i have hs no indication its been updated and still contains an instruction line telling you to install load.sql frm this packages directory.

Am i looking at the right version? sorry for bing a pain, but its tricky to tell.

44 eusdawi@edgedsp6:/home/unix/wickstrom/web/openacs-4-6/packages/openfts-driver/www/doc>cvs log INSTALL
danw@openacs.org's password: 

RCS file: /cvsroot/openacs-4/packages/openfts-driver/www/doc/INSTALL,v
Working file: INSTALL
head: 1.3
branch:
locks: strict
access list:
symbolic names:
        oacs-4-6: 1.3.0.4
        oacs-4-5-final: 1.3
        oacs-4-5-rc-1: 1.3
        don-merge-1: 1.3
        oacs-4-5-beta-1-2: 1.3
        oacs-4-5-beta-1-1: 1.3
        oacs-4-5-beta-1: 1.3
        oacs-4-5: 1.3.0.2
keyword substitution: kv
total revisions: 4;     selected revisions: 4
description:
----------------------------
revision 1.3
date: 2001/09/15 21:31:50;  author: neophytosd;  state: Exp;  lines: +6 -6
branches:  1.3.4;
enhanced search package
----------------------------
revision 1.2
date: 2001/09/04 15:39:51;  author: neophytosd;  state: Exp;  lines: +1 -1
changed gmake-install-headers to gmake install-all-headers
----------------------------
revision 1.1
date: 2001/09/01 20:33:52;  author: donb;  state: Exp;

Added this to the tree ...
----------------------------
revision 1.3.4.1
date: 2002/10/11 17:09:32;  author: danw;  state: Exp;  lines: +5 -27
updated docs and removed load.sql
=============================================================================

Have you updated from cvs lately?

anyway, i have a 4.6 checkout that i regularly cvs update but each time i do it at the moment I'm not getting your changes.

therefore i don't know if maybe my install is screwed? i wouldn't have thought so as its picked up other changes over the past few weeks.

unfortunately i'm stuck at home now and don't have the bandwidth to bring down a new checkout entirely, but i'll try doing that in the morning if you're sure there's nothing else this could be.

i'll do a complete re-install, from pg upwards, tomorrow and follow your advice here exactly.

If could mail me over a copy of the openfts-driver package that you have there, that you know is working that would be a big help. i could at least compare that to what I have here.

Can someone check this.

invalid command name "Search::OpenFTS::Parser::run_parser"
It appears this isn't sourced anywhere.

If you are using AOLserver >= 3.5 you also need to specify your tcl config: --with-tcl=/usr/lib or wherever.

Now everything is fine. I will add a little upgrade howto and I think that the default param in .info should be changed to 0.3.1 instead of 0.2

403 eusdawi@edgedsp6:/home/unix/wickstrom/web/openfts/tcl/examples>./types.tcl 
  1 => Latin word
  2 => Cyrillic word
  3 => Word
  4 => Email
  5 => URL
  6 => Host
  7 => Scientific notation
  8 => VERSION
  9 => Part of hyphenated word
 10 => Cyrillic part of hyphenated word
 11 => Latin part of hyphenated word
 12 => Space symbols
 13 => HTML Tag
 14 => HTTP head
 15 => Hyphenated word
 16 => Latin hyphenated word
 17 => Cyrillic hyphenated word
 18 => URI
 19 => File or path name
 20 => Decimal notation
 21 => Signed integer
 22 => Unsigned integer
 23 => HTML Entity

In my current setup, I have it configured to not index html tags, space symbols, and HTTP head. The openfts driver also allows you to restrict what is shown in a headline display.

I don't recall the search package ever giving a warning for using stopwords for search terms.

Okay, now I understand.  I've never looked at the search package in detail, so I wasn't even aware of that bit of code.  I've fixed the problem, which requires changes to both openfts and the openfts-driver, so I will be releasing a new version of openfts soon.

Specifying all of the stop words in advance is not practical, as there can be hundreds of stopwords, and if your doing multilingual searching, each dictionary can have its own list of stop-words.  I think the way it is implemented now is the most reasonable.

No offence meant, I just assumed you'd read between the lines a bit :o)

Anyway, don't jump all over me, but, as the Search package and the OpenFTS are so inherently intertwined, I just assumed when chnages to one were going on, then someone would be looking after how that affects the operation of the other.

Any idea when you'll make the new versions available Dan?

PS, what I really meant about highlighting keywords in advance is not to necessarilay list them all, just point out on the search screen that such a thing applies.

Its the very first thing that caught out my customer yesterday. Just a minor suggestion. Good help text/instructions can go a long way to even overcoming poor software or design.

http://www.infogettable.net/search/search?q=retires+where+are+you

did not make it into the code base since I'm using 4.6.1 release of oacs and having the same problem with ERROR: Relation 'txt' already exists when installing OpenFTS Driver.

When I manually drop the txt table, the driver install succeeds. I didn't have an older version of OpenFTS installed but had other issues on my install and dropped my OpenFTS driver and reinstalled it. Then I got the error with the txt table existing.