Forum OpenACS Q&A: Example Service Contract for bboard (Search/OpenFTS)

Sorry for bothering you with this again, I am trying to write a service contract for the bboard module.

Therefore I wrote a bboard-sc-create.sql that looks like the following:

select acs_sc_impl__new(
           'FtsContentProvider',                -- impl_contract_name
           'apm_package',                       -- impl_name
           'bboard'                             -- impl_owner_name
);

select acs_sc_impl_alias__new(
           'FtsContentProvider',                -- impl_contract_name
           'apm_package',                       -- impl_name
           'datasource',                        -- impl_operation_name
           'bboard__datasource',                -- impl_alias
           'TCL'                                -- impl_pl
);

select acs_sc_impl_alias__new(
           'FtsContentProvider',                -- impl_contract_name
           'apm_package'                        -- impl_name
           'url',                               -- impl_operation_name
           'bboard__url',                         -- impl_alias
           'TCL'                                -- impl_pl
);

I then loaded the functions via the command:

psql unido-dev -f bboard-sc-create.sql

and got the following output:

NOTICE:  Adding missing FROM-clause entry for 
table "acs_object_id_seq"
psql:bboard-sc-create.sql:5: NOTICE:  
identifier "acs_object__initialize_attributes" will be truncated 
to "acs_object__initialize_attribut"

 acs_sc_impl__new
------------------
            36549
(1 row)

 acs_sc_impl_alias__new
------------------------
                  36549
(1 row)

psql:bboard-sc-create.sql:21: ERROR:  Function 'acs_sc_impl_alias__new
(unknown, unknown, unknown, unknown)' does not exist
        Unable to identify a function that satisfies the given 
argument types
        You may need to add explicit typecasts

Under the tcl directory in a file called bboard-procs.tcl I defined the two procs:

ad_proc bboard__datasource {
        object_id
} {
        @author Gregor Obernosterer
        @creation-date 20-08-2002
} {
        db_0or1row bboard_datasource {
                SELECT
                b.message_id,
                b.forum_id,
                c.item_id as object_id,
                c.title as title,
                c.content as content,
                c.mime_type as mime,
                '' as keywords,
                'text' as storage_type
                FROM
                bboard_forum_message_map b,
                cr_revisions c
                WHERE
                b.message_id = c.item_id
                AND
                item_id = :object_id
                ORDER BY
                c.item_id DESC
                LIMIT ALL

  } -column_array datasource

  return [array get datasource]
}

ad_proc bboard__url {
        object_id
        forum_id  # I also need the forum_id since the search result
                  # page is built up via the message_id that          
                  # corresponds to a forum_id ... or not?
} {
        @author Gregor Obernosterer
        @creation-date 20-08-2002
} {

        set package_id [apm_package_id_from_key bboard]
        db_1row get_url_stub "
                SELECT
                site_node__url(node_id) as url_stub
                FROM
                site_nodes
                WHERE
                object_id = :package_id
        "

        set url "${url_stub}message?
message_id=$object_id&forum_id=$forum_id" #not sure how to built this.
        return $url
}

I suppoes they are not correct - I am a rookie to that at all so I hope not to bother you too much with that. Maybe Neophytos or Dave can give me a hand after looking over this ...

Before I forget, under the search module I found a doc with the title called "How to make an object type searchable?" This doc comprises an example on writing a datasource proc, but misses out the url proc.

Beside all the mistakes I have made in the code above, is there anything else missing (except to bind the impl_name via the service- contract web interface... that's how it is done or...)?

Thanxs for the help. I corrected the syntax error and also changed impl_name to bboard_messages. I was then successfully able to bind the service contract:

Installed Bindings

• 822 FtsEngineDriver, 6174 openfts-driver [Uninstall]
• 841 FtsContentProvider, 3758 note [Uninstall]
• 841 FtsContentProvider, 5660 file_storage_object [Uninstall]
• 841 FtsContentProvider, 881 content_template [Uninstall]
• 841 FtsContentProvider, 6290 news [Uninstall]
• 841 FtsContentProvider, 38449 bboard_messages [Uninstall]

Valid Uninstalled Bindings

• 841 FtsContentProvider, 879 content_revision [Install]
• 841 FtsContentProvider, 880 image [Install]

Invalid Uninstalled Bindings

• None

Orphan Implementations

• 12603 pinds_blog_entries RssGenerationSubscriber

But how do I get the indexer to search through the new service contract, or ist this just a cronjob and I have to wait or change the scheduled time?

Hmmm, this really really really should be in a UI somewhere, perhaps in the admin pages for the openfts package. Any takers?

Well as far as I know there is in the doc section of the search-module a document with the title called 'How to make an object searchable?' - guidelines.html. However, I think it is not up to date, because the FtsContentProvider is comprised of two abstract operations, namely datasource and url. The datasource operation is documented, but not the url one. I would like to update the guidlines and hand it over for proofreading to Dave or Neophytos (as Neophytos is involved in the OpenFTS development).

I think we should have a look on ht://dig and their documentation. Especially hoe to parameterize the search engine and administer the search algorithms such as soundex etc.

In the guidelines document we should also make the users aware that the indexer will only index new entries and that in order to be able to search for old content a function has to be performed on the desired content.

... well my search on the bboard still doesn't work. Maybe my datasource proc and/or url proc in the bboard_procs is still malfunctioning - although the web interface allows me to bind the service contract without any error messages. Maybe anybody has an idea why that could be...

Cheers, Gregor

P.S.: Just a note. From September 17-20 there will be an IT- and Security-fair in Austria called IFABO we will be there and have a booth where we present OpenACS, and our in house developed GPL licensed Intrusion Detection System called EVENT HORIZON. I have free tickets, so if anybody wants to come, just drop me your address and I send you as many entry passes as you need... if there is some interest we can maybe afterwards hold a small social.

Gregor, the impl name has to be the same as the acs object_type of the items you want to index.

So the impl_name should be bboard_message.

One way to check if the items are being indexed is to check the search_observer_queue table. After you add the items to the queue, they should only stay in there for 30 secs or so until the indexer runs again. It is a scheduled proc.

You can also do a select count(*) from txt. This is the actual table that holds the index. The count will tell you how many items have been indexed.

Here are my service contract implementations for forums:

ad_library {

    Forums Library - Search Service Contracts

    @creation-date 2002-08-07
    @author Dave Bauer 
    @cvs-id $Id: 

}

namespace eval forum::message {

ad_proc datasource { message_id } {
    @param message_id
    @author dave@thedesignexperience.org
    @creation_date 2002-08-07

    returns a datasource for the search package
    this is the content that will be indexed by the full text
    search engine

} {
    forum::message::get -message_id $message_id -array message
	set datasource(object_id) $message(message_id)
	set datasource(title) $message(subject)
	set datasource(content) $message(content)
	set datasource(keywords) ""
	set datasource(storage_type) text
	set datasource(mime) "text/html"
    return [array get datasource]
}

ad_proc url { message_id } {
    @param message_id
    @author dave@thedesignexperience.org
    @creation_date 2002-08-07

    returns a url for a message to the search package

} {
    forum::message::get -message_id $message_id -array message
    set forum_id $message(forum_id)
    return "[ad_url][db_string select_forums_package_url {}]message-view?message_id=$message_id"
}

}