Forum OpenACS Q&A: Search package with xowiki

Collapse
Posted by Eduardo Santos on
I'm trying to perform a search inside my xowiki instance, as it should be donne in the ajax search box of xowiki version 0.39. So, I index the xowiki objects and pages using the following commands:

insert into search_observer_queue (select live_revision, now(), 'INSERT' from (
select live_revision from cr_items where content_type like '::xowiki::Page') xowiki);

insert into search_observer_queue (select live_revision, now(), 'INSERT' from (
select live_revision from cr_items where content_type like '::xowiki::Object') xowiki);

insert into search_observer_queue (select live_revision, now(), 'UPDATE' from (
select live_revision from cr_items where content_type like '::xowiki::Page') xowiki);

insert into search_observer_queue (select live_revision, now(), 'UPDATE' from (
select live_revision from cr_items where content_type like '::xowiki::Page') xowiki);

The next step is to perform the search::indexer procedure, wich should index my xowiki pages for search. However, when I try to do that, I get the following error message:

Error: search::indexer: error getting datasource for 1056546
::xowiki::Page: Cannot create object -- illegal name '::' Cannot create object -- illegal name '::'
::xowiki::Package ::xotcl::Class->create
invoked from within
"my create ::$package_id"
(procedure "require" line 10)
::xowiki::Package ::xo::PackageMgr->require
invoked from within
"my require -url $url $package_id"
(procedure "initialize" line 21)
::xowiki::Package ::xo::PackageMgr->initialize
invoked from within
"::xowiki::Package initialize -package_id $package_id -user_id $user_id -parameter $parameter -init_url false -actual_que$ (procedure "instantiate_page_from_id" line 16)
::xowiki::Package->instantiate_page_from_id
invoked from within
"::xowiki::Package instantiate_page_from_id -revision_id $revision_id -user_id 0"
(procedure "::xowiki::datasource" line 4)
invoked from within
"::xowiki::datasource $object_id"
(procedure "AcsSc.ftscontentprovider.datasource.xowiki_page" line 1)
invoked from within
"AcsSc.ftscontentprovider.datasource.xowiki_page 1056546"
("uplevel" body line 1)
invoked from within"uplevel $func_and_args"
(procedure "apply" line 3)
invoked from within
"apply $proc_name $arguments"
(procedure "acs_sc_call" line 6)
invoked from within
"acs_sc_call FtsContentProvider datasource [list $object_id] $object_type"
invoked from within
search::indexer
invoked from within
adp_parse_ad_conn_file
invoked from within
rp_serve_concrete_file /var/lib/aolserver/catir/packages/acs-developer-support/www/shell.adp
invoked from within
rp_serve_abstract_file /var/lib/aolserver/catir/packages/acs-developer-support/www/shell
invoked from within
rp_handler

Any ideas about the problem? I'm using the head version for the search

Collapse
Posted by Gustaf Neumann on
eduardo,

what version of openacs do you have?

the message says essentially, that (some of) your content_revisions do not have package_id entries.

select revision_id, object_type, o.package_id from cr_revisions r, acs_objects o where object_id = revision_id order by revision_id desc limit 10;

There was a bug in earlier versions of openacs 5.2 not propagating the package_id appropriately.

you can run ::xowiki::fix_all_package_ids, which inserts the package_ids manually.

After you run the proc from ds/shell, you will see in the above query the package_ids provided. when you add a new page, check again. if the new revision has the package_id, your installation does not have the problem and you should upgrade.

Collapse
Posted by Eduardo Santos on
Thank you very much Gustaf. The procedure ::xowiki::fix_all_package_ids really fixed the problem, but I still have a concern: I'm using the OpenACS version 5.2.3, which is the last stable one. Is there any patch that I have to apply to correct this problem with the package_id? Is there any other core package for which I'm going to have the same problem?

My concerns have a meaning, because I have a huge and old system. Any upgrade is very difficult to be made, and I usually lose weeks just to test it. So, I don't want to upgrade everything right know. If it's really necessary, I'll do it with the broken package.

Collapse
Posted by Gustaf Neumann on
what happens, if you add a new page, and maybe edit it. do the new entries have the package_id? if yes, than everything is ok, which should be the case in 5.2.3...
Collapse
Posted by Eduardo Santos on
Let me tell you what I did now. It seems like the problem persists.

1 - I've created a new page in xowiki, inserted it in the search_observer_queue table and then I ran the search::indexer. I couldn't find the page in the search with the package_id.

2 - So I ran the ::xowiki::fix_all_package_ids procedure to create the package_id, did all the index stuff again and the search worked for me.

3 - To be sure about the issue I've created another page and looked for it in the acs_objects table. The page doesn't have the package_id for my xowiki instance. The other ones, for wich I had used the ::xowiki::fix_all_package_ids procedure have the package_id for my xowiki instance stored.

My conclusion is that the package_id is not being stored for new pages. Do I have to schedule the ::xowiki::fix_all_package_ids to run everytime I create a new page in xowiki? Can you tell me wich package is responsible for the bug?

Collapse
Posted by Eduardo Santos on
I forgot something. When I edit a page that already had the package_id stored, the new revision loses the package_id, as you presumed.
Collapse
Posted by Eduardo Santos on
Just another question, if you don't mind. In the file search/www/search.tcl, in the line 111 there's the following code:
# TODO calculate subsite or dotlrn package_ids
if {![string equal "this" $scope]} {
    # don't send package_id if its not searching this package
    set search_package_id ""
} else {
    set search_node_id [site_node::get_node_id_from_object_id -object_id $search_package_id]
    if {[string equal "dotlrn" [site_node::get_element -node_id $search_node_id -element package_key]]} {
        set search_package_id [site_node::get_children -node_id $search_node_id -element package_id]
    }
}
What's the purpose for this scope var? I couldn't find any definition for it, and I'm losing the the package_id in the search results page. So, if I have to go to page number two in my results, the next page doesn't take the package_id, and the search is performed in the entire site.
Collapse
Posted by Gustaf Neumann on
This was a very nasty problem that took me quite a while to find. Apparently, the problem exists only on sites updating since many versions, but not on fresh installs of oacs-5-2 from the repository. The same problem exist(ed) on openacs.org, where i first updated openacs to the latest fixes of openacs 5.2. But this did not help either.

Apparently the long running sites have "only" 4 versions of content_revision__new (do a \df content_revision__new), while new installs have 7.

To fix the problem, source from psql packages/acs-content-repository/sql/postgresql/content-revision.sql
and ignore the errors at the end. Do one more ::xowiki::fix_all_package_ids, and you are set.

It looks as if there is somewhere a missing update script.

Collapse
Posted by Gustaf Neumann on
i thought, you are using openacs 5.2? the code above is from 5.3....

the code in the search package is not from me. Apparently it is broken. When a package_id is passed in, it should be passed to next etc. as well.

The code works better, if you comment out 'set search_package_id ""' in the cited code. With this change the package_id is passed correctly in the "next" link, but still not in the link for page 2 etc. is missing.

maybe dave can comment on this.

Collapse
Posted by Eduardo Santos on
Thank you very much Gustaf; it seems to work right now. Aparently you are right about the missing upgrade script, but I couldn't find any new definition for the function content_revision__new in any of the upgrades. I tried to see if in some of the revisions for the file content-revision.sql somebody put more definitions for the function, and I couldn't find it either. The best thing I found is between the dotlrn 2-2-1 final and the HEAD. You can see the diff:

http://cvs.openacs.org/cvs/openacs-4/packages/acs-content-repository/sql/postgresql/content-revision.sql?r1=1.45&r2=1.35

I don't think it's related to this bug anyway, but maybe it's worth to take a look. However, sourcing the file seems to solve everything right now. My search is working just fine.

Collapse
Posted by Eduardo Santos on
I'm using oacs 5-2, but I've installed the search package from the HEAD, because there are a lot of new search procs wich I wanted to use. As it doesn't depend on any other packages, I can use it without any problem.

About the query I told you, I did comment it out. There's no reason for that check to be made anyway, and I don't think some bug is coming from there. About the links for the pages it's an easy fix that needs to be made, and I did it already. Thanks for the help again.

Collapse
Posted by Gustaf Neumann on
good news.

please report the bug + fix for search.tcl via bug-tracker, it might help some pleople and makes sure, changes are nit forgotten.

Collapse
Posted by Eduardo Santos on
Bug reported and patch uploaded in bugtracker:

https://www.openacs.org/bugtracker/openacs/bug?bug%5fnumber=3061