Just to le you know:
We've got our first "Intranet" search engine running. You can see the online demo at http://ptdemo.dnsalias.com/. Log in as "Ben Bigboss".
The permission system finally uses a multi-stage filter architecture:
1. The SQL query uses a "join" to discard items that belong to "business objects" that are not visible to the current user. This is a relatively cheap procedure that can be applied to a huge result set effectively.
2. The remaining objects are joined and ranked by TSearch2
3. What's coming out of this is "paginated" by a "limit - offset". PostgreSQL may still generate the whole result set, but atleast that stays inside the DB.
4. The TCL "db_foreach" look contains object specific calls to the (TCL) permission routines for each object iteratively. We actually receive "limit"*10 rows from the DB in order to be sure to return the whole "limit" number of rows.
We still need to test this with larger result sets, but it already looks quite ok.
Bests,
Frank