Forum OpenACS Q&A: Response to OT: How to search the web...

Posted by John Sequeira on
Google doesn't let you spider their site,  but they offer a SOAP-based API for noncommercial use IIRC.  I've used it via the Perl module,  and it works as advertised.

Also,  check out - they have something like what you're looking for,  but it's not programmable.