Forum OpenACS Q&A: Response to OT: How to search the web...

Collapse
Posted by John Sequeira on
Google doesn't let you spider their site,  but they offer a SOAP-based API for noncommercial use IIRC.  I've used it via the Perl module,  and it works as advertised.

http://www.google.com/apis/

Also,  check out spyonit.com - they have something like what you're looking for,  but it's not programmable.