Forum OpenACS Q&A: Response to ACS/PG+AOLserver3 and robot detection...
The robot updating mechanism is failing with OpenACS 3.2.5, and I don't see any bugs (open or closed) related to this problem. Used to work ok with 3.2.4.
The call to ns_geturl in ad_replicate_web_robots_db seems to fail; it returns 'file does not exist'.
I see that the url for the robot data has changed (now it's http://www.robotstxt.org/wc/active/all.txt) but the file format doesn't seem different, so I presume that ad_replicate_web_robots_db should parse the file correctly -- if it could get it. I've set this new url in the WebRobotsDB parameter in the my server config file.
Any clues about what is going on here? TIA!