Forum OpenACS Q&A: ACS/PG+AOLserver3 and robot detection...

Collapse
1: ACS/PG+AOLserver3 and robot detection...
Posted by Don Baccus on 04/22/00 01:31 AM
Has anyone gotten this to work?  I get an error from nsd.geturl,
trying to connect(), it says:

connect() failed: resource temporarily unavailable

when it tries to hit the web page at info.webcrawler.com.  I checked
at their server's up, that's not the problem.

Presumably some silly configuration problem, anyone want to give this
lazy-assed dude a hint?

Collapse
2: Response to ACS/PG+AOLserver3 and robot detection... (response to 1)
Posted by Michael Feldstein on 06/15/00 08:45 PM
Just curious; did you ever figure this one out?
Collapse
3: Response to ACS/PG+AOLserver3 and robot detection... (response to 1)
Posted by Don Baccus on 06/15/00 09:25 PM
It was a silly configuration problem.  I forget what, exactly.
Collapse
4: Response to ACS/PG+AOLserver3 and robot detection... (response to 1)
Posted by Stan Kaufman on 06/14/01 03:36 AM
Any clue what the silly configuration problem was?

The robot updating mechanism is failing with OpenACS 3.2.5, and I don't see any bugs (open or closed) related to this problem. Used to work ok with 3.2.4.

The call to ns_geturl in ad_replicate_web_robots_db seems to fail; it returns 'file does not exist'.

I see that the url for the robot data has changed (now it's http://www.robotstxt.org/wc/active/all.txt) but the file format doesn't seem different, so I presume that ad_replicate_web_robots_db should parse the file correctly -- if it could get it. I've set this new url in the WebRobotsDB parameter in the my server config file.

Any clues about what is going on here? TIA!