Forum OpenACS Q&A: OpenACS.org not archived on archive.org

Request notifications

Collapse
Posted by Brian Fenton on
Hi, I frequently use the Internet Archive Wayback Machine (www.archive.org) to look at archives of websites. I tried to see if OpenACS.org was archived and it appears that it's not because of a robots.txt file:

http://web.archive.org/web/*/openacs.org/*

We're sorry, access to http://openacs.org/* has been blocked by the site owner via robots.txt.

I think it would be nice if we removed the robots.txt entry for the Wayback Machine. They use the Alexa spider, so we would need to remove the block on Alexa. It's always useful to look back and see your own history.

http://web.archive.org/web/*sa_/http://www.arsdigita.com

:-)

Brian

Collapse
Posted by russell muetzelfeldt on
hmmm... does that robots.txt not also block the google spider? surely the forums (at the very least) should be searchable in google...