Just a warning for people who may not know ....
Initially I had an ad_returnredirect in my index.tcl to redirect to an "about" page if the visitor was not logged in.
However the google web crawler did not follow this redirection. On a google support forum it was pointed out to me that my web server was using the http status 302 to redirect browsers. Apparently web crawlers do not follow 302 redirections. The 302 is generated by ad_returnredirect.
After I changed to use rp_internal_redirect google started following my internal links correctly.
