Forum OpenACS Q&A: Search Engines and registered users

Collapse
1: Search Engines and registered users
Posted by Bob OConnor on 06/06/01 07:13 PM

I am looking for ideas and solutions for this situation:

  • Use OpenACS 😊 and require that everyone log on to see our *fabulous* content. Content can be static ADP pages.

  • Use the email addresses to occasionally send an email newsletter (it's "opt in spam"! ) to tell 'em about new content etc... We don't want unknown users to view our content 'cause...

  • Because only logged on users can see our content, I presume that search engine robots will NOT see the content and therefore we won't appear in search engines for most of our content. (This is a 'bad thing')

So, how do we show unregistered robots but NOT unregistered users our content?

TIA
-Bob

Collapse
2: Response to Search Engines and registered users (response to 1)
Posted by Stephen van Egmond on 06/06/01 07:34 PM
At the moment you decide to reject the unregistered user, take a peek at the user agent. If it's from the search engines you want to let in, let it in.

Make sure to submit a single page to each search engine that links to all the content you want indexed.

WATCH OUT in case the search engines rename their user agents.

Then, watch helplessly as people visit your site from Google, get challenged to register, click back, then click on Google link that shows you the cached copy.

Collapse
3: Response to Search Engines and registered users (response to 1)
Posted by James Punteney on 06/06/01 08:45 PM
While Stephen's method would work you have to be careful with it as some search engines would consider it as Spamming (in the search engine sense).  While I'm sure you aren't planning on using this in a deceitful way, the Search Engines have no way of being able to tell your intentions.  If they do deem it as Spam they might de-list your site from their index, which if you aren't in there anyways might not be that big of a threat.

My suggestion (not knowing the type of site you have) would be to select a few pieces of content that are optimized for your most important key words and make them available to everyone including the Search Engines.  This could help get new users to register as they would be able to sample a small sub-section of your content before they have to register.

Hope that helps,

--James