cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Highlighted
Occasional Advisor

Google indexing in a closed community

Since we have a closed community, we have the challenge of figuring out how to allow Google searches to find TKB articles.  The idea is that someone who isn't logged in to our Community does a Google search, finds an article, but is prompted to log in/create an account before being taken to the article.  

 

The way we manage our permissions is to have a default role that allows all community members to search the TKB. The Default role allows only access to a collection of public information.

 

I've been poring through the permissions but don't see anything that would give us this outcome and am hoping someone else has some widsom you can share.

4 Replies 4
Highlighted
Lithium Alumni (Retired)

The idea is that someone who isn't logged in to our Community does a Google search, finds an article, but is prompted to log in/create an account before being taken to the article.

Hi @lcgolden

 

as far as I am aware - google (and other search engines) only index content that is accessible to the web crawler bot. The bot is equivalent to an anonymous user, so the content should be viewable for an anonymous user to be crawled and indexed by the bot. It's probably (I haven't checked but I would expect so) also part of the search engines's guidelines to only index content that is publicly accessible.

 

You could probably do a customization to display a call to action to invite the user to log in or register, but they shouldn't be prevented from accessing the content if that content has been indexed by a search engine.

 

Alternatively, you could think of publishing public, "teaser" content articles (that can be indexed) and then keep the full content in a private area only accesible upon registration.

 

Hope it helps,


Paolo Tagliaferri
Principal Technical Consultant

Lithium
Highlighted
Lithium Alumni (Retired)

@lcgolden I've no idea if that would work or not, but try allowing bots crawling your TKB folder by updating your robots.txt file

Kudo Monster

"Me want kudo!", "Me eat kudo!", if you appreciate my efforts please give me a kudo ↓
"Om nom nom nom", accept as solution to help others find it faster.
Highlighted

Thanks for the input! I'll see if this can be done by modifying robots.txt. We actually do have a few teaser articles already, but the goal at this point is to have all articles indexed.

Highlighted
Lithium Alumni (Retired)

Hi @lcgolden

 

robots.txt is usually tweaked to prevent Google (and other search engines) to prevent unintentional access/indexing of "private" areas. It is not used the other way around.

 

If the article is not visible for an anonymous user, it will not be indexed and this is exactly how it's supposed to work. If the content is indexed, it is not private anymore by definition.


Paolo Tagliaferri
Principal Technical Consultant

Lithium