Forum Discussion

vaishnavi's avatar
vaishnavi
Expert
10 years ago

Archive boards should not get listed in search engine results

Hi , I have a separate category "Archive" for some boards which are not supported(and archived) now. The archive boards seem to have strong SEO attached to them, so users are still arriving at thes...
  • VarunGrazitti's avatar
    VarunGrazitti
    10 years ago

    vaishnavi - You could get your robots.txt updated and this should solve the purpose. This document should help you. This doc will guide you to basic robots.txt knowledge.

     

    The robots.txt lives on the root, and here is the URL for robots file for Lithium community: http://community.lithium.com/robots.txt

     

    yourcomunity.com/robots.txt

     

    I hope this helps.

     

     

vaishnavi's avatar
vaishnavi
Expert

Thank you Fellsteruk, VarunGrazitti, OlivierS for your help.

 

So there are 2 ways to hide boards from SEO crawlers - robots.txt and google's webmasters tools.

Both the ways would require manual data entry by community admin/moderator.

 

Is there any way where community admin can configure the robots.txt from settings? 

This would be easy for community admin to update robots.txt when a new board is moved udner "Archive" category.

 

how to enable configurable robots.txt in admin?

Any pointers?

 

Regards,

Vaishnavi

 

VarunGrazitti's avatar
VarunGrazitti
Boss
10 years ago

vaishnavi - This cannot be done via admin as this is not a Lithium exclusive thing. You'd need to submit your robots.txt file to lithium support and they will update it on the server for you.

 

For your second question, you wouldn't have to update the robots.txt every time, you just need to disallow Archive category in it and moving forward, everything inside it would be disallowed. Whenever you archive a board, it will be inheriting disallowed property from Archive category. So, it is just a one time change. 

 

I hope this helps.