Guest michelsmith Posted May 25, 2012 Report Share Posted May 25, 2012 The major seo expert use this technique for security purpose also the robot.txt file expert talking about search engine spiders and robots .so please give me suggestion regarding robot.txt file Quote Link to post Share on other sites
rocket976 62 Posted August 23, 2012 Report Share Posted August 23, 2012 the robot txt file can be disregarded it is only a guide line for bots to follow suggested pages. If you can find a link that is not in the robot.txt file there is nothing stopping you from following it.. http://www.robotstxt.org/robotstxt.html Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.