Robots.Txt: stops search engine bots to crawl part of your blog
Using robots.txt file you can give instructions about their site to web robots. While search engine bot comes to your blog they have limited resources to crawl. So after certain crawling they stop and you […]