Pinterest Stumbleupon Whatsapp

With the help of a Robots.txt file, site owners can tell search engines which parts of the site not to look in. Almost all search engines respect this protocol. Although you can create a Robots.txt file yourself, “MN Robots.txt Generator” makes the job much easier.

create a robots.txt file

MN Robots.txt Generator is a free web service that can generate a Robots.txt file for you. You start by setting a default option (set this to either “allow” or “refused”). Next you type in your sitemap URL.

Below that you will find some robots (search engine names) like Google, Yahoo, and MSN Search, amongst others. You can choose the access of each search engine to be set as your default option or specifically “Allow” or “Refuse” its access. You will also find some special search bots like Google Image and Yahoo Blogs for which you perform the same “Allow” and “Refuse” procedure.

Finally you can enter the paths search engines should not look in, and click on the “Create Robots.txt” button. The generated text should be used as your Robots.txt file.

generate a robots.txt file


robots.txt file generator


Check out MN Robots.txt Generator @ MN Robots.txt Generator

Leave a Reply

Your email address will not be published. Required fields are marked *