Trending Now

Latest News – Find Most Trending topics, News Articles, videos, and viral Stories. Articles, Photos, Videos, most frequently read

Tuesday, 19 May 2015

How To Adding Custom Robots.Txt File in Blogger

What is Robots.txt? 

Robots.txt is a content record which contains few lines of straightforward code. It is saved money on the site or blog's server which educate the web crawlers to how to file and slither your web journal in the list items. That implies you can confine any site page on your online journal from web crawlers so that it can't get ordered in internet searchers like your website marks page, your demo page or whatever other pages that are not as critical to get recorded. Never forget that hunt crawlers filter the robots.txt document before slithering any page.

Every site facilitated on blogger have its default robots.txt document which is something resemble this:

User-agent: Mediapartners-GoogleDisallow:User-agent: *Disallow: /searchAllow: /Sitemap: 


This code is partitioned into three segments. How about we first study each of them after that we will figure out how to include custom robots.txt document in blogspot web journals.

  • User-agent: Mediapartners-Google

This code is for Google Adsense robots which help them to serve better advertisements on your online journal. It is possible that you are utilizing Google Adsense on your online journal or not just abandon it as it seems to be.

  • User-agent: *

This is for all robots stamped with reference bullet (*). In default settings our blog's marks connections are limited to ordered via look crawlers that implies the web crawlers won't list our names page joins due to underneath code.

  • Disallow: /search

That implies the connections having pivotal word look recently after the area name will be overlooked. See beneath case which is a connection of name page named SEO.


Furthermore, on the off chance that we evacuate Disallow: /search from the above code then crawlers will get to our whole blog to record and slither every last bit of its substance and pages.

Here Allow:/ alludes to the Homepage that implies web crawlers can slither and record our blog's landing page.

Disallow Particular Post 

Presently assume in the event that we need to bar a specific post from indexing then we can include underneath lines in the code.


Here yyyy and mm alludes to the distributed year and month of the post individually. Case in point in the event that we have distributed a post in year 2013 in month of March then we need to use beneath configuration.


To make this errand simple, you can just duplicate the post URL and expel the web journal name from the earliest starting point.

Disallow Particular Page

In the event that we have to refuse a specific page then we can utilize the same technique as above. Just duplicate the page URL and expel site address from it which will something resemble this:



This code alludes to the sitemap of our online journal. By including sitemap connect here we are basically upgrading our blog's slithering rate. Implies at whatever point the web crawlers examine our robots.txt document they will discover a way to our sitemap where all the connections of our distributed posts present. Web crawlers will think that it simple to creep the greater part of our posts. Thus, there are better risks that web crawlers creep the majority of our blog entries without overlooking a solitary one.

Note: This sitemap will just enlighten the web crawlers regarding the late 25 posts. On the off chance that you need to build the quantity of connection in your sitemap then supplant default sitemap with underneath one. It will work for initial 500 late posts 


On the off chance that you have more than 500 distributed posts in your online journal then you can utilize two sitemaps like underneath:



Adding Custom Robots.Txt to Blogger 

Presently the principle piece of this excercise is the way to include custom robots.txt in blogger. So underneath are ventures to include it.

  • Go to your blogger blog. 
  • Explore to Settings >> Search Preferences ›› Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes 
  • Presently glue your robots.txt record code in the case. 
  • Tap on Save Changes catch. 
  • You are finished! 

Blogger Custom Robots.txt

How to Check Your Robots.txt File? 

You can check this record on your online journal by adding/robots.txt finally to your website URL in the program. Investigate the underneath case for demo. 

When you visit the robots.txt document URL you will see the whole code which you are utilizing as a part of your custom robots.txt record.


Last Words! 

This was the today's finished excercise on the best way to include custom robots.txt record in blogger. I truly attempt with my heart to make this excercise as basic and enlightening as would be prudent. Yet in the event that you have any uncertainty or inquiry then don't hesitate to ask me. Try not to put any code in your custom robots.txt settings without thinking about it. Just ask to me to determine your inquiries. I'll let you know everything in point of interest. Much appreciated gentlemen to peruse this excercise. In the event that you like it then please bolster me to spread my words by sharing this post on your online networking profiles

No comments:

Post a Comment

Your comment can be deleted: If your comments something like these: abusive, harassing, threatening, pornographic, false, misleading, offensive