Bing crawler: bingbot on the horizon

Since our last post in November, the Bing team has been busy rolling out improvements to the Bing web crawler. As a result of this work, we want to announce in advance our plans to change the name of our crawler (aka user agent).

Out of beta with a new name

On October 1st, 2010, we will drop the beta designation from the Bing crawler and change the name of the crawler to reflect Microsoft’s new brand for search. Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:

Mozilla/5.0 (compatible; bingbot/2.0; +

The HTTP header From field will also change as shown below:

From: msnbot(at)

will become

From: bingbot(at)

Robots Exclusion Protocol directives precedence

We want webmasters to know that bingbot will still honor robots.txt directives written for msnbot, so no change is required to your robots.txt file(s).

Please note, however, that if we detect separate sets of directives for bingbot and for any of the older versions of Microsoft search bots (such as msnbot) or a set of directives for all crawlers, the directives for bingbot will take precedence. For example, in the following case, Bing will be authorized to crawl on all whole hosts except on the folder /folder1/, despite the more comprehensive blocking directives for other crawlers:

User-agent: bingbot
Disallow: /folder1/

User-agent: msnbot
Disallow: /folder1/
Disallow: /folder2/

User-agent: *
Disallow: /

How to report issues

We always welcome feedback from the webmaster community. Over the past few months, during the recent development phase, we had different issues reported to us. We want to give our heartfelt thanks to the webmasters who contacted us, and offer our sincere apologies for any issues they may have encountered as a result of the crawler traffic.

If you have any questions regarding BingBot traffic and your site, please contact us at This is the most effective way to reach us. We do our best to respond to your inquiries promptly and report back to you on issues brought to our attention. When you do so, please provide us with a snippet of your logs showing the BingBot traffic as well as a list of the sites served from the same infrastructure, if there is more than one. This way, we can more quickly diagnose and resolve the issues in question.

We look forward to seeing your comments and feedback. Thanks!

Additional bot-related postings:

— Dean Wierman and Fabrice Canel, Bing Crawl and Index teams

Join the conversation

  1. Quality Directory

    The change of name has been long overdue, and many have been complaining that this needed to be done, to reflect the new face of search.

  2. InaiMathi

    MSNbot never (and by never I mean NEVER) crawled my website since Live Search became Bing. I've submitted my site's sitemap several times – with no effect! I mean… c'mon… even Yahoo has indexed it! *sigh* It's quite frustrating… :-(

  3. Quality Directory


    Open a thread in the Webmaster Forum section about the issue you're having, and we will look into it and try to resolve it.

  4. hotels

    This seems much better.

  5. bouka55

    Its good that Bing also supports rel nofollow, its a much easier way to block some pages. using disallow, its easy to make mistakes and drop pages that you want indexed!

  6. Nico26be

    This is good and is exactly what needs to be done!

    Any intel about the performance? Is it gonna change? Is it gonna crawl web sites more often? And more extensively?

  7. GirlyGirl

    Great news! sort of . . . I hope they do more than change the name: I hope they actually improve the way it works, providing a more relevant index like that of Google Caffeine.  But why do we have to wait until October?  This mentality of introducing periodic updates every six months or so is so very typical of the mentality of the Microsoft of old: when companies had to send out updates and did so en masse.  What do companies of new like Google do? They come out with updates almost daily, doing web-based updates.  Of course, Microsoft does this, too, with updates to Office, etc. But, seriously, why wait? Why not put out the update bot (and hopefully an updated search) right now?

  8. Pittbug

    Will the DNS entries for the crawlers also change?

    e.g. currently resolves to:

    Will it change to something like this?


  9. track.wss

    Nice information its very useful for my client thnks for sharing

  10. Standees

    Do you think that bingbot will borrow some of the 'features' of yahoobot or will it be a totally new robot made from the ground up? thanks

  11. ajwatt92

    Good news – hopefully the new crawler will be more efficient than the current one.

  12. sonny.chatrath

    I just logged in to the webmasters tools, and didn't find any pages indexed. What would I need to do in order to get our pages indexed?

  13. Indo Money Online

    awesome posting, I'll try to my site here:

  14. sohel

    Good article and a nice summation of the problem. My only problem with the analysis is given that much of the population joined the chorus of deregulatory mythology, given vested interest is inclined toward perpetuation of the current system and given a lack of a popular cheerleader for your arguments, I'm not seeing much in the way of change.

  15. ClarionCX501

    This is great new if it is implemented properly.  Bing seems to see many of sites more favorably than google and about the same as yahoo.  In any event I am a bing fan!


  16. crozzzlinx

    bing bot is slow but sure, thanks for indexing my blog

  17. Sil

    Bingbot is making all my websites go down when it crawls… Please I need a hint to go through this issue…

  18. Lalacan

    I wonder how often bingbot crawls my blog? How to improve my blog visitor from Bing?


Comments are closed.