New bot work continues at Bing

Back in December of 2008, we alerted you to the possibility that you might start seeing our new web crawler in your referrer logs. Given the recent excitement around the arrival of Bing, we wanted to take a moment to update you on the latest bot news.

As we said earlier, you will see the new Bing crawler user agent string listed as:

msnbot/2.0b (+http://search.msn.com/msnbot.htm).

Note that we didn’t change the bot’s name for Bing. You’ll also see the previous version of the bot as well —

msnbot/1.1 (+http://search.msn.com/msnbot.htm)

— which is still in use, as of today, as Bing’s primary user agent.

We are busy working on our improved crawler and plan to ramp up its workload as we move toward the goal of putting it into full production. As a result, webmasters may begin seeing an increased amount of traffic from the new bot on their sites.

We do not anticipate any problems related to our increasing emphasis on MSNBot 2, but the unexpected can’t always be avoided, no matter how hard you try! As such, we wanted to preemptively alert folks to the most effective way to report bot and crawling issues to Bing’s support team in case they arise. If you have any feedback for us on either the existing or the new MSNBot, or on any other matters pertaining to crawling your website, please post your comments and questions in our crawling/indexing forum. Our Webmaster Center forums moderator will follow-up with you ASAP. Please include in your forum post the identity of the user agent that is the source of your concern so we can help you that much faster. Thank you!

— Rick DeJarnette, Bing Webmaster Center