Crawler improvements—temporary increase in crawl rate

In our continuing efforts to provide great value to our publishers, the Bing Engineering team is constantly testing new features to improve crawl efficiency. Such experiments are vital to our ability to innovate and improve our end user and publisher experience.

Due to an upcoming experiment, a few sites may encounter an intermittent increase (anywhere from 5% to 25%) in crawl requests from MSNBot over the next month (specifically from August 25th through end of September). If you see an increased crawl rate on your site and want to throttle it down, please contact bwmc@microsoft.com as soon as possible, and we will address your concerns. Thanks for your understanding as we try to improve Bing for all of us.

– Sasi Parthasarathy, Senior Program Manager, Bing Engineering

Join the conversation

22 comments
  1. Quality Directory

    BingBot's crawl request is one of the most-complained issues in webmaster circles. I hope this so-called "intermittent increase in crawl requests" doesn't pull down my server.

    However, I welcome the improvement.

  2. casp_40

    the more crawls the merrier, thank you guys so much for finally taking search seriously!

  3. ROIGAIN

    that sounds good…! will be keeping track. Hope this improves the number of pages I have indexed on Bing.

  4. sampatrik1

    This is new thing in search technology But how it will be benifitable  for website ?

  5. azizzii

    that is what we were looking for!!

  6. leonardos1978

    И я присоединяюсь, надеюсь Bing оправдает наши надежды по индексации.

  7. Electric Cigarette LLC

    Good news. I hope all our sites will finally start getting indexed much faster.

  8. waytoaustria.at

    I hope my Site will be finally indexed. I've been waiting more than 8 weeks

  9. JJJXPXP

    In my case i don't see an improvement, my site starting september 1, decreased from more than 40,000 pages crawled per day to 29,000.

    And also pages indexed decreased from 276,000 to 188,000.

    Is this normal?

  10. jhayes101

    This is a great idea, for those concerned about server overload this should not be an issue, if it is maybe its time to look for a new server.  I have even a better idea give us webmasters a little control over the crawl rate like other search engines do, we are the ones that will see issues and can adjust the crawl rate to avoid these issues.  I guess what i am saying one crawl rate for all is not very practical.

  11. neil_meads

    we havent seen any crawl rate increases on our hosting servers, but would welcome a more active bing spider to help list updated pages.

  12. brpstats

    Yeah hope to see our pages back in Bing's search engine. If no changes are made soon to this issue I wondering about Bing's relavency in our emarketing efforts.

  13. hd.shop.online

    Thanks for info, hopefully we can see the improvement soon as possible

  14. ejasoft

    Good news. I hope all our sites will finally start getting indexed much faster.

  15. rickrollingyou

    Good news to all of us. Bing has been the slowest updater to the rankings imho.

  16. viptalal

    wooow

  17. srmisterios

    well, some tips about how can i get more pages indexed? i just have indexed 10-15 pages from my domains, and each one have more than 100 pagtes!

  18. mysitesrock

    Please feel free to experiment on my sites. 12 million pages… & you are crawling 30k/day. I might get them all indexed in a year.  Crank it UP!  :)

  19. Rina As

    my blog has't been in the crawl by bingbot. In fact already entered in the bing blog sitemap webmaster tool. How can I improve my blog crawling?

    thank you

  20. sohel

    Woah this is just an insane amount of information, must of taken ages to compile so thank you so much for just sharing it with all of us. If your ever in any need of  related info, perhaps a bit of  coaching, seduction techniques or just general  tips, just check out my own site!

    http://bobevanscoupon.net

  21. akhmadsudrajat

    After merging with Yahoo, indexing and crawling of my blog (akhmadsudrajat.wordpress.com)  on  BING, really be broken.

    What's wrong?

  22. Nileshg

    Is the crawl rate back to normal now?

Comments are closed.