We at Live Search are pleased to announce another collaboration with Yahoo and Google aimed at making webmasters’ lives easier. Webmasters have long used the Robots Exclusion Protocol (REP) to control how search engines access and display their content. The REP offers an easy and efficient way to communicate with search engines, and is currently used by millions of publishers worldwide.

Over the past few years, we have been working with Yahoo and Google to agree on common ways for webmasters to communicate with search engines. Our previous efforts include support for the Sitemaps protocol (see autodiscovery directive and cross-host sitemap submissions for more info). While most search engines already comply with the REP, this is the first time the three major search engines have come together to detail how we actually implement the protocol. This effort makes it easier for webmasters to know how REP directives will be handled by search providers.

You can view the details of how we implement the REP at Documentation for the Robots Exclusion Protocol.

If you are a publisher with feedback or suggestions for future enhancements to the REP, feel free to contact me at nbuggia@microsoft.com. We value your feedback as we continue to evolve Live Search.

Nathan Buggia, Live Search Webmaster team