As a member of the Live Search Webmaster Team, I'm often asked by web publishers how they can control the way search engines access and display their content. The de-facto standard for managing this is the Robots Exclusion Protocol (REP) introduced back in the early 1990's. Over the years, the REP has evolved to support more than "exclusion" directives; it now supports directives controlling what content gets included, how the...
Read More

If you’re the owner of a large website with lots of content, you’ve probably noticed that up to 10% of your traffic ends up on a “webpage not found” error page due to broken links or misspelled URLs. There are a lot of reasons users who visit your site might reach a 404 page, but how do you keep those customers from abandoning your site? Today we’re announcing the Web Page Error Toolkit, a customizable web...
Read More

The long awaited second annual SMX Advanced is less than one week away and we felt it necessary to tell you that we are a little more than excited about this event.  We can’t wait to have Search Community in our own backyard and because of this excitement we have decided to throw a party. On the night of the 2nd we invite all participants for the conference to join us at the Olympic Sculpture Park for the SMX Advanced 2008 Welcome...
Read More

A few months ago we announced two new features to MSNBot to reduce the burden of crawling on your website. These were part of a series of improvements we’re making to our crawler during the Spring to increase the freshness and breadth of content in our index. As part of these latest improvements, you may notice an increase in the amount of traffic from MSNBot starting over the next couple weeks. If you notice any issues with MSNBot, please...
Read More

Webmasters, you may already have heard, but our friends on the adCenter team launched a new community portal today in an effort to assist and enable advertisers. We want to encourage you, whether you are currently using adCenter or not, to visit.  Features of the site include: Product/Service specific blogs Categorized User Forums Multimedia Distribution including video interviews, audio podcasts and training videos User profiles The...
Read More

Today we’re pleased to announce an update to the Sitemaps Protocol, in collaboration with Google, and Yahoo! This update should help many new sites adopt the protocol by increasing our flexibility on where Sitemaps are hosted. Essentially, the change allows a webmaster to store their Sitemap files just about anywhere, using a reference in the Robots.txt file to establish a trusted relationship between the Sitemap file and the domain or...
Read More

In just hours, we will be heading to sunny Santa Clara California.  If you’re going to be attending SMX West be sure to come see us at one of the Live Search or adCenter sessions, booths or  at the Search Bowl.  Both Nathan and myself along with lots of other folks from Redmond will be making the rounds and enjoying a little bit of sunshine (hopefully!).   SMX West Keynote Panel Brad Goldberg, Keynote Generation...
Read More

Today we’re pleased to announce several improvements in the crawler for Live Search that should significantly improve the efficiency with which we crawl and index your web sites. We are always looking for ways to help webmasters, and we hope these features take us a few more steps in the right direction. HTTP Compression: HTTP compression allows faster transmission time by compressing static files and application responses, reducing...
Read More

That’s a good question, and you’ve come to the right place.  If your site is not performing in Live Search or you are not being indexed by Live Search here are some steps you can take to change course and improve your rank.  First and foremost at Live Search ranking is free and you can’t pay to boost your website’s relevance ranking.  We have a completely automated ranking process which takes into account a...
Read More

Some of you may have noticed Google advertisements showing up in Live Search results today. We have identified and resolved the issue, and you should see these disappear over the next couple days. The issue stems from the way Live Search handles content disallowed by the Robots.txt file. We regularly check the robots.txt file of a site to ensure that we don’t index and cache pages excluded by the webmaster. However, if we do find a link...
Read More