Partnering to help solve duplicate content issues

One of the most common challenges search engines run into when indexing a website is identifying and consolidating duplicate pages. Duplicates can occur when any given webpage has multiple URLs that point to it. For example: URL Description http://mysite.com A webmaster may consider this their authoritative or canonical URL for their homepage. http://www.mysite.com However, you can add ‘www’ to most websites and still get the...
Read More

Optimizing your very large site for search — Part 4

Everyone knows great content is fundamental to the success of your site. It is the reason people are looking for your site and it is why they will stay. In the previous three posts from this series, we have discussed how large sites need to expose fewer URLs, how you can help search engines crawl more efficiently, and how not to unintentionally hide your content from crawlers.
Read More

On the road to Santa Clara

We’re excited to be headed to SMX West and participating in a number of panels and other events next week. We’ll be there to talk about all the usual searchy things you love and we will have plenty of folks (Nathan as well as a few new faces) on hand to help you with your Webmaster Center or search-related questions. If you have a specific question about your site, or you just want to meet the people who work on Live Search, be sure to...
Read More

Optimizing your very large site for Search — Part 3

Working with large sites often means being a part of a large organization, which brings its own set of challenges. Many stakeholders with different agendas or needs influence how sites are structured. Within larger organizations, there are long to-do lists and a lack of understanding of the impact certain designs or architecture choices can have on the ability of the search engine to index the site. In our past two articles on large site...
Read More

Optimizing your very large site for Search — Part 3

Working with large sites often means being a part of a large organization, which brings its own set of challenges. Many stakeholders with different agendas or needs influence how sites are structured. Within larger organizations, there are long to-do lists and a lack of understanding of the impact certain designs or architecture choices can have on the ability of the search engine to index the site.
Read More

Removing content from the Live Search index

We are taking a quick time-out on our series on large site optimization to provide feedback on a question we are getting asked frequently in the Webmaster Center forum. Site owners want to control which content gets indexed. We have worked hard to ensure that we follow and respect the Robots Exclusion Protocol as directed in your robots.txt file.
Read More

Optimizing your very large site for search — Part 2

For the large website, there are many critically important issues in optimizing for search. In Part 1 of this series of posts, we discussed the importance of reducing the number of URLs you expose through canonicalization. But there are other ways to reduce the surface area of your site to search engines and focus on pages that matter.
Read More

Optimizing your very large site for search — Part 1

At Live Search, one of the most common questions we receive from our peers at microsoft.com and msn.com is how to optimize their sites for search. But microsoft.com is unlike most other sites on the Internet. It is huge, containing millions of URLs, and is growing all the time. However, large content sites like microsoft.com and msn.com are not the only sites that can have an infinite number of URLs. There are also large ecommerce sites and...
Read More

Another crawler in your logs

Live Search is committed to continuing to improve how we discover and crawl your content. In the coming weeks, we will be testing an update to MSNBot, which may show up as a new crawler name in your referrer logs. The new crawler user agent string will appear as: msnbot/2.0b (+http://search.msn.com/msnbot.htm) You’ll still see the current version msnbot/1.1 (+http://search.msn.com/msnbot.htm) as Live Search’s primary user agent. We...
Read More

Getting the right kind of links

On the web, links are the coin of the realm and as such, it is probably one of the most talked about subjects in webmaster blogs and at search conferences. In the last few weeks, there have been a few blog posts and "link" related product launches that make it clear there are still some misconceptions about what are effective and beneficial ways to build links. Link building is one of the most important and difficult jobs for a...
Read More