Bing Sitemap Plugin 1.0 Launch

Back in February we launched our beta Bing Sitemap Plugin tool, which enabled anyone to install this on their servers, and have managed sitemaps in place.  Versions are available for IIS and Apache servers, so a wide variety of businesses could benefit from this great feature.  It would create sitemaps for you, and also create a delta sitemap, alerting Bing to your newest content since it last pinged us.

Having both comprehensive and delta Sitemaps provides you with significant benefits, as you will always have a full, up-to-date list of all URLs on your website that search engines can use for deep crawl, as well as a concise Sitemap of URLs that were modified recently, which search engine crawlers can prioritize. This can help in keeping bot traffic bandwidth down. In addition, the Sitemap Plugin automatically adds <lastmod> values to your Sitemap, and generates <priority> values to the Sitemap based on how popular your URLs are.

Now, after having been in beta for just over two months, and with great thanks to our beta program partners, we are announcing the official release of Bing XML Sitemap Plugin 1.0 today. The Bing XML Sitemap Plugin is an open source server-side technology that takes care of generating XML Sitemaps compliant with sitemaps.org for websites running on Internet Information Services (IIS) for Windows® Server as well as Apache HTTP Server.

What’s New in 1.0?

Between the Beta and 1.0 we made a number of improvements based on customer feedback. These include:

- Automatically ping the search engines after each Sitemap update (currently we ping Bing and Google)
- An improved computation algorithm for the <priority> value that is calculated - Multi-machine merge memory improvements and various other improvements
- Custom installation and data directory at installation time

Keeping with the Beta version, the Bing XML Sitemap Plugin 1.0 is available for IIS (6.0 or higher) as well as for Apache Server and is release as an open source project under the Apache License, Version 2.0.

Download the Plugin

You can download the Bing XML Sitemap Plugin 1.0 binaries as well as the source code directly from the Microsoft Download Center or from the Bing XML Sitemap download page in the Bing Webmaster Help Center. 

Bing XML Sitemap Plugin Features Overview

Sitemaps.org Compliant XML Sitemaps

The Bing XML Sitemap Plugin generates XML Sitemaps compliant with http://sitemaps.org/. In doing so, it creates two types of Sitemaps:

1. A comprehensive Sitemap based on recent traffic activity
2. A “delta” Sitemap that contains all pages that changed within a configurable time window.

(When we speak of “Sitemap” we mean the “Sitemap Index” and the actual XML Sitemaps themselves.)

Configurable URL Parameter Handling

By default, query string parameters that are seen by the plugin are not added to the sitemap URLs automatically. However, if your site uses query parameters to uniquely identify content you can easily include each significant parameter to the configuration file called normalization.txt.

Link rel=Canonical Handling

If your pages use link rel=canonical, then the plugin will use the URL value from <link rel=”canonical”> when found in the page’s HTML source as the URL to add to the XML Sitemap. Note: parameters that are part of the canonical URL will not be dropped regardless of whether they are listed in normalization.txt)

How does the plugin handle redirects?

The plugin will only add pages that return HTTP status 200 to the Sitemap. However, when you redirect a page after it was 200 previously, it will still show up in the Sitemap for some time, that is, until the configured time decay has passed (this is configurable using the VisitTimeoutSec setting).

404 Handling

The plugin also generates a Sitemap file that only contains requests that resulted in a 404 on the site. The location of this file is currently commented out in the Sitemap index file because it should not be seen as a regular XML Sitemap, but it can be used by the search engine to inform the index that these URLs are no longer valid.

The Plugin Honors Robots.txt Rules

The XML Sitemap Plugin honors the robots.txt rules you define in your site’s robots.txt in the sense that it will not add blocked URLs to the XML Sitemap.

More flexibility of what gets added: Disallow.txt

For added flexibility you can also add specific disallow rules for the Bing XML Sitemap plugin, using the same syntax as robots.txt in the file disallow.txt. Any disallow rules there will be observed by the plugin in the sense that pages matching the disallowed URL patterns will not be added to the XML Sitemap.

Automatic <priority>: How it is calculated

Since the plugin works off of traffic-based signals, it uses these to help establish a priority value for a given URL in the XML sitemap. Based on the premise that 0.5 represents the average priority and that more important URLs need to get awarded a priority higher than 0.5, the currently used formula to calculate priority is as follows:

aaaabing-sitemap-plugin-1-launch

In other words: the priority of the page is the visit count for the URL divided by 2 times the average visit count for the entire host site.

Additional Configuration Options in Config.ini

The plugin allows for additional configuration post installation. Below are the different settings that you can influence.

Setting
wwwroot

Description
This setting points to the path wwwroot of the site. This value should generally not be changed.

Setting
wwwhost 

Description
This setting points to the path of the content folder for the host. This value generally should not be changed.

Setting
PublishSitemap 

Description
Determines if the XML Sitemap is published to wwwroot (1) or not (0).

Setting
WriteToRobots 

Description
Determines if a sitemap: directive should be written to the host site’s robots.txt file.

Setting
gzip 

Description
Determines if the plugin should decompress compressed pages (1) or not (0). Default = 1. Since we need to decompress pages to calculate a signature, do not change this to 0 in production environments.

Setting
Decompress 

Description
Determines if the plugin can decompress compressed pages. Default = 1

Setting
PingSearchEngines 

Description
Determines if the search engines should be informed (1) or not (0) using a ping to their respective services. Currently the plugin can ping Google and Bing.

Setting
HelpSSP 

Description
Determines if the plugin can write additional debugging information to the Sitemap that helps Bing improve the plugin (1) or not (0).

Setting
MaxSnapshotFileMB 

Description
Maximum file size for the snapshot file used to generate the XML Sitemaps. Default is 4096MB.

Setting
MaxMemoryCacheMB 

Description
Maximum amount of memory the plugin is allowed to use. Default is 32MB.

Setting
SitemapGenerationPeriodHours 

Description
Number of hours between Sitemap generations. Default is the recommended value of 24 (once per day).

Setting
DiskMinFreeMB 

Description
Minimum amount of free disk space to keep. When disk space is smaller than the configured value, the Sitemap Plugin cleans up its temporary data.

Setting
MergeCacheMemoryMB 

Description
Maximum amount of memory used by the merge service to merge the Sitemaps in a multi-server scenario (default is 512MB)

Setting
VisitTimeoutSec 

Description
Longest period of time in seconds a URL remains in the Sitemap without being seeing any traffic. Default is 2592000 which equates 30 days.

Examples of XML Sitemaps Generated by the Bing XML Sitemap Plugin

Our partners at Microsoft use the Bing Sitemap technology at Microsoft, too. Two of our largest properties, MSDN and TechNet, which both contain millions of pages, have been using the technology to generate their Sitemaps since Beta, side-by-side with their prior systems. (Note: You can see a working sample at http://msdn.microsoft.com/sitemaps2/trafficbasedsspsitemap.xml). These revisions, and this tool overall, should provide websites with a much easier way to build and maintain trustworthy sitemaps that are manageable across all scales.

…and since you’ve made it this far, why not check out the Bing XML Sitemap Plugin Downloads page now!

Join the conversation

14 comments
  1. dock-leveler

    I am using blogspot now, do I need this xml sitemap plugin?

  2. Duane Forrester

    This plugin is only for folks who have access to the server hosting their site, dock-leveler.  Unless you can install the plugin on the server, it's not an application for you. ;)  If you CAN do the install, then this is a much more refined, clean way to manage sitemaps.

  3. MenasheA

    Does the plugin creates images sitemaps?

  4. Duane Forrester

    Hey Menashe. Currently the plugin creates sitemaps around site pages, not around image or video URLs.

  5. princeyosefii

    Wow! I had no ideal, so much good stuff and I'm still learning.

    Thanks

  6. material_magic

    I had 1,800 pages indexed and ranked highly on Bing, only to have my entire site currently "unindexed."

    Okay, I am using Lycos Tripod Sitebuilider and have placed the Bing XML file on my web server and I am now "verified." I also just submitted my site to be crawled and indexed on "webmaster," but am unable to submit Bing Sitemap Plugin 1.0 Launch.

    So will be site be reindexed now? If so, is there a general timeframe?

    Thanks!

  7. solilesnet

    Good stuf, but is there any images created on this sitemap?

  8. Webnauts

    I see the XML Sitemap plugin honors the robots.txt directives. Does it do that also with the X-Robots directive for example "noindex"?

  9. ekendra

    Any plans for the versions of sitemap for popular CMS and blogging platform? Wish WordPress sites running in IIS could find this sitemap controlled via WP itself rather than the server directive.

  10. allahditta_4u

    I am using wordpress blog at paid hosting, can i install it on my site.  

  11. rodrigo.stockebrand

    Is this as simple as just dropping the sitemapplugin_code file onto the server? Does Bing automagically then pick up the files and create the sitemaps? Otherwise, can you please advise where to find detailed instructions?

  12. Duane Forrester

    Allahditta – you need to install this on your server, so if you have access to your server, you can install it.  If you do not have access to your server, you will need to contact your hosting company.

    Rodrigo – installation instructions are located here: http://www.microsoft.com/…/details.aspx This product resides on your server, does the work on your server, then ping us when the output is ready.

  13. sini.siva

    I just tried it in our test environment and am seeing some .dat files being generated. Would it be the same in LIVE environment as well?

  14. amjath.khan

    Hi Duane,

    To continue with the above comment, we have been trying it out on our testing environment before moving it to the live sites. Through the process we are getting some .dat files as the output. Could you please tell us if we are doing it right?

    Any help in this regard will be much appreciated!

    Thanks,

    Amjath

Comments are closed.