Search Engine Optimization Best Practices for AJAX URLs

As a member of the Bing Index team I am often asked by web publishers and Content Management Systems (CMS) companies to provide coding best practices for their web sites. Today, I am pleased to share a fundamental improvement to our previous coding best practices related to AJAX.

AJAX URLs

In the past few years, AJAX (shorthand for Asynchronous JavaScript and XML) has become extremely popular with web developers looking to create more dynamic and responsive experiences by exchanging small amounts of data with the web server. Often AJAX is used as a technique for creating interactive web applications, but more than ever people are using AJAX to spice up all their site: what was once limited to some areas of only a handful of websites has grown to become the standard for many web sites and Content Management Systems.

Exchanging data asynchronously requires the execution of scripts when particular page events occur. Search engines face challenges when it comes to interpreting or executing this kind of code. Therefore, content contained within an AJAX-enabled page or control was often not accessible nor indexed except if you were following SEO best practices overly-complicated techniques or protocols.

The new recommended SEO Best Practice for AJAX URLs

Modern browsers including Internet Explorer 10 support an HTML 5 JavaScript function called “pushState”. This pushState function allows web developers to change the entire display URL — with the exception of the domain, port and scheme (http://) portions — using JavaScript.  It also allows web developers to modify the page title and the session history information. This ability is a game changer for AJAX SEO.

The Internet has some great examples of webpages showcasing the use of pushState, such as http://diveintohtml5.info/examples/history/fer.html, and http://blogs.msdn.com/b/ie/archive/2011/10/31/html5-history-in-ie10.aspx is a great place for coding best practices including cross-browser considerations.

Before this pushState function was supported by modern browsers, JavaScript was only allowed to change the URL fragment, that is, anything behind the “#” symbol. AJAX applications were using this fragment to save and restore the state of their application. Since the URL information after the # was never sent by the browser to the server, developers had to rely on overly-complicated protocols such as “crawlable AJAX”, which uses the #! (“Hash bang”) signature.  This was meant to make AJAX development easier in regards to SEO, but it actually complicated things both for search engines as well as webmasters trying to implement, maintain, and debug their AJAX-driven web pages and applications.

With pushShate, we can fully omit the complexity of transforming between “pretty” AJAX URLs and “ugly” static URLs. Search Engines will crawl and index the same URL used by you customers. We are back to business as usual for SEO, including pretty SEO well-understood URLs schema http://domain/path/file?name=value_parameters. This helps you focus on the usual SEO activities (links, page content, etc.) without having to do worry about complex page transformations.

Linking Strategies using pushState

With pushState, you can generate almost any URL you want in the browser’s address bar. However, with such great power comes great responsibility. Here are a few things to keep in mind when exercising that power:

Avoid broken links

A common mistake with pushState is that you may create a broken link in the address bar, remember that people may bookmark, share, link to the link created by pushState also any URL created by pushState should work directly, too, also refresh the whole browser for any URLs created by pushState to verify if this URL exist. A coding best practice to avoid broken links creating URLs with pushState is to limit change to URLs query parameter values.

Search engines discovering this link should be able to see the same content as well (and remember that Bing recommends you avoid cloaking).

Avoid duplicate content

With pushState, it is easy to create new URLs. In fact, it is so easy that a pushState-enabled web site may start to generate way too many URLs. Remember that less is more for your site and that having fewer URLs better. Another tip is to use pushState to inject all useful, significant parameters before the question mark and all — at least from a search engine’s perspective — “useless” parameters after the fragment part of the URL (that is, after the #).

For instance, if you have a music player web application and offer the ability to pause a song and bookmark it, it is highly preferable to link to a URL such as http://www.example.com/musicPlayer.aspx?song=TheCanadaSong#pausedAt=42   instead of http://www.example.com/musicPlayer.aspx?song=TheCanadaSong&pausedAt=42. The latter approach may lead to many duplicate content URLs. The Ignore URL Parameters tool in Bing Webmaster Tools can help mitigate, but prevention is better.

What about Browsers or Crawlers that do not support pushState?

Since not all browsers or even crawlers will support pushState you should make sure that both have a good experience:

• For browsers and crawlers that do not yet support pushState, you should continue to offer links that allow direct navigation to a URL for a given page state.

• For customers with modern browsers that support pushState, you can refresh parts of the web page and update the URL. 

Conclusion

As with our blog post related to building Websites Optimized for All Platforms, we aim for our SEO guidelines to be easy to adopt and to remain valid for years ahead. We feel you should not need to be an expert to create web pages that are search-engine friendly and want to avoid technically overcomplicate solutions. The combination of AJAX and pushState make this possible. 

Related Links

• HTML5 History in IE10

• MSDN Magazine : Building HTML5 Applications : A History (API) Lesson

• SEO And Accessibility With HTML5 PushState

• Dive Into HTML5

• HTML5 WC3 browsers history

 

Fabrice Canel

Principal Program Manager

Microsoft – Bing

Join the conversation

10 comments
  1. davidburgess00

    Interesting article. Thanks for this. More forthcoming than most articles tackling the subject, including on the Google blogs. From considerable experience, you hit the nail on the head with the most important line in your article is this:

    "For browsers and crawlers that do not yet support pushState, you should continue to offer links that allow direct navigation to a URL for a given page state."

    If you're wanting your site to gain any traction from Search Engines, this is essential. Too many developers are creating 'beautiful' sites for clients that unbeknownst to the owner and payer of the re-assuringly expensive bill, are an untamable monsters when it comes to Search.

    IMHO, if you want or may wish to have some traffic from Search Engines in the future, make sure that AJAX is a 'nice-to-have' additional feature and is not baked into the core.

    Keep it coming.

    Cheers

    Dave

  2. Duane Forrester

    Thanks for reading and for your input Dave. :)  I know Fabrice is watching this thread. ;)

  3. bangunrumahelegan

    I was newbie here… Still learn all of it..

  4. mattapitts

    So what about the #!. What if I come up with a solution as suggested by Google to to get pages indexed, will Bing behave similarly. As it stands, Google will take the #! and turn it into ?_escaped_fragment_=. It is up to the dev to return a snapshot based on this. This seems like the most sensible approach, especially because most browsers don't support push state yet. So at the end of the day, its actually more complicated now to add support for push state and #!. IMHO

  5. Icahbanjarmasin

    THANK YOU MY FRIEND.

  6. scriptster

    Does Bing support the scenario supported by Google whereby you insert  into the head of the http://www.example.com/page1.html page and Gbot knows to turn around and request a page at URL http://www.example.com/page1.html?_escaped_fragment_=

    I'm trying to see what else can be done to ensure crawlability of pages with lazy-loaded content (just in case – lazy loading as in you scroll close to the bottom of the page and JS requests a new portion of the page, if there is more content, to be loaded with AJAX and added to the page, kinda like FB does it)

    It seems like  this , then serve to Gbot http://www.example.com/page1.html?_escaped_fragment_= with everyting already loaded  is the way to do it with Google.

    How about Bing?

  7. scriptster

    Sorry, in my previous comment the mega tag got cut off. So, to reiterate, it would be like show "meta name="fragment" content="!" tag in the header, then add  ?_escaped_fragment_= to the URL

  8. douglasit

    Hello, I'm using pushState to create some URL from the website below, I'm using only the (?) not the #! for Google fragment. Do you think the way I'm using the SEO going to work? or I'm going to change to #!?

    URL, with pushState:

    Thanks.

  9. Mihai_searchMaster

    what about AppCache ? I think the meta fragment tag is the best way to indicate that the page/website is ajax

    .   Not surprisingly Bing doesn't support it. Why  don't you get that we have web apps not web pages? I would poke Fabrice in the head "Bing bing bing bing!" until he understands this.

  10. frederic.peters

    I think this is far too complicate for the lambda webmaster. Search engine must adapt to the sites. Not the contrary.  I use jquery to load the page faster: <link removed> This is done in the interest of the visitor. Bing should be able to read and understand the content or provide an EASY way to collaborate.

    Look at the Sitemap protocols, what percentage of webmaster really list their sitemaps in their robots.txt file like it is written in the protocol ? 0,01% ? Maybe less ! Bing should concentrate on giving better results. There are still too many spamdexing.

Comments are closed.