Fixing 404 File Not Found frustrations (SEM 101)

You’ve seen it. So have I. Nearly every person who has actively browsed the Web for more than 15 minutes has seen it. I’m talking about the dreaded 404 File Not Found error. When it occurs, users simply abandon their search on that site and go elsewhere. That’s a potential lost sale, subscription, or download opportunity (aka conversion) for the affected site! It has been estimated that up to 10% of traffic to large websites on the Web is looking for pages that don’t exist, so this is a big problem.

Back in the early days of the Web, when you entered an incorrect URL, you got a nearly blank, stark white screen containing nothing but the simple words, “404 File not found.” Yeah, thanks. That’s really helpful. But back in the day, when people actually wrote webpages in Notepad, that behavior was de rigueur. That was the way things were. Back then. But that was then. This is now. We can do better.

Unusable URLs

There are a great many ways for URLs to be rendered unusable. They can be mistyped or misspelled by the user, the page can be moved, renamed, or deleted by the webmaster, and the URL can be incorrectly written by external webmasters who create the outbound link from their sites to your, which is the most frustrating situation for the webmaster of the intended destination.

So when a potential customer of yours, interested in learning more about what you do or what it is your site has to offer, uses an erroneous URL for a page within your website today, what do they get? Do they get the Web’s equivalent of the blue screen of death, a useless page that stops them dead in their tracks and forces them to move on to a competitor’s site? Or do they get a page with helpful guidance that keeps them in your site, offering to assist them with finding the information they seek?

As webmaster, you control how you rename and remove pages from your site (see information on using redirects to point to moved and renamed pages in an earlier SEM 101 article). But you can’t control the content of your inbound links, nor can you nudge that user who can’t spell or remember your page-naming scheme (yet another good reminder to make the file names of your pages logical and easy to remember). So instead of cursing the darkness of silly users over whom you have no control, instead light a metaphorical candle by creating a custom 404 error page.

Custom 404 error messages

Creating a custom 404 error page is not that hard to do, but so many webmasters overlook doing this small but crucial step. Your custom 404 error page will appear in place of a generic 404 error message when the URL to your site is broken. By anticipating what users will likely want to know when they come to your site, you can proactively give them enough information in the custom error message to keep them in your site, and then provide them with easy means to find the information they want.

OK, so this is a great idea, but how do you do it? Well, I’ve got you covered there. What you do depends on which web server platform you are using to host your site. Let’s take a look at what you need to do to establish this safety net for your users. After all, adding this one little feature to your site might make the difference between a bounce and a conversion!

Dynamic or static?

Before we discuss anything else, you first must decide how you want to approach this situation. You can create a single, static custom 404 error page for your entire site. That will be pretty simple to implement and is likely a good fit for smaller sites, but larger sites might feel constrained by a single, static page. There are technologies available that allow you to create dynamic error pages based on script, and that might suit a larger site’s needs better, but the caveat exists that if the host scripting engine suffers a failure, then you’ll have no custom 404 message at all. Given that this is not a developer blog, I’ll just point all webmasters interested in creating dynamic 404 error pages out to Bing searches for information on both Internet Information Services (IIS) and Apache platforms.

If you’re interested in learning more about basic, static custom 404 pages, follow on. After all, this is SEM 101, right?

Create the custom page content

You need to decide what information and features you want to include in your custom 404 page. I suggest the following:

  1. Use the page template for your website to maintain consistency with your site’s look and feel on this new page.
  2. Include your site’s navigation scheme in this page. If you have created an HTML sitemap page or a dedicated site search box, include access to those features as well.
  3. In the page’s text, first acknowledge that the URL for the page the user was intending to see does not exist. Then offer a quick description of your site’s subject-matter theme and list the products/services/opportunities it offers. Follow that with a suggestion that the reader use the on-page site navigation (menus, sitemap, search box, etc.) to look for the information they are interested in.

Don’t add huge, hard-to-read fonts, auto-play music, videos, or animations, or include anything else that may be off-putting to the first-time reader. Leave off the advertisements as well – this will distract the user from the critical mission at hand – to get the user back onto a real page within your site. Remember, the whole point of a custom 404 error page is to prevent a bounce (aka a single page visit session in which the visitor abandons the site without visiting any other pages). Keep the page clean and easy to read.

Save this custom page file to the root directory of your site (for this discussion, I’ll call the new file 404.htm). Now let’s cover how to employ it on the various web server platforms.

Apache

Apache users can configure a special text file found in the root directory of their site to implement a custom 404 error page. The file, named .htaccess (the dot precedes the file name and contains no typical file name extension), can be edited in Notepad to include the following line (using our sample error page file):

ErrorDocument 404 /404.htm

Of course, you must name and store the custom 404 error page file identified in the location as specified or even it will return a 404 File Not Found error! Use Bing search results to find more information on what to do in Apache for custom 404 messages.

IIS

If you are using IIS, the implementation of a custom 404 page is simple. Here’s how:

  1. First, open IIS and select the website you want to customize.

In IIS versions 5 and 6:

  1. Right-click the website and select Properties.
  2. Click the Custom Errors tab.
  3. Double-click the listing for status code 404 to edit that setting.
  4. In the Edit Custom Error Properties dialog box, set the Message type drop down list to URL.
  5. In the URL text box, per our example earlier, type /404.htm.
  6. Click OK to save your work.

In IIS 7 and higher:

  1. In the right pane, double-click Error Pages.
  2. Double-click the listing for status code 404 to edit that setting.
  3. In the Response Action group, select the option Insert content few static file into the error response.
  4. Click Set.
  5. In the Root directory path text box, either type the physical path of the root directory (up to the portion of the path that branches into a specific locale directory) of the custom error page file or click Browse and navigate to the error file root directory and then click OK.
  6. In the Relative file path text box, type the relative path of the localized error file, which if our example earlier was for US English only, could be written as EN-US404.htm.
  7. If you are implementing localized versions of a custom 404 error page, ensure the Try to return the error file in the client language check box is selected, and then click OK twice to save your work.

IIS 7.0 and higher users can alternatively edit their web.config file to include a snippet of code to accomplish the same task. Using the sample file 404.htm referenced earlier, here is an example code snippet:

<?xml version=”1.0″ encoding=”UTF-8″?>
<configuration>
  <system.webServer>
    <httpErrors errorMode=”DetailedLocalOnly”>
      <remove statusCode=”404″ />
      <error statusCode=”404″ path=”404.htm” responseMode=”File” />
    </httpErrors>
  </system.webServer>
</configuration>

Check out Bing search results for more information on creating custom 404 error pages in IIS.

Bing Web Page Error Toolkit

Aside from creating your own home-grown, dynamic custom 404 page solution for IIS, there is another option for IIS users that kicks it up a notch. The Bing Developer Center team has augmented the IIS experience in creating even more useful customized 404 error pages. Straight from the Bing Developer Center blog, check out the post from this past summer, Customize your 404 error pages with the Bing API Web Page Error Toolkit. The post reveals that the toolkit, built on the Bing API, replaces the default IIS 404 error page with a dynamically created Bing search page containing customized search results derived from keywords extracted from either the source Uniform Resource Identifier (URI) or the HTTP request. This creates a search list of relevant, alternative pages on your site, helping the user more easily and quickly find the information they originally wanted without abandoning their visit to your domain.

The Bing Web Page Error Toolkit is available as a free download. If you’re using IIS, give it a try. Your users will show their gratitude with more conversions and fewer bounces! Note that you will need Microsoft Visual Studio to use this tool.

Test, test, test

Once you have implemented your custom 404 page, test it out. In a browser, type your domain name followed by a short, random string of characters you know does not match any existing file or directory name. If everything was implemented correctly, you should get the custom 404 page you created in response to that bad URL. From that point forward, your potential customers will also see that page, and if the new error page’s content is well done, they will more likely stay on your site. And that’s the goal after all, right?

If you have any questions, comments, or suggestions, feel free to post them in our SEM forum. Later…

– Rick DeJarnette, Bing Webmaster Center

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

UPDATE: Cool news! The Bing Developer Center team has announced today (11/18/2009) the release of the Web Page Error Toolkit for PHP users. For more information, check out the Dev Center blog post titled, Finding the page not found is now even easier.

Rick

Join the conversation

20 comments
  1. wisemx

    Awesome, Awesome, Awesome!  :-)

  2. Anonymous

    Where does the 10% estimate come from?

  3. toscho

    I've set up my 404 pages to send me an email. Most 404s are hack attacks, so I like to react as soon as possible.

  4. Maciej Galecki

    Moreover, if you include tracking codes in such a template, you'll easily find out all the statistical figures required to make improvements, e.g.:

    - number of error page views will help to determine the priority of fixing single issues

    - referral URL will help to find out which external site is linking with a wrong (e.g. mispelled) URL so that a contact with the webmaster (or website owner) is available. This will also help towards SEO…

  5. tarun_juneja

    i will love to implement this on my website.

  6. hotels

    very good

  7. Anonymous

    thanks a lot it's very helpful

    very

  8. Anonymous

    thanks a lot it's very helpful

    very

  9. rickdej

    Vanessa Fox,

    I took the figure from the June 23, 2009 Developer Center blog article titled, "Customize your 404 error pages with the Bing API Web Page Error Toolkit" (I link to it in the above article).

    I just confirmed with the Developer Center team that this figure stems from a study conducted internally at Microsoft. The figure was derived from an analysis that looked at different types of sites across the Internet.

    I hope this helps!

    Rick

  10. rezan123

    thank you for this information

  11. kartiksnj9

    I think the best way to go about to ensure that if anyone or any user enters a wrong URL related to our website he or she should be redirect to a page that states that the URL is not a valid URL and the user should recheck the URL.

    This ensures that the user does not get a bad impression of the site and the custom 404 page can have a link to the home page to the website for the user as well :)

  12. xzooom1

    there is a diffrernt method to redirect the visitor from 404 page to your home page so you will not lost your visitor

    thanx

    —————————————————–

    http://www.arab-max.com/vb

  13. Inkatechnology

    wowow really impressive post thanks

Comments are closed.