Getting out of the penalty box

Life can be cruel. You work hard to create new and compelling content for your site. You’ve studied legitimate SEO techniques. Everything is going well and your site is getting decent page rank scores across the board. Then it hits. A search engine penalty comes out of nowhere and knocks your site out of the index. Talk about a bad day.

To be honest, most webmasters who get penalized know why it happened. They’ve used a myriad of overly aggressive (and often outright malicious) SEO techniques to try to egregiously manipulate their website’s position in search engine rankings. They often go to extraordinary lengths in an effort to game the system rather than do the legitimate work necessary to compete fairly for their page rank. We’re not concerned with those folks in this post. Instead, we’ll talk to the those folks who were working hard and trying to do what was right for their businesses by innocently but actively optimizing their sites who got dinged. It is frustrating, but it happens.

Because there are a small few who will go to such extremes to cheat, search engines have to go to great lengths to protect the integrity of their SERPs. So what happens if you got caught in the effort to stamp out malicious SEO when you are sure you are innocent of any overt crimes? Well, let’s walk through this.

Am I really in the penalty box?

Let’s confirm it. Go to Webmaster Center and log on (or sign in with a Windows Live ID for the first time and then add your site. Note you’ll need to have your site authenticated before you can use Webmaster Center tools).

Once signed in, click the site link you want to check on, and then click the link for the Summary tool. In the Site status section, does it say Blocked: Yes? In this case, No is good. Yes is bad.

How did I get here?

Something on your website was flagged as a problem, either via an automated process, by a review from a person on the Live Search team, or from a report we received from a third party (which was then reviewed by a person on our support team). Perhaps the flagged issue was that you used an optimization technique that you didn’t realize was considered problematic. Perhaps your site was hacked by a malicious third party. Perhaps you were unfortunately corralled by a newly implemented search algorithm that caught something that had been previously undetected. Job #1 right now is to figure out what happened so it can be resolved.

Since the Summary tool confirmed that your site is blocked from the index, the first thing to do is to confirm the issue is not related to malware. In Webmaster Center, check out both the Crawl Issues tool (review the Malware infected issue type) and the Outbound Links tool (select the Show only outbound links to malware option) for possible malware issues. If any malware issues were detected, clean them up immediately.

If malware’s not the issue, then it’s time to carefully review the Live Search Webmaster Center Guidelines. While all are important, pay special attention to the bullet items listed in the section titled, Techniques that might prevent your website from appearing in Live Search results. Posted there are the basic rules that can cause your site to be penalized.

Keep in mind that the basic guidelines between the three major search engines are largely compatible. To get clarity on the issue of search engine guidelines, it would be wise to review the guidelines for both Yahoo! and Google as well.

How did this happen?

Good guys who inadvertently get caught in the penalty box often end up there by one of these three common scenarios:

  1. You noticed that your competitor’s website is ranking well (just where you want your site to be!), so you copied what they were doing with their site. However, you might not realize that the site you copied could be using disreputable technologies and techniques that have yet to be discovered by the search engines. If this is the case, eventually that site’s handy work will be discovered and penalized, and your site may soon be penalized as well. It’s even possible that the site you copied from is already being penalized for some flagged issue and you didn’t recognize that.
  2. You hired a new SEO consultant and they implemented changes to your site with the explicit goal of improving your ranking in search engines rather than helping customers find what they need on your site. Unfortunately, you didn’t realize that some of the tactics they used violated the Live Search Webmaster Center Guidelines. If these tactics are deemed too aggressive, search engine penalties will likely ensue. (A good tip for hiring an SEO is that they should give you a full report of the work they are doing, and they should certify that their work doesn’t violate search engine guidelines.)
  3. Your amazingly clever web engineers devised a creative solution to a technical challenge on your site. Unfortunately, you don’t know what they did, and often times a clever solution can mimic (if not outright be) malicious search engine manipulation. Non-standard tactics, such as cloaking, can raise red flags in some of our automated quality tests. Once discovered, such techniques will often trigger penalties in search engines.

In each of these scenarios, it pays to understand what the "experts" are having you do. If you blindly put your web business in the hands of someone who does not know what they are doing (or worse yet, knows full well and chooses to use malicious technologies and strategies in an attempt to fraudulently manipulate your ranking), your site will be discovered for what it is by the search engines and be penalized.

Where did I cross the line?

SEO work is not easy. Live Search knows that. But there can be a fine line between site optimization and over-optimization (and malicious optimization is just a small stretch beyond that). Keep in mind the following thoughts as you consider what was done to optimize your site:

  • Having lots of valid, relevant links is good. But using hidden links is not. Having tons of artificial (and irrelevant) links gotten through paid link farms or exchanges is worse. Linking out extensively to known web spam sites is worst of all.
  • Using on-page keywords on the body text of your pages is good. Using keywords in your page titles and in links is great. But massive, misleading keyword stuffing in meta tags is not. Keyword stuffing in hidden text is even worse.
  • Providing lots of good, expert content is great. Ripping off someone else’s content or plagiarizing commercial content as your own is not.
  • Having lots of pages of content is good. Having one set of content for users and a completely separate set of content dedicated solely to search engine bots (aka cloaking) is not.
  • Providing valuable content to your users is good. Providing malware or linking out to malware-infected pages is not.

Fix it!

Now that you’ve identified what likely caused the problem, go into your site’s source code and remove the offending issues. Note that the issues mentioned earlier are not only considered to be problems with Live Search. When your site does something that gets you penalized with one search engine, it’s only a matter of time before the other two are likely to act in the same manner. As such, the investment of your time to correct these issues is well worth the effort.

OK, I’ve fixed the problems. Now what?

Once site has been fixed, you can go to the Live Search Support page and request reinclusion into the Live Search index. Fill out the form completely, including the security code from the presented image. Select the Content Inclusion Request option, and be sure to include a clear explanation of what you have done to fix all previous conflicts with the Live Search guidelines. Once the form is completed, click Submit. A member of the Live Search support team will quickly review your request and schedule your site to be recrawled. If the crawler determines that issues have indeed been resolved, then your site will be added back into the index.

Soon you’ll be back and doing better than ever. Talk about a good day!

— Rick DeJarnette, Live Search Webmaster Center