Bing Your Brain: Confirmation Bias and Branding

Last week, SurveyMonkey released a study comparing Bing and Google by separating the search results from the brands. The design was fairly simple: participants were given search results for a specific term and asked to choose which one they preferred. For about half, Bing results were labeled as Bing, Google results labeled as Google. The majority of people preferred the results labeled Google.

The other half of participants, however, were given Bing results labeled as Google, and Google results labeled as Bing. With the brands switched, more people preferred the Bing results labeled Google. But the most interesting result occurs when you look across the two conditions: more people preferred Bing results labeled as Google than Google results labeled as Google.

Confirmation Bias


Before explaining the psychological forces at work here, let me put to rest any concerns of corporate trickery. Bing wasn’t involved in this study in any way. We didn’t even know it was happening until after the results were released, and since Google recently became a prominent investor in SurveyMonkey, it would be hard to argue that SurveyMonkey had a pro-Bing bias. To the best of our understanding from the outside, this was impartial, data-driven research done by an internal team at SurveyMonkey.

So why did people choose based on brand instead of only the quality of the search results? The results of this study demonstrate something psychologists call the confirmation bias, a non-conscious process that works to support our beliefs by actually changing the way we perceive the world. Because some people came in with the belief “Google provides better search results than Bing”, they actually distorted reality to reinforce the belief.

The confirmation bias exists because the human brain generally resists changing its beliefs. And with good reason: if we have a belief like “when it rains, I get wet”, we wouldn’t want a single incidence of that not happening to change the generally correct belief. So three main psychological defenses evolved to prevent our beliefs from changing:

First, when looking for new information, your brain will influence you to search only for evidence that supports your existing beliefs. So in this study, you can imagine someone with a “Google is better” belief looking through the search results specifically to find good results, rather than looking for any results that might be bad.

Second, your brain will actually change what you remember. In the context of this study, imagine asking people to memorize the search results and then recite them. People who believe “Google is better” are more likely to remember more of the good results and fewer of the bad ones when labeled Google. This type of bias recall is particularly problematic, since even when you do have experiences that explicitly counter your beliefs, you rarely remember them long enough to change your opinion.

Third, your brain interprets the information itself in biased ways to support your belief. So imagine that a person with the “Google is better” bias looked at each individual search result. When labeled Google, they would rate each result as good. Then you give them a week to forget them all and have them rate each result, now labeled Bing. They would rate each result as bad. In this way, your brain will use the exact same evidenceto support different conclusions in support of your belief.

These biases explain how in the SurveyMonkey study, someone with a “Google is better” belief would prefer Bing results with a Google label more than any other condition (Bing-Bing, Google-Bing, Google-Google). Because the Bing results are actually better than Google’s, but they have the “Google is better” belief, they will prefer the results the combine the better results with the brand they like.

There are plenty of good reasons that people might have the mistaken belief that Google search results are better. 15 years ago, Google created a revolution in search technology and helped bring search to the forefront of the internet, a truly important milestone in making the web more understandable. As a consequence, many people became daily users of Google, creating a strong habitual belief that “Google search is good search”.

The problem is, because Google was the first major search engine that many people used, when they went to use other search engines, their brains already had a belief to confirm. In the 15 years since Google introduced their page ranking system, other search engines like Bing and DuckDuckGo have brought new innovations to the space. The world has changed, but your brain wants it to stay the same; just like the people in the SurveyMonkey study, the confirmation bias can keep you from finding out the truth about better search results.

Think of it this way: have you ever tried using Bing, not found what you wanted, and then immediately went back to using Google because “Google is better at search”? But then when you use Google and it doesn’t give you the right results, you change your search and try again because you “searched wrong”, rather than giving Bing a try? That’s the confirmation bias: if you were truly trying to find out which search engine was better, you’d give them an equal chance to give you right and wrong answers.

Which leads us back to the SurveyMonkey experiment. The study is actually a trick: it re-labels search results with the wrong brand. But there is another way to test for your own confirmation bias, without doing any re-labelling: just remove the brands and look only at the results. That is what we do at Bing It On. You put in a search term and it will show you to two sets of results: no branding, just results. You pick five sets of results that best answer your queries, and we’ll tell you which search engine they came from, Bing or Google. And because your brain doesn’t know in advance which search engine is which, the confirmation bias can’t distort reality – you get raw, objective evidence of which engine works better for you.

So give Bing It On a try at Maybe you’ll find that you really do prefer Google results, in which case you can thank your confirmation bias for keeping you using it. But if you find that you actually prefer Bing results, you might think about making the switch and stop letting your biases rule your search.

– Matt Wallaert, The Bing Team

Share on twitter Share on facebook Share on linkedin Share on linkedin