Setting the record straight

I was unable to attend the Farsight conference yesterday but
watched events unfold online and wanted to take a moment to share some thoughts
and make sure everyone is clear about a few things.

It was interesting to watch the level of protest and feigned outrage from
Google. One wonders what brought them to a place where they would level these
kinds of accusations.

Before we explore that, let me clear up a few things once and for all.

We do not copy results from any of our competitors. Period. Full stop. We
have some of the best minds in the world at work on search quality and
relevance, and for a competitor to accuse any one of these people of such
activity is just insulting.

We do look at anonymous click stream data as one of more than a thousand
inputs into our ranking algorithm. We learn from our customers as they traverse
the web, a common practice in helping to improve a wide array of online
services. We have been clear about this for a couple of years (see Directions on
Microsoft report, June 15, 2009).

Google engaged in a “honeypot” attack to trick Bing. In simple terms,
Google’s “experiment” was rigged to manipulate Bing search results through a
type of attack also known as “click fraud.” That’s right, the same type of
attack employed by spammers on the web to trick consumers and produce bogus
search results.  What does all this cloak and dagger click fraud prove? Nothing
anyone in the industry doesn’t already know. As we have said before and again in
this post, we use click stream optionally provided by consumers in an anonymous
fashion as one of 1,000 signals to try and determine whether a site might make
sense to be in our index.

Now let’s move the conversation to what might really be going on behind the

Bing was launched nearly two years ago to break new ground and help move the
search industry in new directions. We have brought a number of things to market
that we are very proud of — our daily home page photos, infinite scroll in
image search, great travel and shopping experiences, a new and more useful
visual approach to search, and partnerships with key leaders like Facebook and
Twitter. If you are keeping tabs, you will notice Google has “copied”
a few of these
. Whether they have done it well we leave to customers. But
more importantly, we take no issue and are glad we could help move the industry
to adopt some good ideas.

At the same time, we have been making steady, quiet progress on core search
relevance. In October 2010 we released a series of big, noticeable improvements
to Bing’s relevance. So big and noticeable that we are told Google took notice
and began to worry. Then a short time later, here come the honeypot attacks. Is
the timing purely coincidence? Are industry discussions about search quality to
be ignored? Is this simply a response to the fact that some people in the
industry are beginning to ask whether Bing is as good or in some cases better
than Google on core web relevance?

Clearly that’s a question that will continue in heated debate as long as
there is a search industry. Here at Bing we will continue to focus on our
customers, and try to provide some great innovation for consumers and the

Yusuf Mehdi, Senior Vice President, Online Services Division


Join the conversation

  1. Rizarsky

    Bing, "Powered by Google". It does have a nice ring to it.

  2. HRM

    Mr.Wheeler, Yes, I copied John's answers, but that is only 1 of the 1000 people I usually copy from, and remember, he copied ALL his answers from a textbook!! (by studying day & night – but I dont have time to do all that) I am the underdog, so I use all the tools available at my disposal to come up with the answers – as we all should.

    Everyone copies, I have been since 2005, so It was interesting to watch the level of protest and feigned outrage from John. One wonders what brought him to a place where he would level these kinds of accusations.

    Clearly that’s a question that will continue in heated debate as long as there is Exams

  3. JoeTierney

    "Bing, "Powered by Google". It does have a nice ring to it." > haha zing!

    I have tons of respect for Microsoft as one of the great companies in American history. BUT it is so annoying that you guys never give competitors any props. The iPhone was stupid, Google Apps is stupid, Android is stupid, we're suffering from search overload, we don't know how to talk on our phones, Windows 7 is a better OS for tablets than iOS … I'm not sure you guys could be wrong more often if you tried.

    Microsoft has lost any sense of intellectual honesty. You were copying results – just own up to it. Google has the best search engine – nobody debates that but your overpriced television commercials. Copying them is understandable. You got busted and now you're acting like the spoiled little rich kid who got caught cheating by his teacher but it's somehow the teachers fault. Copying makes sense, spinning it doesn't. Competing is one thing, crying about getting called for a foul is another.

  4. dominicjj

    – Doesn't it bother you that Google can "trick" your results?

    – Don't you find it strange that you use "1000 signals" and yet the Google result is the one that Bing uses?

    – Do you really think Google employees worry about Bing for more than a nanosecond each day when your division, the Online Services Division, flushes half a billion dollars down the toilet every year?

    – Did you know that Google's experiment was to catch you red-handed doing what you've been doing for months (working hard on core relevance eh?) rather than to trick your results once off?

    – Do you know what click fraud actually is? Or did some poor minion have to tell you because you're non-technical?

  5. Luigi Montanez

    When you're using click stream data from Google as a search signal, that's effectively copying from Google. You didn't do it directly or intentionally. But their results showed up in your results. At the end of the day, that's copying.

    Will you blacklist click stream data from Google (and all other search engines)?

  6. jamesmanes

    I believe you guys at Bing. I don't see why you would copy anyone in this field. Keep up the good work Microsoft.

  7. timdot

    Microsoft, just admit to it. Google is better, you know that, so you copied them.

    Then again, you don't like admitting competitors are better now do you? You like using them though! So, tell me, does FreeBSD still power your servers or are you confident your own OS is stable and secure enough yet?

  8. timdot

    jamesmanes: there's a simple reason why they'd copy – Google is better.

  9. nimageofmine

    lol, so many trolls in this blog comments. Do you even understand how complex it is to develop a search engine and how much code lines does it take to develop and manage it ? If Bing would copy all of that, that is the only way, they can copy results from Google.

    Google copied instant scrolling, background image, side bar, etc…just to name a few and these are the features which are clearly visible that they copied it. But no, fanboys can't admit that :). But if Google says without any proof that Bing is copying, they start shouting…probably they are too nervous :)

    I have used both extensively and I hardly find same results for both the search engines. Such an immature statement from a company like Google sounds so weird, they are just turning evil, lol…

  10. creative_minority

    the query that Google is crying about is something for which there is apparently no other signal available and thats why the result was similar to what Google was showing… anyone with above zero IQ couldve understood this… oh wait… ask is also showing as the top result for query "google"… dammit… now they are copying too… :@

  11. Honeybot

    Yet another manager comes up with another lame excuse, earlier there was a Bing "Powered by Google" Team post about this 1000 signal nonsense where stealing results from Google is just a small part and now this Mehdi guy confirms it, just admit it, you were stealing data and displaying it DIRECTLY to your users, Google probably wouldn't have done this honeybot if you were using their data internally to enhance your results, you just stupidly displayed this results directly to users. Seriously Mehdi, can you be any more stupid??

  12. SemSemSam

    Hey Bravooooooo Mehdi, Now it makes sense why you display other's search results on your Bing "Powered by Google" engine.

    "We have been clear about this for a couple of years (see Directions on Microsoft report, June 15, 2009)."

    I have gone through this, there is absolutely nothing about querying Google and displaying their results directly on your user's results pages, are we missing anything here?

    I still don't why you talk about Google copying other technologies when the subject here is clear, Google was concerned about lame Bing displaying Google results, and you talk here about click fraud and what not, it will only show that you guys have no proper explanation on this issue.

    Do you really think people who read this stupid post as stupid as you guys are?

  13. agalande

    Who let the trolls out?

  14. agentfat2004

    You know i did realize lately that the results  from bing were similar to google. What i found with  bing before  the change was that their results were more relevant than google's but  failed to correct  incorrectly  spelt words. I wish they would just fix that portion. Either way there is nothing wrong with what bing is doing, although I would have wanted that to be stated more clearly

  15. Marcelo Camelo

    Google's ROBOT.TXT is set to disallow crawling, so BING can't treat a Google result page as a regular webpage. Even if they did, they must link to the result page, instead of the result itself. That's blatant scraping.

  16. Arrogant Doggle

    Good job Mehdi.

    Don't know if you steal the search or not – improve bing to beat the *** out of becoming smart to loser google. those whiners probably copied and steal enormous amount from different people .. they are crying cause their weapon is used against them now ..

    Google whiners admit it .. your trolling  reaction proves that you took bing more than serious ever.

    Which means bing is in the right path.


  17. kevinkevin

    Let's get it straight. Google toolbar collects clickstream data as much as the IE toolbar collects clickstream data. Special cases of these are where you collect data about a user's navigation behavior using Bing and Google. So? Google uses Bing data just as much as Microsoft uses Google data! The toolbars collect not just your Google click data – both for Microsoft AND Google.

    To all the commenters who don't get it: get it now?

  18. kevinkevin

    Search for hiybbprqag – Google's "honeypot" word.

    Google: 2 pages of results.

    Bing: 9 pages!

  19. alexman

    It really depends how one defines "copying". Let's think about a scenario.

    A student is taking his final exam. Luckily, he sits next to a smart student in his class and he decides to use his classmate's answers as a "signal" to come up with the best answers. For easy questions, he does his own work and for the most part ignore his classmate's answer. For tough questions for which he has no idea what's going on, the "signal" from his classmate's answer becomes so strong that he just puts the exact same one on his answer sheet.

    Suddenly, the professor notices what he's doing and tells him that he gets a zero in the final for cheating. He claims that he's just using his classmate's answers as a "signal" but not copying directly so he shouldn't be penalized. What do you think the professor should do?

    As other have pointed out, if Bing uses Google's search results as a reference to improve its own search algorithms, there's no issue. But using it directly to influence search results? I don't think so.

    This practice may not be illegal, and some people would claim that whether it's ethical or not doesn't matter as long as Bing delivers good search results.

    But it's just lame.

  20. Marcelo Camelo

    @kevinkevin: it's one thing to collect click data, it's another to look at what one search engine returns as a result for a query and add that to your index.

    There is no way Bing can infer from the click stream that those links are good search results for those queries unless they understand that a is a Google search and that the subsequent click is potentially a Google result for that query.

    So, what Bing is effectively doing is saying: "we use Google ranking algorithm as one of our signals".

    You think that's cool? Really? I guess it is if you are used to the Microsoft way of doing things.

  21. Marcelo Camelo

    @kevinkevin I get 4 pages on Google when I search for "hiybbprqag".

  22. kevinkevin

    @Marcelo: click the "post" button once. You will see your post once. Your logic goes both ways. There's a ton more Google toolbars out there than there are IE and Bing toolbar installs. I believe Microsoft – there's 1000 signals, not 1! You think Google, that doesn't care about security or privacy, just doesn't use the toolbar logs? Yes, they damn well do. At Google, they don't scrub out PII information. Google also uses your toolbar logs to improve their results. Are you forgetting that?

    @alexman: the logic goes both ways. Depends on who you call smart. Microsoft has a 100K smart guys. Google has 10K.

  23. Marcelo Camelo

    @kevinkeving, LOL, looks like your advice of clicking the post button once doesn't work, does it?

    Of course Google uses tool bar logs, but not to see what other search websites are returning for user queries and injecting that into their indexes. That's the lame part. It's akin to hiring people to do searches on Google and typing the results into a form so that Bing can show them too!

    PS: let's see how many times this post appears this time.

  24. alexman

    @kevinkevin: the purpose of my scenario is give an analogy to illustrate whether what Bing is doing is "copying" or not but not to point out who's smarter. If Google does the exact same thing with Bing's results using Google Toolbar, that's still copying even if Google is the smarter one.

    And, obviously, Bing won't use Google's search results if they don't think that Google is at least as smart; will you use stuff from a competitor whom you consider dumb?

    Actually, in this incident, what Google doesn't want to see from Microsoft is a simple

    "We messed up. We are sorry. We will make it right."

    And people will soon forget about this copying issue and focus on Google's recent deterioration of search quality. Why hasn't anyone in Microsoft thought of that?

    Anyways, I'll stop here.

  25. Marcelo Camelo


    "Microsoft’s defense boils down to this: Google results are just one of the many ingredients that we use. For two reasons, this argument is not necessarily convincing.

    First, not all of the inputs are necessarily equal. It could be, for instance, that the Google results are weighted so heavily that they are as important as the other 999 inputs combined.

    And it may also be that an even larger fraction of what creates value for Bing users are Google’s results. Bing might consider hundreds of other variables, but these might produce little overall improvement in the quality of its search, or might actually detract from it. (Microsoft might or might not recognize this, since measuring relevance is tricky: it could be that features that they think are improving the relevance of their results actually aren’t helping very much.)

    Second, it is problematic for Microsoft to describe Google results as just one of many “signals and features”. Google results are not any ordinary kind of input; instead, they are more of a finished (albeit ever-evolving) product, one which itself is made up of hundreds of different “signals and features”, which have already been combined together in such a way as to maximize performance in response to the exact same queries that Bing is getting."

  26. orlandomb

    Yusuf Mehdi (Microsoft's Bing Team) said:

    "We do not copy results from any of our competitors […]

    we use click stream optionally provided by consumers in an anonymous fashion"

    Mmm let see:

    i) user U run IE browser

    ii) and search the term "hiybbprqag" with

    iii) present result R to the user

    iv) user U clicks in R

    v) bing send "hiybbprqag -> R" to bing(microsoft) headquarters


    vi) any user AU search the term "hiybbprqag" with

    vii) bing respond with R

    With all my due respect to your "best minds in the world at work

    on search quality and relevance" this is *straight copying* , man.

    But you have all the rigth to call it the way you like, Yusuf.


  27. ethenj

    Microsoft seems to be beating around the bush ("lets move on…") and using propaganda ( "That’s right, the same type of attack employed by spammer") instead of proposing a solution, Google did copy many of the new features of Bing, there's no doubt about that. But Microsoft was caught directly copying google's search results, and we can all see the proof ourselves right now. Google has reason to be angry, as this this is a much more serious issue. This is more than copied feature: It's akin to Ford reverse engineering a Chevrolet engine, and selling the EXACT same engine in thier car. That's just wrong.

    If Google is right, that means ie8 and the bing toolbar are sending google search results from thier users to bing. Intentional or not, Microsoft is going to have to change this if they wan't to avoid more heat.

  28. ethenj

    Hey, why doesn't Microsoft try the same thing with Google Chrome, and see if Bing results show up on Google? I know for a fact that Chrome sends lots of web data to Google.

    Hence this is why I use Chromium as a browser, not Chrome or IE9.

  29. kevinkevin

    @orlandomb: let's do this. Bing "hiybbprqag" and see where it turns up in your results. No, it's not the first link or in the first page just because Google thinks it should be. The news sites have trumped Google's honeypot website ratings. You helped prove your own case – I urge you to check things out for yourself.

    Thank you.

  30. techSage

    To those that aren't literate enough to understand: 1) Only 7-9 of Google's attempts to get their results into the Bing index worked, so obviously there are complex algorithms at play. 2) The data Bing Bar collects is opt-IN and spans everything the opted-in customer clicks on, not just Google links. Google is still part of the web, right? Or are they not relevant enough to be considered by Bing's algorithms? (Careful how you answer…) 3) For those that don't understand Microsoft's model, the Online Services Division, much like the R&D division is not about making money (unlike Google's model where it is ONLY about making money), it exists to build the Microsoft brand.  This has paid off before.  Recall that Xbox was perceived as losing money until recent years.  Now, the brand is not only making great cash, but is being leveraged by Microsoft to make inroads into new markets (mobile gaming; Natural UI) or improve markets that it needs to improve in (mobile computing).  Don't discount Microsoft's mindhive so quickly.  They've beaten large foes before (with both hands tied behind their backs), they can do it again.  And they gotten to the point were they are coming up fighting and they will succeed again.

  31. Marcelo Camelo

    @techSage, Google is still part of the web, but it's not like any other web page. Google makes it's living out of search. It's not cool for a competitor to use Google results as one of its signals, no matter how many other signals it uses. The fact that users opt-in, does not give free pass for Bing to use the output of Google's years of investment. It's not the user's data for them to give permission for Bing to use it.

    Since when making money is part of the problem? Bing is free to make or loose money, and so is Google. It's about using data generated by a superior competitor to improve a inferior product.

    @kevinkevin, you are missing @orlandomb's point entirely. If looking at a Google query/result pair is the only way that Bing used to include a page in it's index, that can only be called copying.

  32. Marcelo Camelo

    So those who say this is cool, you think it would also be cool if we find that Google is scrapping Bing results to use as input to their ranking algorithm?

    Of course not. It would not only be lame and embarrassing for Google but would also be a loss for all users of search since there would be less differentiation between search engines. Spammers would love it, I am sure. Less work for them.

    But hey, everything is fair at the land of Microsoft.

  33. MarkKB

    @Marcelo Camelo:

    If either Bing or Google were 'scrapping' each other's results, I'd criticise them, and rightfully so. Too bad this isn't what happened.

    It seems people aren't really sure of what the term 'click stream data' means. Bascially, the Bing toolbar analyses the links people click on in *any* website that's visited by the user. For example, if lots of people visited a travel site and clicked "asia", Bing would display the Asia link below the main result for that website.

    With websites with a search function (so Wikipedia, Yelp, Amazon, as well as the obvious ones) it picks up the search term as well. This means that if the majority of people searched a term and then clicked on the third, or fifth, or ninth result, those results would rank higher.

    The obvious conclusion to draw is that Bing is merely analysing what keywords people associate links with, since the conclusion Google has presented, that they're doing this specifically to crib off Google, would require that they come up with a ridiculously inefficient method that's varingly inaccurate, all to have it only be a small part of the hundreds of bits of data that make up a ranking. And that is an extremely silly accusation to make.

  34. MarkKB


    Wow. That analogy is so flawed I don't know where to start. Here's a better one:

    Both Google and Microsoft are entering an art competition with a specific subject. As part of his research, Microsoft asks people what techniques they liked about Google's previous artwork. He uses this information, as well as his own knowledge, artistic skill and critiques of his own previous artwork, to make a piece that has a few similarities to the work Google entered, but ultimately is unique. Google notices the few similarities and complains to the competition, saying that Microsoft is copying their work wholesale.

  35. Marcelo Camelo

    @Mark Kéy-Balchin

    Wikipedia and Amazon are not in competing against Bing, and they benefit from Bing showing their pages as result to a use query.

    Google, in the other hand, is a direct competitor to Bing. If Bing displayed Google's page as a result to the query, we wouldn't be having this conversation. The problem is that Bing learned what the response to that query is from Google, but is displaying a direct link to the result, without attribution of any king. Contrast that against the examples you gave and you will see that Google is being ripped off.

    About your analogy, I'd argue that it's more akin to a general knowledge contest and Microsoft is asking people what answers Google gave for each question. For those questions that Microsoft does know something about, the information about Google's answer make little difference. Not so much for those that Microsoft knows nothing about. Which happens to be where Google does much better than Bing, the long tail.

  36. Taranfx

    "Powered by Google", Probably you should copy their ads too, give them something in return.

    How lam3! Bing sucks

  37. Taranfx

    >> Google has “copied” a few of these.


    Didnt you copy those features from first?

  38. HarmoniousDiscord

    I don't understand why there's any debate about this..? Bing blatantly copied search results from Google. It's as simple as that. It's understandable- Google is the biggest, most successful search engine out there- but that doesn't make it right.  

    Also, my goodness- some of you Bing fanboys are pathetic! To quote one of your own, "Google whiners admit it .. your trolling  reaction proves that you took bing more than serious ever. Which means bing is in the right path.". Really? Bing is neat, and has it's uses, but let's be honest here- Bing isn't anywhere remotely close to overtaking Google, and probably never will be. That's why they're gleaning search results from Google- because they need to do so to remain competitive. Get your head out of Micro$oft's ass!

  39. rockinthesixstring

    It's NOT Google's data, and it's NOT Google's results. It's public information posted on the World Wide Web, you idiots. What Microsoft has done is very intelligently tracked users activity (with their permission) and delivered relevant results. By Google employees installing the Bing toolbar and giving Bing permission to view their internet usage, Google gave Microsoft (Bing) permission to do exactly what they did. It's that simple.

  40. Marcelo Camelo

    From Google's terms of service:


    You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of Google services, use of Google services, or access to Google services except as otherwise expressly provided in the Terms or as specifically authorized in a separate written agreement.


  41. MarkKB

    @Marco Camelo:

    "Wikipedia and Amazon are not in competing against Bing, and they benefit from Bing showing their pages as result to a use query."

    That's kinda my point. Google's accusation was a specific one – that it was copying Google's search results wholesale – rather than just detecting search terms on websites and associating them with links.

    And the reason I compared it to an art competition is because of two things:

    1. As with art, there are no right or wrong answers in search. There are results that some people will find more relevent, and others will not, and vice versa. In general knowledge, there's only one right answer.

    2. In my analogy, the subject matter directly analogises the search term, the artwork ia the results, and the individual links are represented by elements of the artwork. As I interpret your analogy, the answer only somewhat analogises the search results and there's no direct analogy to the individual links. Relativitly mimor, I guess, but in my mind my analogy fits better.*

    (I mean, sure, it's my analogy, of course I'd like i better, but it seems to make miore sense.)

  42. MarkKB

    I should quickly add that when I said "associating them with links.", I meant "associating them with links clicked on". Just to clarify.

  43. showkko

    Anyway, Bing uses all kind of  "jumps" on IE, including Google, Yahoo, etc… Right?

    (Though, Google or Yahoo or any other kind of  search engines use only "jumps" from its own search results.)

    And, since any "jumps" on IE Influence Bing's search results, Bing is much fragile against any kind of "social" attacks through IE.

    I understand.

  44. kevinkevin

    @Marcelo: are you from Google? If not, you can stop supporting them. Given that you can't even spell "scraping" correctly, I doubt you work for Google. If you do, shame on you and shame on Google for hiring you. I hope you're not on the spell correction team. Again, it's spelled "scraping" not "scrapping". Those are two different words. Two different meanings. Go Bing it.

    Microsoft did not scrape. Those that don't understand what clickstream data means and how machine learning or feature-weighting works please stop flooding this forum with your nonsense. Search for your favorite word "hiybbprqag" and convince yourself that Bing's not copying Google. Bing has 7 pages of results and Google has 2 last I checked. The orders are different, that potty honeypot page is ranked at different ranks etc. etc.

    Google "was" superior. You guys still have a higher market share. And that's good. Bing's gaining and they're gaining fast. Bing's passed Google in December in relevance KPIs.

    Those working at Google – go copycat Apple and Groupon. You have a tablet and smartphone to build and a new innovative Google Offers to offer the world. You have plenty of loss-making zero-revenue artwork products to build that the world doesn't care about that. Or better: go read through my PII data as your clueless PhD "adult" CEOs ask the world to finger themselves if they are worried about privacy and security, and go get an orgasm.

  45. ymala1

    I love how this debate has brought all the trolls out in droves. Please, Google is very often a victim of the same kind of attack they perpetrated on Bing. Please, the number of times 4chan has pwned Google, in a manner that is essentially the same.

    Search engines rely on user input to help modify their results so that they are more accurate, with the bing bar it happens to monitor user behavior, with assent, wherever they are on the web, to make get better results. Why should bing make a specific exception to this rule for Google? Microsoft couldn't get at Google's actual algorithms even if they wanted to, but they have every right to monitor what users look at and click on, to make their results better. Even if it happens to be on Google's very own website.

    "Google – Pwered by 4chan" – yea, that does have a nice ring to it.

  46. gnibaddab

    Honeypot trap: The fate of espionage in the new world order. John le Carré… over to you.

  47. cool_scientist_273

    Lol. @ all google shills… Are you so pea-brained that u cant understand information?

    1) Bing didn't copy google – rather google pulled  a click -fraud on bing, and that too only 7-9 queries succeeded .

    2) Talking abt copying, its google who's doing it as fast as they can. Image search, background wallpaper and social search. Back when they had smaller market share they made use of similar techniques to copy ask,yahoo and altavista.

    3)The guy who posted the"Bing sting" -Amit sinhal- is a nut-f*** egomaniacal indian( I am an Indian btw). His statement goes like this " Bings result started seeming like stale copies of google's result" My ASS! check it out yourself. …..

    4)They used IE8 with Bingtoolbar installed .. You use the bingtoolbar onlyt if u want to.. Bingtoolbar doesnt come bundled with IE8.

    5)Google is making a mountain out of a mole hill, but any publicity is good publicity for Bing.

    6)To all those who say google has the best algorithm , you know what they say, ignorance is bliss 😀 ..

    @Google – Stop being a crybaby u hyppocrite … U hav used worser methods to improve ur search.

  48. Gaurav Misra

    I don't care what Google says about bing not having good engineering minds. Bing Software-Engineers are some of the best in the world, I mean have you seen the quality of pictures they post on behind that cute little search box which is clearly bigger than google's (Who's the search giant now bi%€#)?

    Google's wants a war? Bing it on!

  49. kiriappeee

    Why don't people get it? IF bing made a program to do a milllion searches per second based on random generated queries and then took Google's results, THAT would be copying. As it is Google just provides 1/1000th of information to Bing for their searches. And that is perfectly ok. You find out what your user wants to see and that's about it. If Google is treated like any other domain so what? Google is just crying out here trying to make it seem like Bing only uses THEM. That's a useless low down dirty attitude to play the game with.

    Furthermore one thing Google failed to mention on their blog was that jut 7-9 experiments worked out of a 100. If I came up with a similar theory at university and brought 7-9 positive results to prove my point I'd be shown the door. The reason the 7-9 worked also is because well it was only their signal to go by. Congratulations to them on finding an outlier weak spot in the system. This probably means that if any of the other 1000 signals was spammed in this way you'd get a similar outcome from them at some point. But what does that prove? Bing steals? Oh pffft. You got to be kidding me. I thought Google people were smart.

    Honestly, Google can stuff their "don't be evil" and "let's be open" crap back where it came from. Sorry to see you guys get hammered so bad by em. But if it means anything to you all, I gave up Google search at the beginning of this year to try Bing out. I won't go back. Keep rocking on.

    ps – Instead of playing make believe tag, they should focus on improving their services. I use Hotmail as my primary accounts and Gmail for a few contacts who wanted a Gmail addy to connect with me. It took me 5 refreshes to get Gmail to open today. And between each refresh was about 45 seconds waiting. So yea. I'm not a fanboy but honestly speaking, this time, I'm siding with MS entirely.

  50. Boaz

    "We don't copy Google's search results, we just watch what users search for on Google, and take note of what they click – we use this data to improve our own search results".

    Hmm…. yeah… big difference, glad you set the record straight.

    And "click fraud"? SERIOUSLY? you might as well have called them terrorists… quick, call homeland security! Google are using BUZZWORDS! you know who also uses BUZZWORDS? Yeah! TERRORISTS! (insert dramatic squirrel).

    But seriously now, I'm a CS undergraduate student and these kinds of stunts, not to mention their wishy-washy half-hearted denial, are the reason I would rather end up waiting tables than working for MS.

  51. collinsauve

    You've been caught cheating red-handed.   You are trying to absolve yourself of blame by crying foul that Google "set you up"?  Now just own up to your mistakes, stop cheating, and go develop your own search engine.

  52. amsmota

    That 1000 signals can  be very well true, but the fact here is, by using the data collected by users they are actually associating some search queries to Google results. Let's see. If you google for "zbrio1234" and you click on the first link, which takes you to, say, you are not actually clicking on that URL but something like So what Bing is getting from their users data is this last one, that is clearly a Google search result. So, Bing is not associating "zbrio1234" with "" but in fact with a Google search result. I'm not saying that that is good or bad or evil. But we must agree that Microsoft record in the industry is not a very nice one, if we think about qdos, netscape, and so on…

  53. mabbo6

    I think it is very surprising that a Senior Vice President at Microsoft is not smart enough to realize that they will lose this PR argument with Google. They look completely stupid and every time they bring up this topic, the only thing anyone will remember is that they copy Google (whether or not it's only part of their algorithm) and they're exposed as copycats and inferior.

    Full stop.

    If they want to fix this, then stop the PR and fix the algorithm.

    Full stop.

  54. NoneOfYourBusiness

    One of your so called "best minds in the world" namely Hugh Williams was caught at least on 3 different occasions plagiarising research whilst at RMIT, I wouldn't be surprised if the same mindset is also to be found in the Big search group. The claim can easily be verified via the office of the Dean of computer science at RMIT.

  55. garciasat1111111

    "You've got questions, we've got dancing paperclips"

  56. Iron Throne

    So Google taking the decision to link to a certain website when someone searches for "hiybbprqag" constitutes an attack! Shocking! Looks like Bing is the most fragile thing on the internet then.

    I understand that Microsoft has to make a statement denying everything just to be able to keep some face for his more hardcore fans, and I understand such fans will buy everything thrown at them, but this is pathetic. The only way Bing could return that exact result to "hiybbprqag" is because you spent your resources in copying Google search results, as there was no other link between the word "hiybbprqag" and that website. And it doesn't matter if usually that is for Bing just 1 of 1.000 factors, or a million factors, in that case was the only factor, because that word doesn't appear anywhere on the website. Trying to label this as "click fraud" is one of the most shameful statements I remember from the IT industry, maybe even worse than copying the results.

  57. sdguero

    Lol. "Anonymous Reporting" that Internet Explorer users "opt in" for. That stuff is on by default, 90% of I.E. users are going to be sending their internet usage data back to Redmond. I'd be surprised and disappointed if Microsoft wasn't using that data to improve their products. Otherwise its just a waste of bandwidth.

    That said, why don't you just admit that you copied google Yusuf? There's nothing wrong with it, and it shows a lot more class than accusing the dominant player in the market with "click fraud." This blog post makes you sound childish.

  58. Imago

    You know what's bothering about all this. It's that Microsoft can't find a way out of it so they use double speak to try to confuse the readers. I've seen the Microsoft fanboys on here. Microsoft… You can't be impressed with your fan base. I've not once seen a Microsoft zealot able to spell. They always use "ur" for your or you're.

    Saying things like "full stop" and saying that your company has "the best minds in the world" working at it is just grand puffery at it's best. You can fool some of the people some of the time but not all of the people all of the time. And if Bing wasn't probing Google it never would have found these results… Plain and simple. "Period. Full stop."

    Just own up to the fact that you guys got caught with your hands in the "honeypot" and now you've found yourself in a sticky situation and will continue to blame anyone else. Google did it! QQ

    So, just own up to the fact that Microsoft is full of rip off artists. Or did all of you forget that you stole the Windows interface from Xerox? (I don't think anyone who worked for that company ever will…)

  59. bobbolin

    It's sad how many people are falling for this publicity stunt from Google, almost as sad as how far Google will go to prove a point that doesn't exist. Spiking the clickstream data with bad results isn't catching anyone "red handed" as some of you keep repeating, it's fraud at best and entrapment at worst. Truly nothing to see here, move along.

  60. DonJuan

    Google is making a mountain out of a molehill.I like the wallpapaer background.And aside form wikipedia, most results are different based on the aforementioned algorithym.I do notice this- that Google is bundled with every single software package out there – free or otherwise.That is a fact.So whats that telling you eh?

    Google are the ones who look like babies in this spat.

  61. hdcs

    I was at a Microsoft Windows 7 Pre-release conference and the speaker there referred to searching on Bing as to "Google-Bing it" They've been aware of their tactics and if anyone here disagrees they that were stealing from Google's search, then you need to read how Google performed the test before you go defending Micro$oft. They're evil and cannot make a good product, even if they steal ideas… Here's the truth:…/microsofts-bing-uses-google-search.html

  62. ecaligo

    Posting a comment so deep in the commenter's well, I remain a pinch of confidence that on such a major blogpost, even comment #64 might be read.

    Regarding the matter in hand – the subject of debate itself being a byproduct, Google's stunt had a sting – it radiates fairness, wit and a general feeling of touché. It leaves your options quite poor- silence meaning admittance on one, and any reaction seeming defensive on the other hand. However, I think a better repitch could be submitted to the public.

    First, saying " we don't. " means nothing to a public  so advertisement-well-trained. if you don't have a wonderful argument to back such statements up – avoid making categorical statements. It's clear an interest is present, and a request to take your word for what it is portraits a lack of honesty.

    Second, the par being techesque, begs a similar angle of defense. It makes the uneducated reader feel involved in the scene, as if he " get's it ". 'Thousands of anonymous click streams' are far too technical for the news community to absorb and transfer, and relays only to the crowd already familiar with the rules of the game, thus not waiting for a clarification from you. Your message could be more clear-cut.

    And last, the mentioning of the timing is indeed the heaviest piece in your arsenal. Being obvious to anyone aware of the context, it still holds the bulk of your ability to turn the wheel back. This being said, the phrasing itself must be done with extreme caution, for the avoidance of a childish image and a hysteric tone to the reaction.

    No disrespect meant, I wish us all a debate searing high.

  63. fanbaby

    Hey, amsmota is on to something. On the result page of Google, every link is an internal redirect. How the heck deos MSFT get the link out of that?

    One more thing, the double posting nightmare of this blog and every msdn blog  is a result of very poor web-dev skills (i.e. using some .net postback sh!t), meaning MSFT has to learn a lot before it can be called an internet company. Oh, and the army of microsoft zombies, sorry developers, crank out every day many sites like this (yes, use the windows skills you have to make web-sites, boo-hoo) hey microsoft developers, here's a hint: learn web development.

  64. kevinkevin

    The point where folks are disagreeing are if capturing google-search clicks is acceptable or not as clickstream data. Those that don't know what clickstream data means (as I can see from some of the posts here: e.g. no, IE does NOT by default send up your click data; no not  90% of IE users have the IE toolbar – it's more like 5% perhaps; no, Google search results are not what is scraped; no, Google search result clicks are NOT highly weighted – if it were, like I said, you would see your honeypot query results in the first page as the first link not on the second or third page as the 20th or 30th link) are just blindly rooting for Google and have been deceived by Google's deceiving PR; those that understand what's going on and are still supporting Google are probably Google employees. It's an important distinction that an open mind would comprehend. Microsoft is not BS-ing; just explaining the difference. It's not copying.

    None of this compares to Google capturing all my PII data, all over the world, spying on me, training on my data and personal life and Googlers checking out who the Pope is secretly communicating with and laughing in their little cubicles. Why don't some of you supporting Google share your Gmail ids so Googlers here can respond back with every embarrassing detail of your personal life? If that's not invasive and wrong, what is? Can't wait for Google to get fined billions of dollars by every government in the world. Wait, Google's spending billions in Government lobbying. Hope you can bribe yourselves out.

    I didn't hate Google before this episode. I loved them. The cheap and childish stunts of these young immature inexperienced high-strung college grads just irks me.

  65. verdelyi

    That's just the warm-up. Roll out search-by date and improve your localized services outside the US, then prepare for the real storm they'll be throwing at you. In most English searches Bing is already as good as Google, if not better. I believe the localization issue is holding back most people.

  66. kevinkevin

    At least Microsoft lets people comment on their blogs. Jeesh: I wish I could speak my mind on the VP blogs of Google! Wait: they don't care about the common man. As the Google CEO says, go screw yourself if you don't like what we do.

  67. suavi

    Collecting information like searched words and results by Bing Toolbar and IE8 is NOT A PROBLEM!

    BUT publishing search results directly from another search engine (copy and paste) in Bing IS A BIG PROBLEM!


    Because, we understood that Bing is collecting searched keywords and results from it's products and they don't   analyzing  relation between keyword and the web site resulted in Google results, Bing is only copying and pasting search results from Google's search results.

    Bing is like blindfolded version of Google!

    I am sure that collecting information like keywords, results is not an illegal issue but copying and pasting search results into your index directly from Google Server's, is an illegal issue, it's a crime!

    Bing busted by Google Engineers.

    Sorry guys but it's a really epic fail that I have ever seen.

  68. aybiss

    This is laughable. For this to be a normal honeypot attack there would have to be an actual site that was being linked to and text on those pages equal to the random gobbledygook Google planted in THEIR SEARCH ENGINE.

    The ONLY way you could get those results is if you READ them FROM GOOGLE. You may have stopped a few old grannies from being quite as outraged, but for everyone else you just confirmed that you'd like to cover this up.

  69. damnmad

    I know it's late to comment, but I just want you Bing people to know blatant lying will not save you. Google's arguments are very strong, and they played a fair game. Don't throw all the "search manipulation" crap at me.

    If this post shows anything, it's that in addition to lack of innovation, you also suffer from a lack of morals.

  70. Cazna

    I think I understand, it is ok to copy someone else's work so long as you do it indirectly. You can copy Google so long as you copy the Google results that somebody else gets.

    This is pretty good, it opens up some opportunities to the rest of us. Now we can copy music so long as we do it from a friends mp3 player not directly from the original source. And we can copy Windows so long as we are not the ones that read the install disk, get someone else to share their drive to you and you are fine. You can also create a competitor to hotmail that looks and acts exactly the same, just as long as you never look at the site yourself, just look at it on someone else's monitor.

    It is incredible that a company who took out a patent on the action of double clicking so nobody else could do it could have such double standards when it comes to intellectual property.

    Oh, and shame on you Google, not giving them 100% good results to copy. That was a dirty trick… a dirty hacker trick… shame on you.

  71. simzaolly

    I'm honestly impressed by the way Bing has responded to the PR Stunt by Google. Kudos to you all. Unlike this incident, hope the search war continues on the innovation front.

  72. retroloop

    That post really was a disgrace, on at least two accounts. Denying that Bing is copying Google is clearly nonsense, whichever way you look at it, and Mr Mehdi clearly should understand that. Instead he spins.

    Google proved without a doubt that Bing is copying their results. They are not doing direct scraping but that the method used is slightly more roundabout, using click stream data analysis is entirely secondary, it should  be called copying because it is copying. Period. Full stop. I am convinced that Google previously hadn't realized what this click stream data analysis actually amounted to and I'd like to believe the same about Bing. But now it is clear, using click stream data in the way Bing does really is a way of copying from their biggest competitor, and so they should admit their mistake and make very sure they don't do that in the future.

    The second point of disgrace is making the entirely irrelevant comparison between what Google did to prove this copying and the practices certain spammers use. Yes, the method is the same, but so what? Intent and result really matters. Google had neither the intent to mess with the actual experience of any Bing users, nor did they actually do that. Rather their methodology depended on not doing so. Spammers are rather the opposite, they are trying to be as disruptive as they possibly can. That is a difference that makes all the difference. And as per above, Mr Mehdi understands that.

  73. amsmota

    I think it's funny how the Microsoft "defenders" say the rest of us just "don't get" it… So I would like to make my point again.  What Bing gets from their anonymous click stream data, is not (in my example)

    "zbrio1234" ->

    but instead

    "zbrio1234" ->

    or at the very least

    "zbrio1234" -> ->

    Now, is just a matter of each one to  "do the math" and reach their own conclusions…

    @Imago, it's not fair to say Microsoft stole from Xerox, Microsoft stole from Apple that in turn had stolen from Xerox…

  74. retroloop


    But that is besides the point, even if the first is all they get the net effect is the same: by analyzing where people click when they have searched using their competitors search engine they effectively copy said competitors (I doubt this is limited to copying Google). There is no two ways about that, they copy the results from Google because of the way they use the click stream data. And up until recently they could plausibly deny that they understood this (as Google themselves clearly didn't), but now they really can't.

  75. retroloop

    Another way of looking at it would be to imagine that Bing would perform actual Google searches, scrape the top results and use that as a 'signal' for their own ranking.  Bet anyone would recognize that as a foul. Their current practice is essentially the same thing, except they have someone else doing their searches for them with the added bonus of having a human actually filtering the Google results for even higher accuracy. Does that difference make the entire process OK? Imagine that in the imaginary scenario they performed the searches manually and had their employees select the best results from Google. Still not alright?

    But what loses them all credibility is denying there is a problem with what they are doing as that means they are clearly intent on continuing.

  76. Explorer7

    Google's bread & butter till date is search ads. It knows Bing is a powerful competitior with Microsoft's muscle behind it. Bing is more than search, it's developiing into an innovative platform on which lots of useful stuff can be build. Google with it's lac of innovation simply copies the useful features of Bing and goes on presenting the list of links. It fears that as soon as more people realize the power of Bing platform, tide will start turning against them. That's why they are using these kind of ploys to bring bad name to Bing.

    You guys go ahead, no one can stop you from innovation & changing the face of search

  77. PeteHat

    "We do not copy results from any of our competitors"

    Yeah, see if you can get anyone to believe that.

  78. kevinkevin

    I'll add one last thing and shut up. Those who understand what's going on and have the background to comprehend it comprehend it – that includes the guys at Google.

    The IE toolbar has been there forever. It's not a new thing. It's been there for many many years. Those that are complaining – did you stop to ask yourselves how come only now Bing's getting better?

    If you're a user X and you had the toolbar and used Google, clicked on a ton of queries, clicked on all those result links on the first page, then Microsoft would learn from 1 user and 1 signal what Google's returning for that query as the top list of results. Bing wouldn't learn the results of each query as answered by Google, at least the entire first page unless this was the case. Think about it once again. Neither the IE nor Google toolbars scrape webpages you're visiting.

    Lastly, Microsoft doesn't copy results. It's not illegal and if they had to do it, they'd have done it by now instead of taking a roundabout route. Microsoft didn't lose billions of dollars being behind.

    What's clear from some of the posts here is people are blinded – at least those that commented, mostly against Microsoft. Logic doesn't work here, you're on the other side. I hope you'd judge things with an open mind and appreciate improvements when they are hard-earned. A Microsoft Senior VP wouldn't get this emotional and passionate if Microsoft was on the wrong side.

  79. domokun

    Wow, it's amazing how you continue to deny the obvious. The shame, it must burn!

  80. retroloop


    First you say "then Microsoft would learn from 1 user and 1 signal what Google's returning for that query as the top list of results" and then you say " Microsoft don't copy results"? That is exactly copying, learning what Google would return on specific queries and using that to enhance their own results on the same queries. And remember, they likely have millions of users providing this kind of feedback, and the queries they are learning are exactly the queries which people actually use.

    If you don't see the problem there you are seriously squinting very hard.

  81. kevinkevin

    Google does the same exact thing with the Google toolbar. Same exact thing – your clicks are recorded and analyzed and fed back into their various systems.

  82. retroloop

    FIrst, let's be clear, there is nothing wrong with using click stream data for ranking (in fact I work for a (unrelated) company which does, just not for web search). The problem here is that Microsoft uses the click stream from their users interaction with their competitors web pages. This means that they are using their competitors results and ranking as a signal for their own results and ranking. If Google actually did this as well they would be equally culpable, but they have clearly denied doing so (

  83. kevinkevin

    Google *does* use click-stream data from the Google toolbar to train their search algos and machine learning models and AI algos. Yes, they do. At least they did that for years until Jan 2010. What do you think they use the data for – search is their bread and butter; search and Ads are Google's mainline products. They might have stopped doing that now, but the fact remains that they did for years – if you clicked on Yahoo or Bing or MSN or altavista search results, you fed them search results data; just as much as they used Goog-411 to collect speech data that they used to train their Speech recognition engines until they had enough training data and discontinued the service.

  84. retroloop


    I have no problem pinning them if they do, but please produce some kind of shred of evidence to support your statements, not only your assurances that they obviously do. What basis do you have for calling "if you clicked on Yahoo or Bing or MSN or altavista search results, you fed them search results data" a "fact"? I mean obviously search is their bread and butter, that's why they are upset with Bing, but if they really do pull the same stunt themselves I would have thought they would keep mum. The risk of getting called seem overwhelming.

    You now seem willing to admit that there actually is a problem, you have moved your defence from claiming no wrongdoing to claiming that Google is equally to blame. Even if you do prove that (which you have not) my original assertment still holds: what Bing is doing is obviously tantamount to copying and certainly wrong. Whether Google is bad as well or not.

  85. Morihei

    This whole thread just proves one thing: 95% of the population just don't get technical stuff. Hardly surprising, but still sad.

    This is why Apple and Google can do no wrong.

  86. hotels

    I like bing and use them when shopping.

Comments are closed.