Building the New Bing

A year and a half ago, I wrote an article suggesting that search was far from a solved problem, and that in fact it was poised for a revolution. I suggested that by applying deep learning models and making search more modern and intuitive, we could significantly improve the experience for our users.

I’m thrilled to see that we are making a large step forward towards this vision with the recent announcement of the new Bing. We’ve received many questions in the last couple of weeks of the current preview, so I’d like to share a little bit more about the new Bing and how it came to be.

Behind the scenes: How the new Bing came to be

Last Summer, OpenAI shared their next generation GPT model with us, and it was game-changing. The new model was much more powerful than GPT-3.5, which powers ChatGPT, and a lot more capable to synthesize, summarize, chat and create. Seeing this new model inspired us to explore how to integrate the GPT capabilities into the Bing search product, so that we could provide more accurate and complete search results for any query including long, complex, natural queries.    

Behind the technology innovation:  Prometheus

The new GPT model represented a breakthrough in Large Language Models (LLMs) but, much like other LLMs, it was trained with data through a given point in time, so we thought that we could make the user experience richer, more relevant, and accurate by combining it with the power of Bing's back-end.

To be more concrete, we developed a proprietary technology we call Prometheus, which is a first-of-its-kind AI model that combines the fresh and comprehensive Bing index, ranking, and answers results with the creative reasoning capabilities of OpenAI’s most-advanced GPT models.  Prometheus leverages the power of Bing and GPT to generate a set of internal queries iteratively through a component called Bing Orchestrator, and aims to provide an accurate and rich answer for the user query within the given conversation context. All within a matter of milliseconds. We refer to this Prometheus-generated answer as the Chat answer.

Selecting the relevant internal queries and leveraging the respective Bing search results is a critical component of Prometheus, since it provides relevant and fresh information to the model, enabling it to answer recent questions and reducing inaccuracies – this method is called grounding.  Put another way, the model reasons over the data provided by Bing and hence it’s grounded by Bing data, via the Bing Orchestrator. The diagram below illustrates how Prometheus works at a high level.

              

The final step is when Prometheus attaches relevant Bing search answers such as weather, stock, sports, news, and so on, to the Chat answer, to provide a richer and more engaging user experience that further leverages the power of Bing. Thanks to the Bing grounding technique, Prometheus is also able to integrate citations into sentences in the Chat answer so that users can easily click to access those sources and verify the information.  Sending traffic to these sources is important for a healthy web ecosystem and remains one of our top Bing goals.  

More to do:  Ensuring a helpful and inclusive experience for all

Even though grounding is a remarkable innovation, it is a new technology and we need to think about how we apply it responsibly, including further reducing inaccuracies and preventing offensive and harmful content. That’s why a key goal of the new Bing preview is to collect user feedback to help us improve Prometheus further.

We’ve already received invaluable user feedback and have learned more in the first two weeks from real users than in several months in the lab, and are putting it into action as we explained in this recent blog. For example, we are planning to 4x increase the grounding data we send to the model to improve the Chat answer accuracy, and we are also considering adding a toggle that gives users more control to optimize their experience depending on their task and goals.

Another important learning is that very long chat sessions can confuse the underlying chat model, which leads to Chat answers that are less accurate or in a tone that we did not intend.  This is why we’ve recently introduced caps in the chat experience which we are expanding with your feedback and as we improve the grounding and tone of the model.

Behind the product innovation: Merging Search and Chat

While Prometheus is a remarkable, pioneering AI-based innovation, from a user experience perspective, it was still not clear how we should integrate its capabilities into Bing.  There were two main points of view:

  • Some in our team felt that search is such an ingrained habit that we needed to keep the UX like today’s web search and simply add the Prometheus-powered Chat answer on the main UX.  Just like any other answer, the Chat answer would earn its place based on its relevance to the query.
  • Others in Bing felt that this was an opportunity to change the search paradigm from the classic web and answers results to a new interactive, chat-based way of searching.

Both schools of thought were partially right, yet neither had the complete picture. The team advocating for classic search was generally using navigational queries to defend their point of view, while the team pushing for the conversational approach would present research-like search sessions such as shopping or travel.  I plotted their queries in a graph like the one below to illustrate this observation.
 

It then became clear that some queries would perform better in the classical or Search mode, while others would do better in a conversational or Chat mode.  Also, some users may prefer one over the other, depending on the query. Therefore, the ideal Bing search product needed to smoothly transition between Search and Chat modes based on the user’s intent and preference. 

Our design team took on the challenge and, after multiple iterations, developed a new UX that unified Search and Chat in a single interface, where users could easily switch back and forth by clicking on UX elements in the page, or by simply scrolling or swiping up and down.

This product or UX innovation is arguably just as important as the Prometheus technology to ensure people can use the product to its full potential in an intuitive way. The video below shows the smooth transition from Search to Chat, and from Chat to Search for a travel search session.


Behind it all: The people

The power of Prometheus and the new Search-Chat interaction model came together along with other innovations which we’ll describe in future blogs.  These have helped us create the new Bing which is bringing search to this next level of innovation. I truly believe that search will never be the same.

Users of the new Bing will get more complete and contextual answers, which in some cases may save them hours of research in a search session, and that means more time back to spend on the things that matter. And we hope along the way you’ll also discover that search can be more interactive and fun.

Although there’s much to learn and improve during and after the preview, it is humbling to see how all these technology and product innovations came together so far in the new Bing. This is also the culmination of many years of work by the Bing team and is a testament to their tenacity and creativity. I want to give a huge thank you to the team who made this possible.

And last but certainly not least, thank you to all our Bing users – from our loyal fans to those who are just trying Bing for the first time. We built this new Bing for you and your feedback is invaluable. Please share it so we can continue to improve.

This is just the beginning – I can’t wait to share more over the coming weeks and months.

Jordi Ribas
Corporate Vice President, Search & AI