Today, in addition to celebrating the first year of Microsoft Copilot, we announced several new Copilot features that are beginning to roll out. We’re excited to give you a more detailed glimpse into some of these features.

GPT-4 Turbo – Soon, Copilot will be able to generate responses using OpenAI's latest model, GPT-4 Turbo, enabling you to tackle more complex and longer tasks such as writing code and more. This model is currently in testing with select users and will be widely integrated into Copilot in the coming weeks.

New DALL-E 3 Model – You can now use Copilot to create images that are even higher quality and more accurate to the prompt with an updated DALL-E 3 model. These capabilities are available to you now by visiting bing.com/create or by prompting Copilot to create an image.

See the below comparison, although to truly appreciate the level of detail of the new model click on the prompt here: photorealistic stegosaurus having its bony plates attended to by a nail salon

Multi-Modal with Search Grounding – We are combining the power of GPT-4 with vision with Bing image search and web search data to deliver better image understanding for your queries. This new capability will be available soon.

Here you can see how we’ve updated Prometheus for multi-modal:

The results are pretty impressive as you can see in the example below. A traditional multi-modal system would be able to describe what's in the image in a generic way, but with search grounding we are able to identify the exact shuttle and the day it was launched.

                               

Code Interpreter - We are developing a new capability that will enable you to perform complex tasks such as more accurate calculation, coding, data analysis, visualization, math and more. We are gathering feedback on these capabilities from a select set of users and plan to make it widely available soon.

Copilot will write the code to answer your complex, natural-language requests, run that code in a sandboxed environment and use the results to give you higher quality responses. You can also upload and download files to and from Copilot, so you can work with your own data and code as well as Bing search results.

Copilot's powerful Python environment runs in a secure sandboxed environment built on top of Azure Container Apps. It provides fast, user isolated environments preloaded with many popular tools and libraries for data science, such as pandas, numpy, matplotlib, sklearn, flask, and more to solve complex problems. You can combine the data Copilot has access to from Bing search and the web with data uploaded in files to get deep, relevant insights as well as beautiful and interactive outputs.

                               

Video understanding and Q&A – Copilot in Edge. You can now summarize or ask questions about a video that you are watching in Edge. For example if you are watching the YouTube video of Satya’s latest Ignite keynote, you can ask Copilot to summarize it for you as you can see in the screenshot below.


We look forward to your feedback!