Bing on Xbox Team: Don’t Be Shy About Talking to the TV


On Tuesday, with the Holiday release of Xbox LIVE we introduced Bing on Xbox which lets you sort through all of your entertainment content – including movies, games, TV shows, apps and music – with just the sound of your voice. With the updates landing in people’s living rooms this week, Michelle Holtmann on the core search team came by to talk with us about what it was like working on the project.

Michelle Brand

Michelle is the Principal Program Manager Lead for Bing and led the collaboration with the Xbox Team to bring these capabilities to market.

Bing Community (BC): Can you talk about how the project was conceived?

Michelle Holtmann (MH): We started with the goal of making the question “where is the remote?” obsolete. With Xbox and Kinect we are collectively working on this magical scenario where “you are the remote” and voice search is the natural extension of that.

BC: Where did the team start with the project?

MH: We started by getting to know the customer. Working closely with the Xbox team, we looked at a tremendous amount of usage data that told us that when it comes to entertainment, people don’t lack selection. The challenge is finding what you want in a timely way.

With that in mind, we focused on surfacing entertainment related content (Movies, TV, Games, Music and other matches) across multiple providers of content (i.e., Netflix, Hulu, Zune, and more …) via their controller, voice or gesture, and making that content actionable thus giving people more time to enjoy that content. No more hunting around and rummaging through multiple applications. Just ask Bing and we’ll find it. How easy and fun is that?


BC: This is new territory. What was it like bringing search to the living room? Was it an adjustment coming from realms people associate with Bing like the Web and mobile?

MH: In the beginning, it was a bit strange to speak to the TV, and we were a little shy about it. We were kind of timid when we were doing our queries. But I’ve got to tell you don’t be shy. It’s designed to recognize: “Xbox, Bing, Batman” and it works.

BC: What were some of the technical challenges you encountered?

MH: Entity matching was one of the challenges we faced. In other words, how can we be sure we got the right “Avatar”?

In addition to machine learning and algorithm tweaking, we utilized human judges to review queries and apply “human truth”. Because an exact title match isn’t always what you’re looking for, we looked at the problem from the experience of the average person, not just the average query.

BC: What did you most enjoy about the project?

MH: I began to feel guilty about coming to work every day working on something so fun. One thing I really appreciate is that thousands of people were behind us. I feel really proud of everyone involved.

BC: Going forward, what are you most excited about?

MH: The vision is there and the opportunities are there. Whatever people can imagine, that is on the list. And, I know it can be done. I’m looking forward to Bing lighting up these experiences for people. This is just a baby step in a long list of progressions ahead.

BC: You got your start in really mission critical work—writing avionic software for Boeing. This seems like a far cry from where you started?

MH: It is, of course. But the one key truth we knew at Boeing is also true in the consumer electronics space. It just has to work. Error tolerance is very low, and if you don’t measure up, you won’t be successful.

TV is mission critical. When you get home from work, TV is mission critical.

BC: A bit off topic but what are you listening to on your way to work?

MH: Rick Springfield. Yes, I did. Big mix, Metallica yesterday, Springfield today.

– Ted Roduner, Bing Community

Share on twitter Share on facebook Share on linkedin Share on linkedin