Testing at Bing provides many fun challenges in an agile environment that ships quickly and frequently. Testing distributed systems like Autopilot that control thousands of machines requires deep engineering knowledge and understanding of algorithms like PAXOS. Testing takes a whole new direction with verifying the output from the 3 new tech majors (Data Mining, Business Intelligence, Analytics/Statistics). A different kind of technical knowledge is required and there is less history, from a testing/verification perspective to rely on. Many Bing testers are breaking new ground in trying different approaches toward verification of machine learning generated results, e.g. Exploratory Test Automation. Testing also takes on a whole new role in the services world as testing in production becomes a form of service monitoring. You can not only do simple functional tests, but even destructive (business continuity) testing and load testing for capacity measurement in a live, running system.
I get to apply my knowledge of Model-Based Testing using Spec Explorer to testing “Cosmos,” one of the world’s largest data storage and computational platforms it enables massively parallel query execution over many petabytes of information. I focus on generating test cases instead of hand crafting them one by one. I also work on studying issues observed in production and looking for patterns that enable us to prevent classes of problems through new tools or tweaked processes. I work with and mentor other Software Development Engineers in Test (SDETs) who are focused on throttling mechanisms for traffic control in Cosmos, more efficient software routing and network awareness in AutoPilot, and making test cases from all parts of Bing part of production monitoring.