Microsoft's AI Panel Beats Doctors 83% to 36% in Complex Case Study

Microsoft’s AI diagnostic system, known as MAI-DxO, just posted a score that might change how we view computer-aided medicine. In a study published this week in Science, the system correctly identified complex medical conditions in a simulation 83% of the time. That is more than double the success rate of human generalist physicians, who managed only 36% correct diagnoses under similar conditions. Mustafa Suleyman, the CEO of Microsoft AI, called the result a significant step toward what the company is calling “medical superintelligence.”

The numbers behind this claim are stark, but they come with specific context. The simulation used complex cases from the New England Journal of Medicine, the kind of tough diagnostics usually handled by specialists or large medical teams rather than individual generalists. When Microsoft tested the system against various underlying AI models, the best results came from OpenAI’s GPT-5 and GPT-5.5. On its own, GPT-5 achieved a 78.9% accuracy rate on these hard cases at a cost of roughly $8,400 per case. The DxO orchestration layer cut that cost by about $3,000 while nudging accuracy up by four points. However, the gains were not universal; for emergency department cases, the system showed little improvement in either accuracy or cost efficiency compared to the base models.

The people driving this effort see this as a solution to a growing mismatch between medical demand and supply. Dominic King, vice president of Health at Microsoft AI, noted that the current challenge in healthcare is balancing the rising need for expertise with the limited availability of doctors. Suleyman, who has spent his career using AI to solve medical problems and previously co-founded DeepMind, believes the technology is ready for real-world testing. He highlighted that the system helps most when working with less capable models, such as Gemini 2.5 Flash, which jumped from 46.4% accuracy alone to nearly 69% with the DxO assistance, though the gap between strong and weak models remained larger than the boost provided by the orchestration.

The stakes are high as Microsoft moves from simulation to live clinical settings. Through a new partnership with Beth Israel Deaconess Medical Center in Boston, the company will evaluate DxO in different hospital environments, including emergency and urgent care. Patients will receive standard treatment while consenting participants also engage with the AI system, allowing researchers to compare recommendations directly against human doctors. King described the plan as the biggest study of its kind, aiming to generate compelling evidence for deploying these systems at scale. This effort coincides with a broader manifesto from Microsoft, defining medical superintelligence as the capacity to surpass human-led health systems in reasoning and coordination across the full arc of care, moving beyond simple benchmarking to integrated clinical support.

Suleyman predicts that healthcare models will see an order of magnitude improvement in capability within the next 24 to 36 months. He envisions a future where AI can stream highly accurate predictions about how a medical record might unfold, providing clinicians with foresight he describes as “gold dust.” While generative AI in healthcare is still in its early stages, Microsoft is positioning itself to lead this transition. The company already sells dictation and radiology tools to clinicians, and it views DxO as a critical expansion of those efforts, aiming to have the state-of-the-art clinical decision support products available over the next few years.