- On 304 complex New England Journal of Medicine cases, MAI-DxO running on GPT-5 reached 83.2% accuracy at $5,431 of estimated tests per case, while physicians reached 36.1% at $4,725.
- On 325 routine emergency department admissions it scored 80.3% against 46.8% for physicians, and the best single physician reached 62.5%.
- Microsoft's orchestration added an average of 7.9 percentage points on complex cases and 4.4 on emergency cases across models, so most of the gain comes from the underlying model.
Microsoft's AI diagnostic system correctly diagnosed 83.2% of the hardest published medical cases against 36.1% for practicing physicians with reference tools, and spent less on tests to get there, according to a peer-reviewed study in Science on October 22. The result gives health systems the first peer-reviewed evidence that AI can manage the whole diagnostic workup, from first question to final answer, and Microsoft is now moving it into a prospective study with real patients at Beth Israel Deaconess Medical Center in Boston.
Microsoft's system beat physicians on accuracy at every cost level tested
The study, led by Harsha Nori and Eric Horvitz with 13 Microsoft co-authors, built a benchmark called SDBench that turns case records into interactive encounters. A diagnostician, human or AI, starts with a short vignette and must ask questions and order tests, each priced from a US hospital's public price list, before committing to a diagnosis. MAI-DxO, the Microsoft AI Diagnostic Orchestrator, simulates a panel of physicians that propose diagnoses, challenge each other and weigh each test's value against its cost.
On the complex cases, physicians working without references averaged 22.3%, and even the strongest physician in the study, at 51.8%, was beaten on both accuracy and cost. A cost-constrained setting of MAI-DxO still reached 79.6% while ordering $1,303 of tests per case, about a quarter of what physicians spent with reference tools. Physicians averaged 11.9 to 18.1 minutes per case.
Emergency department cases make this a hospital operations story
The 2025 preprint of this work used only the curated New England Journal of Medicine cases, which are chosen because they are hard and teach well. The Science paper adds 325 emergency department admissions from the MIMIC-IV-ED database, written in the plain style of everyday electronic health records, where generalist physicians are on home ground. MAI-DxO's accuracy held at 80.3% there, while physicians rose only to 46.8%, a gap the authors say they had expected to be smaller.
| Test set | MAI-DxO, no budget | Physicians, average | Physicians, test cost per case |
|---|---|---|---|
| 304 complex NEJM cases | 83.2% at $5,431 | 36.1% with references, 22.3% without | $4,725 with references |
| 325 emergency admissions | 80.3% at $4,979 | 46.8% | $3,400 |
Source: Nori et al., Science, October 22, 2026. Costs are estimated from Kaiser Permanente Washington's public chargemaster.
The method behind those numbers is the part with commercial weight. MAI-DxO asked a mean of 25.5 questions before ordering its first test on complex cases, against 5.6 for physicians, which means the system substitutes cheap history-taking for expensive testing. In a US system where testing drives a large share of diagnostic spending, a tool that front-loads questions also works as a way to rein in testing costs. Microsoft says it has folded its health teams, including the groups behind Dragon Copilot and PowerScribe, into Microsoft AI, which gives the research a direct route into clinical software that hospitals already buy.
Microsoft's diagnostic orchestrator ran best on OpenAI's GPT-5 and GPT-5.5, so the company is selling clinical judgment built on a partner's model while it trains its own.
Most of the gain belongs to the model, and Microsoft owns the layer above it
The paper's own numbers temper the headline. Across every model tested, Microsoft's orchestration raised accuracy by 4.4 points on emergency cases and 7.9 points on complex ones, which means most of the distance between AI and physicians comes from frontier large language models themselves. The authors also report that the advantage over a plainly prompted model grew when they reran everything on GPT-5.5 without changing prompts. That positions MAI-DxO as a model-agnostic multi-agent system that improves as the models under it improve, at a moment when Microsoft is also training its own frontier base models.
The comparison has limits the paper states plainly. Physicians worked alone on text, without specialists, colleagues or the patient in the room. Both case sets contain only sick patients, so the system was never tested on the healthy and benign visits that fill most clinics, and all 15 authors are Microsoft employees who hold company equity, with eight named on a related patent application.
Real patients at Beth Israel are the next test
Microsoft says it will run a prospective study with Beth Israel Deaconess in which patients interact with the system directly and its conclusions are compared with physicians' independent assessments, and it is building health models with Mayo Clinic. It has also released the SDBench code under an MIT license, so rival labs can now be scored on the same sequential test. Microsoft has announced no commercial release date for MAI-DxO.
The Science paper settles one question and opens a harder one. AI has now outperformed generalist physicians on diagnostic workups in a peer-reviewed benchmark, at lower estimated test cost, using models anyone can rent. The value in healthcare AI will therefore go to whoever proves the result on live patients first and plugs it into hospital workflows, and Microsoft, with Beth Israel enrolled, its own diagnostic layer and the clinical software already on hospital desks, has the shortest path to that point.
Santage is committed to independent, transparent journalism. This article is produced in accordance with Santage's Editorial Standards and aims to provide accurate and timely information. Readers are encouraged to verify information independently.