Microsoft AI System Outperforms General Physicians in Diagnostic Tests
Microsoft's AI system achieved higher diagnostic accuracy than general physicians in two benchmarks, although its performance in clinical settings remains unproven.
Topics
News
- External Review Finds Flaws in OpenAI’s Agent Monitoring System
- Microsoft AI System Outperforms General Physicians in Diagnostic Tests
- How Sundar Pichai Uses AI Agents to Track Google Operations
- Tech Mahindra Partners With SpaceXSI to Scale AI-Powered Coding
- Barclays to Cut 5,000 Jobs as AI Drives Cost Savings
- CERT-In Warns of Compromised eMudhra Certificate Authority Keys
Microsoft’s AI Diagnostic Orchestrator (MAI-DxO) outperformed general physicians in two diagnostic benchmarks, according to research published on Thursday in Science.
The system uses multiple AI agents to assess cases, ask questions, order tests and revise diagnoses as new information becomes available, rather than analyzing all patient information at once.
Researchers evaluated MAI-DxO using Microsoft’s Sequential Diagnosis Benchmark, which simulates the step-by-step process of diagnosing patients. The research was published in Science.
In 304 complex cases from The New England Journal of Medicine, the system achieved 83% diagnostic accuracy, compared with 36% for general physicians using reference materials and 22% without them.
On more routine emergency-department cases from the MIMIC-IV-ED database, MAI-DxO achieved 80% accuracy against 47% for physicians.
Microsoft also assessed diagnostic costs. In one configuration involving complex cases, the system maintained 81% accuracy while selecting tests costing less than half as much as those used in comparison approaches.
Unlike a single AI model, MAI-DxO coordinates agents that propose and challenge diagnoses, identify necessary tests and consider their costs.
The findings do not show that the system is ready to diagnose patients independently. The tests were conducted using historical, structured cases rather than patients receiving care in real time.
Microsoft is now working with Boston’s Beth Israel Deaconess Medical Center on a prospective study in which patients will interact with the system and its assessments will be compared with physicians’ diagnoses.
