Microsoft Unveils Multi-Agent AI System Claimed to Beat Doctors in Diagnostics
Microsoft researchers published a study showing an AI diagnostic tool outperforming human physicians in accuracy, utilizing a multi-agent framework powered by OpenAI models.
Healthcare has long been cited by industry leaders as a prime sector where artificial intelligence could significantly enhance human well-being, and Microsoft AI CEO Mustafa Suleyman believes that milestone has finally arrived. The technology giant recently released findings in the journal Science detailing a newly developed medical diagnostic tool. According to Microsoft, the system demonstrated diagnostic accuracy rates up to four times higher than those achieved by human physicians across multiple testing scenarios.
The Technology Behind the Benchmark
To evaluate the system, Microsoft created a custom benchmark called SDBench, which it intends to open-source. The underlying software, known as the Diagnostic Orchestrator, is designed to emulate a panel of medical experts. Instead of relying on a single prompt, it deploys multiple AI agents to collaboratively evaluate symptoms, review medical histories, determine necessary tests, and formulate a diagnosis.
In evaluations using medical reports from The New England Journal of Medicine, the system achieved roughly 83 percent diagnostic accuracy. By comparison, generalist physicians scored around 36 percent when utilizing reference materials and 22 percent without them. Furthermore, testing against emergency department admissions data from Beth Israel Deaconess Medical Center in Boston showed the AI system reaching 80 percent accuracy, while human doctors averaged 47 percent. The automated framework also tended to arrive at conclusions with lower estimated costs than human practitioners.
Powered by OpenAI technology, the Diagnostic Orchestrator was initially built around GPT-4.1, though researchers noted that subsequent testing indicated multi-agent collaboration continues to provide distinct advantages as underlying foundational models improve.
What it means for developers
The advancement of multi-agent orchestration frameworks points toward increasingly complex, specialized reasoning tasks that require coordinated AI systems rather than isolated model calls. Developers looking to experiment with advanced agentic architectures, reasoning models, and multi-step workflows can try top AI models cheaply through one API at https://apixoai.online. Building resilient applications often means testing various frontier models to find the right balance of cost and capability for complex domains.
Path to Clinical Validation
Despite the promising benchmark scores, Microsoft acknowledges that extensive real-world testing remains necessary before deployment. The current assessments relied entirely on historical case studies, leaving actual clinical efficacy in live environments unverified. Suleyman emphasized that formal clinical validation represents a significant leap forward, though he views the study as a major step along that path.
Microsoft plans to initiate hospital testing alongside Beth Israel Deaconess Medical Center within the next few months to observe how the tool functions in active clinical settings. While commercialization details remain unspecified, the company is also looking at broader healthcare challenges, including care coordination, treatment decisions, and administrative workflows. Suleyman maintains that human clinicians will continue to play a central role, expressing hope that automation will reduce administrative burdens and allow practitioners to spend more time directly with patients.
Source: Microsoft says its AI tool is better than doctors at diagnosing conditions — The Verge AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
