Skip to content

AI Doctors: Will They Beat Humans at Diagnosis?

Grid of brain MRI scan images
Photo: Pexels

Key takeaways

  • In controlled tests, AI has matched or beaten doctors on some diagnosis tasks. A 2026 study in Science reported 83.2% accuracy on hard published cases versus 36.1% for generalist physicians.
  • But in a randomized trial, doctors who were given an AI chatbot did not diagnose better than doctors without it, even though the chatbot alone scored higher.
  • Most of these results come from simulated cases, text chats or benchmark sets, not real patients in real clinics. The researchers themselves say more research is needed.
  • No study here says AI should replace your doctor. Treat AI health tools as a second opinion, not a diagnosis.

This article is general information about published research, not medical advice. If you have symptoms, see a qualified clinician.

Computer desk laptop stethoscope
A person holding a digital tablet
Doctor typing on keyboard with stethosco
Photos: Pexels (free to use)

Why this question matters

Diagnostic mistakes are a real problem in medicine, and AI tools are improving quickly. So researchers have started testing them head to head with doctors. The results are more nuanced than headlines suggest. Here are four studies, from oldest to newest, with the key numbers.

1. Reading medical images (2019 review, The Lancet Digital Health)

This systematic review looked at deep learning, the AI behind image-reading tools, compared with health professionals. It searched more than 31,000 studies and included 82. Only 14 compared AI with professionals on the same set of cases and had been tested on outside data. In those 14:

  • Sensitivity (catching people who have the disease): 87.0% for AI versus 86.4% for health professionals.
  • Specificity (correctly clearing people who don’t): 92.5% for AI versus 90.5% for professionals.

The authors concluded AI performance was “equivalent” to health professionals. They also flagged that few studies tested their models on outside data, and that poor reporting was common, which “limits reliable interpretation.”

2. Doctors using a chatbot (2024 randomized trial, JAMA Network Open)

Fifty physicians (family medicine, internal medicine and emergency medicine) were randomly assigned to diagnose up to six case write-ups either with a large language model (LLM) chatbot plus normal resources, or with normal resources only. The results:

  • Doctors with the chatbot: median score of 76%.
  • Doctors without it: 74%. The difference was 2 percentage points and not statistically significant.
  • The chatbot alone scored 16 percentage points higher than the doctors using only conventional resources.

The authors’ conclusion was that giving physicians an LLM “did not significantly improve clinical reasoning,” and that the chatbot alone did better than both groups of doctors, indicating a need for “technology and workforce development” to make human-AI teamwork work. In plain words: owning a good tool is not the same as using it well.

3. A chat-based AI doctor (2025, Nature)

A research system called AMIE was designed for diagnostic conversation. It was compared with 20 primary care physicians in a randomized, double-blind crossover study of text-based consultations with trained actors playing patients, across 159 case scenarios from Canada, the UK and India. Specialist physicians rated AMIE as better on 30 of 32 measures, including diagnostic accuracy, and the patient-actors rated it better on 25 of 26 measures.

The authors’ own warning: the study “should be interpreted with caution.” Doctors used text chat, which is “unfamiliar in clinical practice,” and the paper says more research is required before AMIE could be used in real-world settings.

4. A “panel of AI physicians” (2026, Science)

Researchers built an AI system that acts like a panel of physicians, asking for information step by step and choosing tests, and tested it on an interactive benchmark. It reached:

  • 83.2% accuracy on New England Journal of Medicine clinicopathological conference cases, versus 36.1% for generalist physicians.
  • 80.3% accuracy on emergency department admission records, versus 46.8% for generalist physicians.

The authors note these results came in “a controlled benchmark setting,” and describe them as showing promise for supporting more accurate, lower-cost diagnoses, not for replacing physicians. A benchmark is not how a real clinic works.

What the limits are

  • Most tests are not real life. The studies above used case write-ups, records, text chats with actors or benchmark sets. Real patients are messier, and a doctor can examine, listen and follow up.
  • Few studies test AI on outside data. The 2019 review found few did, which limits how far to trust a headline accuracy number.
  • Accuracy is not the same as better care. The 2024 trial found that giving doctors AI did not improve their scores in that test.
  • Human-AI teamwork needs work. That is the 2024 trial’s own conclusion.

What is allowed in the real world

The FDA keeps a public list of AI-enabled medical devices that it has authorized in the United States. The FDA says that these devices “have met the FDA’s applicable premarket requirements,” and notes that the list is not comprehensive. A general-purpose chatbot is not the same as an authorized medical device.

What this means for you

  • Use AI health tools as a second opinion at most. The research shows promise, but none of it says to skip your doctor.
  • Take any AI answer to a clinician if it concerns symptoms or treatment.
  • Ask how a tool was tested. Was it checked on real patients, or on practice cases?
  • Protect your privacy. Be careful about pasting medical records into a chatbot.

For a step-by-step look at what your phone shares, see our guide to phone privacy settings.

Frequently asked questions

Is AI better than doctors at diagnosis? In some controlled tests, yes. In others, a doctor with AI did no better than a doctor without it. None of these tests proves AI is better in everyday care.

Which AI results are the strongest? The review of image-reading AI found it equivalent to health professionals, though few studies were rigorous. The 2026 Science study reported 83.2% accuracy against 36.1% for generalist physicians, but on benchmark cases.

Can I use a chatbot to diagnose myself? No study here supports that. See a clinician for symptoms.

Will AI replace doctors? The sources here don’t say so. The authors call for more research and better human-AI teamwork.

Information checked on: November 13, 2026. This field changes quickly, so check current sources.

Sources

  • A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health, 2019
  • Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial. JAMA Network Open, 2024
  • Towards conversational diagnostic artificial intelligence. Nature, 2025
  • Sequential diagnosis with language models. Science, 2026
  • US Food and Drug Administration, List of Artificial Intelligence-Enabled Medical Devices