ChatGPT Outperformed Doctors in Diagnosing Diseases and Medical Conditions, Says Study

The study also found that doctors who used ChatGPT to diagnose diseases scored higher than those who did not use the chatbot.

Written by Akash Dutta, Edited by Manas Mitul | Updated: 19 November 2024 19:38 IST

ChatGPT Outperformed Doctors in Diagnosing Diseases and Medical Conditions, Says Study

Photo Credit: Unsplash/Levart_Photographer

The findings of the study were recently published in the JAMA Network Open journal

Highlights

The experiment included 50 residents and attending physicians
Doctors were given six case histories to diagnose
ChatGPT alone scored 90 percent on the test

ChatGPT was able to outperform human doctors in diagnosing diseases and medical conditions in a study. The findings of the study were published last month and highlighted that artificial intelligence (AI) chatbots might be more efficient in analysing patient histories and conditions and provide more accurate diagnoses. While the study aimed to understand if AI chatbots could help doctors provide better diagnoses, the results unexpectedly revealed that OpenAI's GPT-4-powered chatbot performed much better when performing without human assistance compared to when paired with a doctor.

ChatGPT Outperforms Doctors in Diagnosing Diseases

The study, published in the JAMA Network Open journal, was conducted at the Beth Israel Deaconess Medical Center in Boston by a group of researchers. The experiment aimed to find out if AI can help doctors better diagnose diseases compared to traditional methods.

According to a New York Times report, the experiment involved 50 doctors who were a mix of residents and physicians attending the medical college. They were recruited through multiple large hospital systems in the US and were given six case histories of patients. The subjects were reportedly asked to suggest a diagnosis for each of the cases and provide an explanation for why they favoured or ruled certain diagnoses out. Doctors were said to also be graded based on whether their final diagnosis was right.

[Sponsored] Samsung Galaxy S24: The Ultimate Camera Phone With Smarter On-Device AI

To evaluate each of the participants' performance, medical experts were reportedly selected as graders. While they were said to be shown the answers, they were not told if the response came from a doctor with access to AI, just the doctor, or from only ChatGPT.

Further, to eliminate the possibility of unrealistic case histories, the researchers reportedly picked case histories of real patients that have been used by researchers for decades but have never been published to avoid contamination. This point is important because ChatGPT cannot be trained on data which has never been published.

The findings of the study were surprising. Doctors who did not use any AI tool to diagnose the case histories had an average score of 74 percent whereas those physicians who used the chatbot scored 76 percent on average. However, when ChatGPT alone analysed the case histories and provided diagnosis, it scored an average of 90 percent

While various factors could have impacted the outcome of the study — from the experience level of the doctors to individual biases with certain diagnoses — the researchers believe the study highlights that the potential of AI systems in medical institutions cannot be ignored.

Comments

For the latest tech news and reviews, follow Gadgets 360 on X, Facebook, WhatsApp, Threads and Google News. For the latest videos on gadgets and tech, subscribe to our YouTube channel. If you want to know everything about top influencers, follow our in-house Who'sThat360 on Instagram and YouTube.

Further reading: ChatGPT, OpenAI, AI, Artificial Intelligence

Akash Dutta Email Akash Dutta

Akash Dutta is a Senior Sub Editor at Gadgets 360. He is particularly interested in the social impact of technological developments and loves reading about emerging fields such as AI, metaverse, and fediverse. In his free time, he can be seen supporting his favourite football club - Chelsea, watching movies and anime, and sharing passionate opinions on food. More