AI outperforms resident physicians in emergency medicine exams, study shows

Doha, Qatar: A unique study conducted in Qatar reveals that ChatGPT, a generative artificial intelligence chatbot, has demonstrated remarkable profici...

Doha, Qatar: A unique study conducted in Qatar reveals that ChatGPT, a generative artificial intelligence chatbot, has demonstrated remarkable proficiency in theoretical aspects of emergency medicine (EM), outperforming resident physicians in examination settings.

The study, titled “Performance of ChatGPT in Emergency Medicine Residency Exams in Qatar: A Comparative Analysis with Resident Physicians,” has sparked interest within the readers. It is one of the most-read articles on the Qscience website this month.

Published in the recent edition of the Qatar Medical Journal, the study underscores the growing potential of artificial intelligence (AI) as a supplementary tool in medical education. The findings suggest that AI could play a crucial role in enhancing learning and assessment methods in medical fields, particularly in emergency medicine.

The study, conducted by a team of experts in Qatar, focuses on evaluating ChatGPT’s performance in EM residency examinations. By comparing the AI’s results to those of resident physicians, the research highlights the potential of AI to contribute to medical training and assessment.

The examination format consisted of multiple-choice questions (MCQs) designed by the same faculty responsible for Qatari Board EM exams. In August 2023, a retrospective descriptive study with a mixed-methods design was carried out. The performance of 238 emergency department residents from various postgraduate years (PGY1 to PGY4) was assessed. The participants’ examination scores were then compared with those of ChatGPT, which completed the same exams.