OpenAI's Language Model, ChatGPT 4.0, Passes Neurology Exam – An Indication of Future Clinical Applications
Summary:
OpenAI's language model, ChatGPT 4.0, successfully passed a neurology exam with an 85% success rate in a study performed by researchers from the University Hospital Heidelberg and the German Cancer Research Center. The model outperformed the average human score and an older version of itself, demonstrating its potential for clinical applications in neurology. However, there were shortcomings in tasks requiring higher cognitive functions, and experts advise caution in implementing such models in practice without further refinements.
OpenAI's cutting-edge language model, ChatGPT 4.0, has demonstrated its applicability in neurology by achieving an 85% success rate on a test that mirrors a clinical neurology exam. Researchers from the University Hospital Heidelberg and the German Cancer Research Center, who conducted the study, suggest that a few more adjustments could make these models greatly useful in the field of neurology. The findings of the trial were disclosed on December 7. In the test, which took place on May 31, two models, ChatGPT 3.5 and ChatGPT 4.0, were posed questions from the American Board of Psychiatry and Neurology exam, along with a handful from the European Board for Neurology.
In contrast to its predecessor, ChatGPT 3.5, which achieved a score of 66.8% by correctly answering 1306 out of a set of 1956 questions, the updated version, ChatGPT 4.0, secured an impressive 85% by getting 1662 questions right. This surpassed the average human performance of 73.8%. The 4.0 model excelled particularly in questions related to behavior, cognition, and psychology, effectively passing the neurology test, which considers a score of 70% as the threshold to pass. However, both versions were less capable when it came to tasks demanding a higher level of cognition in contrast to simpler tasks.
According to the research team, these models, with some improvements, could greatly aid in clinical neurology. However, due to weaknesses in handling complex cognitive tasks, usage of these models in regular practice needs to be coupled with caution. Dr. Varun Venkataramani, one of the study's authors, clarified that this study is primarily a representation of the potential capabilities of LLMs, although they still need enhancements for efficient application in neurology.
AI technology has been making significant contributions in healthcare, ranging from acting as an aid in the discovery of cancer treatment for AstraZeneca, to mitigating the problem of antibiotic overprescription in Hong Kong.
Published At
12/11/2023 4:57:53 PM
Disclaimer: Algoine does not endorse any content or product on this page. Readers should conduct their own research before taking any actions related to the asset, company, or any information in this article and assume full responsibility for their decisions. This article should not be considered as investment advice. Our news is prepared with AI support.
Do you suspect this content may be misleading, incomplete, or inappropriate in any way, requiring modification or removal?
We appreciate your report.