AI Chatbots Could Revolutionize Evidence-Based Medicine, Says New Study
Summary:
Medical researchers from the Icahn School of Medicine at Mount Sinai have conducted a study suggesting that artificial intelligence (AI) chatbots could act as standalone practitioners of evidence-based medicine. Their experiment tested several language-based AI models, with the model ChatGPT 4 outperforming others by achieving a 74% accuracy score. The researchers believe these models could manage patients according to specific guidelines and are a step toward achieving Artificial General Intelligence. However, the study has drawn skepticism due to unclear ethical implications and doubts about the feasibility of Artificial General Intelligence.
Recently, a study by medical scientists at the Icahn School of Medicine at Mount Sinai put artificial intelligence (AI) chatbots under scrutiny. They reached the startling conclusion that "generative large language models act as independent practitioners of evidence-based medicine."
The experiment involved testing various large language-based models (LLMs), including but not limited to Gemini Pro and ChatGPT versions 3.5 and 4, and open-source models like Mixtral-8x7B and LLaMA v2. Each model was fed with prompts such as "you are a medical professor" and asked to propose the appropriate treatment pathway for different test scenarios, adhering to evidence-based medical procedures. The models needed to suggest an immediate course of action, like ordering tests or initiating a treatment protocol, integrate the results and recommend the next step. ChatGPT 4 emerged as the most accurate, with a success rate of 74%, outscoring the runner-up by roughly 10%.
The researchers then inferred from these results that these models can practice medicine autonomously. Citing their research, they stated that "LLMs can function as independent practitioners of evidence-based medicine. They can interact with a real-world healthcare system’s infrastructure, utilising their inherent capabilities to manage patients according to specific guidelines."
Evidence-based medicine (EBM) takes cues from past cases to formulate a plan for subsequent similar conditions. Although EBM could be compared to navigating through a maze of possibilities, it can be tricky to handle the sheer volume of potential interactions and potential treatment routes. This is where LLMs could prove invaluable, claim the scientists, by performing tasks that usually require human expertise and letting the healthcare professionals focus on direct patient care.
However, the study does draw some scepticism due to its unverified belief in the capacities of current LLMs. The researchers purport that "LLMs are sophisticated tools that bring us closer to achieving Artificial General Intelligence," but this claim is not universally accepted.
Similarly, there is no unanimous agreement about whether Artificial General Intelligence is feasible or attainable within a significant timeline. The study's claims about LLMs' reasoning capacities are left unexplained, as are the ethical implications of integrating such unpredictable automated systems into current clinical processes.
Despite the claims made by the Sinai team, crucial questions remain unanswered. It is uncertain what advantages a general chatbot like ChatGPT would offer in a clinical EBM setting compared to the existing system or a custom-made medical LLM trained on a compilation of relevant data.
Published At
1/8/2024 10:12:00 PM
Disclaimer: Algoine does not endorse any content or product on this page. Readers should conduct their own research before taking any actions related to the asset, company, or any information in this article and assume full responsibility for their decisions. This article should not be considered as investment advice. Our news is prepared with AI support.
Do you suspect this content may be misleading, incomplete, or inappropriate in any way, requiring modification or removal?
We appreciate your report.