US EditionAI Chatbots Give Misleading Medical Advice 50% of the Time, Study Finds
Researchers found 50% of answers from five popular chatbots were problematic, and Grok had the highest rate at 58%, according to BMJ Open.
- A new BMJ Open study found half of medical information provided by five major chatbots was "problematic," with nearly 20 per cent deemed highly problematic. Researchers evaluated ChatGPT, Gemini, Meta, Grok, and DeepSeek across five health categories.
- Chatbots often "hallucinate," generating incorrect medical responses due to biased training data and a tendency to prioritize user beliefs over truth. These systems lack clinical judgment and are not licensed to provide professional health advice.
- Grok returned the most problematic responses at 58 per cent, followed by ChatGPT at 52 per cent and Meta at 50 per cent. Previous work found only 32 per cent of citations from specific models were accurate.
- Experts warned that expanding chatbot use in medicine requires "diligent oversight" to prevent misinformation amplification. Researchers emphasized the need for public education and professional training to ensure chatbots support rather than erode public health.
- Despite these limitations, more than 200 million people weekly ask ChatGPT health questions. The study authors concluded developers must reevaluate how these tools are deployed in public-facing health communication.
25 Articles
25 Articles
Popular chatbots often provide inaccurate and incomplete medical information
A substantial amount of medical information provided by 5 popular chatbots is inaccurate and incomplete, with half of the answers to clear evidence based questions "somewhat" or "highly" problematic, show the results of a study published in the open access journal BMJ Open.
Five widely used artificial intelligence chatbots often produced problematic answers to health and medical questions, according to a recent study. The researchers, in a study published yesterday in BMJ Open, tested the chatbots Gemini, DeepSeek, Meta AI, ChatGPT and Grok with 50 questions in five categories prone to misinformation. These included cancer, vaccines, stem cells, nutrition and athletic performance. The questions, asked in February 2…
AI Chatbots Mislead In 50% Cases, Struggle With Complex Clinical Diagnosis: Study
A new study finds AI chatbots often fail at clinical reasoning, missing key diagnoses and giving misleading advice. Experts warn these tools cannot replace doctors, especially in complex medical decision-making.
Chatbots based on artificial intelligence (IA) are providing problem medical advice to users about half of the time, according to a new study, highlighting the risks to health of this technology that is becoming increasingly essential every day. Puberness has happened more and more quickly in girls: this is normal? What are the causes? Doctors explain five tips to help remember how to work best researchers in the United States, Canada and the Un…
Substantial amount of medical information provided by popular chatbots inaccurate and incomplete
A substantial amount of medical information provided by 5 popular chatbots is inaccurate and incomplete, with half of the answers to clear evidence based questions “somewhat” or “highly” problematic, show the results of a study published in the open access journal BMJ Open. Continued deployment of these chatbots without public education and oversight risks amplifying misinformation, warn the researchers.
Coverage Details
Bias Distribution
- 40% of the sources are Center
Factuality
To view factuality data please Upgrade to Premium
