AI chatbots give risky medical advice in half of cases, study finds

Artificial intelligence chatbots are providing problematic medical advice in about half of user interactions, raising fresh concerns over the safety of widely used digital tools that millions rely on for health guidance, a new study has found.

The research, published in BMJ Open, evaluated five major AI platforms: ChatGPT, Gemini, Meta AI, Grok and DeepSeek, across a range of medical questions. Researchers found that roughly 50 percent of responses were problematic, including nearly 20 percent classified as highly problematic.

The study, conducted by researchers from the United States, Canada and the United Kingdom, tested the systems with 10 questions spanning five health categories. It found that chatbots performed better on straightforward, closed-ended questions, particularly in areas such as vaccines and cancer.

However, accuracy dropped significantly when users asked open-ended questions or sought advice on more complex topics like stem cells and nutrition.

Read also: Nigerian businesses must not miss out on the global AI Chatbot revolution

Despite these gaps, the systems frequently delivered answers with a high level of confidence, which researchers warned could mislead users into trusting inaccurate or incomplete information. None of the platforms provided fully complete or accurate reference sources in their responses, and only two instances of refusal to answer were recorded, both from Meta AI.

The findings come at a time when generative AI tools are increasingly being used for personal health queries. According to OpenAI, more than 200 million people ask health and wellness-related questions on ChatGPT each week, underscoring the scale of reliance on such tools.

However, experts caution that these platforms are not licensed medical professionals and lack the clinical judgment required for diagnosis or treatment recommendations. The study warns that without proper oversight and public education, AI chatbots could amplify misinformation and pose risks to users seeking medical advice.

The authors said the results highlight important behavioral limitations in current AI systems and called for a reassessment of how such tools are deployed in public-facing health communication. They added that chatbots can produce authoritative-sounding but potentially flawed responses, making it difficult for users to distinguish reliable information from misleading guidance.

Read also: Moniepoint AI chatbot ‘M’ to empower Nigeria’s informal economy

The report also comes as AI companies expand their footprint in healthcare. OpenAI earlier this year introduced new health-focused tools for users and clinicians, while rivals are rolling out similar offerings, intensifying competition in the emerging AI-driven healthcare space.

The rapid adoption of AI in health advice reflects both opportunity and risk. While chatbots can improve access to basic health information, particularly in underserved regions, weak safeguards could undermine trust and expose users to harm if inaccurate advice is followed.

For countries like Nigeria, where access to healthcare professionals remains uneven, the growing use of AI chatbots could fill critical gaps, but only if supported by strong regulation, clear disclaimers and improved digital literacy, experts say.