AI chatbots ‘highly vulnerable’ to repeating false medical information, experts warn

AI chatbots are frequently prone to repeating false and misleading medical information, according to new research.

Experts have warned of a “critical need” for stronger safeguards before the bots can be used in healthcare, adding models not only repeated untrue claims but also “confidently” expanded on them to create explanations for non-existent medical conditions.

The team from the Mount Sinai School of Medicine created fictional patient scenarios, each containing one fabricated medical terms such as a made-up disease, symptom, or test, and submitted them to leading large language models. In a study published in journal Communications Medicine, they said that the chatbots “routinely” expanded on the fake medical detail, giving a “detailed, decisive response based entirely on fiction”.

Read full article.

Source: The Independent, 7 August 2025