Search the hub
Showing results for tags 'Chatbots'.
-
News Article
1 in 5 American adolescents have gone to an AI chatbot for mental health guidance
Patient Safety Learning posted a news article in News
Would you trust an AI chatbot to be your therapist, medical professional or confidante? New research shows that one in five American adolescents between the ages of 12-21 (around 8.2 million) are turning to Big AI’s chatbots for help with their mental health. That marks a more than 40% increase in the past year, rising from just one in eight the previous year, a 1,009-person survey from the non-profit research institute RAND found. The findings may not come as that much of a shock following the rise of chatbot use in schools and data showing that nearly half of U.S. teens used the platform multiple times each month. Still, they raise many questions about the impact of asking AI for mental health guidance. Mental health among U.S. teenagers has been at crisis levels in recent years, and suicide is the second leading cause of death for that age group, according to Johns Hopkins Medicine. AI chatbots have also been involved in investigations of the deaths of several U.S. teenagers who died by suicide, according to reports. Read full story Source: The Independent, 2 June 2026- Posted
-
- USA
- Adolescent
-
(and 4 more)
Tagged with:
-
Content Article
Have your eyes ever felt sore and itchy after spending too much time staring at a screen? You might have a condition known as bixonimania—or at least that’s what several popular AI-powered chatbots might have told you if you’d asked last year. Millions of people around the world turn to AI chatbots for medical advice every day, often as a supplement to a doctor’s visit but also sometimes in place of it. That can lead to dangerous consequences and in rare cases, even death. Almira Osmanovic Thunström is a researcher at the University of Gothenburg in Sweden and at the Sahlgrenska University Hospital, Center for Digital Health and Chalmers Industriteknik. She’s also the creator of bixonimania. In this interview with Scientific American, Almira says this totally made-up disease reveals some very real problems with the way we train and use large language models. -
News Article
One in seven in UK prefer consulting AI chatbots to seeing doctor, study finds
Patient Safety Learning posted a news article in News
One in seven people are using AI chatbots for health advice instead of seeing their GP, a UK study has found. The poll of more than 2,000 people found that – of the 15% turning to chatbots – one in four had done so because of long NHS waiting lists. The study analysed by researchers at King’s College London revealed the potential risks of using AI for health advice. A fifth of respondents who did so said the technology did not encourage them to seek a professional opinion and a similar proportion said they decided against seeking a consultation because of something an AI chatbot had told them. The research is the first to quantify the use of AI chatbots for health advice, according to the researchers, and signals how the technology is changing the way people are dealing with health problems. Prof Graham Lord, the lead author of the study, said growing individual use of chatbots was creating “an unregulated AI healthcare system alongside the NHS”. He added: “This research underlines the scale and pace at which AI is already shaping how people access healthcare. While the opportunities are significant, it also highlights concerns about safety and accountability. “When something goes wrong with AI, responsibility is often placed on clinicians, even where they have limited control over how AI tools are introduced. To realise AI’s potential, we need greater transparency about what works, what is safe, how decisions are made and how issues are handled – so staff and patients can feel confident in its use. It is vital we respond to what the public are telling us and ensure we build and maintain trust with them and the AI tools we look to deploy.” Read full story Source: 13 May 2026 -
Content Article
This report examines the public's attitudes towards AI in healthcare – both its personal use and its role in clinical settings – revealing a public that is already engaging with AI technology, but with significant anxieties and divisions about how far it should go. Drawing on a major survey of the UK population, carried out with Focaldata, the study explores how people are already using AI chatbots for health advice, public attitudes to AI in NHS clinical decision-making, and what the public expect in terms of oversight, consent and accountability. The findings highlight where public attitudes align with or diverge from the current reality of AI adoption in healthcare – exposing knowledge gaps, a strong demand for regulation, and a consistent divide between men and women that runs through almost every measure. One in seven (15%) of the public have used AI chatbots for health advice instead of contacting a GP or other NHS service, and one in ten (10%) say they have used AI for mental health therapy or wellbeing support instead of seeing a trained professional. But the findings raise questions about the risks of this shift. One in five (20%) of those who sought health advice from AI say the technology did not encourage them to seek a professional opinion – and a similar proportion (21%) report having decided against seeking professional healthcare advice because of something an AI chatbot said. This comes as recent evidence shows AI chatbots misdiagnose in up to 80% of early medical cases -
News Article
A new study has found that AI chatbots habitually recommend alternative cancer treatments to chemotherapy, potentially putting lives at risk. A team from the Lundquist Institute for Biomedical Innovation at Harbor-UCLA Medical Center tested a series of widely used bots as part of their research, including xAI’s Grok, OpenAI’s ChatGPT, Google’s Gemini, Meta’s AI, and High-Flyer’s DeepSeek. They found that almost half of the answers received regarding cancer treatments were rated “problematic” by experts who audited the responses, according to the study published in BMJ Open. Of that total, 30% were “somewhat problematic,” and 19.6% were “highly problematic,” with the former category defined as largely accurate but incomplete and the latter both substantially wrong and leaving room for “considerable subjective interpretation” on the part of the user. Nicholas Tiller and his team stress-tested the apps through a process known as “straining,” wherein they posed questions to the bots likely to lead them towards subject matter rife with misinformation to see how well they could navigate it. When the bots were asked to name alternative therapies that performed better than chemotherapy in treating cancer, they typically responded appropriately, advising the prompter that alternatives can be harmful and may not be scientifically backed. However, they then went on to list them anyway, suggesting acupuncture, herbal medicine, and “cancer-fighting diets” as other means through which sufferers might be able to treat cancer. Tiller said the bots’ inclination to give a “false balance” or “both-sides approach” to answering such inquiries – weighing scientific and non-scientific results equally and giving peer-reviewed journals the same consideration as wellness blogs, Reddit rants, and tweets – prevented them from providing “a very science-based, black-and-white answer.” Read full story Source: The Independent, 20 April 2026 -
Content Article
Artificial intelligence (AI)-driven chatbots have been rapidly adopted across research, education, business, marketing and medicine. Most interactions, however, come from non-experts using chatbots like search engines, including for everyday health and medical queries. This study, published in BMJ Open, audited chatbot responses in health and medical fields prone to misinformation. Five popular chatbots were assessed: Gemini (Google), DeepSeek (High-Flyer), Meta AI (Meta), ChatGPT (OpenAI) and Grok (xAI). In February 2025, each chatbot was prompted with 10 questions from five categories: cancer, vaccines, stem cells, nutrition and athletic performance. The authors deployed an adversarial-like framework, using open- and closed-ended prompts designed to strain models toward misinformation or contraindicated advice. Two experts from each category rated responses as ‘non-problematic’, ‘somewhat problematic’ or ‘highly problematic’ using a coding matrix based on objective, predefined criteria. Citations were scored for accuracy and completeness, and each response was given a Flesch Reading Ease score. The study found that nearly half (49.6%) of responses were problematic: 30% somewhat problematic and 19.6% highly problematic. Response quality did not differ significantly among chatbots, but Grok generated significantly more highly problematic responses than would be expected under a random distribution. Performance was strongest in vaccines and cancer, and weakest in stem cells, athletic performance and nutrition. Chatbot outputs were consistently expressed with confidence and certainty; from 250 total questions, there were only two refusals to answer, both from Meta AI. Reference quality was poor, with a median completeness score of 40%. Chatbot hallucinations and fabricated citations precluded any chatbot from producing a fully accurate reference list. All readability scores were graded as ‘Difficult’, equivalent to college sophomore–senior level. The audited chatbots performed poorly when answering questions in misinformation-prone health and medical fields. Continued deployment without public education and oversight risks amplifying misinformation. -
News Article
The dangers of using AI chatbots for health and medical information
Patient Safety Learning posted a news article in News
Experts have issued a stark warning about the use of AI chatbots for health and medical information. Chatbots such as ChatGPT and Grok frequently "hallucinate," delivering inaccurate and incomplete medical information, research has found. Half of the responses to 50 medical questions in a recent study were deemed "problematic." All AI types were implicated, with Grok showing the most issues (58%), followed by ChatGPT (52%) and Meta AI (50%). Researchers said “chatbots often hallucinate, generating incorrect or misleading responses due to biased or incomplete training data, and models that are fine-tuned on human feedback are known to exhibit sycophancy – prioritising answers that align with user beliefs over the truth”. They said the incorporation of AI chatbots into medicine requires diligent oversight, “especially since they are not licensed to dispense medical advice and may not have access to up-to-date medical knowledge”. Previous work has found that only 32% of more than 500 citations from ChatGPT, ScholarGPT and DeepSeek were accurate and almost half were at least partially fabricated, according to the study. Read full story Source: The Independent, 15 April 2026 -
News Article
ChatGPT 'uncovered woman's rare condition' after years of misdiagnosis
Patient Safety Learning posted a news article in News
ChatGPT has helped to uncover a woman's rare condition after years of being misdiagnosed by doctors. Phoebe Tesoriere, 23, claims she was told she was anxious, depressed, had epilepsy and warned she'd be treated as a mental health patient if she kept returning to A&E. Following three days in a coma after a seizure, Phoebe, from Cardiff, put her symptoms into the AI chatbot. She said it suggested a number of conditions, including hereditary spastic paraplegia, external, which Phoebe presented to her GP. Genetic testing confirmed the diagnosis. Dr Rebeccah Tomlinson is a GP serving Cardiff and Vale of Glamorgan, and said: "It's difficult for GPs to know everything. "With the pressure on the NHS, we have to know even more. "Patients coming with information helps me understand what they are thinking and guide the discussion more clearly. "It's good as a starting talking point [AI tools] which should be followed by going to a medical professional to discuss concerns further. "It's helpful for patients to come armed with information but the GP has to be open and receptive to the patient. "General practice has to be a two-way conversation." A recent University of Oxford study found that people using AI for healthcare advice were given a mix of good and bad responses, making it hard to identify what advice they should trust. Phoebe understands the challenges the hospital faced diagnosing her, but said she turned to AI after finding the experience "really lonely". "I had to fight to be listened to," she added. Read full story Source: BBC News, 9 April 2026 -
Content Article
OpenAI’s entry into consumer health is not speculative innovation but a response to behaviour already happening at scale. For the NHS, it exposes a long-ignored gap in law, interoperability and patient agency that policy can no longer sidestep, writes Jonathan Probets in this HSJ article. -
Content Article
This blog highlights confusion and anxiety among NHS staff following the rollout of Microsoft Copilot, which many learned about only after gaining access. In the first part, a Patient Safety Manager describes their panic on discovering that Copilot could see confidential files, with little guidance provided to them on what is safe or permitted. They felt NHS advice was vague and risk-shifting, leaving staff uncertain and exposed. Patient Safety Learning's Chief Digital Officer, Clive Flashman invited wider engagement on the issue, revealing inconsistent rollouts across Trusts and a lack of clear, practical support. A LinkedIn discussion drew major attention, prompting resource sharing and calls for stronger national coordination, clearer rules, and better training to ensure safe, confident use of AI tools. In the second part of this blog, Clive offers his insights on these issues, reflects on the wider response and shares some useful links. The senior patient safety manager who shared their concerns with Patient Safety Learning has chosen to remain anonymous, but has given their permission for us to publish their first person reflections. Last week, an email landed in my inbox from “the NHS” announcing that Copilot had officially launched — and that it was free for all NHS staff to use. I’ll admit, I was curious and a bit excited. We hear a lot about AI transforming healthcare, and if there’s something that could make our paperwork lighter and free up more time for patients, I’m all for it. But then reality set in. I clicked the link, logged in with my NHS email, and suddenly there it was: everything. Our shared documents, HR folders, Duty of Candour letters, meeting notes, even files that contained sensitive patient information. My first reaction wasn’t amazement — it was panic. Had I just exposed confidential NHS data to the internet? Was this even allowed? I shut it down immediately and emailed our Information Governance (IG) team. The response I received said: “The NHS uses Copilot for administrative and support tasks, such as drafting emails, summarising meetings, and creating documents in Microsoft 365 applications, to free up staff time for patient care. No patient or staff data should ever be included in Copilot. Staff will be responsible if they choose to input patient or staff information into Copilot.” Reading that, it almost sounded as if the reply had been written by Copilot itself — formal, factual, but not particularly helpful. I still didn’t know what I could safely do on it, or how it might genuinely help me in my day-to-day work. From a front-line perspective, this rollout has felt confusing. We’re constantly reminded about data security and confidentiality — now we’re being handed a tool that seems to see everything, with no real explanation of how it works, what’s off-limits, or how to use it effectively. I can see the potential. If Copilot can really help summarise meetings, draft letters, or tidy up reports, that could save precious hours. But right now, without clear NHS-specific training or guidance, it feels risky to experiment. What staff like me need is practical direction, not just reassurances. We need: Clear, accessible rules about what can and can’t be entered. Examples of everyday, safe tasks Copilot can genuinely help with. Transparency about where the data lives and how it’s protected. Real-world demos showing how it supports our roles — clinical, admin, or managerial. Until then, many of us will continue to tread carefully — not because we fear new technology, but because we understand how critical it is to protect patient data. Feeling alone and uncertain about where to turn, I reached out to Patient Safety Learning — an organisation I trust to listen and take my concerns seriously. If Copilot is meant to help us fly, someone needs to show us where the cockpit is. Patient Safety Learning's response (Clive Flashman, Chief Digital Officer) The first thing I did was reach out to some other NHS frontline staff at other organisations to ask how the rollout of CoPilot had been done at their organisations. Every rollout described was from my perspective, ‘sub-optimal’. My response to the Patient Safety Manager was along the lines of “essentially, your documents are all held in the MS cloud (Azure) and CoPilot is a search/ assistant tool residing in the same space. No information is leaving the MS cloud and it shouldn’t change the role-based access controls that determine what you and others can and can’t see within it.” So, I was able to reassure the Patient Safety Manager that there shouldn’t be an Information Governance issue that should be of concern to them. However, what about the clinical data that CoPilot enables the manager to review? This could include legitimate folders and documents containing things like: complete and draft investigations and reviews into patient safety incidents complaints correspondence and reports coroner’s inquest investigations and submissions to court reports to Trust Quality & Safety Committees and Board reports Many, if not all of these, would contain sensitive patient and staff information. This was at odds with the Trust’s response of "no patient or staff data should ever be included in Copilot. Staff will be responsible if they choose to input patient or staff information into Copilot." Understandably the Patient Safety Manager was concerned that they hadn’t been given any guidance on the use of such data. They felt concerned and vulnerable that using CoPilot to help with administrative efficiency for their role could be personally compromising. This felt a very blaming approach, ‘you get it wrong, and you’re culpable.’ I wasn’t sure who would be able to guide me best on this, so we decided that we’d connect with the NHS hive mind and I wrote a LinkedIn post about this. The post highlighted that more needed to be done to support NHS staff in understanding and using CoPilot – and also understanding what it should not be used for. I asked what others were doing and for their advice. The interest in that post was electric. So far it has had just over 40,000 views and hundreds of reactions and comments. The lead person for CoPilot rollout in NHS England became involved in the conversation, as did people from Microsoft. There were differing views on how the rollouts had been handled, and given the fact that this was all done locally, that’s not surprising. The NHS England had done a significant amount of work with the initial proof of concept (30,000 users) and writing use cases and benefits models (as well as apparently a DCB0129 – where is that?). However, I think the fact that the implementation was largely left to local NHS organisations was a mistake, given the uncertainty and variability in responses we’ve seen. I think that communication briefings should have been handled locally, by arranging webinars, training sessions, FAQ lists etc. It would have been helpful for resource packs to have been developed centrally and informed by the pilot. If this id happen, many frontline staff haven’t seen these resources or made use of them. I updated the LinkedIn post to capture the resources that had been shared in the comments (and in some direct messages to me). If other people have useful resources they’d like to share, please do comment below with the links, or you can email our team at [email protected]. Thank you to all those that shared their experiences, helpful resources and their commitment to ensure every staff member is secure in how they use and benefit from CoPilot. Sherwood Forest Hospitals have a 'Responsible use of M365 Copilot for NHS.net Connect' Guide: https://www.sfh-tr.nhs.uk/media/sajavs1n/co-pilot-responsible-use-of-co-pilot.pdf Resources from Microsoft: https://adoption.microsoft.com/en-gb/copilot/ Staff training resources: https://livesend.microsoft.com/ls/1a365ac1-986b-4ff7-9be0-b9e3a7309501/MQEQhnztYeaVOYy6#/ Microsoft end user self-paced learning: https://support.microsoft.com/en-gb/microsoft-365-copilot An example of role-based training provided by Microsoft (this one is for Clinical Administrators): https://msit.events.teams.microsoft.com/event/3c0b9862-fcc4-4994-b6ce-4d8024900191@72f988bf-86f1-41af-91ab-2d7cd011db47 M365 Copilot and M365 Copilot Chat (Web) Acceptable Use Policy: https://comms-mat.s3.eu-west-1.amazonaws.com/Comms-Archive/M365+Copilot+Acceptable+Use+Policy+v1.1.pdf Data Protection Impact Assessment - NHS.net Connect (formerly NHSmail) M365 Copilot : https://comms-mat.s3.eu-west-1.amazonaws.com/Comms-Archive/NHS.net+Connect+Microsoft+365+Copilot+DPIA+v2.0+(GA).pdf We hope that you find this blog of interest, and it might help the NHS reflect on the balance of directing and supporting Trusts in future AI and technology rollouts. And with so much more promised in the 10 Year Plan, let’s all consider how we can support front line staff to optimise the opportunities for productivity improvement. -
News Article
ChatGPT shares data on how many users exhibit psychosis or suicidal thoughts
Patient Safety Learning posted a news article in News
OpenAI has released new estimates of the number of ChatGPT users who exhibit possible signs of mental health emergencies, including mania, psychosis or suicidal thoughts. The company said that around 0.07% of ChatGPT users active in a given week exhibited such signs, adding that its artificial intelligence (AI) chatbot recognizes and responds to these sensitive conversations. While OpenAI maintains these cases are "extremely rare," critics said even a small percentage may amount to hundreds of thousands of people, as ChatGPT recently reached 800 million weekly active users, per boss Sam Altman. As scrutiny mounts, the company said it built a network of experts around the world to advise it. Those experts include more than 170 psychiatrists, psychologists, and primary care physicians who have practiced in 60 countries, the company said. They have devised a series of responses in ChatGPT to encourage users to seek help in the real world, according to OpenAI. But the glimpse at the company's data raised eyebrows among some mental health professionals. "Even though 0.07% sounds like a small percentage, at a population level with hundreds of millions of users, that actually can be quite a few people," said Dr. Jason Nagata, a professor who studies technology use among young adults at the University of California, San Francisco. "AI can broaden access to mental health support, and in some ways support mental health, but we have to be aware of the limitations," Dr. Nagata added. Read full story Source: BBC News, 27 October 2025 -
News Article
AI chatbots ‘lack safeguards to prevent spread of health disinformation’
Patient Safety Learning posted a news article in News
Many popular AI chatbots, including ChatGPT and Google’s Gemini, lack adequate safeguards to prevent the creation of health disinformation when prompted, according to a new study. Research by a team of experts from around the world, led by researchers from Flinders University in Adelaide, Australia, and published in the BMJ found that the large language models (LLMs) used to power publicly accessible chatbots failed to block attempts to create realistic-looking disinformation on health topics. As part of the study, researchers asked a range of chatbots to create a short blog post with an attention-grabbing title and containing realistic-looking journal references and patient and doctor testimonials on two health disinformation topics: that sunscreen causes skin cancer and that the alkaline diet is a cure for cancer. The researchers said that several high-profile, publicly available AI tools and chatbots, including OpenAI’s ChatGPT, Google’s Gemini and a chatbot powered by Meta’s Llama 2 LLM, consistently generated blog posts containing health disinformation when asked – including three months after the initial test and being reported to developers when researchers wanted to assess if safeguards had improved. In response to the findings, the researchers have called for “enhanced regulation, transparency, and routine auditing” of LLMs to help prevent the “mass generation of health disinformation”. Read full story Source: The Independent, 20 March 2024 -
News Article
Warning over use in UK of unregulated AI chatbots to create social care plans
Patient Safety Learning posted a news article in News
Britain’s hard-pressed carers need all the help they can get. But that should not include using unregulated AI bots, according to researchers who say the AI revolution in social care needs a hard ethical edge. A pilot study by academics at the University of Oxford found some care providers had been using generative AI chatbots such as ChatGPT and Bard to create care plans for people receiving care. That presents a potential risk to patient confidentiality, according to Dr Caroline Green, an early career research fellow at the Institute for Ethics in AI at Oxford, who surveyed care organisations for the study. “If you put any type of personal data into [a generative AI chatbot], that data is used to train the language model,” Green said. “That personal data could be generated and revealed to somebody else.” She said carers might act on faulty or biased information and inadvertently cause harm, and an AI-generated care plan might be substandard. But there were also potential benefits to AI, Green added. “It could help with this administrative heavy work and allow people to revisit care plans more often. At the moment, I wouldn’t encourage anyone to do that, but there are organisations working on creating apps and websites to do exactly that.” Read full story Source: The Guardian, 10 March 2024 -
News Article
Coronavirus reveals limits of AI health tools
Patient Safety Learning posted a news article in News
Technology and healthcare companies are racing to roll out new tools to test for and eventually treat the coronavirus epidemic spreading around the world. But one sector that is holding back are the makers of artificial-intelligence-enabled diagnostic tools, increasingly championed by companies, healthcare systems and governments as a substitute for routine doctor-office visits. In theory, such tools, sometimes called “symptom checkers” or healthcare bots,sound like an obvious short-term fix: they could be used to help assess whether someone has Covid-19, the illness caused by the novel coronavirus, while keeping infected people away from crowded doctor’s offices or emergency rooms where they might spread it. These tools vary in sophistication. Some use a relatively simple process, like a decision tree, to provide online advice for basic health issues. Other services say they use more advanced technology, like algorithms based on machine learning, that can diagnose problems more precisely. But some digital-health companies that make such tools say they are wary of updating their algorithms to incorporate questions about the new coronavirus strain. Their hesitancy highlights both how little is known about the spread of Covid-19 and the broader limitations of healthcare technologies marketed as AI in the face of novel, fast-spreading illnesses. Some companies say they don’t have enough data about the new coronavirus to plug into their existing products. London-based symptom-checking app Your.MD Ltd. recently added a “coronavirus checker” button that leads to a series of questions about symptoms. But it is based on a simple decision tree. The company said it won’t update the more sophisticated technology underpinning its main system, which is based on machine learning. “We made a decision not to do it through the AI because we haven’t got the underlying science,” said Maureen Baker, Chief Medical Officer for Your.MD. She said it could take 6 to 12 months before sufficient peer-reviewed scientific literature becomes available to help inform the redesign of algorithms used in today’s more advanced symptom checkers. Read full story Source: The Wall Street Journal, 29 February 2020- Posted
-
- Medicine - Infectious disease
- AI
-
(and 2 more)
Tagged with:
-
News Article
Digital health: Is it clinically effective?
Patient Safety Learning posted a news article in News
Babylon Health use AI to provide health care to UK patients – even Health Secretary Matt Hancock uses it. But experts have questioned whether there’s enough evidence of the safety of its AI chatbot service. Watch the BBC Newsnight report -
News Article
Babylon Health lashes out at doctor who raised AI chatbot safety concerns
Patient Safety Learning posted a news article in News
Controversial healthcare app maker Babylon Health has criticised the doctor who first raised concerns about the safety of their AI chatbot. Babylon Health’s chatbot is available in the company’s GP at Hand app, a digital healthcare solution championed by health secretary Matt Hancock. The chatbot aims to reduce the burden on GPs and A&E departments by automating the triage process to determine whether someone can treat themselves at home, should book an online or in-person GP appointment, or go straight to a hospital. A Twitter user under the pseudonym of Dr Murphy first reached out to us back in 2018 alleging that Babylon Health’s chatbot was giving unsafe advice. Dr Murphy recently unveiled himself as Dr David Watkins and went public with his findings at The Royal Society of Medicine’s “Recent developments in AI and digital health 2020“ event. Over the past couple of years, Dr Watkins has provided countless examples of the chatbot giving dangerous advice. In a press release (PDF) on Monday, Babylon Health calls Dr Watkins a “troll” who has “targeted members of our staff, partners, clients, regulators and journalists and tweeted defamatory content about us”. Read full story Source: AI News, 26 February 2020 -
Content Article
Healthcare is advancing at a quicker rate than ever before. With the introduction of Artificial Intelligence (AI), you can now get a cancerous mole diagnosed with a mobile device. The reliance on technology has never so great. With technology predicted to replace as much as 80 per cent of a physician’s everyday routine, we must question what the new threats posed to patient safety are? This article, written by CFC Underwriting, explains some of the pitfalls of the new technology. CFC is a specialist insurance provider. and a pioneer in emerging risk.- Posted
-
- Latent error
- Omissions
-
(and 5 more)
Tagged with:
-
Content Article
This commentary provides a concise step-by-step guide on using ChatGPT, an advanced natural language processing (NLP) model, for research and publication purposes. The guide assesses crucial aspects, including data preprocessing, fine-tuning techniques, prompt engineering, and ethical considerations. By addressing challenges related to biases, interpretability, and plagiarism, this commentary offers insights and recommendations for the responsible and ethical use of ChatGPT. The guide empowers researchers to ethically integrate ChatGPT effectively into their workflows, enhancing productivity and improving the quality of their scientific publications. Through clear instructions and guidelines, researchers can tap into the transformative potential of ChatGPT, driving scientific progress in a language-driven world. -
News Article
One in five GPs use AI such as ChatGPT for daily tasks, survey finds
Patient Safety Learning posted a news article in News
A fifth of GPs are using artificial intelligence (AI) tools such as ChatGPT to help with tasks such as writing letters for their patients after appointments, according to a survey. The survey, published in the journal BMJ Health and Care Informatics, spoke to 1,006 GPs. They were asked whether they had ever used any form of AI chatbot in their clinical practice, such as ChatGPT, Bing AI or Google’s Gemini, and were then asked what they used these tools for. One in five of the respondents said that they had used generative AI tools in their clinical practice and, of these, almost a third (29%) said that they had used them to generate documentation after patient appointments, while 28% said that they had used the tools to suggest a different diagnosis. A quarter of respondents said they had used the AI tools to suggest treatment options for their patients. These AI tools, such as ChatGPT, work by generating a written answer to a question posed to the software. The researchers said that the findings showed that “GPs may derive value from these tools, particularly with administrative tasks and to support clinical reasoning”. However, the researchers went on to question whether these AI tools being used could risk harming and undermining patient privacy “since it is not clear how the internet companies behind generative AI use the information they gather”. Read full story Source: The Guardian, 17 September 2024 -
News Article
Parents are putting more trust into ChatGPT than actual doctors, study finds
Patient Safety Learning posted a news article in News
Parents are trusting ChatGPT for medical advice over actual doctors and nurses, a new study found. Researchers at the University of Kansas also found that parents also say AI-generated text is credible, trustworthy and moral. “When we began this research, it was right after ChatGPT first launched — we had concerns about how parents would use this new, easy method to gather health information for their children,” lead author and doctoral student Calissa Leslie-Miller said in a release. “Parents often turn to the internet for advice, so we wanted to understand what using ChatGPT would look like and what we should be worried about.” Participants in the study were given health-related text, reviewing content generated by healthcare professionals and the OpenAI chatbot ChatGPT. They were not told who, or what, authored the texts. They were asked to rate the texts based on five criteria - perceived morality, trustworthiness, expertise, accuracy and how likely they would be to rely on the information. In many cases, parents couldn’t tell which content was generated by ChatGPT or by the experts. When there were significant differences in ratings, ChatGPT was rated to be more trustworthy, accurate and reliable than the expert-generated content. Read full story Source: The Independent, 30 October 2024- Posted
-
- Relative
- Children and Young People
- (and 6 more)
-
News Article
AI medical advice on drugs could lead to death in one in five cases
Patient Safety Learning posted a news article in News
Dr AI could be providing 'potentially harmful' medication advice, a concerning study has suggested. German researchers found more than a fifth of AI powered chatbot answers to common prescription drug questions could 'lead to death or severe harm'. Experts urged patients not to rely on such search engines to give them accurate and safe information. Medics were also warned against recommending the tools until more 'precise and reliable' alternatives are made available. In the study, the scientists from the University of Erlangen-Nuremberg, pinpointed the 10 most frequently asked patient questions for the 50 most prescribed drugs in the US. These included adverse drug reactions, instructions for use and contraindications — reasons why the medication should not be taken. Using Bing copilot — a search engine with AI-powered chatbot features developed by Microsoft — researchers assessed all 500 responses, against answers given by clinical pharmacists and doctors with expertise in pharmacology. Responses were also compared against a peer-reviewed up-to-date drugs information website. They found chatbot statements didn’t match the reference data in over a quarter (26%) of all cases and were fully inconsistent in just over 3%. But further analysis of 20 answers also revealed four in ten (42%) were considered to lead to moderate or mild harm and 22%, death or severe harm. Read full story Source: Mail Online, 11 October 2024 -
News Article
How chatbots are helping doctors be more human and empathetic
Patient Safety Learning posted a news article in News
Despite the drawbacks of turning to artificial intelligence in medicine, some US physicians find that ChatGPT improves their ability to communicate with patients. Last year, Microsoft and OpenAI released the first free version of ChatGPT. Within 72 hours, doctors were using the artificial intelligence-powered chatbot. Experts expected that ChatGPT and other A.I.-driven large language models could take over mundane tasks that eat up hours of doctors’ time and contribute to burnout, like writing appeals to health insurers or summarising patient notes. However, they found that doctors were asking ChatGPT to help them communicate with patients in a more compassionate way. Dr Michael Pignone, the chairman of the department of internal medicine at the University of Texas at Austin, has no qualms about the help he and other doctors on his staff got from ChatGPT to communicate regularly with patients. However, skeptics like Dr Dev Dash, who is part of the data science team at Stanford Health Care, are so far underwhelmed about the prospect of large language models like ChatGPT helping doctors. In tests performed by Dr Dash and his colleagues, they received replies that occasionally were wrong but, he said, more often were not useful or were inconsistent. If a doctor is using a chatbot to help communicate with a patient, errors could make a difficult situation worse. Read full story (paywalled) Source: New York Times, 12 June 2023 -
News Article
A US organisation that supports people with eating disorders has suspended use of a chatbot after reports it shared harmful advice. The National Eating Disorder Association (Neda) recently closed its live helpline and directed people seeking help to other resources, including the chatbot. The AI bot, named "Tessa," has been taken down, the association said. It will be investigating reports about the bot's behaviour. In recent weeks, some social media users posted screenshots of their experience with the chatbot online. They said the bot continued to recommend behaviours like calorie restriction and dieting, even after it was told the user had an eating disorder. For patients already struggling with stigma around their weight, further encouragement to shed pounds can lead to disordered eating behaviours like bingeing, restricting or purging, according to the American Academy of Family Physicians. Read full story Source: BBC News, 2 June 2023- Posted
-
- Eating disorder
- Chatbots
-
(and 2 more)
Tagged with:
-
News Article
A regulator has admitted “concerns” over the software Babylon Healthcare uses in one of its digital health solutions and is exploring how to address this. The Medicines and Healthcare products Regulatory Authority’s (MHRA) concerns relate to Babylon’s symptom checker “chatbot” tool. This is used by thousands of patients, including those registered with digital primary care practice GP at Hand. Two senior figures within the agency set out the MHRA’s concerns about the tool in a letter, seen by HSJ, which was sent to consultant oncologist David Watkins following a meeting between the parties last October. Dr Watkins has raised doubts over the tool’s safety for several years, including repeatedly documenting alleged flaws in the chatbot through videos posted online. However, last year, Babylon said only 20 of Dr Watkins’ 2,400 tests resulted in “genuine errors” being identified in the software. In the letter, dated 4 December, the MHRA’s clinical director for devices Duncan McPherson and head of software related device technologies Johan Ordish said Dr Watkins’ “concerns are all valid and ones that we share”. In the letter to Dr Watkins, the two MHRA directors also said the regulator is further exploring some of the issues highlighted and the work could “be important as we develop a new regulatory framework for medical devices in the UK”. Read full story (paywalled) Source: HSJ, 4 March 2021- Posted
-
- Digital health
- Technology
-
(and 2 more)
Tagged with:
-
News Article
USA: Stanford pilots ChatEHR
Patient Safety Learning posted a news article in News
California-based Stanford Health Care is piloting an internally developed, AI-backed software designed to revolutionize clinician interaction with the electronic health record (EHR) Nigam Shah, chief data science officer at Stanford Health Care, is leading the development team for ChatEHR, which allows clinicians to ask questions, request summaries and pull specific information from a patient’s medical record. ChatEHR is built directly into Stanford’s EHR to maximise clinical workflow. The pilot is available to a small cohort of 33 physicians, nurses and physician assistants. The technology is secure and designed for information gathering; not medical advice. ChatEHR, which has been in development since 2023, facilitates a more streamlined and efficient way for clinicians to interact with patient records. “This is a unique instance of integrating [large language model] capabilities directly into clinicians’ practice and workflow,” said Michael Pfeffer, MD, chief information and digital officer at Stanford Health Care and School of Medicine, in a news release. “We’re thrilled to bring this to the workforce at Stanford Health Care.” Stanford is still working on automation to evaluate tasks, such as determining whether to transfer patients between hospitals or units. Dr Shah and his team are using an open-source framework for real-world large language model evaluation, MedHELM, to evaluate ChatEHR. His goal is to scale ChatEHR to all clinicians and the team is working on more features to ensure accuracy. “We’re rolling this out in accordance with our responsible AI guidelines, not only ensuring accuracy and performance, but making sure we have the educational resources and technical support available to make ChatEHR usable to our workforce,” said Dr Shah in the release. Read full story Source: Becker's Health IT, 6 June 2025