Search the hub

Have your eyes ever felt sore and itchy after spending too much time staring at a screen? You might have a condition known as bixonimania—or at least that’s what several popular AI-powered chatbots might have told you if you’d asked last year. Millions of people around the world turn to AI chatbots for medical advice every day, often as a supplement to a doctor’s visit but also sometimes in place of it. That can lead to dangerous consequences and in rare cases, even death. Almira Osmanovic Thunström is a researcher at the University of Gothenburg in Sweden and at the Sahlgrenska University Hospital, Center for Digital Health and Chalmers Industriteknik. She’s also the creator of bixonimania. In this interview with Scientific American, Almira says this totally made-up disease reveals some very real problems with the way we train and use large language models.

Artificial intelligence (AI)-driven chatbots have been rapidly adopted across research, education, business, marketing and medicine. Most interactions, however, come from non-experts using chatbots like search engines, including for everyday health and medical queries. This study, published in BMJ Open, audited chatbot responses in health and medical fields prone to misinformation. Five popular chatbots were assessed: Gemini (Google), DeepSeek (High-Flyer), Meta AI (Meta), ChatGPT (OpenAI) and Grok (xAI). In February 2025, each chatbot was prompted with 10 questions from five categories: cancer, vaccines, stem cells, nutrition and athletic performance. The authors deployed an adversarial-like framework, using open- and closed-ended prompts designed to strain models toward misinformation or contraindicated advice. Two experts from each category rated responses as ‘non-problematic’, ‘somewhat problematic’ or ‘highly problematic’ using a coding matrix based on objective, predefined criteria. Citations were scored for accuracy and completeness, and each response was given a Flesch Reading Ease score. The study found that nearly half (49.6%) of responses were problematic: 30% somewhat problematic and 19.6% highly problematic. Response quality did not differ significantly among chatbots, but Grok generated significantly more highly problematic responses than would be expected under a random distribution. Performance was strongest in vaccines and cancer, and weakest in stem cells, athletic performance and nutrition. Chatbot outputs were consistently expressed with confidence and certainty; from 250 total questions, there were only two refusals to answer, both from Meta AI. Reference quality was poor, with a median completeness score of 40%. Chatbot hallucinations and fabricated citations precluded any chatbot from producing a fully accurate reference list. All readability scores were graded as ‘Difficult’, equivalent to college sophomore–senior level. The audited chatbots performed poorly when answering questions in misinformation-prone health and medical fields. Continued deployment without public education and oversight risks amplifying misinformation.

Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. The authors of this study, published in Nature, tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM or a source of their choice (control). T Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. The authors identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, they recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare.

On the 23 January 2023 the Minister for Mental Health and Women’s Health Strategy, Maria Caulfield MP, announced the commencement of a rapid review into patient safety in mental health inpatient settings in England. The review Chair, Dr Geraldine Strathdee, was asked to consider how improvements could be made to the way that data and information is used in relation to patient safety in mental health inpatient care settings and pathways, including for people with a learning disability and autistic people. This report contains the findings of this review and an associated set of recommendations. Below is a brief summary of the main recommendations set out in this report. The findings and recommendations in full can be read here. Recommendation 1 NHS England should establish a programme of work, co-produced with experts by experience and key national, regional and local leaders, including Care Quality Commission (CQC), Integrated Care Systems (ICSs), provider collaboratives, independent safeguarding bodies, professional bodies, provider representatives and third sector organisations, among others, to agree how to make sure that providers, commissioners and national bodies are ‘measuring what matters’ for mental health inpatient services, and can access the information they need to provide safe, therapeutic care. Recommendation 2 Digital platforms that allow the collection of core patient information and associated data infrastructure must allow submissions into relevant national data sets, directly or through other interoperable platforms, and facilitate data flows between systems of different local provider organisations to support joined-up understanding of care pathways. These systems should allow the data collected to be made available to different decision makers, including CQC, at the appropriate level of aggregation and without requiring duplicative submissions, and allow benchmarking across trusts and independent sector providers. NHS England’s Transformation Directorate should scope out options for how this ambition could be delivered, including cost implications and a value for money assessment to help providers meet this aim specifically for mental health, including specific ways in which mental health electronic patient record improvement and data sharing can be prioritised and interdependencies with other systems and programmes of work. Recommendation 3 ICSs and provider collaboratives should bring together trusts and independent sector providers, along with other relevant stakeholders such as independent safeguarding bodies, across all healthcare sectors to facilitate the cross-sector sharing of good practice in data collection, reporting and use. Recommendation 4 The Department of Health and Social Care, in partnership with NHS England and CQC and supported by key experts from across governmental and non-governmental organisations, should convene all the relevant organisations who collect and analyse mortality data to determine what further action is needed to improve the timeliness, quality and availability of that data. Recommendation 5 Provider boards should take the following actions to improve their capacity to identify, prevent and respond to risks to patient safety: Every provider board should urgently review its membership and skillset and ensure that the board has an expert by experience and carer representative. Every provider board should ensure that its membership has the skills to understand and interpret data about mental health inpatient pathways and ensure that a responsive quality improvement methodology is embedded across their organisations. CQC should assess and report on whether the membership of the boards of providers of mental health inpatient services includes experts by experience (including carer) representatives and whether boards are maintaining an appropriately high level of data literacy and quality improvement expertise on mental health inpatient pathways among their membership as part of their assessments. Every provider board should urgently review its approach to board reports and board assessment frameworks to ensure that they highlight the key risks in all of their mental health inpatient wards, as set out in the safety issues framework, and that they support the board to take action to mitigate risks and improve care, including both quantitative data and qualitative ‘soft intelligence’ such as feedback from patients, staff and carers. NHS England should review and update the guidance on board assessment frameworks. Recommendation 6 Trust and provider leaders, including board members, should prioritise spending time on wards regularly, including regular unannounced and ‘out-of-hours’ visits, to be available to and gather informal intelligence from staff and patients. Recommendation 7 All providers of NHS-funded care should review the information they provide about their inpatient services to patients and carers annually and make sure that comprehensive information about staffing, ward environment, therapeutic activity and other relevant information about life on the wards is available. CQC should assess the quality, availability and accessibility of this information as part of their assessment of services. Recommendation 8 ICSs and provider collaboratives should map out the pathway for all their mental health service lines to establish which parties need access to relevant data at all points on the pathway and take steps to ensure that data is available to those who need it. Recommendation 9 ICSs will develop system-wide infrastructure strategies by December 2023 and the mental health estate needs to be fully incorporated and represented in these strategies and in subsequent local action plans. This recommendation is for local ICSs to review the mental health estate to inform these and future strategies, recognising there are evidence-based therapeutic design features that can contribute to reducing risk and improving safety. Recommendation 10 Providers should review their processes for allowing ward visitors access to mental health inpatient wards with a view to increasing the amount of time families, carers, friends and advocates can spend on wards. The Department of Health and Social Care should consider what more can be done to strengthen the expectation for all health and care providers in England to allow visiting. Recommendation 11 All providers of NHS-funded care should meet the relevant core carer standards set by the National Institute for Health and Care Excellence (NICE) and Triangle of Care, England. Regulators, including CQC and professional regulators, should consider how to monitor the implementation of these carer standards, especially where there is greater risk of unsafe closed cultures developing. ICSs should consider how to routinely seek carer feedback. Inpatient staff training programmes should identify how they can benefit from carer trainers. For patients detained under the Mental Health Act, families and carers should be part of all detention reviews. Recommendation 12 Professional bodies, such as the Royal Colleges, should come together across healthcare sectors to form an alliance for compassionate professional care. This multi-professional alliance should: work together and learn from each other to identify ways to drive improvement in the quality of compassionate care and safety across all sectors, including mental health services, and how they can support staff to provide it along with their specialist data units, where they exist, contribute to the work set out in recommendation. Recommendation 13 Except where specified, these recommendations should be implemented by all parties within 12 months of the publication of this report. Government ministers, through the Department of Health and Social Care, should review progress against these recommendations after 12 months.

Healthcare systems rely on self-advocacy from service users to maintain the safety and quality of care. Systemic bias, service pressures and workforce issues often deny agency to patients at times when they need to have most control over representation of their story. This drives diagnostic error, treatment delay or failure to treat important conditions. In maternal care, perinatal mental health and thrombosis are significant challenges. With funding from SBRI Health care, Ulster University and Southern Health and Social Care Trust are developing an NLP powered platform that will empower mothers to be more active agents in their perinatal care. Download the poster below.

Hindsight bias (colloquially known as ‘the retrospectoscope’) is the tendency to perceive past events as more predictable than they actually were. It has been shown to play a significant role in the evaluation of an past event, and has been demonstrated in both medical and judicial settings. This study in Clinical Medicine aimed to determine whether hindsight bias impacts on retrospective case note review, through an internet survey completed by doctors of different grades. The authors found that in some cases, doctors are markedly more critical of identical healthcare when a patient dies compared to when a patient survives. Hindsight bias while reviewing care when a patient survives might prevent identification of learning arising from errors. They also suggest that hindsight bias combined with a legal duty of candour will cause families to be informed that patients died because of healthcare error when this is not a fact.

EasyFOI is an email address compiler designed to help you send identical freedom of information requests to multiple organisations. Journalists, researchers and ordinary members of the public use the FOI act every day to request all kinds of information from statutory public bodies. You may want to request the same information from different organisations. But it can be hard to find a central list of every public body in the country, let alone their FOI inboxes (which don't tend to follow a standard format). EasyFOI is here to make that easier. Instead of searching for each organisation's contact details, or compiling your own database, you can use this simple tool to copy the appropriate email address for every relevant organisation straight into your device's clipboard. You can also use the EasyFOI generator to help you write your request in seconds. The EasyFOI database doesn't yet cover all public bodies. But it's expanding all the time, and currently includes more than 1,000 organisations.

This document from the Patient Safety Authority outlines final recommendations to acute care facilities in the USA regarding patient weights. The Patient Safety Authority is responsible for submitting recommendations to the Department of Health (Department) in the US for changes in health care practices and procedures which may be instituted for the purpose of reducing the number and severity of serious events and incidents. Having accurate patient information (for example, age, allergies, laboratory results) helps practitioners select medications, doses and routes of administration. One vital piece of information, the patient's weight, is especially important, because it is used to calculate the appropriate dose of a medication (for example, mg/kg, mcg/kg, mg/m2). A prescribed or dispensed medication dose can differ significantly from the appropriate dose because of missing or inaccurate patient weights. Patients in oncology treatment, patients with renal insufficiency, or who are elderly, paediatric or neonatal are at greater risk for adverse drug events, because they are more vulnerable to the effects of an error, and their weight may change frequently over short periods of time. Formulas such as the Cockcroft-Gault equation, which is used to calculate creatinine clearance to aid in the dosing of medications, and the Harris-Benedict formula, which is used to calculate basal metabolic rate, rely on knowledge of an accurate patient weight. Also, both height and weight are needed to use nomograms to determine body surface area and body mass index, for example, when calculating doses for chemotherapy. In the United States, most patients are weighed in pounds. But weighing and documenting patients' weights in pounds introduces the need to convert the weight into kilograms—an error-prone process—to conduct weight-based and other dosing. Another risk when measuring the patient's weight in pounds is failing to convert the weight into kilograms but recording that weight in kilograms (that is, documenting a weight of 200 lbs. as 200 kg instead of 90.9 kg), resulting in more than two-fold dosing errors. This document recommends a number of procedures to ensure accurate patient weights. Further reading Weight-based medication errors: How to tip the scale in the right direction

The United States is one of only three countries in the world that does not use the metric system. Yet, every single medication prescribed today is based on it. In addition to dosages based on the metric system, some doses are also very dependent on patient weight. These include blood thinners, certain antibiotics, chemotherapy agents, and many pediatric doses. The very young, very old, and people with certain medical conditions are at the highest risk of experiencing harm because their bodies are more sensitive to the effects of an error. Calculations made with incorrect weights can have devastating, if not fatal, consequences. Further reading Patient Safety Authority Department of Health: Final recommendation to ensure accurate patient weights

A team of scientific experts has joined forces from across the world to help fight the spread of misinformation about the COVID-19 vaccines. Together they have created a unique online guide, led by the University of Bristol, to arm people with practical tips combined with the very latest information and evidence to talk reliably about the vaccines, constructively challenge associated myths, and allay fears. With the race on to vaccinate as many people as possible soonest in the wake of a more virulent virus strain, they’re appealing to everyone, from doctors to politicians, teachers to journalists and parents to older generations, to understand the facts, follow the guidance, and spread the word.