Clive Flashman

The advance of artificial intelligence (AI) has seen the emergence of digital diagnostic tools, with some claiming a more accurate diagnosis than a human. But what challenges does this present to patient safety? In this blog, Clive Flashman, Patient Safety Learning's Chief Digital Officer, looks at some of these new digital tools that are becoming  increasingly available not only to clinicians but also for patients, and highlights some of the risks that they bring and considerations that need to be thought through. This blog has been published as part of a series for World Patient Safety Day 2024 and the theme of Improving diagnosis for patient safety. #WPSD24, World Patient Safety Day 2024, WPSD 2024. Why diagnosis is important When you have been feeling unwell for a while despite the over-the-counter remedies that you’ve bought, you try and make an appointment to see your GP. Note that I said ‘try’; getting a GP appointment these days is a bit like a long distance run, with no guarantee that you will cross the finish line anywhere near the time you’d hoped for. When you do see your GP, they will draw on their medical school training, their years of experience, their knowledge of you as their patient (or at least what the medical record they hold about you contains) and, from all of that, determine what might be wrong with you. They might need some additional evidence before confirming that diagnosis—so they might request that you have a blood test or an MRI, etc. Once those results are back, you will be contacted again by the GP (or one of their staff) to tell you what that means for you. Was the original diagnosis correct? Has it changed? How will you be treated? Diagnosis is the starting point for therapeutic treatment The diagnosis is the starting point for therapeutic treatment. Get that wrong and, like a long line of dominos, everything else will fall out of place. We are reliant on this diagnosis to make our recovery (if that is possible) or to at least return to some form of wellness. It is pivotal in the patient’s care pathway and there are around 1.5 million[1] primary care consultations a day, of which 45% are with a GP. It is also critical that the diagnostic test is carried out and the results delivered promptly to give a timely and more accurate diagnosis. Sadly, as of May 2024, 1.66 million[2] people are on the waiting list for a diagnostic test—the highest figure since the current data series started being collected in January 2006. A new model for diagnoses In recent years, advances in AI have seen the emergence of digital diagnostic tools. Perhaps the most well known of these is Babylon Health, once valued at $4.2 billion, which collapsed in 2023 having already pulled out of the NHS contracts it had previously won.[3] Its main competitor for general AI-driven health diagnosis is Ada Health, which features a link to a medical journal (Rheumatology International) on its website that claims that “Ada was more accurate than physicians in suggesting the correct final diagnosis (54% of cases vs physicians' 32%)”.[4] Personally, I’m not sure that even a 54% diagnostic accuracy rate is that great, but compared to an awful 32% for human doctors, it is clearly an improvement. One of the most successful areas where AI has been involved in diagnosis is in the interpretation of medical images. Typically, two radiographers would review the same X-ray and come to individual conclusions about what is being shown, then compare those conclusions to make a final judgement. Studies have shown that where one of those radiographers is replaced by an AI equivalent, the diagnostic accuracy rate is at least as good if not better than before.[5] Many of these tools are for use by clinicians. They are expected to overlay their own judgement on top of the recommendations provided by the digital solution. However, increasingly digital diagnostic tools are being aimed at both patients and clinicians (Ada Health being the most obvious one). For example: MiiCare - blends smart home surveillance, an AI-driven voice assistant and vital signs monitoring to create a unique at-home virtual care solution. Healthy.io – offers standardised digital wound management services that help clinicians make better care decisions using the smartphone camera to accurately capture wounds and analyse their progress. Qure.ai – uses AI algorithms for medical imaging to identify and localise abnormalities on X-rays, MRI and CT scans. Odin Vision – helps clinicians to detect and diagnose polyps during colonoscopy procedures. PocDoc – looks to leverage the ubiquity of smartphones, turning them into personal diagnostic devices able to detect a range of major diseases from a pinprick of blood. Zio by iRhythm – helps clinicians and patients to quickly spot and confirm heart arrythmias. Binah.ai – uses the smartphone camera to calculate vital signs. Its software looks at the region around the eyes, where the skin is a bit thinner, and analyses the light reflecting off blood vessels back to the lens. Canary Speech – uses the same underlying technology as Amazon’s Alexa to analyse patients’ voices for mental health conditions. eMoodie Minds – a digital-first mental health assessment tool. Feebris – used with a digital stethoscope for earlier diagnosis of childhood pneumonia. Home: the new setting for diagnostic tests With the increasing availability of home-based tests from companies such as Thriva, PocDoc, Pinpoint (blood testing) and Healthy.io (urinalysis) —which can all link to smartphone apps to leverage AI algorithms, potentially support diagnoses and track conditions — diagnostic testing at home is becoming more prevalent. This frees up clinician time and clinic space. However, it relies on the patient to perform the test accurately and submit the reading in a complete, timely and accurate manner. Regulatory and ethical guardrails There are a significant number of hurdles that digital health technology suppliers in the UK have to jump before their solutions can be used by doctors or patients. The Medicines and Healthcare Products Regulatory Agency (MHRA) sets strict guidance on the evidence that is required and, in some cases, software will be classed as a medical device[6] and treated with the same regulatory rigour. NICE expects digital health technologies to have involved patients and healthcare professionals in their design and testing, and also to have gathered increasing amounts of evidence about efficacy.[7] This can be anything from a simple impact evaluation to a full randomised control trial, depending on the level of clinical judgement and recommendations the digital solution will be providing. NHS England demands that all NHS buyers of digital health technologies will complete a clinical safety case[9] for each solution they buy and also to complete the DTAC (Digital Technology Assessment Criteria), which considers: clinical safety (again) data protection clinical assurance interoperability usability and accessibility. Challenges However, patients are not trained how to interpret health data[8] and are also potentially at risk if seeing a diagnosis for the first time without adequate support in place. People who are newly diagnosed with a condition generally want to talk it through with a clinician, discuss treatment options, understand the impact it may have on their lives. Being left alone with your diagnosis, and having to then proactively make appointments to discuss it, is not what patient groups might consider the best approach to patient-centred care. Passive data collection, such as through a clinically certified wearable device or skin patch, can be relied upon to provide reasonably accurate data. However, where the patient is expected to conduct their own diagnostic test using their phone, or gathering blood or urine, the reliability of the data collected might be lower. There are also people who are not able to use these types of technologies or tests at home. They may be one of a group of people who are digitally excluded or have impairments (fine motor skills, cognition, visual, etc.) that mean that they cannot use the necessary items. In the same way that many people hate the push by supermarkets to move to self-checkouts where the burden is placed on the shopper to scan their own goods, some people also recoil at the thought of having to do their own medical tests and interpret the results. Conclusion It is inevitable that we will move to see more digital health diagnostics used by healthcare professionals and patients. However, we should not forget that this will not be appropriate for some people and offer other options for them to gain a formal diagnosis. People using digital diagnostic tools should be able to call on support where they need it and the guardrails that we have in place should be continuously reviewed so that they deal with new and innovative technologies before they cause significant harm to users. It is my view that these tools should be smart enough to recognise when a new (significant) diagnosis has been given to the patient and, in those cases, immediately contact a clinician to advise that support should be provided. If that cannot be done, then new diagnoses of this nature should not be communicated directly to a patient. References BMA. Pressures in general practice data analysis, 26 July 2024. Kirk-Wade E, Harker R, Stiebahl S. NHS key statistics: England. House of Commons Library, 16 July 2024. The Fall of Babylon Is a Warning for AI Unicorns. Wired, 19 September 2023. Graf M, Knitza J, Leipe J, et al. Comparison of physician and artificial intelligence-based symptom checker diagnostic accuracy. Rheum Int, 2022:42(12):2167-2176. doi: 10.1007/s00296-022-05202-4. Epub 2022 Sep 10. King's College London. AI trained on X-rays can diagnose medical issues as accurately as doctors, 11 December 2023. MHRA. Guidance. Software and AI as a Medical Device Change Programme - Roadmap. Medicines and Healthcare products Regulatory Agency, 14 June 2023. NICE. Evidence standards framework (ESF) for digital health technologies. NIHR Evidence: Health information: are you getting your message across? June 2022; doi: 10.3310/nihrevidence_51109 NHS England. Digital clinical safety assurance, v 1.1, 28 July 2023. Share your insights What do you think about the digital developments in health that Clive talked about in this blog? Do you have an experience to share as a patient, or as someone who works in this area? If you'd like to share your insights around digital health and patient safety, get in touch with the team at [email protected]. Have you been affected by a late diagnosis? Or perhaps you have insights to share on diagnostic safety through the work that you do. If you would like to write a blog or share your thoughts, experiences or resources through the hub please get in touch with our team at [email protected] or add your comments to our community forum page.

NHS England recently issued a national patient safety alert to all trusts providing maternity services after faults were discovered in IT software that could pose “potential serious risks to patient safety”. In this short blog, Clive Flashman, Patient Safety Learning’s Chief Digital Officer, calls for a closer look at the reasons into this and what we can learn from it. Electronic patient records (EPRs) are a way of managing clinical information with the intention of making this information more easily accessible for use by healthcare professionals. In A plan for digital health and social care published in June 2022, the Department of Health and Social Care set a target that all NHS Trusts should have an EPR system by March 2025.[1] In November last year, NHS England announced it was on course to meet this target, stating that 90% of NHS trusts have now introduced these new systems.[2] Although EPRs have the potential to significantly improve patient care and information handling, there are also a number of patient safety risks associated with their implementation and use. In a recent blog reviewing the recurring themes identified through their safety investigations, the Health Services Safety Investigations Body (HSSIB) identified a number of these:[3] Interoperability – problems stemming from EPRs being unable to exchange or make use of information from other IT systems used by trusts. Usability testing of software – EPR systems being introduced without appropriate testing of how easy or difficult healthcare professionals tasked with using these find them to operate. Standards and standardisation – EPR systems in use not incorporating human factors engineering principles in their design to ensure they are the best they can be. The themes listed above have been highlighted in recent media coverage, including recently a number of patient safety incidents and a patient death related to the introduction of a new EPR at the Royal Surrey Foundation Trust and St Peter’s Hospital Foundation Trust.[4] Patient Safety Alert Euroking is an EPR provided by Magentus Software. According to an NHS England National Patient Safety Alert issued on the 7 December 2023, the Euroking EPR has been found to process information incorrectly — overwriting the existing record with new information and incorrect storage and display of safeguarding information, which could lead to “incorrect management of the pregnancy and subsequent harm”.[5] The Euroking EPR is used in the maternity departments of at least 15 NHS trusts according to information held by HSJ.[6] NHS England has said there is “currently no evidence of actual harm being reported as a result of these issues" and the Trusts involved have 6 months to deal with it. The alert tells Trusts to review their data within the system and consider if Euroking meets their maternity services requirements and is safe, looking at replacement systems if deemed necessary. However, what is unclear from this Alert is whether NHS England had already spoken to the vendor and asked for a fix to be created and pushed out as soon as possible. Commenting in response to the Alert, Euroking have said that they were working with NHS England on issues concerning the data fields in the system and have provided their customers with ”support and information”. More robust response needed At Patient Safety Learning we don’t believe this is an adequate response to the issues raised in this Patient Safety Alert. A more robust response could have involved NHS England having been clear about the changes that were needed from a patient safety perspective and, subsequently, having worked with Euroking to make these changes and then push out this fix to all existing users as soon as possible. If that had been done, this Patient Safety Alert could have looked quite different, focused on the need to update the software first by a certain date, then check each data field and add back in any overwritten data if still available. This would also mean that the need to potentially consider moving to a new system, a potentially expensive and disruptive change, could then have been a lower priority. Although this issue and subsequent Alert relates to a specific vendor and system, we also believe it would be useful for all trusts to check that the same issue is not affecting similar systems (all EPRs, not just their Maternity one) from other vendors. It would have been helpful for NHS England to have annexed their alert with a testing process to support trusts in this. References 1. Department of Health and Social Care. A plan for digital health and social care; 29 June 2022. 2. NHS Digital. 90% of NHS trusts now have electronic patient records; 16 November 2023. 3. Helen Jones. Electronic patient record systems: recurring themes arising from safety investigations. HSSIB; 19 December 2023. 4. Alison Moore. Patient died and 30 harmed after new IT system launch. HSJ; 11 October 2023. 5. NHS England. National Patient Safety Alert: Identified safety risks with the Euroking maternity information system; 7 December 2023. 6. Joe Talora. NHSE warns widely used EPR could pose ‘serious risks to patient safety’. HSJ; 8 December 2023.