Digital diagnosis—what the doctor ordered?

Summary

The advance of artificial intelligence (AI) has seen the emergence of digital diagnostic tools, with some claiming a more accurate diagnosis than a human. But what challenges does this present to patient safety? In this blog, Clive Flashman, Patient Safety Learning's Chief Digital Officer, looks at some of these new digital tools that are becoming  increasingly available not only to clinicians but also for patients, and highlights some of the risks that they bring and considerations that need to be thought through.

This blog has been published as part of a series for World Patient Safety Day 2024 and the theme of Improving diagnosis for patient safety. #WPSD24, World Patient Safety Day 2024, WPSD 2024.

Content

Why diagnosis is important

When you have been feeling unwell for a while despite the over-the-counter remedies that you’ve bought, you try and make an appointment to see your GP. Note that I said ‘try’; getting a GP appointment these days is a bit like a long distance run, with no guarantee that you will cross the finish line anywhere near the time you’d hoped for.

When you do see your GP, they will draw on their medical school training, their years of experience, their knowledge of you as their patient (or at least what the medical record they hold about you contains) and, from all of that, determine what might be wrong with you. They might need some additional evidence before confirming that diagnosis—so they might request that you have a blood test or an MRI, etc.

Once those results are back, you will be contacted again by the GP (or one of their staff) to tell you what that means for you. Was the original diagnosis correct? Has it changed? How will you be treated?

Diagnosis is the starting point for therapeutic treatment

The diagnosis is the starting point for therapeutic treatment. Get that wrong and, like a long line of dominos, everything else will fall out of place.

We are reliant on this diagnosis to make our recovery (if that is possible) or to at least return to some form of wellness. It is pivotal in the patient’s care pathway and there are around 1.5 million[1] primary care consultations a day, of which 45% are with a GP. It is also critical that the diagnostic test is carried out and the results delivered promptly to give a timely and more accurate diagnosis. Sadly, as of May 2024, 1.66 million[2] people are on the waiting list for a diagnostic test—the highest figure since the current data series started being collected in January 2006.

A new model for diagnoses

In recent years, advances in AI have seen the emergence of digital diagnostic tools. Perhaps the most well known of these is Babylon Health, once valued at $4.2 billion, which collapsed in 2023 having already pulled out of the NHS contracts it had previously won.[3] Its main competitor for general AI-driven health diagnosis is Ada Health, which features a link to a medical journal (Rheumatology International) on its website that claims that “Ada was more accurate than physicians in suggesting the correct final diagnosis (54% of cases vs physicians' 32%)”.[4]

Personally, I’m not sure that even a 54% diagnostic accuracy rate is that great, but compared to an awful 32% for human doctors, it is clearly an improvement.

One of the most successful areas where AI has been involved in diagnosis is in the interpretation of medical images. Typically, two radiographers would review the same X-ray and come to individual conclusions about what is being shown, then compare those conclusions to make a final judgement. Studies have shown that where one of those radiographers is replaced by an AI equivalent, the diagnostic accuracy rate is at least as good if not better than before.[5]

Many of these tools are for use by clinicians. They are expected to overlay their own judgement on top of the recommendations provided by the digital solution.

However, increasingly digital diagnostic tools are being aimed at both patients and clinicians (Ada Health being the most obvious one). For example:

• MiiCare - blends smart home surveillance, an AI-driven voice assistant and vital signs monitoring to create a unique at-home virtual care solution.
• Healthy.io – offers standardised digital wound management services that help clinicians make better care decisions using the smartphone camera to accurately capture wounds and analyse their progress.
• Qure.ai – uses AI algorithms for medical imaging to identify and localise abnormalities on X-rays, MRI and CT scans.
• Odin Vision – helps clinicians to detect and diagnose polyps during colonoscopy procedures.
• PocDoc – looks to leverage the ubiquity of smartphones, turning them into personal diagnostic devices able to detect a range of major diseases from a pinprick of blood.
• Zio by iRhythm – helps clinicians and patients to quickly spot and confirm heart arrythmias.
• Binah.ai – uses the smartphone camera to calculate vital signs. Its software looks at the region around the eyes, where the skin is a bit thinner, and analyses the light reflecting off blood vessels back to the lens. 
• Canary Speech – uses the same underlying technology as Amazon’s Alexa to analyse patients’ voices for mental health conditions.
• eMoodie Minds – a digital-first mental health assessment tool.
• Feebris – used with a digital stethoscope for earlier diagnosis of childhood pneumonia.

Home: the new setting for diagnostic tests

With the increasing availability of home-based tests from companies such as Thriva, PocDoc, Pinpoint (blood testing) and Healthy.io (urinalysis) —which can all link to smartphone apps to leverage AI algorithms, potentially support diagnoses and track conditions — diagnostic testing at home is becoming more prevalent. This frees up clinician time and clinic space. However, it relies on the patient to perform the test accurately and submit the reading in a complete, timely and accurate manner.

Regulatory and ethical guardrails

There are a significant number of hurdles that digital health technology suppliers in the UK have to jump before their solutions can be used by doctors or patients. The Medicines and Healthcare Products Regulatory Agency (MHRA) sets strict guidance on the evidence that is required and, in some cases, software will be classed as a medical device[6] and treated with the same regulatory rigour.

NICE expects digital health technologies to have involved patients and healthcare professionals in their design and testing, and also to have gathered increasing amounts of evidence about efficacy.[7] This can be anything from a simple impact evaluation to a full randomised control trial, depending on the level of clinical judgement and recommendations the digital solution will be providing.

NHS England demands that all NHS buyers of digital health technologies will complete a clinical safety case[9] for each solution they buy and also to complete the DTAC (Digital Technology Assessment Criteria), which considers:

• clinical safety (again)
• data protection
• clinical assurance
• interoperability
• usability and accessibility.

Challenges

However, patients are not trained how to interpret health data[8] and are also potentially at risk if seeing a diagnosis for the first time without adequate support in place. People who are newly diagnosed with a condition generally want to talk it through with a clinician, discuss treatment options, understand the impact it may have on their lives. Being left alone with your diagnosis, and having to then proactively make appointments to discuss it, is not what patient groups might consider the best approach to patient-centred care.

Passive data collection, such as through a clinically certified wearable device or skin patch, can be relied upon to provide reasonably accurate data. However, where the patient is expected to conduct their own diagnostic test using their phone, or gathering blood or urine, the reliability of the data collected might be lower.

There are also people who are not able to use these types of technologies or tests at home. They may be one of a group of people who are digitally excluded or have impairments (fine motor skills, cognition, visual, etc.) that mean that they cannot use the necessary items.

In the same way that many people hate the push by supermarkets to move to self-checkouts where the burden is placed on the shopper to scan their own goods, some people also recoil at the thought of having to do their own medical tests and interpret the results.

Conclusion

It is inevitable that we will move to see more digital health diagnostics used by healthcare professionals and patients. However, we should not forget that this will not be appropriate for some people and offer other options for them to gain a formal diagnosis. People using digital diagnostic tools should be able to call on support where they need it and the guardrails that we have in place should be continuously reviewed so that they deal with new and innovative technologies before they cause significant harm to users. It is my view that these tools should be smart enough to recognise when a new (significant) diagnosis has been given to the patient and, in those cases, immediately contact a clinician to advise that support should be provided. If that cannot be done, then new diagnoses of this nature should not be communicated directly to a patient.

References

1. BMA. Pressures in general practice data analysis, 26 July 2024. 
2. Kirk-Wade E, Harker R, Stiebahl S. NHS key statistics: England. House of Commons Library, 16 July 2024. 
3. The Fall of Babylon Is a Warning for AI Unicorns. Wired, 19 September 2023. 
4. Graf M, Knitza J, Leipe J, et al. Comparison of physician and artificial intelligence-based symptom checker diagnostic accuracy. Rheum Int, 2022:42(12):2167-2176. doi: 10.1007/s00296-022-05202-4. Epub 2022 Sep 10. 
5. King's College London. AI trained on X-rays can diagnose medical issues as accurately as doctors, 11 December 2023. 
6. MHRA. Guidance. Software and AI as a Medical Device Change Programme - Roadmap. Medicines and Healthcare products Regulatory Agency, 14 June 2023. 
7. NICE. Evidence standards framework (ESF) for digital health technologies.
8. NIHR Evidence: Health information: are you getting your message across? June 2022; doi: 10.3310/nihrevidence_51109
9. NHS England. Digital clinical safety assurance, v 1.1, 28 July 2023. 

Share your insights

What do you think about the digital developments in health that Clive talked about in this blog? Do you have an experience to share as a patient, or as someone who works in this area? If you'd like to share your insights around digital health and patient safety, get in touch with the team at [email protected]. 

Have you been affected by a late diagnosis? Or perhaps you have insights to share on diagnostic safety through the work that you do. If you would like to write a blog or share your thoughts, experiences or resources through the hub please get in touch with our team at [email protected] or add your comments to our community forum page.