One size does not fit all. How AI and better data can help us embrace complexity in diagnosis and treatment

Summary

Orthopaedic surgeon Sunny Deo has spent three decades diagnosing and treating knee joint issues. In this blog, Sunny argues that the healthcare community needs to take a more nuanced approach to diagnosis and decision making so that it can provide patients with safer, more appropriate treatment options.

He reflects on why medicine prefers simple answers and looks at how this affects patient care. He goes on to explore how better data collection and the use of artificial intelligence (AI) could provide a more accurate picture of complexity and allow treatment options to be better tailored to individual patients’ needs.

Content

"To know the patient that has the disease is more important than to know the disease that the patient has." William Osler, father of modern medicine, 1849-1919.

Diagnosis is the process of identifying the nature of an illness or other problem by examining the symptoms and objective findings from investigations. In modern medicine, it is a key focal point of the assessment and management of all patients. A huge amount of clinical medicine training is focused on the art and science of obtaining a diagnosis, and this focus continues into medical practice.

The ease of getting to a diagnosis ranges from the glaringly obvious, the so-called ‘spot diagnosis’, through to cases that are very difficult to solve. In between these extremes there is a range from delayed to missed to incorrect diagnosis. The aim of doctors over the centuries has been to work out diagnoses from patients’ symptoms, presenting features (clinical signs) and, in the past century or so, from the evidence of clinical investigations. Quite often, symptoms, signs and investigations produce consistent patterns, and it is these patterns that are taught to medical and other healthcare professionals.

This is how diagnoses and outcomes are portrayed in television series or films—just think back to the last episode of Casualty or Grey’s Anatomy you watched. It's also how things often appear in internet searches and on websites and social media.

Seeking simple answers to complex questions

However, the reality is different. When a patient is sitting in front of me, what I hear and observe may not exactly be what the textbooks, evidence or research tells me I should be seeing. But because we are wired and trained to recognise patterns, we tend to look for diagnoses and solutions that fit within the well-worn narrative. What if the pattern doesn’t fit the actual diagnosis? There are classic presentations for nearly every condition, and these are what you tend to find at the start of a Google search or when using NHS Choices. The expectation of typical symptoms sometimes means we ignore what we might see as annoying variance, superfluous detail or the patient embellishing the truth.

This discordance then causes tension with a very basic trait of humans: when we’re faced with a difficult problem, we still seek the simplest solution. This is an evolutionary feature hardwired into us to optimise survival chances. It means we often believe there is a truth to be found that will provide us with a definite answer. From this answer we will come to the best, and ideally only, ‘correct’ solution. Patients who don’t fit the set patterns of diagnosis may then run into trouble when we offer them what is considered to be the ideal treatment.

This is an important problem in clinical thinking, language and practice. As a medical community, we tend to create oversimplified approaches based on research that looks for binary answers to complex questions. This research evidence may be based on a small, highly selective ‘typical’ patient cohort, but its findings and conclusions are then translated on to the entire population. This approach results in poor patient outcomes and experience for a small but significant proportion of patients.

Pathways designed for ideal diagnoses can cause harm to patients

Over my 30 years as an orthopaedic surgeon, 15 as a knee specialist, I have seen that the assessment and treatment of any given condition isn't quite as predictable as we would like it to be. While many patients fit the pattern we are expecting, some do not. I would empirically put the proportion at 60:40, but some unpublished research we did a decade ago suggested the proportion of truly ‘typical’ case presentations for a common condition is much lower.  For example, we found that in the case of suspected meniscal tear, this diagnosis actually applied to only 33% of patients with a variety of other diagnoses accounting for the rest.  

It gets worse when large organisations start to lump patients into a category by condition in a ‘one diagnosis fits all’ strategy. When this approach is taken, there are winners and losers. The winners are those patients whose condition very closely matches the classic presentation of a given condition in isolation.

Let’s take the example of knee osteoarthritis—patients with the ‘right type’ of symptoms, physical signs and x-ray changes are generally more likely to do well. Their recovery is more likely to sit within the knowledge base of treating the condition that has evolved over the past half-century. In contrast, patients whose symptoms and test results fall outside of this category may be less likely to do well or recover in the predicted timeframe. This also applies to patients with additional diagnoses or conditions, often termed comorbidities, which may interact, usually in a bad way, with the condition at hand.

Failure to consider other diagnoses, either by over-focus on one condition causing wilful ignorance, inattention or lack of attention, may lead to unexpected poor outcomes from a given treatment. It may also mean that the symptoms from the condition that the patient presents with are worse than expected. This doesn’t mean that they won't gain any benefit from a particular treatment, but the risks and potential outcomes may not be communicated adequately by the patient’s healthcare team, if at all.

For example, for patients with painful knee osteoarthritis, the current diagnosis to treatment logic runs like this:

1. Knee osteoarthritis is a painful condition.
2. Total knee replacement surgery is a validated safe procedure with significant improvements in quality of life. Other treatment options do not produce as much positive therapeutic benefit compared to total knee replacement surgery.
3. Therefore, total knee replacement surgery is the only treatment for painful knee osteoarthritis.

However, there are patients for whom knee replacement surgery is not a safe or practical option, and these patients may benefit from alternative treatments that are not currently offered as they are seen as providing limited benefit. This may be because the participants in trials undertaken over the years had varying diagnoses, meaning that true comparisons of alternative options may have had additional interacting diagnoses or failed to account for differing severity.

Understanding the spectrum of complexity

As healthcare professionals, we have a duty to diagnose patients as accurately as possible. In orthopaedics, if treatments go wrong or are poorly undertaken, it may lead to prolonged or permanent pain or disability, and we obviously want to avoid this as much as possible. Incomplete identification and documentation of all relevant symptoms and health conditions can potentially lead to an increased risk of treatment failure and complications. Our priority should be to identify these diagnoses or diagnostic clusters as accurately as possible. I think these are basic principles we need to apply to create better systems and improved care for as many patients as possible.

In my view, there are grades of ‘atypical patients’ and I have devoted the past decade to trying to demonstrate this, with surprisingly stiff resistance from peer-reviewed journals and funding organisations. I have tried to move away from lumping all patients into a single category. I have done some research on seemingly straightforward soft tissue problems and osteoarthritis in the knee. My initial analysis suggests that we need to collect more detailed and accurate data, rather than simplifying data into minimum datasets. This is where AI can really come into its own, not as a diagnostic tool initially, but as a powerful aid to unlocking and interpreting some of the diagnostic interactions that create problems for patients. However, the use of AI does need to be undertaken with extreme care and consideration, and this isn’t always happening currently.

To offer healthcare that is truly person-centred, we need to look beyond our well-worn simple answers and solutions. By using better data and new machine learning tools to understand the nuances of each person’s condition and how it relates to their wider health, we can offer treatment options that are safer, kinder and more cost-effective. 

Share your views

We would love to hear your views on the issues highlighted in Sunny’s blog

Are you a clinician who would like to share your experiences? Do these challenges resonate with you?

Or are you a patient who has experienced complications because of poor, missed or inadequate diagnosis?

Add your comment below (you will need to be a hub member and signed in) or contact us at [email protected] and we can share your story anonymously.

Related content on the hub:

• Using data to improve decision making and person-centred care in surgery: An interview with Sunny Deo and Matthew Bacon
• Diagnostic errors and delays: why quality investigations are key

About the Author

Sunny Deo is a Consultant Orthopaedic and Trauma Surgeon at Great Western Hospital NHS Foundation Trust, where he has been the Clinical Lead for the Trauma and Orthopaedic Department since 2011. Prior to this, Sunny trained and worked at hospitals across the UK and in Cape Town and Vancouver. He specialises in knee problems including sports injuries, arthritic conditions and patients with problematic joint replacements. Sunny has extensively researched a range of subjects, including clinical and diagnostic complexity, outcomes following knee replacements, peri-articular fractures, knee ligament injuries, hip fractures and paediatric fracture care, publishing over 40 peer-reviewed journal articles and presenting at many national and international meetings.