Emerging safety issues in artificial intelligence

Artificial Intelligence is creating a lot of buzz in the US and around the world. 

This perspective from the US site AHRQ Patient Safety Net explores a range of issues that could affect the uptake artificial intelligence systems in health care. What do hub members think? Are we destined to encounter Hal (from 2001: a Space Odyssey) or Samantha (from Her)?  

Emerging safety issues in artificial intelligence

@lzipperer You might be interested in the new NHSX report out last week which gives an overview of the current state of play of data-driven technologies within the UK health and care system.

https://www.pslhub.org/learn/organisations-linked-to-patient-safety-uk-and-beyond/government-and-alb-direction-and-guidance/nhs-x/nhs-x-artificial-intelligence-how-to-get-it-right-october-2019-r849/

 

Thanks @lzipperer I was just drafting an email to get it on the hub. Super speedy - you beat me to it! Thank you

@Clive Flashman and I are in active discusion with @Gary Saunders at NHSX and their CEO Matthew Gould. There is clearly a huge amount of work to do to embed patient safety into this field. NHSX's Mission 4 (of 5) is on Patient Safety. A commendable ambition but, in light of this report, one that is just starting to be scoped. I'm redaing it now and see the need for patient safety issues to be quantified and deisgned into their development programmes.

We'd welcome reflections on challenges, risks, opportiunities and insights from elsewhere in healthcare and other industries. Your thoughts will inform our discussions with NHS

Thank you in advance. Helen

@Andrea D you might be intersted too

Thanks for drawing my attention to this @HelenH. It's interesting that among the aims of the new AI Lab, are two that are loosely linked to safety of some kind, albeit not Patient safety as we would normally view it. It is mentioned in the Exec Summary as being of relevance to the section on Governance (Chapter 3). 

In Chapter 2, the limitations of the current governance framework based on a survey are said to be "perhaps limiting innovation and potentially risking patient safety". I'm not clear where the evidence comes from to make that latter point, or if it is being loosely associated with the perceived issues related to collection, management and use of patient data. This is just one aspect of patient safety - there is a lot more that is not being considered here.

In Chapter 3, (from the beginning) one of the reasons given for the need for ethics & regulation is patient safety. The algorithmic considerations also mention safety, which in my view is much more fundamental to the whole discussion around patient safety and AI. There should be an awful lot more written about this in Chapter 3, the ommission is significant.

Perhaps one of the most telling statements is in the appendixed case study on Genomics England: "...safety is crucial and it is vital that the system is able to guarantee the integrity in the diagnoses and treatment plans which are delivered to patients, whether these are facilitated by AI, or more traditional methods."