Equity in medical devices: independent review - final report (DHSC, 11 March 2024)

Summary

A core responsibility of the NHS is to maintain the highest standards of safety and effectiveness of medical devices available for all patients in its care. Evidence has emerged, however, about the potential for racial and ethnic bias in the design and use of some medical devices commonly used in the NHS, and that some ethnic groups may receive sub-optimal treatment as a result. In response to these concerns, the UK Government commissioned this independent review on equity in medical devices.

In its final report, the Review sets out the need for immediate action to tackle the impact of ethnic biases in the use of medical devices. Its findings and recommendations have also been published in a short animation. The Government’s response to the Review's 18 recommendations has also been published alongside its final report.

Content

The report makes 18 recommendations:

Immediate mitigation for pulse oximeters

Recommendation 1

Regulators, developers, manufacturers and healthcare professionals should take immediate mitigation actions to ensure existing pulse oximeter devices in the NHS can be used safely and equitably for all patient groups across the range of skin tones.

This requires action on several fronts as follows:

• MHRA should strengthen its guidance for patients and caregivers using oximeters at home, and for healthcare professionals, on the accuracy and performance of pulse oximeters. This should include guidance on taking and interpreting readings from patients with different skin tones. Renewed efforts should be made to promote this guidance to health professionals throughout the NHS, patients and the public
• health professionals should advise patients who have been provided with a pulse oximeter to use at home to look at changes in readings, rather than just a single reading, to identify when oxygen levels are going down and they need to call for assistance. Patients should also be advised to look out for other worrying symptoms such as shortness of breath, cold hands and feet, chest pain and fast heart rate
• clinical guideline developers and health technology assessment (HTA) agencies such as the National Institute for Health and Care Excellence (NICE) should produce guidance on the use of pulse oximeters, emphasising the variable accuracy of readings in patients with darker skin tones, and recommend the monitoring of trends rather than setting absolute thresholds for action
• Health Education England (part of NHS England) and the respective agencies in the devolved nations should educate clinicians about how the technology of pulse oximeters works, and advise that treatment should not be withheld or given on the basis of absolute thresholds alone. Clinicians should be trained to monitor trends rather than absolute thresholds for action
• manufacturers of pulse oximeters must update their instructions for use to inform patients and clinicians about whether the device is ISO compliant, the limitations of their model of pulse oximetry and any contra-indications, and its differential accuracy in patients with different skin pigmentation
• MHRA should issue updated guidance to developers and manufacturers on the need to make the performance of their device across subgroups with different skin tones transparent.

Recommendation 2

MHRA and approved bodies for medical devices should strengthen the standards for approval of new pulse oximeter devices to include sufficient clinical data to demonstrate accuracy overall and in groups with darker skin tones. Greater population representativeness in testing and calibration of devices should be stipulated.

The approach should include:

• MHRA and UK-approved bodies following the US FDA in requiring manufacturers to obtain validity data from a diverse subject pool with a:
• large number of participants
• diverse range of skin tones
• clinically relevant range of oxygenation levels
• manufacturers and research-funding bodies commissioning studies that include the population upon which the device will be used, subjects with a diverse range of skin pigmentations and critically unwell subjects with poor perfusion. Validation of devices should be conducted in the intended use population and setting, such as at home or in an intensive care unit
• manufacturers of medical-grade pulse oximeters being required to comply with BS EN ISO 80601-2-61:2019 (medical electrical equipment - particular requirements for basic safety and performance of pulse oximeter equipment) to gain market approval
• healthcare equity impact assessments being essential requirements for developing or supplying pulse oximeters in the UK in order to identify whether mitigating actions are needed to ensure they are fit for purpose for all racial and ethnic groups and people of varying skin tones. Making these assessments an essential requirement is in line with technological progress and international best practice.

Recommendation 3

Innovators, researchers and manufacturers should co-operate with public and patient participants to design better, smarter oximeters using innovative technologies to produce devices that are not biased by skin tone.

This could include:

• developing enhanced algorithms for oximeter device software to address measurement bias
• exploring the use of multi-wavelength systems, which measure and correct for skin pigmentation, to replace conventional 2-wavelength oximeters.

Equity of optical medical devices

Recommendation 4

The professional practice bodies in the UK, such as the Royal Colleges, should convene a task group of clinicians from relevant disciplines - including medical physicists, public and patient participants, developers and evaluators - to carry out an equity audit of optical devices in common use in the NHS, starting with dermatological devices, to identify those at particular risk of racial bias with potential for harm, which should be given priority for further investigation and action.

Recommendation 5

Renewed efforts should be made to:

• increase skin tone diversity in medical imaging databanks used for developing and testing optical devices for dermatology, including in clinical trials
• improve the tools for measuring skin tone incorporated into optical devices.

This will require a concerted effort on several fronts, including:

• encouraging links between imaging databank compilers, professional bodies, optical device developers and clinicians to develop and improve accessibility of imaging data resources that reflect skin tone diversity within the population, such as in databanks for skin cancer diagnosis
• MHRA providing strengthened guidance to developers and manufacturers on improving skin tone diversity in testing and development of prioritised optical devices. MHRA is already working towards such guidance as part of its programme on pulse oximeters
• research funders supporting additional incentives and patient-centred approaches to address logistical, financial and cultural barriers that limit participation of minority ethnic groups in clinical studies of optical devices
• researchers and dermatologists developing more accurate methods for measuring and classifying skin tone, which are objective, reproducible, affordable and user-friendly. Current practice of using uncertain descriptors of ancestry, ethnicity or race to define patients with dark skin tones is ambiguous and problematic. In its discussions on updating standards, MHRA is examining which measures would be most appropriate, with the aim of agreeing a consensus. This work is to be commended.

Recommendation 6

Once in use, optical devices should be monitored and audited in real-world conditions to evaluate safety performance overall and by skin diversity. This will ensure any adverse outcomes in certain populations are identified early and mitigations implemented.

This requires a whole-system approach and should include:

• commitment from manufacturers at the pre-qualification stage to fund and facilitate the establishment of registries for collecting data across all population groups on patient demographic characteristics, use and patient outcomes, following deployment of the technology
• HTA agencies (such as NICE, the Scottish Health Technologies Group and Health Technology Wales) being provided with access to post-deployment monitoring and adverse effects data as part of their assessments of optical devices. This data should be considered alongside the wider evidence when determining the value of the optical device for NHS use
• NHS Supply Chain, National Services Scotland, NHS Wales Shared Services Partnership, Northern Ireland Procurement and Logistics Service and other contracting authorities including a minimum standard of device performance across subgroups of the target population, which will make transparent any equity impacts as part of the pre-qualification stage when establishing national framework agreements. Manufacturers need to declare whether they have considered minimum standards for equity
• DHSC and the devolved administrations updating the national pre-acquisition questionnaire used by NHS trust electrical biomedical engineering teams when buying medical equipment to include a minimum designated standard for equity as part of the pre-purchase validation checks
• the approved body conducting regular surveillance audits of prioritised optical devices. The audits should include data submissions from the manufacturer and the Medical Device Safety Officers or Incidents and Alerts Safety Officers networks (representatives from NHS trusts in charge of reporting on safety), and should include data from the MHRA Yellow Card scheme for reporting adverse incidents and the Learn from patient safety events service. These audits should include an evaluation of differential safety by ethnic group
• the continued strengthening of MHRA’s vigilance role, as specified in the Cumberlege report’s recommendation 6, which called for substantial improvements in adverse event reporting and medical device regulation with an emphasis on patient engagement and outcomes
• better routine capturing of ethnicity data in electronic healthcare records, alongside better collection and collation of data on medical devices in use. This would enable MHRA to conduct more rapid studies to build the evidence when a hypothesis about potential inequity in an optical device is made.

Recommendation 7

A review should be conducted by the relevant academic bodies of how medical education and continuing professional development requirements for health professionals currently cover equity issues arising in the use of medical devices generally and skin diversity issues in particular, with appropriate training materials developed in response.

This should include:

• undergraduate and postgraduate medical and allied health professions training, including teaching clinicians about clinically relevant conditions where disease presentation differs between White and ethnic minority patients
• clinicians being made aware that, when using dermoscopy or other medical devices to examine skin lesions, clinical signs may differ according to skin tone, and their training should include images of skin lesions in all skin tones
• clinicians receiving training in identifying potential sources of bias in medical devices and how to report adverse events to MHRA
• where new devices are introduced into clinical practice, organisations and clinicians using the new devices, ensuring there is sufficient training to acquire skills and competencies before the device is used

Preventing bias in AI-assisted medical devices

Recommendation 8

AI-enabled device developers and stakeholders, including the NHS organisations that deploy the devices, should engage with diverse groups of patients, patient organisations and the public, and ensure they are supported to contribute to a co-design process for AI-enabled devices that takes account of the goals of equity, fairness and transparency throughout the product’s lifecycle.

Engagement frameworks from organisations such as NHS England can help hold developers and healthcare teams to account for ensuring that existing health inequities affecting racial, ethnic and socio-economic subgroups are mitigated in the care pathways in which the devices are used.

Recommendation 9

The government should commission an online and offline academy to improve the understanding among all stakeholders of equity in AI-assisted medical devices.

This academy could be established through the appropriate NHS agencies, and should develop material for lay and professional stakeholders to promote better ways for developers and users of AI devices to address equity issues, including:

• ensuring undergraduate and postgraduate health professional training includes the potential for AI to undermine heath equity, and how to identify and mitigate or remove unfair biases
• producing materials to help train computer scientists, AI experts and design specialists involved in developing medical devices about equity, and systemic and social determinants of racism and discrimination in health
• ensuring that clinical guideline bodies identify how health professionals can collaborate with other stakeholders to identify and mitigate unfair biases that may arise in the development and deployment of AI-assisted devices
• encompassing an appreciation of AI within a whole-system and lifecycle perspective, and understanding the end-to-end deployment and potential for inequity.

Recommendation 10

Researchers, developers and those deploying AI devices should ensure they are transparent about the diversity, completeness and accuracy of data through all stages of research and development. This includes the sociodemographic, racial and ethnic characteristics of the people participating in development, validation and monitoring of product performance.

This should include:

• the government resourcing MHRA to provide guidance on the assessment of biases that may have an impact on health equity in its evaluation of AI-assisted devices, and the appropriate level of population detail needed to ensure adequate performance across subgroups
• encouraging the custodians of datasets to build trust with minoritised groups and take steps with them to make their demographic data as complete and accurate as possible, subject to confidentiality and privacy
• developers, research funders, regulators and users of AI devices recognising the limitations of many commonly used datasets, and seeking ones that are more diverse and complete. This may require a concerted effort to recruit and sample underrepresented individuals. We commend initiatives internationally and in the UK (such as the National Institute for Health and Care Research-led INCLUDE guidance) to encourage the development and use of more inclusive datasets. Data collection by public bodies must be properly resourced so that datasets are accurate and inclusive
• dataset curators, developers and regulators using consensus-driven tools, such as those by STANDING Together, to describe the datasets that are used in developing, testing and monitoring
• regulators requiring manufacturers to report the diversity of data used to train algorithms
• regulators providing guidance that helps manufacturers enhance the curation and labelling of datasets by assessing bias, being transparent about limitations of the data, the device and the device evaluation, and how to mitigate or avoid performance biases
• regulators enforcing requirements for manufacturers to document and publicise differential limitations of device performance and, where necessary, place reasonable restrictions on intended use
• making sure that the Health Research Authority and medical ethics committees approving AI-enabled device research do not impose data minimisation constraints that could undermine dataset diversity or the evaluation of equity in the outcomes of research.

Recommendation 11

Stakeholders across the device lifecycle should work together to ensure that best practice guidance, assurance and governance processes are co-ordinated and followed in support of a clear focus on reducing bias, with end-to-end accountability.

This should include:

• MHRA adjusting its risk assessment of AI-assisted devices so that all but the simplest and lowest-risk technologies are categorised under Class IIa or higher, including a requirement for their algorithms to be suitable for independent evaluation, the use of a test of overall patient benefit that covers the risks of biased performance, and a requirement for manufacturers to publish performance audits with appropriate regularity that include an assessment of bias
• supporting health professionals’ involvement early in the development and deployment of AI devices. We commend the use of ethical design checklists, which may assist in the quality assurance of these processes
• manufacturers adopting MHRA’s Good Machine Learning Practice for Medical Device Development: Guiding Principles
• all stakeholders supporting MHRA’s Software and AI as a Medical Device Change Programme Roadmap, such as promoting the development of methodologies for the identification and elimination of bias, and testing the robustness of algorithms to changing clinical inputs, populations and conditions
• placing a duty on developers and manufacturers to participate in auditing of AI model performance to identify specific harms. These should be examined across subgroups of the population, monitoring for equity impacts rather than just unequal performance.

Recommendation 12

UK regulatory bodies should be provided with the long-term resources to develop agile and evolving guidance, including governance and assurance mechanisms, to assist innovators, businesses and data scientists to collaboratively integrate processes in the medical device lifecycle that reduce unfair biases and their detection, without being cumbersome or blocking progress.

Recommendation 13

The NHS should lead by example, drawing on its equity principles, influence and purchasing power, to influence the deployment of equitable AI-enabled medical devices in the health service.

This should include:

• NHS England and the NHS in the devolved administrations including a minimum standard for equity as part of the pre-qualification stage when establishing national framework agreements for digital technology
• NHS England updating the digital technology assessment criteria used by health and social care teams when buying digital technology to recommend equity as part of the pre-purchase validation checks
• working with manufacturers and regulators to promote joint responsibility for safety monitoring and algorithm audits to ensure outcome fairness in the deployment of AI-assisted devices. This will require support for the creation of the right data infrastructure and governance.

Recommendation 14

Research commissioners should prioritise diversity and inclusion. The pursuit of equity should be a key driver of investment decisions and project prioritisation. This should incorporate the access of underrepresented groups to research funding and support, and inclusion of underrepresented groups in all stages of research development and appraisal.

This should include:

• requiring that AI-related research proposals demonstrate consideration of equity in all aspects of the research cycle
• ensuring that independent research ethics committees consider social, economic and health equity impacts of AI-related research.

Recommendation 15

Regulators should be properly resourced by the government to prepare and plan for the disruption that foundation models and generative AI will bring to medical devices, and the potential impact on equity.

A government-appointed expert panel should be convened - made up of clinical, technology and healthcare leaders, patient and public involvement representatives, industry, third sector, scientists and researchers who collectively understand the technical details of emerging AI and the context of medical devices - with the aim of assessing and monitoring the potential impact on AI quality and equity of LLM and foundation models.

Future proofing: polygenic risk scores

Recommendation 16

The focus of PRS studies should be widened beyond genetic diversity to include:

• the contribution of the social determinants of health - including lifestyle, living and working conditions, and environmental factors such as air pollution - to overall disease risk
• how these affect the predictive potential of PRS among different ethnicities and socio-economic groups.

Developments with this wider research focus should aid the refinement of overall risk assessments so that they better reflect the role that PRS play alongside non-genetic risk factors.

Recommendation 17

National research funders should commission a broad programme of research and consultation with the public, patients and health professionals to fill the gaps in knowledge and understanding concerning PRS. The programme should cover:

• the public’s understanding of the nature of genetic risk and the meaning of the PRS they are presented with
• explorations of how health professionals interpret these risks, and can best communicate and support people in understanding the results of their PRS
• the research programme should cover impacts on diverse population subgroups, and be informed by extensive engagement with the public and patients to gain their perspectives.

Results from this research programme, together with actions on recommendation 16, should feed into the development of clinical applications for PRS medical devices covered in recommendation 18.

Recommendation 18

UK professional bodies - such as the Royal Colleges and health education bodies across the UK - should develop guidance for healthcare professionals on the equity and ethical challenges and limitations of applying PRS testing in patient care and population health programmes.

The guidance should:

• include the interpretation of risk scores, communicating risk to patients and the public, and counselling and support
• be informed by extensive public and patient engagement.