Artificial intelligence and patient safety in healthcare: Insights and recommendations from HETT 2025 roundtable

Summary

The 10 Year Health Plan for England sets an ambition of the NHS becoming the most AI-enabled care system in the world. Subsequent discussions around this have often centred on how new technologies powered by AI have the potential to improve patient care and outcomes. However, there has been significantly less attention on the implications of this for patient safety.

This report, authored by Grayson Katzenbach, an attendee at the roundtable, and Patient Safety Learning’s Chief Digital Officer Clive Flashman, considers how AI can be deployed safely across the health and care system. It draws on a discussion at the Healthcare Excellence Through Technology (HETT) 2025 roundtable on this topic, which convened leaders from healthcare, academia, law and industry. Their discussion explored how innovation, governance and culture must evolve together to ensure that AI maintains and strengthens, rather than compromises, patient safety. 

Content

The report (attached below) summarises key themes that emerged from the HETT roundtable discussion on Tuesday 7 October 2025 and highlights five cross-cutting priority actions and recommendations emerging from this.

1. Strengthen safety culture and leadership alignment

Actions:

• National health and care leaders should embed AI safety governance within existing patient safety frameworks, ensuring consistent oversight across sectors.
• Provider boards and senior leadership teams should assign clear roles for AI oversight, learning and accountability.
• Professional regulators and safety bodies should align expectations for digital safety leadership across care settings.

Recommendations:

• Embed AI safety within broader organisational safety culture and learning systems.
• Link leadership accountability for digital safety to existing performance and governance metrics.

2. Build workforce capability and human factors competence

Actions:

• Education and professional bodies (e.g., GMC, NMC, Royal Colleges) should define core AI literacy and human factors competencies.
• Health and care organisations should incorporate digital safety and automation-bias training into staff development.
• Industry partners should co-design training materials and user-centred design standards for safe implementation.

Recommendations:

• Make AI and data science literacy a standard element of clinical and care curricula.
• Embed human factors principles into technology design, procurement and deployment throughout the health and care system.

3.  Improve data quality and representativeness

Actions:

• National data and standards bodies should align data quality requirements across health, private and social care providers.
• Provider organisations should implement continuous feedback loops to identify and correct bias in AI outputs.
• Developers and suppliers should publish evidence on data representativeness and model performance.

Recommendations:

• Establish a sector-wide approach to improving data completeness, coding accuracy and population representativeness.
• Promote cross-organisational data sharing and transparency to support collective learning on bias and model drift.

4.  Embed continuous governance and lifecycle assurance

Actions:

• Regulators and system leaders should develop post-deployment monitoring requirements for AI systems in all care settings.
• Developers and provider organisations should share responsibility for ongoing surveillance, incident reporting and performance assurance of AI-based systems to ensure transparent, fair and sustained accountability.
• National policy bodies should clarify where accountability lies for ongoing surveillance of deployed systems.

Recommendations:

• Adopt adaptive, proportionate regulation that supports both innovation and accountability.
• Require regular post-deployment reviews and transparent reporting to sustain public and professional confidence.
• Extend Patient Safety Incident Response Framework (PSIRF) principles of learning from harm and near miss events to digital and AI systems to embed shared responsibility between developers and providers.

5. Align innovation pace with system readiness

Actions:

• Policy and funding bodies should link innovation incentives to demonstrated safety, benefit and operational readiness.
• Developers and providers should conduct staged roll-outs and real-world evaluations before large-scale deployment.

Recommendations:

• Ensure that patient and service safety remains the rate-limiting factor in AI adoption.
• Reward measured, evidence-led implementation rather than rapid scale-up driven by market or policy pressure.
• Promote adaptive evaluation models that enable learning while reducing risk.

Attachments