The value of lived experience

The workshop reinforced a key message, that the future of AI in health and care cannot be shaped by technical expertise alone. Creating spaces where patients and service users work alongside regulators and Accredited Registers supports safer innovation. Lived experience brings vital insight into how systems work in practice, where risks can emerge, what the public want and need from regulation, and how to build trust. It also helps support the safe and reliable integration of AI.

The workshop was designed with participation in mind. Patients and service users took part alongside regulators and accredited registers on an equal footing. In a space that can sometimes feel highly technical, the workshop showed that meaningful public involvement is both possible and necessary.

Participants with lived experience engaged confidently with topics such as assurance, transparency and accountability. Discussions also covered how regulation, standards and guidance are experienced by the people they are meant to serve.

A consistent message throughout the day was that patients and service users are not just observers of AI policy and regulation, they are essential partners in getting it right.

Their contributions raised practical questions and real-world examples, and kept the focus on how AI-enabled decisions can affect people’s lives, access to services and confidence in care.

Equity, transparency and trust

Patients, service users and members of the public highlighted several issues that deserve particular attention as AI becomes more common across health and care.

If engagement only reaches the most confident, connected or well-resourced groups, AI tools and the rules around them risk being shaped by a narrow range of experience. True inclusion means actively involving people who are often overlooked, so innovation serves everyone and not just those who are easiest to reach. To support safe and fair innovation, tackling inequality needs to be built into every stage, from development, to procurement and service design, to long-term monitoring after deployment. Fairness and equity must be central, not an afterthought.

Avoiding harm requires more than technical fixes. It also needs careful scrutiny of the data that feeds AI systems. That includes the data used to train models and the data used in designing health and care services. It is also essential to be clear about which outcomes are being measured and how success is defined.

Trust depends on clarity, and it is important to give consideration to how AI is integrated in health and social care. Patients and service users should not feel like they are interacting with a 'black box'. It should be clear when AI is being used, what role it is playing in someone’s care and what options are available if something feels wrong. Empowering people helps them remain partners in their own health and care journey.

The workshop highlighted challenges but also the opportunities for health and social care improvement presented by AI. As we navigate this technological transformation, patients and service users should remain empowered through co-production, helping to shape the standards, guidance and regulation that govern how AI is designed, deployed and monitored in practice.

To find out more about the workshop and read the report, visit: Artificial intelligence - how to guide and regulate for health and social care professionals using AI

• the navigators: steering AI strategy and oversight for health systems
• the change-makers: stakeholder engagement and workforce development
• the guardrails: legal, policy and guideline structures for AI in health
• the backbone: health data governance for trustworthy AI
• the catalysts: leveraging AI for health requirements
• the gatekeepers: tackling adoption barriers

The conversations around AI governance across various health care organisations reveal a critical need for guidance that is practical, scalable, and centred on patient outcomes. Following are four key takeaways from the Leadership Alliance AI Accelerator that health systems can apply when developing or refining their AI governance structures:

1. Take a broad, integrated governance approach.
2.  Build governance with scalable capabilities and clear accountability.
3. Prioritise patient outcomes over model performance.
4. Prepare for regulatory gaps and build internal oversight.

From predictive analytics to clinical decision support, AI is increasingly being integrated into patient safety strategies. But while its potential is vast, its implementation demands careful scrutiny. The question is not just whether AI has a place in patient safety—I feel it clearly does—but whether it should redefine the roles of governance and safety professionals—I don’t agree. Can systems such as AI truly safeguard patients or will human oversight remain indispensable?

There is great promise with the use of AI in patient safety and it offers some advantages, such as:

• Early detection of risk: System learning algorithms can analyse vast datasets to identify patterns that signal deterioration, infection risk or medication errors often before clinicians detect it.
• Streamlined workflows: AI-powered tools can automate routine tasks, such as documentation, triage and scheduling, reduce human error and free up clinical time.
• Decision support: AI can provide real-time recommendations based on evidence-based guidelines, helping clinicians make safer, faster decisions.

These innovations are already being piloted with some healthcare providers with, so far, promising results.

However, we need to proceed with caution. Despite its promise, AI is not a single solution. There are some risks that should be in the forefront of our minds:

• Bias and inequity: Algorithms trained on incomplete or biased data can impact on decision making.
• Transparency and accountability: AI decisions can be too structured, making it difficult to trace errors or assign responsibility.
• Overreliance: There's a danger that clinicians may defer too readily to AI, undermining clinical judgment.

I feel these concerns reiterate the need for robust governance frameworks and continuous oversight. AI must be seen as a tool and not a replacement for the wealth of experience that is out there. Governance in healthcare can never become obsolete; AI may become our new colleague that continues to need support and guidance. Patient safety professionals will continue to play a crucial role in:

• Validating AI tools: Ensuring algorithms are clinically sound, ethically designed and rigorously tested.
• Monitoring outcomes: Tracking the real-world impact of AI on patient safety and intervening when necessary.
• Educating staff: Helping clinicians understand AI outputs and integrate them responsibly into care.

AI is undeniably part of the future of patient safety in healthcare, which we should welcome, but it is not a single system approach. It is one supportive component of a broader system that blends technological innovation with professional judgment and experience.

Related reading on the hub:

• New AI system to identify patient safety issues announced: Patient Safety Learning’s initial reflections
• One size does not fit all. How AI and better data can help us embrace complexity in diagnosis and treatment
• From pain to progress: How NHS trusts are tackling the complaints crisis with AI

More blogs from Ian:

• Integrating patient safety into pre-registration education

The scale of the problem

In January 2025, Healthwatch England published A pain to complain, revealing a deeply critical picture of NHS complaints handling. Nearly one-quarter of adults experienced poor NHS care in the past year, yet more than half took no action at all. Of those who did act, fewer than one in ten made a formal complaint—a significant drop from 39% in 2014.

The reasons paint a picture of institutional failure: 34% believed the NHS wouldn't use their complaint to improve services, 33% doubted they'd receive an effective response and 30% felt their concerns wouldn't be taken seriously. For those who did complain, over half remained dissatisfied with both process and outcome, while 43% waited more than six months for responses.

Perhaps most troubling was the revelation that NHS organisations aren't systematically learning from complaints, with traditional manual analysis missing critical patterns and leaving dangerous gaps in safety oversight.

Breaking through: real solutions from the frontline

Against this backdrop, I've been working with forward-thinking NHS trusts recently using our Patient Experience Quality AI and Learning tool (QUAIL) to systematically improve and uncover hidden insights in patient feedback. These collaborations offer hope that the crisis isn't insurmountable.

Uncovering the invisible

Working with one major NHS trust, we deployed QUAIL's AI-powered analysis to their existing patient feedback and revealed important gaps between what they thought was true and what was taking place. In one example, traditional manual analysis had identified just two end-of-life care complaints over nearly a 2-year period. The AI-led analysis revealed 44 such cases—meaning 95% of these important patient and family experiences had been invisible to improvement efforts.

In another example within cardiology services, complaints showed minimal service accessibility issues. But when we used QUAIL to analyse Patient Advice and Liaison Services (PALS) data, a different picture emerged: complaints represented only 1% of actual patient concerns about accessing care. The remaining 99% had been hidden in hundreds of PALS enquiries that couldn't be manually themed at scale and were not visible to the organisation in a way they could act upon.

From hours to minutes

At another NHS organisation, we tackled the time pressure preventing quality responses. Complaints officers were spending 2–4 hours crafting each response letter. Through intelligent automation generating high-quality draft responses in seconds, complaints breaching the response deadline dramatically decreased freeing up staff time to focus on the human investigation, contributing towards actual quality improvement and not peripheral administrative tasks.

The ripple effect

The benefits extended far beyond individual departments. CEOs are incorporating insights into governance meetings, while analysis supported Patient Safety Incident Response Framework (PSIRF) implementation. Crucially, QUAIL enabled us to link patient complaints directly to specific action plans and improvement initiatives. Instead of generic responses, trusts could implement targeted interventions—from additional administrative staff recruitment to new telephony systems—based on precisely identified patterns.

Response times improved and time spent preparing governance reports reduced by 80% due to having relevant information available dynamically, at the touch of a button.  Most importantly, complaints were now driving measurable service improvements rather than disappearing into administrative processes.

Addressing Healthwatch's challenge

The NHS trusts we've worked with have embraced AI directly to address the systemic failures identified in the Healthwatch report by building confidence through visible pattern identification, improving responsiveness through streamlined processes, enabling genuine learning through insight-driven quality improvement, and ensuring systematic rather than biased analysis.

A new paradigm

My work with these NHS trusts demonstrates that the crisis identified by Healthwatch isn't insurmountable. We can create systems where every patient voice is heard, understood and acted upon.

This isn't about replacing human judgement with AI—it's about empowering that judgement with better insights, faster responses and deeper understanding of patient needs.

As Healthwatch concluded: "We must treat feedback, concerns and complaints as 'gold nuggets' that drive improvements to care." The NHS organisations I've worked with show us how to mine those nuggets effectively, transforming patient voices from administrative burden into the driving force for better, safer care.

Further reading on the hub:

• How do we harness technology responsibly to safeguard and improve patient care?
• Patient Safety Spotlight Interview with James Munro, Chief Executive of Care Opinion
• One size does not fit all. How AI and better data can help us embrace complexity in diagnosis and treatment
• AI in healthcare: Shifting the frame from ‘what’ to ‘how’

• Interest in the use of artificial intelligence (AI) in health care is growing rapidly, but if it is to be accepted, and its benefits fully realised, it must command the confidence of patients, the public and NHS staff. To help understand public and staff attitudes towards the use of AI in health care, the Health Foundation commissioned a survey, conducted in June and July 2024, of 7,201 nationally representative members of the public (aged 16 years and older) and 1,292 NHS staff members.
• There is, on balance, support from both the public and NHS staff for the use of AI in health care, indicating a broadly receptive environment. Over half of the UK public (54%) and three-quarters of NHS staff surveyed (76%) said they support the use of AI for patient care, and an even greater proportion said they support the use of AI for administrative purposes (61% of the public and 81% of NHS staff surveyed).
• Despite this, a significant minority of the public is currently not supportive. For example, around 1 in 6 members of the public and around 1 in 10 of the NHS staff we surveyed think that AI will make care quality worse. Among the public, young people (aged 16–24 years) are less likely to believe that AI will improve care quality compared to other age groups, and women are less likely to believe that AI will improve care quality compared to men. This highlights the need to engage closely with the public and staff in order to understand and address concerns.
• One area of concern is AI’s potential impact on the social and relational aspects of health care. Over half of the public (53%) think AI will make them feel more distant from health care staff, while nearly two-thirds of the NHS staff surveyed (65%) think AI will make them feel more distant from patients. These results suggest that AI technologies will need to be designed and used in ways that protect or even enhance the human dimension of care. 
• The public are also concerned about the potential impact of AI on decision-making accuracy. For example, nearly a third (30%) of the public think that the main disadvantage of AI will be that health care staff will not question the outputs of AI systems, and so may miss errors. The public are also much more likely to support the involvement of AI in decision making where AI outputs are checked by a human. These results suggest the public strongly value keeping a human in the loop for many uses of AI in health care.
• While the NHS staff we surveyed are, on balance, looking forward to using AI as part of their roles (57% agreeing compared with 17% disagreeing), this is not equally felt across all occupational groups. For example, medical and dental staff are more positive than clinical staff such as health care assistants and health care support workers. In helping health care workers adjust to the rise of AI, policy makers and NHS leaders need to consider how its impact will vary across different roles and tailor engagement and support accordingly.

1. The Bias challenge: AI can introduce or perpetuate biases that society finds unacceptable.
2. The Privacy challenge: AI can allow individuals to be identified and personal information about them to be used in ways beyond what the public wants.
3. The Misrepresentation challenge: AI can allow the generation of material that deliberately misrepresents someone’s behaviour, opinions or character.
4. The Access to Data challenge: The most powerful AI needs very large datasets, which are held by few organisations.
5. The Access to Compute challenge: The development of powerful AI requires significant compute power, access to which is limited to a few organisations.
6. The Black Box challenge: Some AI models and tools cannot explain why they produce a particular result, which is a challenge to transparency requirements.
7. The Open-Source challenge: Requiring code to be openly available may promote transparency and innovation; allowing it to be proprietary may concentrate market power but allow more dependable regulation of harms.
8. The Intellectual Property and Copyright Challenge: Some AI models and tools make use of other people's content: policy must establish the rights of the originators of this content, and these rights must be enforced.
9. The Liability challenge: If AI models and tools are used by third parties to do harm, policy must establish whether developers or providers of the technology bear any liability for harms done.
10. The Employment challenge: AI will disrupt the jobs that people do and that are available to be done. Policy makers must anticipate and manage the disruption.
11. The International Coordination challenge: AI is a global technology, and the development of governance frameworks to regulate its uses must be an international undertaking.
12. The Existential challenge: Some people think that AI is a major threat to human life. If that is a possibility, governance needs to provide protections for national security.