Artificial Intelligence and the practice of patient safety: GPT-4 performance on a standardized test of safety knowledge (26 September 2024)

Summary

This study highlights the potential of generative artificial intelligence (AI) to improve patient safety in healthcare.

Content

Published in The Joint Commission Journal on Quality and Patient Safety, this study tested the widely used AI model GPT-4 on the Certified Professional in Patient Safety (CPPS) exam, where it answered 88% of questions correctly. Researchers believe AI could help reduce medical errors, estimated to cause 400,000 deaths annually, by assisting clinicians in identifying and addressing safety risks in hospitals and clinics.

The study marks the first in-depth test of GPT-4’s capabilities in patient safety, focusing on its performance in key areas, including risk solutions, measuring performance, and systems thinking. GPT-4 excelled in areas such as patient safety and solutions, but showed weaknesses in culture and leadership domains, especially when multiple correct answers were possible.

The study authors suggested that AI has promise in helping doctors better recognise, address, and prevent mistakes or accidental harm in hospitals and clinics.