Analysing hospital responses to patient-reported safety incidents online using large language models

Summary

Patient experiences reported online can reveal safety incidents in healthcare organisations, but they can only translate into quality improvement if organisations engage with them. Accordingly, healthcare organisations’ responses to online feedback offer a new data source on corrective culture, namely, the norms and practices for identifying, interpreting, and acting on safety incidents.

This study explored the feasibility of using a large language model to reliably code these responses for two candidate indicators of corrective culture: engagement and organisational learning.

The results indicate that large language models can reliably code engagement in these responses, while their ability to code organisational learning remains inconclusive.

Organisational culture in healthcare is usually assessed with staff surveys, which rely on self-report. Large language models could complement surveys by measuring observed behaviour in naturally occurring data, although whether engagement reflects corrective culture requires external validation.