Designing in risk: Measuring safety part 3

Summary

The relationship between management and the workforce, in very simplistic terms, can be considered one of reward in return for effort. The contracted effort is communicated through a roster. In organisations that have a continuous operation, blocks of effort are distributed to maintain the flow of output. The organisation of effort, then, is a legitimate function of management. 

Norman's previous blog looked at performance variability under normal conditions. In this blog, Norman looks at the impact of physiological states and how management’s organisation of effort degrades decision-making.

Content

Fatigue

The chart below shows pilot fatigue measured using the Samn-Perelli Scale (S-PS).[1] The S-PS has 7 intervals and a score of 4 indicates the onset of fatigue. The data shows how fatigue increases across the first and second sectors of the day, but, also, that fatigue is significantly higher during night-time operations.

A study[2] of urology surgeons using the S-PS, reported that fatigue, as measured pre- and post-operation, increased by 67.95% across the four procedures undertaken in the day. Another study[3] looking at 29 ICU doctors found that the median S-PS score at the start of a day shift was 3 and 4 at the end; however, at the start of a night shift the median was 3 and at the end it was 5. Pilots with less than 6 hours of sleep before a duty started the day with an S-PS score of 4. In a risk assessment of night flights to Queenstown Airport, New Zealand, it was suggested that pilots with an S-PS of 4 or greater should be prohibited from flying.[4] 

Fatigue affects error rates. The Line Operations Safety Audit (LOSA)[5] shows that crew that slept for 6 hours or less before a duty committed more errors. In a study[6] of crew flying night cargo operations, crew acclimatised to the local day but flying during their local night had an error rate of 13.18/sector. However, crews who were flying at night in a different time zone but operating on their home daytime body clock had an error rate of 5.4 errors/sector. It is well-understood that performance is degraded during the 'window of circadian low' – that phase of the circadian cycle when humans are supposed to be sleeping – but in my previous blog, I made the point that raw error rates are not necessarily the issue, rather it was how errors shape the operation.

Fatigue and decision-making

The table below shows error outcomes across consecutive flights. An ‘additional risk’ is where, in dealing with the initial error, the crew either committed a subsequent error or the consequence was a ‘Undesired Aircraft State’ (UAS). It is common to see improved performance on the second sector as crew build familiarity but there is a sharp fall-off in performance on the third sector, including a significant increase in the number of mistakes made by crew.  

Mistakes in this context are errors of decision-making. In short, fatigue affects judgement. We see the same in other domains: in finance, traders make riskier trades when fatigued.[7] This data on fatigue and error points to job design and staff deployment as risk factors.

Organisational responses to self-management of fatigue

Workers absent themselves from the workplace for a variety of reasons. It could be for genuine ill-health, no-notice personal needs and disaffection (morale). Or it could be personal fatigue management. Again, the control of unplanned absence is a legitimate management activity. Workforce absenteeism places an increased burden on the attending workforce and adds to fatigue. The graph below shows the absence rate for a group of pilots and the percentage of pilots who did not take a single day of unplanned absence in a year.  

The absence management rules were changed to address the problem. The next graph shows how the duration of absences changed in response to the new policy:

Pilot absence episode duration (days)

The data suggests that management and workforce exist in a dynamic relationship and management’s attempt to exert control results in a corresponding response. The deployment of the workforce is a legitimate management function, but the way contracted effort is utilised shapes safety. Shift duration and timing induce fatigue and, importantly, fatigue can result in riskier decisions. In the previous blog, decision-making in normal operations was also seen to affect risk. 

Conclusion

In this series of blogs, I have suggested that to understand safety we need to look at the factors that increase risk. Risk is a function of the tension between organisational controls and the need for flexibility that flows from variability in the workplace. Three areas of interest have been suggested: the preparation of staff for work, their control and, finally, their deployment. To understand ‘what goes on here’ we need to better understand the dynamics of these three domains.

References

1. Samn SW, Perelli LP. Estimating aircrew fatigue: A technique with application to airlift operations. Brooks Air Force Base. San Antonio, TX. Report No: SAM-TR-82-221, 1982.
2. Petrut B, et al. Mental fatigue evaluation of surgical teams during a regular workday in a high-volume tertiary healthcare center. Urol Int 2020; 104(3-4): 301–308.
3. Bihari S, et al. ICU shift related effects on sleep, fatigue and alertness levels. Occup Med (Lond) 2020; 70(2):107-112.
4. Navigatus Consulting (2017). Queenstown Airport Night Operations Foundation Safety Case.
5. Klinect JR. Line Operations Safety Audit: A Cockpit Observation Methodology for Monitoring Commercial Airline Safety Performance. Unpublished PhD thesis, 2005. University of Texas. Unpublished PhD thesis. University of Texas.
6. MacLeod N. Crew Resource Management Training: A Competence-based Approach for Airline Pilots. CRC Press, 2021.
7. Dickinson DL, Chaudhuri A, Greenaway-McGrevy R. Trading while sleepy? Circadian mismatch and mispricing in a global experimental asset market. Exp Econ 2019; 23:526–553.

Further reading from Norman

• Can you measure safety? Part 1
• Errors as clues in the search for safety measures: Measuring safety part 2

About the Author

Norman MacLeod served for 20 years in the RAF involved in the design and delivery of training in a variety of situations. He stumbled across 'CRM' in 1988 while investigating leadership in military transport aircraft crews. From 1994, he worked around the world as a consultant in the field of CRM in commercial aviation, latterly employed as the Human Factors Manager for a blue chip airline in Hong Kong. Now semi-retired, he is one of the Patient Safety Partners at James Cook Hospital in Middlesborough.