Search the hub

The relationship between management and the workforce, in very simplistic terms, can be considered one of reward in return for effort. The contracted effort is communicated through a roster. In organisations that have a continuous operation, blocks of effort are distributed to maintain the flow of output. The organisation of effort, then, is a legitimate function of management.  Norman's previous blog looked at performance variability under normal conditions. In this blog, Norman looks at the impact of physiological states and how management’s organisation of effort degrades decision-making. Fatigue The chart below shows pilot fatigue measured using the Samn-Perelli Scale (S-PS).[1] The S-PS has 7 intervals and a score of 4 indicates the onset of fatigue. The data shows how fatigue increases across the first and second sectors of the day, but, also, that fatigue is significantly higher during night-time operations. A study[2] of urology surgeons using the S-PS, reported that fatigue, as measured pre- and post-operation, increased by 67.95% across the four procedures undertaken in the day. Another study[3] looking at 29 ICU doctors found that the median S-PS score at the start of a day shift was 3 and 4 at the end; however, at the start of a night shift the median was 3 and at the end it was 5. Pilots with less than 6 hours of sleep before a duty started the day with an S-PS score of 4. In a risk assessment of night flights to Queenstown Airport, New Zealand, it was suggested that pilots with an S-PS of 4 or greater should be prohibited from flying.[4] Fatigue affects error rates. The Line Operations Safety Audit (LOSA)[5] shows that crew that slept for 6 hours or less before a duty committed more errors. In a study[6] of crew flying night cargo operations, crew acclimatised to the local day but flying during their local night had an error rate of 13.18/sector. However, crews who were flying at night in a different time zone but operating on their home daytime body clock had an error rate of 5.4 errors/sector. It is well-understood that performance is degraded during the 'window of circadian low' – that phase of the circadian cycle when humans are supposed to be sleeping – but in my previous blog, I made the point that raw error rates are not necessarily the issue, rather it was how errors shape the operation. Fatigue and decision-making The table below shows error outcomes across consecutive flights. An ‘additional risk’ is where, in dealing with the initial error, the crew either committed a subsequent error or the consequence was a ‘Undesired Aircraft State’ (UAS). It is common to see improved performance on the second sector as crew build familiarity but there is a sharp fall-off in performance on the third sector, including a significant increase in the number of mistakes made by crew. Mistakes in this context are errors of decision-making. In short, fatigue affects judgement. We see the same in other domains: in finance, traders make riskier trades when fatigued.[7] This data on fatigue and error points to job design and staff deployment as risk factors. Organisational responses to self-management of fatigue Workers absent themselves from the workplace for a variety of reasons. It could be for genuine ill-health, no-notice personal needs and disaffection (morale). Or it could be personal fatigue management. Again, the control of unplanned absence is a legitimate management activity. Workforce absenteeism places an increased burden on the attending workforce and adds to fatigue. The graph below shows the absence rate for a group of pilots and the percentage of pilots who did not take a single day of unplanned absence in a year. The absence management rules were changed to address the problem. The next graph shows how the duration of absences changed in response to the new policy: Pilot absence episode duration (days) The data suggests that management and workforce exist in a dynamic relationship and management’s attempt to exert control results in a corresponding response. The deployment of the workforce is a legitimate management function, but the way contracted effort is utilised shapes safety. Shift duration and timing induce fatigue and, importantly, fatigue can result in riskier decisions. In the previous blog, decision-making in normal operations was also seen to affect risk. Conclusion In this series of blogs, I have suggested that to understand safety we need to look at the factors that increase risk. Risk is a function of the tension between organisational controls and the need for flexibility that flows from variability in the workplace. Three areas of interest have been suggested: the preparation of staff for work, their control and, finally, their deployment. To understand ‘what goes on here’ we need to better understand the dynamics of these three domains. References Samn SW, Perelli LP. Estimating aircrew fatigue: A technique with application to airlift operations. Brooks Air Force Base. San Antonio, TX. Report No: SAM-TR-82-221, 1982. Petrut B, et al. Mental fatigue evaluation of surgical teams during a regular workday in a high-volume tertiary healthcare center. Urol Int 2020; 104(3-4): 301–308. Bihari S, et al. ICU shift related effects on sleep, fatigue and alertness levels. Occup Med (Lond) 2020; 70(2):107-112. Navigatus Consulting (2017). Queenstown Airport Night Operations Foundation Safety Case. Klinect JR. Line Operations Safety Audit: A Cockpit Observation Methodology for Monitoring Commercial Airline Safety Performance. Unpublished PhD thesis, 2005. University of Texas. Unpublished PhD thesis. University of Texas. MacLeod N. Crew Resource Management Training: A Competence-based Approach for Airline Pilots. CRC Press, 2021. Dickinson DL, Chaudhuri A, Greenaway-McGrevy R. Trading while sleepy? Circadian mismatch and mispricing in a global experimental asset market. Exp Econ 2019; 23:526–553. Further reading from Norman Can you measure safety? Part 1 Errors as clues in the search for safety measures: Measuring safety part 2

In a three-part series of blogs for the hub, Norman Macleod explores how systems behave and how the actions of humans and organisations increase risk.  In part 1 of this blog series, Norman suggested that measuring safety is problematic because the inherent variability in any system is largely invisible. Unfortunately, what we call safety is largely a function of the risks arising from that variability. In this blog, Norman explores how error might offer a pointer to where we might look.  Safety as risk propagation It is common in safety management to talk in terms of hazards. We can identify three classes of hazards: substances or objects that could cause loss or harm; engineered situations where humans engage in activity involving known hazards but under controlled conditions; acts by individuals that inadvertently expose the operation to a hazard (we might call these ‘errors’). Controls are put in place to contain hazards but controls are designed by humans and are fallible. Healthcare is an example of a hazardous condition: things are done to patients that would be illegal if inflicted upon a healthy person. Procedures act as controls in these situations but there is always a tension between work-as-imagined (WAI) and work-as-done (WAD). WAI describes the least-risky solution to a problem that will work in most circumstances (or, at least, those envisaged by the procedure designers), whereas WAD reflects the inherent flexibility needed in the real world. In a study of maritime accidents,[1] it was found that collisions have occurred between ships actively trying to follow the ‘rules of the road.’ Procedures contain affordance spaces, or lacunae, that must be filled by actors applying expertise. Procedures, or rules, form a hierarchy. At the top there are rules about goals: ‘first, do no harm.’ Then there are IF-THEN rules that aid decision-making: IF <symptom> THEN <condition>. The lowest order of rules are task prescriptions: step 1, step 2, step n. As we ascend the hierarchy, actors need more extensive training to cope with the lacunae that invariably exist. Many airlines use a process called the Line Operations Safety Audit (LOSA).[2] Trained observers monitor flight crew under normal flight conditions and log departures from procedures, crew responses and subsequent outcomes. In most cases, 95% of errors are inconsequential: error is very much noise in the system. LOSA can let us see what happens when crew attempt to fill in the gaps in procedures. The observer can tag an error as 'intentional’ (an INC) if certain criteria are met and figures of between 8.8% and 26.4% of INC errors have been seen. However, ‘Intentional’ errors are usually attempts to adapt to local circumstances or to solve problems. These departures from prescribed activity reflect system buffering. The outcome of an error can be categorised in LOSA as ‘inconsequential’, can trigger an additional error or results in an ‘Undesired Aircraft State’ (UAS) if the observer feels that safety has been jeopardised. In one study I looked at UASs arising from INCs versus non-intentional errors. INCs were twice as likely to result in a UAS. I then looked at who committed the error. For INCs, captains accounted for 91.66% of UASs compared with 40.6% when the error was non-intentional. The data suggests that agents actively choose courses of action that contravene procedures to maintain the flow of work but those decisions increase risk. Captains are over-represented in the data because they are the primary decision-makers in the team. Ironically, compliance with procedures is often the starting point for any safety investigation. However, rather than police ‘compliance’, organisations should probably find ways to capture variability and render it as knowledge. What error does To view error simply as failure, however, is to miss the fact that they change the work process in a way that needs to be addressed if safety is to be maintained. This can happen in one of three ways. First, they reduce performance margins. Even slight departures from the optimum aircraft configuration mean that, should a subsequent event occur, the crew have less flexibility to respond. In the flight data shown in the previous blog, an aircraft operating in the outer bands of the distribution is migrating towards the margins of the safe space. Something as commonplace as a change in windspeed or direction could result in a critical outcome. Second, error transfers risk when my action affects others. For example, passengers have been killed when aircraft have flown into turbulence. If a pilot delays or fails to turn on the seat belt sign in time the cabin crew and passengers are exposed to risk because they will not have taken steps to protect themselves (such as sitting down or fastening seat belts). Sometimes, and in contravention of procedures, pilots start the ‘after landing’ checklist early to save time. This usually results in pausing the checklist while air traffic control issues directions to the terminal building. LOSA shows that crew then often forget to finish the checklist and aircraft park with the weather radar still turned on, exposing the ground handlers to a radiation hazard. Finally, separation reduction describes the condition where aircraft are placed in closer proximity to hazardous objects (other aircraft, the ground) than was intended. Again, should something happen, the crew will have less time to react. Error, then, can reveal how the risk profile is shaped by the deliberate actions of crew. What goes on here? This examination of normal work suggests two candidate domains for measures of safety. First, what is the organisation’s understanding of the utility of its control structures (policies and procedures, codes of conduct)? How well-written and comprehensive are the structures? Where are the contradictions and ambiguities that flow from multiple stakeholders in the process of oversight? Second, what is the skills mix of those required to work within the system, recognising the need to cope with the variability inherent in the real world. Does the organisation have a competence model for the different functions in the system? What are the risks associated with substituting staff (bank staff, staff on loan)? Conclusion In this post I have looked how workplace variability shapes risk. I have suggested two key aspects of the structure of an organisation – control and competence – that could be candidates for measuring ‘safety’. In my final blog I want to explore how organisations actively design unsafety into their operations. References Belcher P. ‘A Sociological Interpretation of the COLREGS”. Journal of Navigation, 2002; 55(02): 213-224. Klinect JR, 1st Klinect JR. Line Operations Safety Audit: A Cockpit Observation Methodology for Monitoring Commercial Airline Safety Performance. Unpublished PhD thesis, 2005. University of Texas. Unpublished PhD thesis. University of Texas. Read part one and part three of Norman's blog series.

In a three-part series of blogs for the hub, Norman Macleod explores how systems behave and how the actions of humans and organisations increase risk.  He argues that, to measure safety, we need to understand the creation of risk. In this first blog, Norman looks at the problems of measuring safety, using an example from aviation to illustrate his points. In the final paragraph of his seminal 2005 paper, 'Evaluating the Quality of Medical Care',[1] Donabedian suggests that instead of asking "What is wrong: and how can we make it better?" we should, more often, ask "What goes on here?" The author identifies three areas of enquiry: process, outcomes and structure. He also recognises that care episodes are not discrete: instead, they form chains of events involving multiple actors. The issues raised in the paper apply equally to the problem of measuring safety. Vincent, Burnett and Carthey[2] offer a definition of patient safety as: "The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare." The authors also suggest that quality deals with the intended results of the healthcare system whereas safety looks at the ways the system can fail to function. Leveson, though, observes that, in engineering, reliability is not the same as safety: and we could substitute quality for reliability.[3] Safety has been described as a "dynamic non-event" (Weick) in that it is "an ongoing condition in which problems are momentarily under control […]".[4] Implicit in this position is that the absence of failure does not mean that an entity is safe. Another view is that safety is the "freedom from [a level of] risk which is not tolerable".[5] These approaches shift the focus from outcomes to the domain of structure and how it shapes processes. This suggests that measures of safety should address the issue of ‘control’ in the workplace. We particularly want to understand the distribution of risk and how it becomes ‘intolerable.’ Understanding ‘What goes on here?’ A patient entering the healthcare system experiences episodes of care, each of which is intended to remediate the patient’s condition in some way. Despite being highly proceduralised, the inherent variability in each patient requires treatment to be adaptive because, in short, no two patients are the same. Equally, the condition of the healthcare worker introduces variability. As a result, there are multiple pathways that can lead to the same safe outcome. The range of different ways an episode can unfold can be described as ‘buffering’: the system has the capacity to cope with variability and still function as intended. Unfortunately, each variation in the delivery of a specific episode carries with it a degree of risk, which is often not apparent unless something goes wrong. Occasionally activity will exceed the system’s buffering capacity. We can hypothesis a point where a process transitions from safe to unsafe: the resources available to restore the process to a safe state have been exhausted. We are particularly interested in how systems behave in these boundary states. Finally, we want to know how a system fails. Is the outcome inconsequential, recoverable but with additional intervention, or catastrophic? A system’s response to failure can be described as its tolerance. These concepts are illustrated using output from an aircraft’s digital flight data recorder (DFDR): Figure 1: Li L. CityU, Hong Kong. Personal communication. The graph depicts an aircraft during the final approach.[6] Approaching the runway, the pilot lifts the nose to stop the rate of descent. Power is reduced, the aircraft settles on the runway and the nose is lowered again. This change in attitude is recorded in flight data as the pitch angle. The graph shows the pitch angle of 300 aircraft during the final mile of the approach to touchdown and then shows the aircraft on the runway and slowing down. The dark blue band shows the central 50% of data points, those closest to the planned approach path, with the outer, lighter bands showing 20% either side (some data is lost in the processing). All these approaches were successful and the data shows the range of solutions to the problem of attitude control on final approach: the buffering. Airline safety management systems are required to track parameters out of tolerance and the chart shows the angle that would trigger a Flight Data Monitoring (FDM) alert. We can see the gap between ‘normal’ and what would trigger a safety alert. Put another way, it shows how close the system is operating to a safety trigger but without knowing it. The graph reveals the ‘what goes on here’ that would normally be invisible. The red line on the graph is the data for a specific flight that did result in an investigation. The outcome was a ‘hard landing’. Hard landings can trigger a mandatory maintenance inspection (lost productivity while the aircraft is being checked), damage to the aircraft structure and even a collapsed undercarriage. These are the outcomes that could arise from the same initial problem. The result, in this case benign, illustrates the tolerance in the system. Conclusion To measure safety we, first, need to understand performance variability (buffering), behaviour at the boundaries (opportunities to recover) and tolerance (how failure propagates). Having said that measures of outcome are not useful indicators of safety, the first problem we face is that safety reflects performance in a space that is not easily open to inspection. If that is the case, then we need to look for surrogates that can reliably stand in for direct measures of safety. In part 2 of this blog, I will look at how error may offer insight into system’s behaviour. I would love to hear your feedback on this blog and how you 'measure safety'. Please add your comments below (you will need to be a hub member and signed into the hub to comment). References Donabedian A. Evaluating the Quality of Medical Care. The Milbank Quarterly 2005; 83 (4):691-729. Vincent C, Burnett S, Carthey J. The Measure and Monitoring of Safety. The Health Foundation Spotlight, 2013. Leveson N. Engineering a Safer World. MIT Press. 2011. DOI: https://doi.org/10.7551/mitpress/8179.001.0001 Weick KE. Organizational culture as a source of high reliability. California Management Review 1987: 29 (2): 112-128. Li L. CityU, Hong Kong. Personal communication. Read part two and part three of Norman's blogs. Further blogs from Norman: What is a ‘safety management system’? Error isn’t a problem – the problem is the word ‘error’

A ‘Just Culture’ aims to improve patient safety by looking at the organisational and individual factors that contribute to incidents. It encourages people to speak up about their errors and mistakes so that action can be taken to prevent those errors from being repeated.  Adam Tasker and Julia Jones are graduate medical students at Warwick Medical School. They wanted to explore doctors’ perceptions of culture and identify ways to foster a Just Culture, so they conducted a qualitative research study at one of the hospitals where they were doing their medical training. We asked them about why Just Culture is important in the health and care system, and what they discovered from their research.

This free webinar will uncover the intricacies of accident investigation from a human factors perspective. It will feature examples from rail, air and maritime from our speakers who are all specialist human factors investigators. Hear first hand how they tackle investigations and get insights into this vital work that lead to improvements in safety across all travel sectors. Will Tutton will briefly mention the Herald of Free Enterprise, but will mainly talk about the cargo vessel Kaami, which ran aground in Scotland in March 2020. The investigation focused on front line operators. Lisa Fitzsimons will talk about common themes relating to human performance and organisational factors which emerge when investigating the technical aspects of an air accident, drawing upon several recent examples. Becky Charles will discuss track worker safety and specifically about an incident which occurred at Margam, UK in July 2019 where two trackworkers were struck and fatally injured. Register

This free webinar will explore near misses in three different sectors and how controls can, or cannot, be developed to prevent future events. It will start with an introduction to the concept of near misses in healthcare and the challenges faced in learning from these near misses to improve safety. You will then hear how near misses are approached in rail and nuclear and how controls are developed in their processes. At this event, you’ll: Gain valuable insights from all three sectors: healthcare, rail and nuclear. Hear discussion about defining near misses with respect to controls. Learn how to build barriers in systems. Who will this be of interest to? This webinar will be of interest to anyone involved in the management of safety events in their industry/ organisation, and especially human factors practitioners, safety investigators, policy leads and regulators. Register

Join the Airport Training Experts Miroslav SPAK and Frederic Rooseleer for a 90 minutes live training session. Enhanced operational efficiency and sustainability can be achieved by optimising the current operations through implementation of advanced solutions maximizing utilisation of the airport capacity. These solutions have been initially developed and validated under SESAR, and recently packaged by EUROCONTROL to cover Runway Performance, Surface Management and Total Airport Management. This webinar will provide information on the total airport management building blocks and also a review of the key runway performance solutions available for deployment as well as their benefits, supporting implementation needs and reference material. Register