Designing for Human Failures

As I have said countless times, we ALL make errors.  This means in situations where those errors carry more severe consequences, such as industrial workplaces, then as safety engineers, it is our role to help the business DESIGN SAFE work tasks.  In HF engineering, we have eight (8) system design principles:

  1. Accurate mental models
  2. Managing information
  3. Reducing complexity
  4. Visibility
  5. Constraining behavior
  6. Design for errors
  7. Standardization
  8. User-centered design

A good work process should not allow people to make mistakes easily. This may sound obvious, but all too commonly, system design is carried out without feedback from its potential users, which increases the chance that the users will not be able to interact correctly with the work tasks. Following HF engineering design principles can minimize the potential for these human errors.

Accurate mental models

There is often a discrepancy between the state of a system and the user’s mental model of it. This common cause of erroneous behavior arises because the user’s model of the system and the system itself will differ to some extent since the user is rarely the work task designer. Problems that can arise due to this discrepancy are illustrated by the Three Mile Island incident. In this incident, the system had been designed to display whether the valves had been instructed to be open or closed. The most obvious interpretation to the user was that the display reflected the actual status of the system. Designers need to exploit the natural mappings between the system and the expectations and intentions of the user. 

Another example of the importance of user familiarity with the work task is demonstrated by a laboratory study that examined how useful it was to give staff an overview of a fictitious petrochemical plant’s structure and day-to-day functioning. One group was given rules about which buttons to press if a dangerous situation arose; another was given the rules and an overview of the workings of the plant. Both groups were equal in their ability to deal with the expected problems. Still, when new problems arose, only the group which understood the plant’s functioning could deal with the situation.

Managing information

As our brains are easily distracted and can overlook necessary tasks, it makes sense to put information in the environment to help us carry out complex tasks. For example, the omission of steps in maintenance tasks is cited as a substantial cause of nuclear power plant incidents.  When under time pressure, technicians will likely forget to replace nuts and bolts. A very simple solution to this problem would be to require technicians to carry a hand-held computer with an interactive maintenance checklist which specifically required the technician to acknowledge that certain stages of the job had been completed. It could also provide information on task specifications if necessary. This would also allow a reduction in paperwork and hence time pressure.

Reducing complexity

Making the structure of tasks as simple as possible can avoid overloading the psychological processes—the more complex the task specifications, the more chances for human error. Healthcare systems in the US are currently addressing this issue. With the realization that a leading cause of medical error in the United States was related to errors in prescribing drugs, a program was undertaken to analyze and address the root causes of the problem. A computerized drug selection and bar-coding system reduced the load on memory and knowledge on the part of the prescriber, and errors of interpretation on the part of the dispenser, resulting in an overall reduction in prescription errors. Examples such as this emphasize the fact that reducing task complexity reduces the chance of accidents.

Visibility

The user must be able to perceive what actions are possible in a system and, furthermore, what actions are desirable. This reduces demands on mental resources in choosing between a range of possible actions. Perhaps even more important is sound quality feedback, which allows users to judge how effective their actions have been and what new state the system is in as a result of those actions. An example of poor feedback occurred during the Three Mile Island incident; a poorly-designed temperature gauge was consistently misread by experienced operators (they read 285 degrees Fahrenheit as 235 degrees), which led them to underestimate the severity of the situation.

Constraining behavior

If a system could prevent a user from performing any action which could be dangerous, then no accidents would occur. However, the real world offers too complex an environment for such a simplistic solution. In an industrial operation, a procedure that could be beneficial at one stage in the process may be disastrous at another. Nevertheless, it is possible to reduce human error by carefully applying “forcing functions.” A good example of a forcing function is found in the design of early cash machines. People used to insert their cards, request cash, take it, and walk away, leaving their cash cards behind. It was a natural enough response, as the main objective of the action had been achieved: obtaining money. The task was thus mentally marked as complete before all necessary transaction stages were carried out. After a great deal of thought, the systems designers came up with a very simple solution that has been effective ever since: as the target objective of the task was to obtain money, placing this stage at the very end of the transaction would avoid the problem. Hence, the card is now given back before the money is. Functions such as this relieve the user of the responsibility of deciding what actions are appropriate whilst interacting with the system and are very effective in preventing dangerous incidents.

Design for errors

In safety-critical systems, such as industrial plants, numerous safety systems are in place which can mitigate accidents. One approach is “defense in depth” (implementing many independent systems simultaneously); another is “Fail-safe” system design. However, designers must assume that mistakes will occur, and so any useful system must make provision for recovery from these errors. Another consideration is that the design should make it difficult to enact non-reversible actions.

Although this is an underlying design principle, it must be applied carefully. For instance, most home computers have a “recycle bin” or “trash folder”, in which all deleted files are stored. They are recoverable from here, but files cannot be recovered when this folder is emptied. Attempts to empty this folder result in a message asking the user to confirm the deletion. The problem is that the user is often asked to confirm such requests and, just like the train drivers with the AWS system, learns to associate the appearance of the warning message with the pressing of the ‘OK’ button. The result is that the pop-up messages may not be read, and files are occasionally accidentally destroyed. A safer option would be to use this type of pop-up box less regularly and to require different user input each time.

Standardization

When systems are necessarily complex but have been made as accessible and easy to use as possible, and errors are still being made, standardization is sometimes used to make the situation predictable. It has been suggested that medicine is one of the areas most amenable to standardization. For instance, resuscitation units in emergency rooms vary considerably in their design and operation. This diversity, coupled with the movement of staff between hospitals, means that errors can be made and life-threatening delays can occur. Another example where standardization might be of use in medicine is across different equipment brands since staff often do not have training in all the available designs. If all hospital equipment had standard placement and design, then all staff could locate and operate equipment efficiently.

One problem with standardization is that if any advances in design or usage are made, it is very costly to re-implement standardization across all affected personnel/locations. Also, a standardized system may be ideal for one set of tasks but very inefficient for another set. Such practical considerations have tended to limit the application of standardization as an approach for reducing human errors.

User-centered design

Another fundamental principle of design is that it should be centered around the user at all stages, from initial conception, evolution, and testing to implementation. In practice, however, systems designers are often given a brief, create the system and impose it upon the users without appropriate feedback. This can result in unexpected behavior and over-reliance on manuals that have been written by the system designers from their own perspective. Systems designed in this way will be opaque to the end user, which can hinder effective interaction. Designers of computer interfaces often fall into this trap.

 

 

Source: https://www.hse.gov.uk/humanfactors/resources/risk-assessment.htm

 

Scroll to Top