Keywords

1 Introduction to Human Factors Engineering

Human factors engineering is a well-established scientific discipline that studies the functional capabilities and limitations of humans in order to design and optimize systems, processes and technology to reliably obtain a desired outcome (Lee et al. 2017). It incorporates principles and methods from disciplines such as industrial systems engineering, cognitive psychology, and computer science to analyze and model human-system interactions and to support system designs which meet quantifiable needs of the users and which support work in ways that are effective, efficient, and safe.

Human factors engineering has had a major influence on the design of systems and workflows in a wide range of safety critical industries including nuclear power, military and defense, and aviation. By understanding human capabilities, limitations, and common pathways for error, systems can be designed to prevent errors and—importantly—mitigate their effects, thus reducing harm to users and others who may be affected. In health care, the benefits of human factors engineering design approach extend to keeping patients free from error-based harm, to improving care through more efficient and effective workflows, to protecting staff members from fatigue and injury. Human factors engineering is particularly important in the successful integration of new technology into an existing work system. Recent examples include the use of drones in military and civilian applications and the emergence of self-driving cars that may share the road with human-driven cars. In each case human factors methods and principles are being applied to analyze the implications for the humans in the system, and to design effective user interfaces and work flows to enhance overall safety of operations (Casner et al. 2016; Roth and Pritchett 2018).

An important strength of human factors engineering is the focus on a broader context within which a system workflow or device operates (Carayon 2006). This includes describing specific physical, cognitive, and perceptual capabilities and limitations of the populations of system users involved; understanding and formally modeling the purposes and tasks being performed; mapping task requirements to human-system capabilities; and considering relevant aspects of the physical environment and work situations in which the system will be deployed. For example, a portable intravenous pump undergoing testing may work flawlessly in a simulated environment with experienced intensive care unit nurses, however that same pump may present significant hazard when an alarm goes off at home with a patient that misplaced their reading glasses.

This chapter introduces core concepts and methods from the discipline of human factors and describes how they can be applied to the study and improvement of clinical workflow. We begin by presenting a set of core human factors concepts (or human factors ‘lenses’) that are important to adopt when trying to identify sources of problems and opportunities for improvement to healthcare-related workflows. This is followed by description of specific human factors methods that can be used to analyze and improve workflow.

2 Applying Human Factors Lenses to Workflow Analysis and Design

When considering the application of human factors to the healthcare environment including health IT systems it is important to have a context within which to work. The following core Human Factors concepts and theoretical perspectives will aid the reader in applying a human factors lens when analyzing or trying to identify improvements to specific workflows and situations. These include situations where healthcare organizations may be trying to understand the factors that are contributing to performance problems or errors and how they can be mitigated; as well as situations where organizations are trying to develop and/or introduce new health IT and monitor and manage its impact on performance and satisfaction. There can be many points where there is value in adopting a ‘human factors lens’—early in the process when requirements for a health IT system are being defined, during design in determining whether the system being developed will work as imagined, and after implementation, to understand and address human performance problems that emerge (e.g., near misses, adverse events, productivity bottlenecks).

2.1 Supporting ‘Work as Done’ Versus ‘Work as Imagined’

A core precept of human factors is that it is important to begin any analysis or design project by studying how work is actually done, in all its messiness. Too often there is a significant gap between the way in which leaders believe the work is performed at the front line, and the way in which it actually occurs. Some authors refer to this as the different between ‘work as imagined’ and ‘work as done’ (Hollnagel et al. 2013; Braithwaite et al. 2017). Clinical work is fundamentally collaborative, involving multitasking, frequent interruptions, time-pressure, and incomplete, ambiguous, time-lagged information. Problems arise when there is a disconnect between the realities of the work ‘as done’ and the assumptions underlying the Health IT system (i.e., work as imagined). A case in point is decision-support tools where the implicit assumption is often that of a single decision-maker deciding at a particular point in time, with the all the information in hand. This contrasts with the demands of actual work practice, with the result that such tools are less likely to be adopted in real clinical settings (Wears and Berg 2005).

The rapid adoption of electronic health records in the United States since the Health Information Technology for Economic and Clinical Health Act (HITECH) in 2009 has introduced technology with variable degrees of success and unintended consequences (Bernstam et al. 2010). Often problems arise because of a mismatch between the implicit model of the work inherent in the HIT and the actual complexities of the clinical work environment. As Wears and Berg (2005) put it, the problem is not one of ‘not developing the systems right’ but rather of ‘not developing the right systems’.

Good design and implementation needs to go beyond a narrow focus on the technology to be implemented. A sociotechnical lens is required that includes examining the characteristics of the organization to be supported (the people, values, norms, and culture), the technical environment in which the new system is to be inserted (the equipment, processes, procedures, and physical facilities), and the work demands and complexities that healthcare practitioners face. Only through this type of broad perspective will the gap between work as imagined and work as done be narrowed.

2.2 Addressing Context Independent vs. Context Dependent Design Elements

One of the significant challenges when introducing technology into any complex environment is addressing both its usability and usefulness. Usability is defined as how intuitive a tool is, how easy it is to learn and to use by the intended user. In contrast, usefulness refers to the extent to which the device, technology or workflow provides meaningful improvement in performance by the intended user under anticipated working conditions. To highlight the differences between usability and usefulness, one can imagine a new application within the electronic health record (EHR) is tested in a lab and found to be intuitive to use with few errors by the user (usability). But when used by nurses in the emergency department who are frequently interrupted and multi-tasking across many patients the application becomes burdensome to enter data and found to have limited usefulness in the clinical environment due to missing critical information from other parts of the EHR.

Usability is generally affected by context independent features of a design often framed as design “heuristics” including making system status visible, providing meaningful and rapid feedback, maintaining consistency in indications and actions, using language and labels, and supporting error recovery (Nielsen 1995). These design principles are largely independent of the content and context of the interface or device being investigated. A recent human factors review of electronic medical record and electronic health record systems found that there were extensive usability issues (Zahabi et al. 2015). These authors noted that these often resulted from a lack of application of standard human factors usability guidelines including: using simple natural dialogue, speaking the user’s language, minimizing memory load; providing feedback and good error messages; maintaining consistency in design and error prevention (Molich and Nielsen 1990).

Human factors engineering provides extensive guidelines for identifying and correcting these ‘context-independent’ aspects of design. There are well established rules and guidelines that have been agreed upon for decades in the human factors and associated literature regardless of the application, from medical device to electronic airplane dashboard. For example, yellow text on a white background provides less contrast than black text and will be more difficult for the user to interpret. In addition, providing a list of choices on a display that are only separated by one pixel is more likely to lead a user to make an accidental selection if they are distracted or slip. While the rush to implement health IT systems may not followed many of these guidelines, the incorporation of User-Centered Design principles and human factors engineers in the design and certification of EHRs in the United States has begun to standardize the approach and remove these basic design errors that can lead to patient harm (Tolley et al. 2018).

In contrast to usability, usefulness of a health IT system is based on context -dependent design considerations that rely on an understanding of the purposes of system implementation, user goals, and context of use (Hettinger et al. 2015). For example, when placing an electronic order for a patient, providers frequently need to refer to previous laboratory values to make the most appropriate choice. A well designed computerized provider order entry (CPOE) system would not only allow the user to view previous orders while placing a new order, but may make specific values more salient based on the current order selections. For example, a radiology test with intravenous contrast requires normal kidney function to prevent serious adverse events. Relying on the provider to remember the results of prior tests of kidney function or requiring them to navigate away from the ordering screen and potentially get distracted on another task will lead to the predictable error of ordering the wrong test or a delay in care. It would be preferable to display prior kidney function values on the screen used to order radiology tests.

Context dependent design is much more challenging and requires in-depth study of the users and their workflow in the environment where the work will be performed. This entails anticipating the needs of the users based on the context of use and making it easier for users to make the correct decision or action. Effective design requires consideration of both context independent and context-dependent aspects, and an interactive process that allows both usability and usefulness in context to be assessed.

2.3 Engineering for Resilience

Resilience Engineering offers a complementary human factors lens through which to examine clinical workflow (Fairbanks et al. 2014). Instead of focusing on the rare errors and failure modes, it encourages examining the adaptive behavior of individuals in the everyday context that keep things from going wrong, and how these behaviors can be better supported and more widely adopted (Braithwaite et al. 2015).

The basic premise of Resilience Engineering is that healthcare is a very complex process that presents multiple challenges. The different policies, procedures, patients, staff members and other various components interact in such a manner that there are often unanticipated outcomes when trying to change clinical workflow and that no one individual in the system has a clear understanding of all the components and how they interact with each other. However, humans are incredibly adaptable and often serve to hold the system together. For example, if a particular component in the system is not working correctly, e.g. the CT scanner stops working, then it is the humans that will develop the work arounds to get other testing, transfer patients to a facility that has the necessary equipment or delay the testing in those patients that have less time sensitive conditions until the equipment is working again. Without humans, the brittle interconnected system of electronic orders and medical equipment would grind to a halt until the equipment could be repaired, causing potential serious delays in acutely ill patients.

Resilience Engineering seeks to learn from the positive everyday behaviors of the humans in the system that keep the system going and prevent harm. In effect, instead of focusing only on the rare cases of errors and system breakdowns, it asks why more errors aren’t happening and what can be done through better designs and workflows to enhance positive behaviors across users and not just the individuals that are anticipating the hazards through previous experience and institutional knowledge (Braithwaite et al. 2015).

2.4 Guiding the Co-evolution of Technology and Work Practice

A core Human Factors precept with extensive empirical support is that when new technology is introduced it inevitably changes work practice, sometimes in unanticipated ways. People adapt to the new health IT and learn to use it in ways that were not necessarily envisioned by the system developers. These new and unanticipated uses can in turn trigger a need for new technology development. This dynamic cycle of technology development and user adaptation has been referred to as the task-artifact cycle, to emphasize that how tasks are performed and the artifacts that support them co-evolve over time (Carroll and Rosson 1992; Carroll and Campbell 1998). This implies a need to continue to track the impact of a new health IT system after it is introduced to identify emerging practices and changing needs.

New technology cannot simply be ‘dropped’ into a work context. Rather, its impacts on the larger work context and organization needs to be tracked and unanticipated reverberations need to be recognized and addressed (Woods 2002). As Wears and Berg (2005) noted, the introduction of new health IT cannot be thought of in isolation, but rather as part of the larger context of organizational change. This includes recognizing that there will be a period of exploration and mutual learning involving users and system developers (Wears and Berg 2005). New workflows will emerge and additional support needs will be identified. This in turn will trigger new design cycles—be it through changes in training, workflow or design changes to the IT system. For example, the patient tracking boards (i.e., dry erase white boards) in emergency departments (EDs) originally were developed independently across organizations by the front line users. For example research by Bisantz et al. (2010) noted that with the transition to electronic information systems (EDIS) that attending physician workflow with resident physicians and students was no longer supported. Specifically, the method by which case presentation, attending exam and final note had been tracked on the dry-erase board with a series of colors and symbols was no longer supported (Bisantz et al. 2010). Attending physicians adapted by using paper notes kept in the pocket to track this information (new ‘home grown’ artifact). Because the information was no longer publicly displayed, residents and nurses were not able to maintain awareness of where the attending physician was in their workflow. An unintended consequence was that patients were sometimes discharged before the attending physician evaluation and plan was complete. This task-artifact loop spurred EHR design changes. More recent EHRs used in clinical practice have been observed using these findings to incorporate the tracking of resident/attending workflow and note status in a more comprehensive manner.

2.5 Adopting a Patient Safety Transformational (PST) Prevention Model

Human factors approaches are intended to anticipate and prevent or mitigate the use errors before they can occur and cause potential harm. This is analogous to the patient safety transformational (PSF) model that has been used in cardiovascular care. The PST model distinguishes primary prevention—prevention before the hazard occurs; secondary prevention—prevention after the hazard occurs but before the patient is harmed; and tertiary prevention—prevention after the harm event has occurred but during the critical time that an intervention could improve a patient’s outcome. The aim is to design for primary prevention whenever possible, followed by secondary, and then tertiary prevention.

Cardiovascular care for patients has undergone major changes since the 1950s when researchers were just starting to understand the link between heart disease and risk factors that we now take for granted like diabetes, hypertension and hypercholesterolemia (Dawber et al. 1951). As a result of this improved depth of understanding and new methods for diagnostic testing, medicine went from a model of waiting for patients to have heart attacks to actively trying to prevent cardiovascular disease through life style modification (primary prevention) and aggressive management of chronic disease (secondary prevention). While there is still significant effort in tertiary prevention, reducing the long term impact of the heart attack once it occurs through rapid cardiac angioplasty and bypass surgery, there is considerable effort to prevent the patient from ever needing those dramatic efforts.

In stark contrast to changes made in cardiovascular disease, healthcare safety and operations often focus on the critical events that demonstrate breakdowns and try to improve their systems from one adverse event to the next. Using processes like Root Cause Analysis (RCA) often lead to brief analysis of adverse events that culminate in short term fixes such as disciplining those involved and training the other team members to vigilant instead of implementing sustainable and effective changes to the clinical workflow of the front-line staff (Hettinger et al. 2013). By taking a similar primary/secondary/tertiary prevention approach as that taken in cardiovascular care, the hazard under investigation may be designed out of the system. For example, a surgical department investigates a retained piece of medical equipment despite performing a surgical count of equipment and a post-operative x-ray at the completion of the case. In an effort to prevent future cases the organization decides to apply the PST prevention concept instead of a traditional model of referring the involved staff to their respective peer review committees and sending a memo to staff to be more vigilant. They find multiple pieces of equipment and disposables that are not visible on x-ray and develop a plan to replace them, removing them from circulation in the operating rooms (primary prevention). Furthermore, they investigate technology that will allow wireless scanning and counting of surgical equipment to remove a foreign body before the end of surgery (secondary prevention). Finally, after reviewing clinical data they determine that most retained foreign body cases are in surgical cases that are either long duration or complex with many pieces of equipment. They develop a clinical workflow so that these cases are pre-operatively identified as high risk and streamline a process for getting post-operative x-rays looking for foreign bodies before the patient leaves the operating room (tertiary prevention).

The PST prevention model can be embraced in the health IT system development process, before any adverse event has occurred. For example, the use of robust user centered design processes during the formative development period is likely to prevent many hazards from making it into the system (primary prevention) or catch the hazards during usability testing with representative end-users (secondary prevention). The use of EHR safety surveillance during the post implementation period for health IT system can then catch hazard and harm events where the contribution of the health IT system may be unrecognized (tertiary prevention). One of the benefits of human factors approaches is that it provides methods to catch and correct problems during different phases of design and implementation—before there is opportunity for harm. Without designing for primary, secondary and tertiary prevention in clinical workflow, individual healthcare providers are destined to make the same errors over and over again.

3 Human Factors Methods for Analyzing and Improving Workflow

Evaluating, designing andoptimizing clinical workflow is a critical part of providing safe and effective care to patients. The section above presented some core human factors concepts that are intended to provide guiding perspectives when trying to identify sources of problems and opportunities for improvement to healthcare-related workflow. A common thread across the multiple lenses presented is the need to understand the broader context of work, the complexities that can arise, and the cognitive and collaborative demands they impose, when trying to understand or improve workflow. This includes cases where an organization is trying to understand why problems or errors are occurring and develop mitigations. As well as cases where an organization is trying to design new health IT or insert new systems developed by vendors so as to improve performance.

In this section we provide brief descriptions of some core human factors methods that can be used to analyze the context of work and the impact of new technologies on work. These include methods that can be used early in the analysis process when one is trying to understand sources of performance problems and define requirements for more effective support, methods that can be used during design when a team is trying to determine whether the health IT system being developed will work as imagined, and methods that can be used after a system is implemented to understand and address human performance problems that are identified (e.g., near misses, adverse events, productivity bottlenecks). As we introduce each method we will highlight the types of analyses and stages of technology design and introduction for which they are best suited. We will also briefly describe their strength and limitations.

The review of human factors methods provided below is necessarily selective. We focus on methods for uncovering information about workflow and the context of work, particularly the cognitive and collaborative demands of work that can lead to performance problems, as well methods for evaluating and guiding the design new HIT systems as part of the development cycle. Broader surveys of human factors methods and more in-depth descriptions of the methods described below can be found in the literature (Bisantz et al. 2015; Bisantz and Roth 2008; Hettinger et al. 2017; Lee et al. 2013; Lowry et al. 2014; Stanton et al. 2017).

It is important to note for the reader that while each of the methods are covered individually below, in practice researchers will use a combination of methods to obtain a richer picture of the workflow of interest and the broader context in which it is imbedded than would be possible with any single method. For example researchers will often combine interviews and focus groups with observational studies (Militello et al. 2014) as well as with artifact analysis (Xiao et al. 2010).

These methods can be effectively used by multiple types of organizations and stake-holders and tailored to the scope, size, and budget of the project. This includes technology vendors who may be trying to develop and upgrade health IT systems for applications across multiple hospitals, clinical organizations (e.g. ambulatory clinics, hospitals, larger healthcare systems) that might be trying to roll-out and manage new health IT systems to minimize error, and improve performance, satisfaction and safety, as well as individual healthcare researchers or leaders who may be trying to examine sources of problems or errors and identify appropriate solutions.

3.1 Interviews and Focus Groups

Interviews and focus groups are among the most common methods for learning about workflow and obstacles to effective performance (Bisantz et al. 2015). They are particularly useful during the early stage of information gathering to get an overview of the ideal workflow and obtain multiple perspectives on challenges and barriers to effective performance that may result in a disconnect between work as imagined and work as practiced. Interviews and focus groups can also assist in tertiary prevention when analyzing an adverse event that has occurred and safety experts are attempting to assess the severity of hazard for future patients and the potential frequency with which they may occur.

Interviews using human factors methodologies frequently employ a semi-structured format to ensure that key topics (e.g., previously identified key pieces of a workflow or known work-arounds) are discussed, while remaining flexible enough for the interviewer to discover new information and allow the participant to guide the discussion based on their experience with the process, system and culture. This facilitates learning the true work as performed versus work as imagined discussed previously. As one example, McDonald et al. used a semi-structured interview approach to map the clinical workflow for high-risk patient monitoring at five specialty clinics (pulmonary medicine, breast cancer, gastroenterology, urology and otolaryngology). Based on the interviews they were able to identify (1) the steps that were most critical, time-intensive, and risky from a patient-safety perspective; (2) critical data elements needed for effective monitoring of high-risk patients; and (3) candidate technical and organizational interventions to address the identified workflow vulnerabilities (McDonald et al. 2017).

Focus groups also employ semi-structured interview questions but allow the participants to clarify and build upon each other’s comments, enabling a richer, more nuanced, construction of the workflow. A critical decision is whether to mix individuals from different backgrounds (e.g., different job positions; experience levels; status in the organization) in one focus group. An important consideration is to ensure that everyone feels free to express themselves openly. One example where this concern came up is in a focus group conducted seeking to understand communication patterns between nurses and physicians (Benda et al. 2017). In this study, separate focus groups with nurses, residents and attending physicians were chosen because of anticipation of different perspectives based on both roles and experience level between and among nurses and physicians. Indeed during focus group interviews residents and attending physicians expressed very different views. Attending physicians were more likely to discuss the importance of two-way communication and listening to nurses as their eyes and ears within the ED. In turn nurses talked about strategies for guiding less experienced residents, given the formal hierarchy relationship.

Interviews and focus groups, in general, require less expertise and time to conduct than some of the following methods. However, lack of appropriate preparation for both techniques are likely to result in less helpful data collected. Further, focus groups often require two moderators—one to conduct the focus group and one to record the discussions. The use of audio and/or video recording devices can help reduce the number of personnel used but require a significant amount of resources to turn the recordings into usable data. Audio/video recordings can also negatively impact the participant’s willingness to share more controversial views and observations.

3.2 Critical Decision Method

One of the most powerful methods for learning about the demands in the environment and the strategies that people have developed for coping with them is to ask them to describe a specific past challenging situation they personally experienced and how they handled it (Flanagan 1954). The critical decision method (CDM) is a widely used structured interview technique that builds on this approach (Klein et al. 1989). It was initially developed to understand the decision making process of firefighters when making rapid decisions with limited access to information that could have life-threatening consequences. It consists of a trained individual in the method conducting a structured interview with a single participant, typically an expert in the workflow under consideration. The method involves having the individual go through the incident in progressively deeper passes to understand the decisions that were made, the information that was used and alternative events that could have occurred and how they were avoided (Crandall et al. 2006).

CDM has been used in multiple high-risk settings, including urban and wild land firefighting, military command and control, and software engineering. It has been extensively used in health care, including to study the perceptual cues used by experienced neonatal intensive care unit nurses; (Crandall and Getchell-Reiter 1993) and to compare the strategies employed by physicians of different levels of expertise for early recognition of sepsis (Patterson et al. 2016). The results have been used to propose improvements to workflow, new forms of decision-support, and new training.

More recently a variant of CDM has been developed as a means to identify resilient behavior and workflows by healthcare providers. For example, Hegde and colleagues are developing a lesson-sharing tool called Resilience Engineering Tool to Improve Patient Safety (RETIPS) based on CDM interviews of nurses and physicians that focus on examples of resilient behavior (Hegde et al. 2014, 2015). The intent was to collect a corpus of cases that demonstrate how people adapt in everyday clinical work to perform effectively and avoid harm to patients under challenging conditions as a means of generating safety lessons.

While CDM is powerful method for collecting information on workflow challenges and the adaptive strategies that individuals develop in response, it has some limitations. In particular it requires significant training and expertise to conduct CDM interviews. Often CDM interviews are conducted by trained human factors consultants and there are short-courses offered in the methodology. In addition there have been efforts to adapt the methodology to on-line questionnaires (Hegde and Jackson 2017).

3.3 Observations

One of the most useful human factors techniques for studying workflow is to conduct observations in the actual work context or in a close analogue such as a high fidelity simulator (Roth and Patterson 2000). Observing individuals and teams working in their work environment allows the analyst to document the range of complexities that arise that challenge work flow and the various adaptations and work arounds that individuals have developed to cope with demands, overcome obstacles, fill in gaps and otherwise contribute to the overall safety of the system (or not).

Observational studies involve having one or more observers unobtrusively shadow individuals as they go about their work. The goal is to observe the activities and communications that occur without getting in the way, serving as a source of distraction, or otherwise influencing the behavior of the individuals being observed. The observer typically records their observations in real time either in free form or using a predefined set of coding categories (Bisantz et al. 2015). These are then analyzed after the fact using qualitative grounded theory methods and/or quantitative methods (e.g., recording and analyzing the frequency of different types of occurrences).

Often the observational team will include a behavioral scientist (e.g., a human factors specialist) with knowledge and skill in observational methods, and a second individual with knowledge and expertise in the domain of practice being observed (e.g., a physician or a nurse in studies of health care environments). For example, a study examining workflow challenges in complex surgeries had a two-person observation team in the operating room that included a practicing surgeon and a human factors specialist (Christian et al. 2006). The surgeon could draw on their surgical knowledge to interpret what was observed while the human factors specialist could draw on their cross-domain knowledge of human performance drivers and systems challenges to point to patterns of behavior and systems problems whose significance might not be recognized by the surgeon. Both took notes in real-time during the surgery being observed which were then combined to obtain a more complete and accurate description of what took place.

Whenever feasible, observations are coupled with opportunistic interviews that occur during periods of low workload or at the end of a shift. This allows for the subject to answer clarifying questions or provide elaborations or confirmations of what was observed without interfering with the work. In some cases, if the environment allows, the sessions are audio or video recorded for later review and analysis. For example, a study examining inter-operative deviations in care had video-recordings made of ten high acuity operations. These were then transcribed and analyzed by a multidisciplinary team consisting of surgeons and human factors specialists (Hu et al. 2012). This resulted in more complete data capture than would be possible when relying solely on real-time observations. In another study, Tiferes et al. used video- and audio-recordings of robotic assisted surgeries to code and characterize verbal and non-verbal communication among members of the surgical team (Tiferes et al. 2018).

Observational studies are useful early in an investigation when trying to understand the work as actually done (as opposed to the work as imagined). This includes situations where human performance problems have been identified and there is a need to understand why they are occurring and what can be done to reduce the problem. One good example was an observational study that was conducted to understand the ‘counting protocol’ used by nurses to keep track of surgical objects (needles, sponges, instruments) during operations in order to reduce the risk of leaving a foreign object in the patient (Dierks et al. 2004). Hospital leadership wanted to understand why surgical objects were sometimes left in patients in spite of having the counting protocol. The observational study showed that the counting protocol was difficult to perform and documented multiple factors that contributed to challenges in maintaining an accurate count (e.g., incomplete surgical kits; shift changes in the middle of surgery; differences in counting conventions across nurses). Further it showed that the counting protocol itself had unanticipated negative consequences that in some cases compromised patient safety. Complications in the count, which occurred in six of the nine observed surgeries, triggered activities to reconcile the source of the inconsistency. This drew attention away from the ongoing surgery, resulting in delays and additional risk to the patient. The study led to numerous recommendations for improving performance ranging from increasing standardization to eliminating the count through use of new technologies for keeping track of surgical objects.

Observational studies are also useful after a new system is put in place to understand the impact of the new system on practitioner workflow. This includes tracking whether the system is being used in the manner envisioned by the developers, whether it is having the positive effects anticipated, and whether any new issues are emerging. For example, an observational study was conducted to understand use of Electronic Health Record (EHR) systems in primary care outpatient clinics (Flanagan et al. 2013). The study identified mismatches between the EHR system designs and the demands of outpatient settings that led to a variety of work arounds (some paper-based and some computer-based) intended to improve efficiency and support memory and awareness of the healthcare practitioners. These pointed to limitations of the EHRs that contributed to their lack of use and opportunities for improvement. Another study examined the impact of the introduction of EHRs on nurse physician verbal communication in emergency departments (Benda et al. 2017). The goal was to understand the content and pattern of physician-nurse communication given the availability of EHRs. Among other things the study identified the situations where verbal communication continued to be needed in spite of the availability of the information in the EHR. For example, verbal communication was used to draw the attention of the provider to important patient status information that might otherwise not be salient, as well as to confirm that the provider was aware of the information. The results pointed to opportunities to improve EHR systems.

Observational studies have also been used to examine the impact of new technology such as surgical robots, on operating room workflow, teamwork and patient safety. For example, observational studies have been used to document workflow disruptions in robotic surgeries, the factors contributing to them and the impact on safety (Catchpole et al. 2018). Catchpole and colleagues observed 89 robotic surgeries and documented 4229 flow disruptions, defined as deviations from the natural progression of the operation. The researchers found that flow disruptions rates due to problems in communication and coordination were comparable to those for other types of surgeries. In contrast flow disruption rates due to equipment problems (e.g. improper insertion of the camera; fogging of the endoscope) were much higher pointing to opportunities to improve performance through changes in training, equipment or workflow.

Observational methods have also been used to explore verbal and non-verbal aspects of team communication in robotic surgery where the surgeon sits at a robot console away from direct view of the patient on the operating table (Tiferes et al. 2016). The authors documented numerous types of verbal and non-verbal interaction between the surgeon and the physician assistants located by the patient. This included use of the robotic tool itself as a means of non-verbal communication (e.g., positioning and zooming the camera to draw the attention of the physician assistant to a particular location). This last example illustrates how new technology results in new adaptations and uses unanticipated by the system developers. The authors pointed to how the results could be leveraged to design more effective team training for robotic surgeries.

While observational studies are a powerful tool for understanding the actual demands of work, they have some limitations. First they are time and labor intensive, both in terms of the time required to conduct the study and the time required to analyze the results. Second, they require expertise in performing observational studies. Their success depends on the skills of the observers and the representativeness of the sample of observations (Roth and Patterson 2000). Third, there is a potential that the presence of the observer to impact the workflow or get biased results, for example if the individuals being observed are concerned that they are being evaluated or that they may be reported if they deviate from prescribed policies and procedures. Finally, while the approach is useful for studying every day work, it is not suitable for studying rare events that by definition would be unlikely to be observed during any particular observation period.

3.4 Artifact Analysis

One of the best ways to gain insights into how work is actually performed and the requirements for more effective support is to examine the tools (‘artifacts’ ) currently in use (Xiao 2005). Artifacts include formal aids provided and sanctioned by the institution such as procedures and checklists (e.g., formal OR checklists) as well as ‘home grown’ artifacts that practitioners have developed on their own initiative to support their own work (Xiao et al. 2009).

‘Home-grown’ artifacts developed by practitioners can highlight mismatches between the formal systems in place and the requirements of the work (Bisantz et al. 2010; Xiao 2005). They provide a window on the cognitive and collaborative aspects of work that need to be supported and the information needed to effectively support work. Artifacts can be simple, low tech, items such as ‘sticky-notes’ and paper-based ‘cheat sheets’ (also sometimes called ‘brain sheets’) that practitioners routinely use to support memory and situation awareness. Increasingly one also finds highly sophisticated computer-based visualizations and decision aids developed by computer-savvy practitioners to facilitate their own work (Xiao et al. 2009). For example, Roth and colleagues examined work practice in a military airlift organization (Roth et al. 2006). They documented a variety of new computer-based visualizations; local databases; and decision-aids that were developed as ‘home-grown’ artifacts to compensate for limitations of the formal computer-systems in place.

Analysis of participant-developed artifacts can provide a rich source of information to guide design of new HIT. For example, Bauer, Guerlain and Brown studied the use of paper-based patient flow sheets in pediatric intensive care (Bauer et al. 2006). Positive features identified included that it was portable, that it supported easy comparison of information and that it allowed for free-form annotation. Based on these observations the researchers were able to specify important functions that electronic system s should continue to support including the need to allow for flexible rather than sequential data entry; the need to allow users to optionally leave data fields unfilled; and the need to support unstructured annotations. At the same time the researchers were able to identify ways that an electronic system could improve on the paper flow sheets, including automatic calculations that were done manually with the paper form.

Similarly, Gurses, Xiao and Hu studied the paper-based clipboard created by nurse coordinators to compensate for inadequate support of the formal hospital information system (Gurses et al. 2009). Nurse coordinators painstakingly created clipboards that synthesized and reorganized information obtained from multiple disparate sources to better support their fast-paced work demands. The authors recommended modifications to the hospital information system to allow users to create and print tailored single page views that could provide ‘at a glance’ summaries of key information.

One of the most studied home-grown artifacts in healthcare is the dry erase white board (Wears et al. 2007a; Bisantz et al. 2010; Pennathur et al. 2011; Patterson et al. 2010; Xiao et al. 2007). Dry-erase status boards arose spontaneously and became ubiquitous in the ED in the mid 1980s as a means to track patients (Wears et al. 2007a). Dry erase status boards have largely been replaced by electronic systems, however, as mentioned above, not all of the functions supported by the dry-erase status board were successfully transferred to the electronic versions. While the electronic versions support basic information exchange functions (e.g., patient demographics; location; caregiver assignments), they are less effective at directing attention, maintaining awareness of provider work flow status, and coordinating work across providers (Bisantz et al. 2010; Pennathur et al. 2007). For example, as mentioned earlier, attending and resident physicians used hand drawn symbols to track (and allow others to see) their patient specific workflow status with the dry erase status board but this was not supported with the electronic version. Similarly, with the dry-erase status board it was possible to provide information about the overall ED (e.g., whether an ED pharmacist) and to annotate and track aspects of medical care by making annotations outside the matrix structure (e.g., notes at the top, lines along the side). This flexibility was no longer supported by the electronic versions.

Comparison of dry-erase status boards and electronic versions led Bisantz et al. to draw several conclusions and recommendations (Bisantz et al. 2010). Most importantly, it is not sufficient to reproduce the literal format of an existing technology. Mimicking the matrix format and basic information of the dry-erase status boards failed to support the variety of cognitive and collaborative functions that the dry-erase status boards supported. System developers need to gain a deeper understanding of the demands of the work, how existing artifacts support work and where they fall short in order to develop a firm foundation for new health IT design. In particular, the fact that dry-erase boards are highly flexible, easy to tailor, and easy to simply walk up to and input information of any kind without having to first log in, and without being limited with respect to what can be entered and where it can go, turned out to be critical elements contributing to their success (Wears et al. 2007b). The results of the analyses provided the foundation for a more extensive project to design and evaluate improved display concepts for ED status displays (Guarrera et al. 2015).

Artifact analysis provides an important window on the multiple, often subtle, demands of work. As such it is a valuable tool for health IT developers trying to gather user support requirements. Its primary limitation is the risk of adapting too literally superficial aspects of the artifact (e.g., the particular format used; the specific bits of information included) without fully appreciating all of its functionality and the full range of cognitive and collaborative support it provides. This risk can be mitigated by coupling artifact analysis with other human factors techniques such as work practice observations and practitioner interviews to obtain a richer understanding of the demands of the work environment, how the artifact supports work, and limitations of the artifact that can be overcome through effective use of new technology (e.g., automating computations, synthesizing information).

3.5 Work Oriented Evaluations

Health IT systems are often plagued with usability problems that make them difficult to use adding to inefficiency and potential for error (Zahabi et al. 2015). Of even greater concern, they may not provide effective support for the cognitive and collaborative work of the healthcare providers. One way to overcome this problem is to encourage multiple work-oriented evaluation cycles as part of the system design process.

Traditionally a distinction has been made between two types of user evaluations: formative evaluation and summative evaluation (Neilsen 1994). Formative evaluations are designed to provide feedback with respect to what aspects of the system design work well and which can be improved—that is they are intended to be learning opportunities. There are a variety of approaches to formative evaluation ranging from fast and relatively low-cost heuristic evaluations that consist of structured reviews by usability experts, to more formal usability tests that bring in representative users to exercise the system. Usability tests typically collect both performance data (e.g., number of key strokes, time to complete a task, errors) and user feedback data (e.g., via structured questionnaires). Summative evaluations are designed to provide an overall assessment of the system. They are typically conducted at the completion of a system development process to establish that the system meets pre-defined evaluation criteria.

A work-centered evaluation is an example of a usability test approach that is work-oriented (Truxler et al. 2012; Roth and Eggleston 2010). The focus is on insuring that the health IT supports the cognitive and collaborative work of the healthcare practitioners. Work-centered evaluations are designed to be diagnostic. They are intended to not only provide an overall assessment of the usability and usefulness the health IT system, but to also provide detailed a detailed assessment of: (1) which cognitive and collaborative activities the health IT supports well and which less so; (2) which features of the health IT system are useful to the health practitioners and which less so; and (3) which features of the health IT are easy to use (usable) and which less so. These provide important information to guide health IT design course correction.

Work-centered evaluations couple elements of both formative and summative evaluations (Roth and Eggleston 2010). From a summative perspective the aim is to evaluate the design against a predefined set of cognitive performance support objectives that the system is designed to meet (Clark et al. 2017). For example a cognitive performance support objective might be ‘identify hold-ups in the care of an individual patient’. Work-centered evaluations include explicit metrics to establish whether these cognitive performance support objectives have been met. These metrics include performance on test cases that are representative of the cognitive and collaborative challenges that arise in that work context that the HIT is intended to support. For example, if an HIT system is to support ‘identifying hold-ups in the care of an individual patient’ then one or more of the test cases would involve recognizing that there is a ‘hold up’ preventing progress in the flow of care of a particular patient and being able to identify what that hold up was (e.g., the attending is waiting to hear back from a consulting physician). Work-centered evaluations also collect direct user feedback on whether that cognitive performance support objective has been met. This feedback is typically obtained via rating questions on a final questionnaire that is administered after all test cases have been completed. For example, the test participant might be asked to rate on a nine-point scale whether they feel that the health IT effectively supports ‘Identify hold-ups in the care of an individual patient’.

Work-centered evaluations also include a formative evaluation aspect—an opportunity to discover need for additional improvement. The evaluations are designed to catch any usability problems that need to be addressed prior to final implementation. This is accomplished by identifying any confusions, difficulties or usability errors that test participants make during the test cases portion of the evaluation, as well as via usability rating questions included on the final questionnaire. Work-centered evaluations are also designed to probe for additional work demands not previously identified that may signal new cognitive performance support requirements and propel further design innovation. Previously unrecognized work demands and additional cognitive performance support requirements are typically elicited via open-ended questions on the final questionnaire as well as via end of session verbal debriefs. This includes explicitly asking participants to consider situations beyond the ones sampled in test cases, and indicate any ones they feel the health IT might not handle well, as well as any situations where the health IT would be particularly helpful.

A work-centered approach was used to evaluate an Emergency Department information System (EDIS) prototype designed to support awareness of the overall ED state and flow of patients through the ED, patient care, staff workload, and available resources (Clark et al. 2017). Participants performed patient planning and orientation tasks using the EDIS displays. They then rated the ability of the EDIS to support the work-oriented cognitive needs of emergency clinical staff that were identified as part of the cognitive analysis that drove the system design (i.e., the cognitive performance support requirements). The questionnaire employed a nine-point rating scale with ‘9’ indicating ‘extremely effective’. Example cognitive performance support questions include ability to ‘Identify bottlenecks or holdups preventing overall patient flow through ED’; ‘Maintain awareness of overall acuity of patients waiting and currently being treated’; and ‘Provide support for prioritizing your tasks’. The participants also rated the usability, usefulness, and predicted frequency of use of specific system components.

Overall mean ratings were positive (i.e., mean above 5) for cognitive performance support objectives, usability, usefulness, and frequency of use, indicating that the EDIS prototype would provide effective cognitive support for emergency medicine staff. At the same time, the evaluation generated diagnostic information regarding which aspects of the EDIS displays were most useful, where there were issues in usability, and the extent to which the displays supported the cognitive work of different types of providers. For example, in some cases mean usefulness scores were significantly higher than mean usability scores (e.g., for the waiting room and patient progress displays) suggesting that while waiting room and patient progress information is useful to ED staff members, the information could be displayed better.

The study also illustrated the diagnostic power of cognitive performance support oriented questions. For example, the question ‘provide support for prioritizing your tasks’ received significantly lower mean ratings (5.9 on a nine-point scale) than many other of the questions (all with mean ratings above 7). This result made sense because while the researchers identified the need to support individual task prioritization as an important requirement for the ultimate full system, this particular cognitive task was beyond the design goals of the prototype being tested. The evaluation also revealed that Nurse and Physician provider roles had significantly different perceptions of the usability and usefulness of certain EDIS components, suggesting that they have different information needs while working.

In summary key elements of work-centered evaluations include: (1) An explicit articulation and test of the cognitive performance support requirements underlying the aiding system that are used to guide the selection of test cases and test measures; (2) test participants that are representative of the target user population; (3) test cases that reflect the range of cognitive and collaborative complexity that arises in the work context; and (4) multi-faceted assessment measures, including objective measures of performance as well as a final user-feedback questionnaire that addresses usability and usefulness of the aiding system. A main strength of the approach is its work-oriented focus. A primary limitation is that it can be resource intensive to design, implement, and analyze.

3.6 Task Analysis

There are a variety of human factors task analysis methods used to formally describe work activities. These methods decompose work in terms of goals, tasks, and sub-tasks. Requirements for successful task completion are identified, including knowledge or skills, equipment, or information needs, and opportunities for error or other performance limiting factors are made explicit. The granularity of decomposition depends on the needs of analysis, and can range from high-level activities (e.g., “order medication”) to keystroke or mouse-click level actions. In some cases, time estimates are associated with activities in order to predict task completion times.

Hierarchical Task Analysis (HTA) is a common task analysis method that begins by decomposing task goals, hierarchically, into subtasks and actions (Kirwan and Ainsworth 1992; Stanton 2001). A distinguishing feature of HTA is the articulation of plans, which describe the manner in which subtasks and activities are executed. For instances, activities can be performed sequentially, subject to if-then or branching conditions, or performed iteratively until some stopping condition is met. Each node that has been decomposed into lower level actions is provided with a plan. The HTA method therefore supports a description of activity in a way that is reflective of predictable situational conditions or more flexible choice of strategy.

The family of GOMS task analytic methods (task Goals, Operators or actions, Methods or sequences of actions, and Selection rules to choose the appropriate Method) includes operators that describe cognitive, perceptual, and motor actions at the keystroke level of detail along with the times associated with the operators. GOMS models can be used to model predictable sequences of actions, including interactions with health IT systems such as electronic health records (John and Kieras 1996). Models can be used to compare task times across different systems (during procurement) or to understand impacts of operational change. A number of architectures influenced by GOMS have been implemented which support computational modeling of human activities (Byrne 2009).

Data necessary to complete task analyses (regardless of form) comes primarily from observation or interviews to allow the work tasks, performance indicators, and support requirements to be identified. Task times can be obtained through measurement, and in some cases (e.g., perceptual, cognitive, or keystroke level GOMS operators) from the published literature. Results for task analyses can be used in design (i.e., to insure critical information is present, to identify and mitigate likely sources of error, to understand when activities exceed perceptual capabilities), in system procurement (i.e., to compare times or skill requirements for critical activities), and in training (i.e., to document required knowledge and skills). For example, hierarchical task analysis was used to compare interactions with across two different drug infusion pumps in order to predict potential user errors (Chung et al. 2003). Importantly, however, task analyses are limited by the degree to which tasks are predictable a priori, and therefore are best applied to well-defined, repeated tasks (e.g., entering a medication order) rather than complex higher level tasks (e.g., diagnostic decision-making). Such complex work activities should be analyzed using other methods, such as the critical decision methods (described above) and related cognitive task analysis techniques (Bisantz and Roth 2008).

3.7 Cognitive Informatics Techniques

The development of cognitive informatics presents new opportunities to interface with human factors engineering principles. Whereas many of the previously mentioned methods and techniques can be challenging to gather data on more than 10–20 participants, the use of cognitive informatics can allow for observations across thousands of users and millions of interactions. Cognitive informatics goes beyond just measuring clicks and mouse movements, but seeks to both identify and understand the circumstances of a particular action or outcome across large numbers of users. Adelman et al. were able to identify instances of where medical providers ordered a test on a wrong patient by creating algorithms based on provider workflow (Adelman et al. 2013). The authors were able to significantly reduce the incidence of these errors by having ordering providers re-identify their patients with each order. Follow up work by Green et al. was able to replicate the work, but noted that the change in workflow increased workflow by 4.1–4.9 s per order. A reduction in wrong patient orders of almost 25% was sustained at 2 years after the implementation (Green et al. 2015). Yet further analysis of their implementation, extrapolated across the national healthcare system would require 400 additional full time emergency physicians and 900,000 extra hours of checking to make sure that the order is placed on the correct patient (Wears 2015). While the intervention is effective, future research is needed to better understand the human factors engineering principles behind why users order on the wrong patient. It could due to patient names on the screens being next to each other, interruptions, or errors in the health IT systems that route users to the wrong patient despite making the correct selection or some combination of other causes. Each of these require different interventions and improvements to the EHR workflow to design the errors out of the system. For this problem and many others, the use of cognitive informatics with human factors engineering is critical to identifying the underlying reasons for the errors and inefficiencies, and to help prioritize the most frequent and potentially catastrophic events from impacting our patients and clinicians.

4 Conclusion

This chapter provided an introduction to human factors perspectives and methods. Key methods include semi-structured interviews and focus groups, critical incident analyses, observational methods, artifact analyses and cognitive informatics approaches. Multiple health care examples of applications of these methods were provided to illustrate the power of studying work as practiced to identify sources of complexity that create risk as well as adaptive behavior of healthcare providers that contribute to system resilience and enhance safety. The examples also illustrated how human factors methods can be leveraged to identify opportunities for improvement whether through training to disseminate and reinforce effective strategies or through technology enhancements. A key point is the need to include multiple opportunities to collect information on the usability and usefulness of new technologies throughout the development process, up to and including fielding of systems in the actual work environment.

An important point to stress is that the human factors methods are appropriate for use by multiple types of organizations and stake-holders, and can be and tailored to the scope, size, and budget of the project. This includes technology vendors who may be trying to develop and upgrade health IT systems for applications across multiple hospitals, clinical organizations (e.g. ambulatory clinics, hospitals, larger healthcare systems) that might be trying to roll-out and manage new health IT systems to minimize error, and improve performance, satisfaction and safety, as well as individual healthcare researchers or leaders who may be trying to examine sources of problems or errors and identify appropriate solutions.