Background

The United States, like other nations, experiences both natural and manmade disasters such as fires, storms, floods and terrorist attacks. Periodic outbreaks of contagious illness pose threats of a pandemic influenza crisis. Many health professionals including physicians and those at their work sites are unprepared for a disaster crisis or sometimes even for common emergencies. A national survey of community health centers reported that only 9% of 193 respondents were prepared for a community disaster and others indicated that training was a high priority [1–3].

The Health Resources Services Administration (HRSA) and the office of the Assistant Secretary for Preparedness and Response (ASPR) administered the national Bioterrorism Training and Curriculum Development Program from 2003 to 2008 to train health professionals in all hazards preparedness [4] Academic institutions primarily were awarded $118 million to develop curriculum and provide continuing education for professionals to identify and respond to disasters. Initially, the national program focused on didactic or knowledge based training. In the last 2 years of the program, ASPR emphasized skills based approaches, specifically exercises and drills.

The California Statewide Area Health Education Center (AHEC) administered through the University of California, San Francisco’s School of Medicine, was funded to develop the cal-PEN (California Emergency Preparedness Network) with six of its 12 community AHECs. The cal-PEN developed six educational modules and delivered all hazards preparedness training to over 25,000 health professionals at various settings, predominantly in medically underserved areas throughout California. The modules covered general disaster preparedness, bioterrorism, chemical and radiological events, emerging infections, diseases following a disaster, and pandemic influenza [5].

In the last 2 years of the program, cal-PEN developed and conducted table top exercises where health professionals gathered at a conference table, assigned roles to play in a specific disaster scenario, enacted the scenario, and evaluated together their approaches and needs for improvement. The health professionals’ skills development focused on activating site specific emergency plans, use of incident command system (ICS), and integrating site plans with community and medical emergency response.

In California, the AHEC organization at both state and community levels was partnered with community health centers (CHCs) where health professionals care for the medically underserved. Thus, the clinics were a major focus and core partners in implementation of the cal-PEN exercises. Four of the cal-PEN AHECs in San Diego, Los Angeles, San Jose, and northern California conducted table top exercises to train health professionals in skills for disaster preparedness and expanded their typical geographic regions to do so. California’s community clinics had a mandate to prepare, document, and implement written disaster plans. At the time that cal-PEN implemented trainings, most clinics had not developed their plans or were in the beginning stages and certainly had not tested or practiced them [6].

We describe here the trainings and discuss the effects for learners and for clinics’ emergency plans, along with challenges and successes from the process.

Methods

Training Materials

Based on California’s guidelines for community clinic emergency plans and resources from the Hospital Bioterrorism Preparedness Program, the cal-PEN medical director developed a scenario for two exercises on pandemic influenza [7] Exercise One was used for facilities that had no emergency plan, or were developing one, and focused on needs and components for the plan. Exercise two added information about activating an incident command system, community outreach, and patient surge capacity. The exercise was adapted to the readiness and interests of participants who also chose to use scenarios of earthquakes, fires, and floods. Table 1 shows the learning objectives for these two exercises which emphasize when and how to activate an emergency plan, report an event, handle the problem, and coordinate a response. Participation in the cal-PEN module on general disaster preparedness was a prerequisite to the exercise. This introduction included the basics of an all hazards approach to disasters, the importance of a family disaster plan, the National Incident Management System (NIMS), certification in ICS 100, and a discussion about hazards and vulnerabilities and references to state based plan resources.

Table 1 Learning objectives for table top exercises knowledge
Full size table

Trainers and Trainings

Coordinators from each of the four AHECs arranged with clinics or other groups of health professionals in counties surrounding their respective regions to schedule the exercises on-site. Coordinators recruited 23 local health professionals with interest and expertise in emergency preparedness as trainers to facilitate the exercises. The coordinators and the multidisciplinary regional trainers were prepared by the cal-PEN medical director to conduct the exercises. The 23 trainers were 10 physicians, five registered nurses, one pharmacist, and seven others with varied health professional backgrounds. Periodically, each of the four coordinators served as the facilitator. Both coordinators and trainers also had experience administering the cal-PEN educational modules to health professionals in their regions. To aid in launching the exercises, a coordinators’ and trainers’ manual, the cal-PEN web site, and evaluation materials were developed. Additionally, cal-PEN’s annual faculty development workshop convened all staff and trainers and provided hands on technical assistance and feedback. Program progress and opportunities to critique a mock exercise were part of this experience.

To conduct the exercise, the facilitator and coordinator outlined the scenario and identified roles to be assumed by participants. After the scenario was presented with critical questions included, the group responded, and their responses were recorded. Issues that were important to address but not directly related to the exercise process were listed in a ‘parking lot’ so that they could be discussed at a later time. After the exercise, an after action report (AAR) was completed by the facilitator. The report included group debriefing about the experience and how well they met the criteria for an emergency plan. After completing the AAR, the group developed an action plan together.

Evaluation

Based on a community needs assessment, learning objectives and an evaluation plan were developed. This plan guided the development and monitoring of the program. Also, objectives and evaluation materials were adapted for the exercises. Data collected included:

  • participants’ background information

  • assessments of learners about the educational objectives and quality of the experience

  • AARs completed by the trainees as a group and by facilitators

  • follow up interviews at three months completed by the facilitator or coordinator

  • debriefings of the coordinators about arranging and conducting exercises

  • interviews post disaster of trainees completed by our program evaluator

  • reports completed quarterly by the AHECs about coordinators’ activities, partnerships, contacts, and issues or obstacles encountered.

Initially, when a site was contacted to schedule an exercise, the coordinators completed an assessment of the status of the site’s emergency plan with the emergency manager or other administrator. They used a checklist of 15 components for an emergency plan (Table 2). Coordinators repeated this same assessment on-site before beginning the exercise, and for some, repeated the assessment once again 3 months after the exercise to note any changes. Following the exercise, the facilitator and the group completed the AAR together indicating whether the criteria for their emergency plans were met, not met, not observed, or not applicable during the exercise. Comments were noted for each while identifying strengths and weaknesses and plans for improvement within each site’s plan.

Table 2 Numbers and percentages of 15 criteria in 23 CHCs’ emergency operations plans at three intervals: initial contact, at time of exercise, and 3 months after exercise
Full size table

During the three month follow up interviews, the coordinator or facilitator identified weaknesses or plans for improvements listed at the time of the AARs and any corrective actions taken. Technical support was provided where needed.

Results

Participants and Sites

A total of 90 table top exercises were conducted in 18 of California’s 58 counties, both rural and urban. Of these, 77 (86%) used Exercise One, and 13 (14%) used Exercise Two. Three-fourths (66, 73%) of the exercises were conducted at community clinics, including 51 at community health centers, nine at rural hospital clinics and six at other types of clinics including two school based clinics. Other sites were organizations (22, 24%) that included six health departments, two Medical Reserve Corps (MRCs), two family medicine residency programs, one nursing home, and six others. All exercises were conducted at participants actual work sites.

Of the 1,496 health professionals and others attending the exercises, 1,319 (88%) completed background information and 1,211 (81%) completed learner evaluation forms. Table 3 shows the types of health professionals and support staff who participated. The professionals most frequently represented were nurses (19%), clinic administrators (15%), physicians (13%), and medical assistants (10%),

Table 3 Types of health professionals participating in the 90 table top exercises
Full size table

Trainee Evaluation

Trainees’ assessments as to how well the exercises addressed the educational objectives, the information relevance and the overall quality of the speaker and program were very favorable with 98–99% (N = 1,176) of participants rating these measures as good to excellent. In addition, 92–98% of these participants rated their knowledge and skills gained from the exercises as good to excellent.

Effects on CHCs Emergency Plans

With the exception of two initial pilots, all sites (N = 88) submitted information about their emergency plans. Approximately one-third (N = 30) of the 88 sites had the post exercise three month follow-up assessment. (This was not possible with other sites because the federally funded cal-PEN program ended before these could be accomplished or, in some cases the clinic emergency manager was no longer with the clinic or the clinics were responding to fire disasters in both northern and southern areas of the state and could not respond.) Of these 30 sites having post exercise assessments, 23 (77%) were federally qualified CHCs; others were groups of people in various organizations.

To assess the effects of the table top exercise on the CHCs we compared the initial baseline plan assessment, the assessment at the time of the table top exercise, and the one completed three months after the exercise. (Table 2) Most clinics (91%, N = 21) made improvements in their emergency plans by adding new components before or after the exercise with statistically significant changes in two-thirds of the 15 plan criteria (P = .001–.046). All clinics had one or more components, a total of 129 (37%), in place at the time of the coordinators’ baseline assessments. At the time of the exercise, the clinics had 158 (46%) components in place. Three months after the exercise they had a total of 254 plan components in place, essentially twice as many than at baseline assessment. As indicated previously, some of these changes occurred between the time coordinators made initial contacts and the time of the exercise itself. Almost half (11, 48%) of the clinics added one to six components during this period suggesting that the coordinators called attention to the ingredients in a plan when they gave the exercise materials to the clinic in preparation for the exercise. Three months after the exercise, 20 clinics (87%) had added one to 14 additional components to their plans.

Table 2 shows also the types of changes the clinics made in their plans. Initially at baseline assessment, most clinics had incident command systems (13, 56%), an internal communications process (16, 70%), or an evacuation plan (22, 96%) Few had hazard and vulnerability, risk assessment, triggers for the plans, a process to communicate with outside stakeholders, guidelines for shelter, emergency operation center forms or procedures, or pandemic flu annex. These elements dramatically improved from baseline assessments to the 3 months follow-up and appeared in the plans two to three times more frequently.

The clinics made other changes as noted by the coordinators conducting the follow-up assessments: updating or modifying their plans, improving communications, identifying or obtaining new resources, and instituting or planning further training. Clinics updated their plans, developed family plans, ICS/NIMS staff certifications and improved policies, inventories, closing and shelter procedures and provisions for new employees. To improve communications, they established emergency phone numbers or phone trees, updated contact sheets, established a disaster committee, or coordinated with other clinics in their networks. New resources included supplies, more security, respiratory stations, new phones, alternate sites, stockpiles, multiple language pamphlets, local pharmacy, personal kits and emergency plans for each employee, a clinic emergency preparedness manual, and templates for emergency plans. Some instituted further training with fire drills, safety and disaster trainings, more of the cal-PEN modules, or more exercises and drills.

Some clinics reported that the exercises alerted them to their lack of personal preparedness for their staff to cope with a disaster and their lack of capacity to handle a mass surge of patients.

Post Disaster Interviews

Following the devastating fires in San Diego County, Fall, 2007, our program evaluator captured qualitative data through interviews with seven health professionals from community health centers (three RNs, three clinic administrators, one local cal-PEN staff) who had participated in cal-PEN exercises and asked each if their training had made a difference in their abilities to participate in the disaster response and relief, and if so, how. All responded that the training helped enormously and provided specific examples:

  • “we had to evacuate our building because of smoke and knew what to do and who to contact-we felt prepared”

  • “staff knew what roles they had and how to take care of their families first and return to work”

  • “the exercises helped us brainstorm what we needed to build into our process and policies”

  • “we learned that our clinic needed a facility for both children and pets”

  • “cal-PEN put community health centers on the map for disaster planning in San Diego County”

One CHC was destroyed by an internal fire shortly after participating in the table top exercise. Staff reported that the exercise prepared them by establishing an emergency communications system which enabled timely and efficient contacts. Four CHCs who had declined to participate in the table top exercises requested cal-PEN assistance after the disastrous fires.

Effects on Emergency Plans for All Sites

Using the criteria for an emergency plan listed in Table 2, a comparison was made for each of the 88 sites regarding the status of these components at the time the coordinator made initial contact to arrange for the exercise and again at the time of the exercise itself. Between these intervals all sites made improvements in their emergency plans with some or all of these plan criteria suggesting again that the coordinators’ initial contacts heightened the awareness of what is needed in an emergency plan and focused attention on them.

After action reports were completed at the end of each exercise for 88 (98%) of the 90 sites. Table 4 shows the degree to which the exercise performance measures were met by participants. Approximately three-fourths or more of sites (72–90%) were able to describe events and steps necessary to activate their emergency plans, actions taken once the plan is activated, roles of individuals, internal and external communications needed, how to participate in a coordinated response, and who is responsible for oversight of the plan. Fewer sites were able to explain how to correct the plan if needed (69%), to lessen the spread of disease to staff, patients, and families (62%), or to plan for a surge of infectious patients (42%). However, for sites that did not meet these measures for their emergency plans, most did not discuss these topics in their exercises.

Table 4 Exercise outcomes from after action reports for 88 Sites showing the degree to which they met or did not meet criteria for their emergency plans
Full size table

At the end of the exercise, participants identified strengths, weaknesses, and plans for improvements. The strengths and weaknesses reported related to policies and procedures about plan criteria, personnel issues about staff responsibilities for aspects of the plans, and the need for external or internal resources to support them. Needs for improvements noted additional training, equipment, communications, family disaster plans, and community relationships including alternate care sites. Some clinics reported that the AARs helped them reinforce their progress.

Challenges and Approaches

Coordinators reported several challenges. Time constraints and lack of resources in busy clinics were cited as the greatest barriers. Some clinics were apprehensive about the table top exercises because they had not paid attention to their (state required) emergency plans or did not have one. Others were overwhelmed at the daunting task of preparing a formal plan and whether they had adequate funds to support it. The coordinators utilized several approaches to minimize these challenges. They established a one to one relationship with clinic managers whose job responsibilities included emergency preparedness. In some cases, where CHCs had multiple clinics under one corporate structure, coordinators brought together key staff from their multiple sites for the exercise. In many cases, coordinators provided technical assistance and capacity building in response to clinics’ expressed needs and provided tools, templates for the plans, and information about proper procedures, or resource people to assist them in further developing their plans. Some coordinators scheduled exercises during spring and summer months when clinics were less overwhelmed with sick patients and/or holidays. They noted a need for ongoing attention to train new staff and update plans because of consistent personnel turnover particularly in underserved sites.

Benefits and Successes

The exercises improved the emergency plans in all sites, and remarkably so within CHCs as noted in the post exercise follow up. Our coordinators and trainers reported numerous successful outcomes. The exercises provided tools and resources for clinics during and after trainings and gave them access to resources that are typically available to hospitals. They enabled trainings customized at the workplace for clinic staff in settings where they potentially might apply the information. The process itself provided interdisciplinary training where all members of the health care team collaborated and experienced organizational change, while uncovering problems that could be addressed together. Care was taken to avoid fault finding and blaming which helped create a safe learning environment in which staff, clinicians, and leadership could participate and collaborate. The exercises heightened clinic staff awareness about the importance of individual emergency plans with their own families in order to remain at work in the event of a disaster. This fostered a personal sense of buy-in and support. The format of the exercises offered the advantages of delivering training in any stage of the plan or with topics that were relevant to local community vulnerabilities and rural needs in diverse facilities. Flexible approaches were important in reaching a variety of health professionals in work sites or other groups and making the training interesting and relevant for clinical practice. Coordinators provided individual clinic support and technical assistance after the training was completed. Testing our conclusions with a few participants after a real time disaster reinforced the importance of these exercises.

All of the cal-PEN sites where exercises were conducted were either clinics or organizations that had strong partnerships with clinics, mainly through the AHEC program. The exercises and technical support contributed to the enhancement of long standing relationships As in most community based programs, trust is important and likely facilitated access to busy clinicians and their motivation to change their clinic systems. Following the cal-PEN activities clinics continued to report additional training needs in disaster preparedness. Given the often stressful conditions of increasing patients, insufficient clinicians, and inadequate financial support under which CHCs operate, scheduling of trainings can be difficult. Clinic leaders indicated the importance of considering these issues in planning trainings with clinics.

These issues are critical to consider since there are increasing expectations and roles for community clinics to have during a disaster. Often, they serve as a necessary link between the community and hospitals, schools, law enforcement or social service agencies.

Due to the fact that the professional groups and clinics self selected or were recruited to participate, it is possible that these clinics were more motivated to make changes and improvements than others might be. Data are not available regarding how many chose not to participate when contacted by coordinators. For clinics, the technical assistance provided by the coordinators added incentives to develop or improve plans that were already required.

While the national program and the federal funding for cal-PEN were discontinued, aspects of the program in California were sustained. One coordinator continued to conduct emergency preparedness training in other local jobs. cal-PEN prepared a group of community based trainers who had interest, expertise, and motivation to learn more, and most continued to teach emergency preparedness in similar or other venues. One of the AHECs received county emergency funding and continued preparedness training as a focus. The statewide AHEC program maintained a web site offering resources including the six educational modules, related handouts, and posters. The cal-PEN exercises made the AHEC program more visible as an educational delivery system and particularly one for their CHC partners.

Conclusions

The table top exercises provided a means for clinics and others to develop or improve emergency plans and begin a dialogue before an emergency happens. They raised health professionals’ awareness about disaster preparedness and improved their skills. The clinics improved their capacities to plan a coordinated response to emergencies. The training model used here could be used for other clinics for preparation of emergency plans or further training for emerging threats. The AHEC program with its well established academic-community partnerships, particularly with CHCs, provided a respected educational delivery system Evidence here suggests that exercises do improve the knowledge and skills of health professionals and empower their work sites to be prepared for any emergency.