Coaching Teachers to Detect, Prevent, and Respond to Bullying Using Mixed Reality Simulation: an Efficacy Study in Middle Schools
- 2.1k Downloads
Teachers play a critical role in bullying prevention but often struggle to identify bullying and intervene effectively, particularly in real time and in the classroom. The coaching literature suggests that ongoing, tailored supports are often needed to promote changes in teacher practices. The purpose of this study was to examine the efficacy of an adapted version of the Classroom Check-Up (CCU) coaching model, which utilized mixed reality simulation to allow teachers to practice addressing the detection of, prevention of, and responding to bullying in the classroom. The sample included 78 teachers within 5 middle schools, randomized to either receive coaching or serve as a comparison teacher. Teachers provided survey data regarding their perceptions about the occurrence of bullying and how they respond to bullying. Classroom observations of teacher preventive and responding practices as well as student aggression were conducted by trained observers. Regression analyses examining differences between intervention and comparison teachers, following coaching provided during one school year, indicated that intervention teachers were more likely to report responding to bullies with referrals to counselors and other staff, to intervene with the victims and perpetrators, and report that they do not perceive adults at their school do enough to address bullying in schools. Survey data regarding detection approached significance; however, observational data regarding preventive practices and responding showed no differences. Together with prior research findings showing that the intervention was acceptable and feasible, these results demonstrate the promise of this preventive intervention for increasing teachers’ responding to and detection of bullying in classrooms.
KeywordsBullying prevention and intervention Coaching Teacher-focused Mixed reality simulator
Bullying is the most commonly reported discipline problem in public schools (Zhang et al. 2016) and therefore is widely experienced by school-aged youth. Bullying has received considerable attention in the USA, as reflected in the media and state policies. It has been defined as “any unwanted aggressive behavior(s) by another youth or group of youths … that involves an observed or perceived power imbalance and is repeated multiple times or is highly likely to be repeated” (p. 7; Gladden et al. 2014). Nearly all states mandate professional development to address bullying in schools (Cornell and Limber 2015), typically with the goal of encouraging the implementation of student-focused prevention curricula (Frey et al. 2000; Second Step; Grossman and Neckerman 1997; e.g., OBPP; Olweus 2005) or school-wide climate and organizational approaches (e.g., Positive Behavioral Interventions and Supports; see Bradshaw 2013). However, research indicates that teachers have difficulty detecting and responding effectively to bullying (Bradshaw et al. 2007; Demaray et al. 2013). This may vary based upon the form (e.g., some forms of bullying are harder to detect than others) but also may stem, in part, from the fact that professional development related to bullying has typically utilized a workshop or information-sharing approach. Such an approach is typically conducted without ongoing supports to enable teachers to prioritize and develop their skills in detecting, preventing, and responding to bullying behaviors in school (Reinke et al. 2011b).
In contrast, the teacher professional development literature suggests that the traditional “sit-and-get” approach is not likely to produce sustained behavior change in teachers; rather ongoing, tailored supports are needed to promote lasting and meaningful changes to teacher behavior practices (Joyce and Showers 1981; Pas et al. 2014). Although coaching has been widely implemented and researched in relevant education areas (e.g., behavioral classroom management), it has not yet been leveraged to promote teachers’ focus on the social-emotional needs of the students, such as addressing bullying (e.g., see Bradshaw et al. 2018c; Reinke et al. 2011b). The occurrence of bullying is quite harmful both to classroom and school climate as well as to individuals directly engaged in its perpetration, those who are victims of it, and for those who witness it (National Academies of Sciences 2016). This makes bullying a very sensitive topic to address with teachers (Bradshaw et al. 2013). Based on the coaching and professional development literature related to classroom management (Kraft et al. 2018; Pas et al. 2014), it is reasonable to expect that coaching would be a promising approach for improving teachers’ implementation of bullying prevention efforts. Thus, the purpose of the current study was to examine the effects of an adapted version of the Classroom Check-Up (CCU; Reinke et al. 2008) coaching model integrated with mixed reality simulation as a means for providing guided practice to address bullying in the classroom. This efficacy trial was a part of the model development process to establish preliminary evidence for further research.
Role of Teachers in Addressing Bullying in Schools
Despite growing concern about the issue of bullying (e.g., Waasdorp et al. 2017), the effects of bullying prevention programs are relatively modest (Bradshaw 2015, 2017; Hahn Fox et al. 2012). Moreover, the uptake of the available research-based programs is varied, with many schools and districts lacking resources to provide quality training and implementation supports to teachers. Yet extant research suggests that core components of effective prevention programs include systemic, classroom-based supports including the consistent use of preventive classroom management and responsive disciplinary strategies (Bradshaw 2015; Ttofi and Farrington 2011), as well as training with an adequate amount of time and intensity dedicated to it (Farrington and Ttofi 2009). Often in bullying prevention programming, the burden falls to the teachers to implement a curriculum aimed at changing student behaviors. In fact, research suggests that bullying is more likely to occur in the classroom given students spend more time in the classroom than any other school setting (Bradshaw et al. 2007). Yet little training is provided on how to manage bullying in real time and without disrupting class time. Teachers are often overwhelmed with their multitude of tasks and are focused on teaching academic curriculum (Jennings and Greenberg 2009), leaving little time to focus on social difficulties and students’ peer relationships.
Further, teachers underestimate the prevalence of bullying (e.g., Bradshaw et al. 2007), expect bullying conflicts can be independently resolved by students (e.g., Newman 2003), and respond differentially based on the form and perceived danger or seriousness of aggression. For example, teachers are more likely to intervene to overt bullying (e.g., physical and direct verbal), and less likely to with covert bullying (e.g., relational aggression; Begotti et al. 2017; Yoon et al. 2016). It is possible that this, in part, is driven by the perception that covert forms of bullying are not as harmful or damaging and the extent to which the specific situation evokes empathy from the teacher (Begotti et al. 2017; Troop-Gordon and Ladd 2015). Consistent with the health beliefs model (Becker 1974), such beliefs can undermine teachers’ motivation and drive to implement effective and often time-consuming prevention activities in the classroom. These attitudinal barriers to implementation are not specific to bullying and represent a larger issue in school-based programming (Kraft et al. 2018), particularly related to non-academic issues such as social, emotional, and behavioral supports (Domitrovich et al. 2008). Moreover, this disconnect between teachers’ perceptions of bullying and students’ perceptions of bullying can reduce students’ likelihood of going to adults for help or being responsive to adults’ prevention and engagement efforts (Bradshaw et al. 2007; Waasdorp et al. 2011).
Supporting Teachers’ Implementation of Classroom-Based Supports
A number of school-wide and classroom-focused programs have been developed to address bullying (Bradshaw 2017; Evans et al. 2014; Nickerson 2017), yet few programs include supports to adequately assist teachers in implementing the programming in their classrooms. To better support teachers’ implementation of evidence-based practices, Reinke and colleagues developed the Classroom Check-Up coaching model (CCU; Reinke et al. 2011a). The CCU was originally created to promote improved teacher classroom management strategies, while actively addressing the common barriers to implementation faced in the coaching and consultation literature, including knowledge, attitudes, and motivation for change and skill development. The CCU has been adapted to support a range of classroom programs (Bradshaw et al. 2018b; Pas et al. 2016a; Reinke et al. 2012).
The CCU utilizes a staged problem-solving approach. It includes rapport building through an initial interview, data collection, performance feedback, goal setting and action planning, implementation, and monitoring, all for the purpose of improving teacher classroom management knowledge and skills. Further, the CCU incorporates a communication technique called motivational interviewing (Miller and Rollnick 2002) that was developed in the counseling field to explicitly target improving motivation for change.
Motivational interviewing builds upon the assumption that ambivalence to change is normal; in other words, though many people desire change, change is hard and is something people often avoid or do not sustain. In the practice of coaching teachers, recognizing ambivalence as normal means that the “expert” (i.e., coach) avoids confronting teacher ambivalence about implementing new practices (e.g., by telling teachers they need to change and use new practices). Instead the coach helps to elicit and reflect back the change talk that the teacher uses. For example, a coach listens for a teacher’s frustration with the status quo (“I just cannot keep doing this”) or frustration with their situation (“I cannot stand how negative students are with one another”) and reflects it back for the teacher to hear. This communication technique helps a teacher move from ambivalence, to recognizing the desire for change in their classroom, and this motivates action.
Research on the original CCU indicated that its use was associated with an increase in teacher use of specific classroom management strategies (e.g., use of behavior-specific praise) and a decrease in student disruptions (Reinke et al. 2008). There is some extant research on CCU adaptations that have also provided promising evidence. For example, a recent study adapted the CCU to address teachers’ use of culturally responsive practices in classrooms, and similarly demonstrated improved teacher practices and acceptability (see Bradshaw, Pas et al. 2018; Pas et al. 2016b).
Technological Enhancements to Coaching
The use of a mixed reality (i.e., part real person, part electronic) simulation technology, TeachLivE© (Dieker et al. 2008), is a promising method for providing the opportunity for guided practice to teachers (e.g., Dieker et al. 2014; Elford et al. 2013; Larson et al. submitted for publication; Pas et al. 2016a). Such guided practice (i.e., with or without technology) is largely absent from extant coaching and consultation approaches. More typically, teachers practice new skills within the classroom and are possibly later observed and given feedback by a coach. The topic of bullying is sensitive in nature; moreover, bullying may be unlikely to occur during a finite and brief classroom coaching session. Thus, TeachLivE© has the potential to be very beneficial for learning how to handle relatively low frequency but highly sensitive and potentially harmful behaviors like bullying, by ensuring that the teacher and coach can witness bullying instances during a set observational time and allowing for coach feedback. Therefore, providing guided practice on how to handle these challenging behaviors in such a controlled environment optimizes the experience for both the coach and the teacher, without exposing students to harm.
TeachLivE© allows the teacher to teach a lesson to five computer-generated, animated middle school-aged student avatars, while the avatars display poor classroom behaviors that are specified by the coach; this can easily include socially unfavorable peer interactions (e.g., bullying; Bradshaw et al. 2018a). The avatar students react to the human teacher in real time through a trained, live voice-actor (i.e., interactor; see Dieker et al. 2008; http://teachlive.org). Prior research on TeachLivE© has demonstrated that it is an acceptable, efficient, and potentially efficacious tool for changing teacher practice (Dieker et al. 2014; Elford et al. 2013; Pas et al. 2016a). In one research study, the CCU was integrated with the TeachLivE© (Dieker et al. 2008) mixed reality simulation technology to assist special educators; specifically, this study found that the participating teachers showed improvements in their use of classroom management strategies and students demonstrated behavioral improvements (Pas et al. 2016a). No such research has been conducted using this innovative coaching and simulated guided practice opportunity in relation to bullying.
The Current Study
The overarching goal of the current study was to shift the typical bullying prevention programming focus from student skill development to teacher skill development as a means for addressing bullying in schools; as such, our focus on teachers and behavior management in relation to bullying represents a novel approach within the bullying literature. Further, it expands the application of the coaching literature, which has typically focused on outcomes like academics (Kraft et al. 2018; Scott et al. 2012), the implementation of specific interventions (e.g., Becker et al. 2013; Pas et al. 2015a), and behavior management (Reinke et al. 2008), to address the issue of bullying. This study leveraged the Classroom Check-Up process and materials and adapted them to reflect bullying-specific knowledge, skills, and strategies. The Classroom Check-Up was selected as the coaching model because of its overlapping focus on behavioral management and the motivational interviewing approach, which helps to address attitudes and beliefs that may make bullying programming and implementation more challenging. The TeachLive© mixed reality simulator was integrated into the coaching to provide teachers with guided practice in skills and strategies related to the detection, prevention, and responding to specific bullying events that occur in the classroom.
Improve teachers’ detection, as demonstrated through increases in teacher-reported prevalence of bullying, increases in the degree to which teachers found bullying to be problematic in the school, and the degree to which they felt teachers in the building were responsive enough to bullying.
Change teachers’ responses to bullying, as demonstrated through teacher-reported ability to engage in concrete responses to bullying (e.g., seeking assistance from mental health professionals, intervening directly when witnessing bullying). Research suggests teachers often do not know what to do in response to bullying behaviors (Bradshaw et al. 2013; Bradshaw et al. 2018c).
Increase teacher use of preventive strategies by the intervention teachers, as demonstrated through teacher self-report and observed teacher practices.
Ultimately, the intervention would improve the distal outcome of student aggressive behavior. Earlier studies of the CCU have demonstrated improvements in student, non-aggressive, behavior (Bradshaw et al. 2018b; Pas et al. 2016a). Therefore, we were interested in whether this intervention resulted in reductions in student aggression.
Teacher demographic characteristics
Native American/American Indian
9 or more years
Teachers were recruited for participation in September and October of 2016. Once teachers consented to participate, baseline teacher surveys were administered and classroom observations were conducted (i.e., that fall). Data were completed by and for all 78 teachers. Teachers were then randomized to the intervention (referred to from here on as the Bullying CCU or BCCU) or the comparison condition. The BCCU intervention began in January of 2017 and was implemented through April 2017. Spring data collection was conducted in May through June of 2017 and 75 teachers completed the survey. Observations were conducted in all classrooms. The Institutional Review Board at the authors’ institutions approved this study; all teachers volunteered for participation and provided written consent.
Coaches were three doctoral-level practitioners who had a background in teaching or school/clinical psychology. All coaches were women. The coaches were trained and supervised by the lead author on this study and contributed to the development and refinement of this model.
The BCCU is grounded on the theory that foundational to successfully addressing bullying in schools is a positive classroom climate, positive behavior supports, and strong teacher-student relationships. Though these preventive elements are seen as necessary (e.g., Bradshaw 2013; Waasdorp et al. 2012), they are not sufficient to fully address bullying. Teachers also need the skills to detect bullying on an ongoing basis and respond effectively in real time. As noted above, these are skills that teachers currently lack and need support for, and the BCCU was developed in response to this need. See Bradshaw et al. (2018c) for additional information on the BCCU.
As mentioned earlier, the Classroom Check-Up (Reinke et al. 2011a) includes a staged problem-solving process for the teacher to select a goal for improvement in their classroom and to design and implement a plan, which was applied in this adaptation. Whereas the focus of the original CCU was on classroom management and climate, it was adapted in the current study to include explicit data collection and feedback points regarding the teachers’ detection, prevention, and responding to bullying. As briefly described above, in a typical implementation of the CCU, the teacher would practice their targeted skill within the classroom and get continued support and feedback from their coach. In this BCCU version, guided practice was provided in the mixed reality simulator TeachLivE©. Teachers were offered three sessions within the simulator to practice skills they selected within their individualized goal setting and action planning. All teachers were also exposed to practice and live feedback regarding each domain of detection, prevention, and responding to bullying. The guided practice in the teaching simulator is both a novel and important component of the coaching, as it ensured that all elements could be practiced in the presence of a coach and that teachers could receive immediate feedback and practice again. Further, research indicates that the majority of students perceive that current teacher responses to bullying often worsen the situation rather than make it better (Bradshaw et al. 2007); therefore, practice within the regular classroom could be perceived as harmful. Finally, coaches distributed four psychoeducational materials (i.e., two-sided tip sheets referred to as “Bullying Bulletins”) addressing the definition of bullying as well as strategies for detecting, preventing, and responding to bullying (i.e., one bulletin per topic). These were discussed by the coach and teacher to ensure that teacher knowledge of the topic was addressed. As in the original CCU, the motivational interviewing was embedded within the materials (e.g., interview process and within the action planning tool) to ensure it was utilized to help shift teacher attitudes and motivation.
Consistent with best practice in bullying prevention (e.g., Bradshaw 2013, 2017), during the coaching sessions, teachers were taught to demonstrate perspective taking by first labeling the bullying behavior, and expressing empathy and apologizing to victims of bullying in the classroom (i.e., in instances where the victim was easily identified); this direct and immediate response served to acknowledge the behavior, check in with the victimized student, reflect a sense of caring, and to model these effective and direct social-emotional responses to all students. Teachers were coached to do this in a time efficient, yet sincere, manner. For example, teachers were encouraged to use statements like, “I am sorry that happened in my classroom; I know that if someone said that to me, I would feel hurt.”
In summary, the BCCU intervention included the following five steps: (1) an assessment of the classroom through the CCU motivational interview, teacher completion of a checklist, and coach-conducted classroom visits to observe teacher and student behaviors; (2) an integrated feedback session drawing from all data sources; (3) goal setting and the development of an action plan addressing the two teacher-selected goals; (4) ongoing support via guided practice and performance feedback (Mesa et al. 2005) within the TeachLivE© mixed reality simulator; and (5) faded coach support. It is important to note that the initial feedback session was based on findings from the objective data collection from step 1, and the coach used motivational interviewing to provide it in a non-judgmental and empowering manner. For example, consistent with the CCU model, coaches did not tell teachers what to work on, but rather presented the data and asked what the teacher would like to work on. Similarly, during the guided practice, the coach collected additional objective data and immediately provided feedback to the teachers. Teachers could then apply this information during their second opportunity in the simulator based on the data. The personalized coaching was provided during one school year and required approximately four months to complete.
A prior study published from this project focused on feasibility (summarized elsewhere; Bradshaw et al. 2018c) and indicated that the coaching meetings (i.e., interview, feedback and action planning, and mixed reality simulator sessions) required less than 4 h of the teachers time, spread across up to 4 months. Fidelity to the coaching model (i.e., engaging in every step of the interview, feedback session, and goal setting and action planning process) was upheld for 95 to 100% of teachers, as reported by coaches. The greatest variability in fidelity related to planning for and engaging with the simulator. With regard to the dosage teachers received (i.e., time spent by teachers in coaching), the greatest variability came from the total time spent in coaching as well as time spent in the simulator (i.e., M = 61.95 min, SD = 23.37) and in follow-up feedback sessions (i.e., M = 19.86 min, SD = 18.12). Two teachers (5.3%) never attended the simulator, four (10.5%) attended once, 14 (36.8%) attended twice, and 18 (47.4%) attended all three times. In many cases, the coach provided content from the third session within the second session, when it was clear that the teacher would only be able to come twice. Therefore, coaches reported that 78% of teachers engaged in the preventive strategy practice typically allocated for the third session, whereas 92–97% of teachers practiced all other targeted skills, including detection, responding, and strategies specifically selected for the teachers’ action plan. Teachers provided positive feedback about the process, the coach, and the utility of the coaching in improving their knowledge. For example, nearly all teachers reported feeling that the coach delivered their support, recommendations, and technical assistance clearly and concisely and that the coaching helped to build their capacity to implement evidence-based strategies.
The teacher-report data assessed bullying consistent with the recommendation of Olweus (1993) and the Centers for Disease Control and Prevention (Gladden et al. 2014); the survey included a definition of bullying, which read, “A person is bullied when he or she is exposed, repeatedly and over time, to negative actions on the part of one or more other persons. Bullying often occurs in situations where there is a power or status difference. Bullying includes actions like threatening, teasing, name-calling, ignoring, rumor spreading, sending hurtful emails and text messages, and leaving someone out on purpose.” This teacher-report measure has been used extensively in several prior studies of bullying (Bradshaw et al. 2007; Bradshaw et al. 2013; Waasdorp et al. 2011; Waasdorp et al. 2017). It included the following items which were analyzed in the current study.
Perception of Bullying Prevention Efforts and Prevalence
Teacher detection was measured on the survey through two variables reflecting the broader perception of bullying climate and efforts for prevention (Bradshaw et al. 2014) as well as a series of items regarding witnessing of bullying. The first item asked if “Bullying is a problem at this school.” Participants responded using a 4-point Likert scale from 1 (not a problem) to 4 (a large problem). The next item assessed the perception of others intervening with bullying. Specifically, “Adults at this school are doing enough to try to stop/prevent bullying” on a 4-point Likert scale from 1 (strongly disagree) to 4 (strongly agree). These items were included in this study as important precursors to teacher behavioral change.
The teachers’ perception of his/her classroom prevalence of bullying was assessed with one item asking “Where have you seen students being bullied during the past 30 days? In your classroom” (yes = 1/no = 0). This item was included as prior research indicates that teachers underestimate the prevalence of bullying (Bradshaw et al. 2007), and therefore, we would expect that teacher assessment of bullying prevalence may increase after psychoeducation and coaching addressing improved detection of bullying.
Perceptions of bullying and prevalence and responses to witnessing bullying for intervention teachers
Teasing, picking on, making fun
Pushing or shoving
Hitting, slapping, kicking
Witness in my classroom
Intervened with the bully
Intervened with the victim
Talked to other staff
Talked to admin
Talked to bully’s parent
Talked to victim’s parent
Referred to guidance counselor
Ignored it/did nothing
Responses to Witnessing
Participants were asked “When you have seen bullying in the past 30 days, how did you respond? (check all that apply)” and responded using a yes/no for each of 9 behavioral responses (Bradshaw et al. 2007). These items were included to assess teachers’ behavioral changes in responding to bullying. See Table 2 in the results section for all included responses.
Participating teachers also responded to a series of demographic questions used as covariates, including grade taught, years in their role, gender, and race/ethnicity.
We utilized the Assessing School Settings: Interactions of Students and Teachers (ASSIST; Rusby et al. 2001) classroom observation that has been used in prior studies of adaptations of the CCU (i.e., Bradshaw et al. 2018c; Pas et al. 2016a) to measure teacher responding as well as prevention. This measure includes event-based tallies as well as Likert-type global ratings. The tallies of focus in this study were the following: (a) proactive behavioral management (i.e., the verbal and physical demonstrations of behavioral expectations prior to a behavior becoming a problem), (b) approval (i.e., recognition of students’ performance through the use of a tangible item, verbal praise, approving gestures, or physical contact), (c) proportion of time that the teacher did not respond to aggression (i.e., a non-response tally was coded each time aggression was observed, see below, but the teacher did not respond within 5 s and could include instances when the teacher did not see or hear the aggression or actively ignored it; the proportion is non-response tallies divided by aggression tallies), and (d) student aggression (i.e., coded both through a physical aggression code, which was any aversive physical contact, and a verbal and relational aggression code, which was verbal disapproval or critical judgment by any student or any action that could harm a peer’s relationship or social standing). Due to low base rates, the observations of student aggression were dichotomized into 0 (none) or 1 (1 or more instance).
The original ASSIST also included global rating scales, which were expanded upon for the purpose of this study to specifically assess teacher use of positive behavior supports. The positive behavior supports global scale of the ASSIST was comprised of items typically assessed by implementation measures of a positive behavioral approach (i.e., Schoolwide Evaluation Tool [SET]; Horner et al. 2004). The scale included five items about the presence of (a) 3–5 positively stated expectations posted in the classroom, (b) a classroom-specific behavioral matrix, (c) teacher reference to the expectations, (d) evidence of the presence of a reinforcement system to reward positive behaviors, and (e) use of the reinforcement system. Items were coded as 0 for not observed, 1 for partially observed, and 2 for observed. The Cronbach’s alpha (α) = 0.72 for the baseline data in this study. Item scores were averaged into a scale.
Given the focus on teacher social-emotional responses to bullying, we included one item, “when aggression occurred, teacher acknowledged student’s feelings (e.g., stating that the behavior is hurtful or offensive)”. Each item had a 5-point Likert scale, ranging from never (scored 0) to almost continuously/often occurred (scored 4). See Debnam et al. (2015) and Pas et al. (2015b) for additional information about the ASSIST. Data were collected in classrooms on three different occasions at baseline (i.e., fall/winter) and following the intervention (i.e., spring). Observers provided complete data during each observation conducted.
Univariate general liner models (GLM) were conducted in SPSS to examine general perceptions of bullying, the observational tallies of proactive behavior management and approval, and global ratings of positive behavior supports and teacher responding. The binary teacher survey outcomes (i.e., prevalence in the classroom, forms witnessed, and responses to bullying) and dichotomized observational aggression tallies were anlayzed using logistic regressions; therefore odds ratios are reported for these outcomes. Covariates in all models included fall baseline and race (White = 0, all other races/ethnicities = 1), years teaching (4 or more = 1, 3 or less = 0), gender (coded 1 for male), and grade (dummy coded with 7th as the reference group; two teachers taught multiple grades and were coded for all dummy codes as 1). Because no aggression was observed in over half of the classrooms (i.e., 42 out of 78), data are missing for these classrooms on the teacher non-response to aggression (i.e., the denominator for these classrooms was 0). Thus, we present descriptive data but not adjusted regression models for this outcome.
Detection of Bullying
Using the teacher survey items regarding the forms of bullying, we assessed teachers’ detection of bullying. While controlling for school and teacher race, years teaching, gender, and grade, logistic regression models showed that intervention teachers had marginally significantly higher odds of reporting a series of forms of bullying; none of these findings reached significance. There were trends detected on the items regarding witnessing (a) teasing, picking on, making fun; (b) hitting, slapping, or kicking; and (c) witnessing ignoring/leaving out. A non-significant trend was also seen for witnessing bullying in the classroom, suggesting a possible increase in teachers’ reported witnessing. See Table 2 for all intervention estimates.
Further, teachers were asked about their perception of the bullying efforts in the school. The GLMs examining these items indicated that intervention teachers had significantly lower scores, and thus more disagreement, that adults at the school are doing enough to stop/prevent bullying F = 8.83, p = .004 (Mcontrol = 2.83 [SE = .095] Mintervention = 2.50 [SE = .096]). There was also a trend for perceptions that bullying is a problem at the school, with intervention teachers perceiving that bullying is more of a problem F = 3.41, p = .069 (Mcontrol = 2.90 [SE = .097]; Mintervention = 3.16 [SE = .098]).
Responses to Bullying
These outcomes were assessed both by teacher reports and observational items. Controlling for school and teacher race, years teaching, gender, and grade, logistic regression models examined for differences in teacher-reported responses when witnessing bullying. Results indicated that intervention teachers had statistically significantly higher odds of responding to bullying witnessed by talking with other staff (OR = 3.96, p < .05). Intervention teachers also had significantly higher odds of reporting intervening by referring to a guidance counselor (OR = 5.87, p < .05). Further, intervention teachers had statistically significantly higher odds of reporting that they would intervene with the bully (OR = 4.83, p < .05) and with the victim (OR = 3.51, p = .05). See Table 2 for all estimates.
Observational measures of prevention, responding, and student aggression
Adjusted post mean (SE) control
Adjusted post mean (SE) intervention
Proactive behavioral management (tally)
Positive behavior supports (global scale)
Teacher acknowledges student feelings in response to aggression (global item)
Aggression (dichotomized tally)
Prevention of Bullying
Teacher prevention of bullying was assessed through observations in the classroom. Specifically, observers tallied proactive behavior management and instances of approvals and also rated the presence of positive behavioral supports in the classroom. No intervention effects were detected in regression analyses of these tallies or this global scale. See Table 3 for a full listing of findings.
Observers tallied all instances of aggression in the classroom using the ASSIST, and the presence of aggression was dichotomized. No intervention effects were detected in the logistic regression analysis testing this outcome. See Table 3 for a full listing of findings.
This study examined a novel, teacher-focused bullying intervention that adapted existing coaching and professional development tools (i.e., the Classroom Check-Up coaching process and materials and the innovative mixed reality simulator, TeachLive©) to address bullying in schools. This 78 teacher, within-school randomized controlled trial was utilized to establish the preliminary evidence of promise for this newly -developed approach, following coaching provided to teachers over the course of 4 months within one school year. Of interest were the proximal outcomes of teacher detection, prevention, and responding to bullying, as collected through teacher surveys and classroom observations. These were key outcomes, given the theoretical framework of the intervention, which emphasizes these three teacher skills. Further, it is hypothesized that improvements in these skills would in turn impact observed student aggression and therefore this was measured as a distal outcome.
Detection of bullying is a pre-requisite for teachers to respond. Yet research demonstrates that teachers underestimate the prevalence of bullying (Bradshaw et al. 2007; e.g., Demaray et al. 2013), which illustrates a gap in teacher detection that needs to be addressed through programming and intervention. Further, research also indicates that teachers often do not respond or ineffectively respond to bullying (Bradshaw et al. 2007; Demaray et al. 2013). The results of the current study suggested that the coaching improved teachers’ responding to, and may have improved the detection of, bullying. With regard to responding to bullying, the survey findings indicated that intervention teachers were more likely to talk with other school staff and refer to a guidance counselor in response to witnessing bullying. Teachers also responded that they were more likely to intervene both with the perpetrator and victim of bullying. Prior research has suggested that teachers are more likely to intervene with the perpetrator than the victims of bullying (Burger et al. 2015). As such, a major focus of the BCCU was on providing teachers with training on strategies for helping student victims of bullying. In focus groups with teachers who had participated in the intervention, many stated that addressing the victim was the most powerful new knowledge and behavior that they had acquired. Although there were not statistically significant differences on the classroom observational measures to support that these teacher behaviors had emerged, it is possible additional coaching was needed to result in observed behavior change; the BCCU was very low burden and only required about 4 h of active teacher time (Bradshaw et al. 2018c). Further, the tallying of non-response to aggression and rating of teacher responding with feelings were anchored to very low base rates of observed aggression, which is discussed in more detail below.
With regard to detection, intervention teachers tended to have higher odds of reporting observing hitting, slapping, or kicking; teasing; and ignoring/leaving out, although these effects did not reach statistical significance. This trend is important to note, as literature suggests that teachers struggle more so in identifying more covert and relational forms of aggression/bullying (e.g., exclusion, rumor spreading) and are less likely to perceive these covert forms warrant adult intervention as compared to more overt (e.g., physical) forms (Bauman and Del Rio 2006; Chen et al. 2015; Doll et al. 2004). Further, studies show that teachers have difficulty deciphering between play and real fighting (e.g., Richards 2016; Schåfer and Smith 1996). It is possible that due to the intervention, behaviors that teachers would otherwise label as “play fighting” and ignore were labeled as real fighting by intervention teachers after receiving the coaching.
These trends for teachers’ increased witnessing of bullying occurred within the context of unchanged observer tallies of student aggressive behaviors. As noted above, although there was a very low base rate for aggression in all classrooms (i.e., M = 0.37 events per 15 min with 53.8% of classrooms having no instances coded), it is noteworthy that the significant differences did not emerge on student aggression between intervention and comparison classrooms. This could suggest that any potential increase in witnessing by teachers is not the result of increased student aggression in this study, ruling out an iatrogenic effect of the intervention. Although we hypothesized decreases in aggression for intervention teachers, it is likely that a longer time frame (both observational time in the classroom or time lapse after the coaching ended) than was feasible in this study is needed, as is significant teacher behavior change. Given the low observed base rates of aggression as tallied by the observers, there is a bit of a floor effect for this particular outcome. Also as suggested above, though bullying and aggression are prevalent behaviors, they are not high frequency behaviors that can easily be captured in a finite observational measurement session, and so it is possible that this limited our ability to detect differences on this outcome. Further, the potential of a floor effect could be related to the presence of the external ASSIST observers in the classroom, which may have caused students to reduce their problem behavior.
Related to detection, coached teachers expressed concerns that adults at the school were not doing enough to address and prevent bullying in their school, following the intervention. This could have arisen from (1) a greater awareness of its occurrence and that such behavior should not be ignored, (2) teachers’ own reflection that they were not responding to bullying as much as needed prior to the coaching, or (3) their observation of others’ inaction that only gained their attention following the intervention. Finally, intervention teachers had a trend toward viewing the overall bullying climate as worse as compared to comparison teachers. This finding aligns with the trends of increased detection of bullying both in the classroom and in specific and varying forms.
Finally, we collected observational data regarding teacher positive behavior supports and did not detect significant differences. This is in contrast to earlier CCU findings that have shown impacts on the utilized tally measures. In post hoc explorations of the global rating items specifically, we found that there were higher ratings of the presence of 3–5 behavioral expectations posted in the classroom. It is worth noting that this is the one item that reflects a permanent product, and therefore is the most easily observable. Other global items represent skills that may be utilized, but were not observed in the limited time frame (i.e., three observations of 15 min). As compared to other coaching components, providing opportunities for practice of preventive strategies had lower fidelity (i.e., 78% of teachers practiced prevention in the simulator). As such, the non-significant findings for these strategies could relate to teachers’ lower exposure to practice. Additional conceptualization of what comprises fidelity in this model, which includes (a) how the coaches adhered to the coaching protocols (i.e., with low variability), (b) teacher dosage of time spent in coaching overall and in specific activities, and (c) the how goals were set and met within the coaching process are areas for additional research, to help further parse out the mixed pattern of findings presented here.
Limitations and Future Directions
Although there are many strengths of this study, including its development and rigorous experimental testing of a novel coaching intervention to address bullying, there are also important limitations to note. The study included a sizable number of teachers (i.e., 78) given the developmental stage of this work and in comparison to extant coaching literature. However, the BCCU would benefit from testing in a larger sample, including more schools from more than one school district. Though the intervention included some brief psychoeducation to teachers, largely in the form of tip sheets and basic one-on-one training, this may be more efficient and potentially impactful as a whole-school training that is complemented by individualized coaching through the BCCU. The current study’s within-school randomization, which was utilized to optimize statistical power, presented challenges to delivering group-based professional development and excluding or minimizing contamination for control teachers. Future research will embed the BCCU coaching into a whole-school approach, that utilizes a group-based (i.e., school-level) randomized trial to test the BCCU as part of a multi-component bullying prevention program.
The inclusion of classroom observations is a strength of this study, but there are limitations to the measure. Despite the high prevalence of bullying in schools, the base rates in classrooms are still relatively low given the finite amount of time used for observational measurement. Thus, having the power to detect differences on observed student aggression is poor. Though there are multiple extant measures of observations of bullying, at the time of the study development, we were unaware of any that were specifically designed for classrooms. Many focus on more unstructured environments such as the playground (e.g., Frey et al. 2005). There was little control over the timing of the classroom observations, as times were randomly selected. Observers were told to observe each classroom at least once during a transition time, when the classroom was the least structured and most likely to allow for aggression and bullying to emerge, but it is possible this was not sufficient. Additional control over the timing of observations within classrooms may be important. This also suggests that a contrived classroom-based group work activity that could increase the likelihood of the occurrence of negative social interactions as well as the addition of student self-report would be beneficial. The observed teacher responding by tallies and the one global item were limited given these were anchored to the low base rates of aggressive behavior observed within the classroom.
This study provides some preliminary evidence of promise that this novel and innovative coaching intervention improves teacher responding to bullying and may also improve detection of bullying and related behaviors. As compared to most bullying prevention interventions, the BCCU intervention was teacher-, not student-, focused and emphasized both classroom management and teacher demonstration of social-emotional skills as a means for detecting, preventing, and responding to bullying. This was balanced with an understanding, based on focus groups and extant literature (e.g., Herman et al. 2018), that time is a major stressor for teachers. Thus, the coaches emphasized to teachers the use of efficient, though immediate, responding so that academic time would not be lost. The coaching integrated guided practice using mixed reality simulation, so as not to rely on in-classroom practice and experiences only. This is a major shift from current teacher (pre- and in-service) training and coaching utilized in the field. Mixed reality simulation has the potential for accelerated learning that can promote the uptake of new strategies. It also allowed for repeated exposure, which could optimize the reinforcement of new skills and data suggest that teachers found this technology as acceptable (e.g., Bradshaw et al. 2018c).
We would like to thank the coaches who helped to develop and implemented this intervention, Kristine Larson, Stacy Johnson, Dana Marchese, and Lauren Kaiser. We also thank Wendy Reinke for providing consultation regarding the adaptation of the Classroom Check-up for this project.
Support for this research study comes from funding provided by the National Institute of Justice (Grant # 2015-CK-BX-008).
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they have no conflicts of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent was obtained from all individual participants included in the study.
Ideas expressed here do not represent the position or policy of the National Institute of Justice.
- Becker, K. D., Darney, D., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2013). Supporting universal prevention programs: a two-phased coaching model. Clinical Child and Family Psychology Review, 16(2), 213–228. https://doi.org/10.1007/s10567-013-0134-2.CrossRefPubMedPubMedCentralGoogle Scholar
- Bradshaw, C. P. (Ed.). (2017). Handbook on bullying prevention: a life course perspective. Washington DC: NASW Press.Google Scholar
- Bradshaw, C. P., Sawyer, A. L., & O'Brennan, L. M. (2007). Bullying and peer victimization at school: perceptual differences between students and school staff. School Psychology Review, 36(3), 361–382.Google Scholar
- Bradshaw, C. P., O’Brennan, L., Waasdorp, T. E., Pas, E. T., Blumenstyk, J., Bartolo, D., & Leff, S. S. (2018a). The new frontier: Leveraging innovative technologies to prevent bullying. In A. Vazsonyi, D. Flannery, & M. Delisi (Eds.), Cambridge Handbook of Violent Behavior and Aggression (2nd Ed; pp. 724–735). Cambridge:Cambridge University Press.Google Scholar
- Bradshaw, C. P, Pas, E. T., Bottiani, J., Debnam, K. J., Reinke, W., Herman, K., & Rosenberg, M. (2018b). Promoting cultural responsivity and student engagement through Double Check coaching of classroom teachers: An efficacy study. School Psychology Review, 47, 118–134. https://doi.org/10.17105/SPR-2017-0119.V47-2.
- Bradshaw, C. P., Waasdorp, T. E., Pas, E. T., Larson, K. E., & Johnson, S. (2018c). Coaching teachers in detection and intervention related to bullying. In J. Gordon (Ed.), Bullying Prevention and Intervention at School: Integrating Theory and Research into Best Practices (pp. 53–72). Cham: Springer International Publishing.Google Scholar
- Burger, C., Strohmeier, D., Spröber, N., Bauman, S., & Rigby, K. (2015). How teachers respond to school bullying: an examination of self-reported intervention strategy use, moderator effects, and concurrent use of multiple strategies. Teaching and Teacher Education, 51, 191–202. https://doi.org/10.1016/j.tate.2015.07.004.CrossRefGoogle Scholar
- Dieker, L. A., Rodriguez, J. A., Lignugaris/Kraft, B., Hynes, M. C., & Hughes, C. E. (2014). The potential of simulated environments in teacher education: current and future possibilities. Teacher Education and Special Education: The Journal of the Teacher Education Division of the Council for Exceptional Children, 37(1), 21–33. https://doi.org/10.1177/0888406413512683.CrossRefGoogle Scholar
- Doll, B., Song, S., & Siemers, E. (2004). Classroom ecologies that support or discourage bullying. In D. L. Espelage & S. M. Swearer (Eds.), Bullying in American schools: a social-ecological perspective on prevention and intervention (pp. 161–183). Mahwah: Lawrence Erlbaum Associates Publishers.Google Scholar
- Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K., Buckley, J. A., Olin, S., et al. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: a conceptual framework. Advances in School Mental Health Promotion, 1(3), 6–28. https://doi.org/10.1080/1754730X.2008.9715730.CrossRefPubMedPubMedCentralGoogle Scholar
- Elford, M., Carter, R. A., Jr., & Aronin, S. (2013). Virtual reality check: teachers use bug-in-ear coaching to practice feedback techniques with student avatars. Journal of Staff Development, 34(1), 40–43.Google Scholar
- Farrington, D. P., & Ttofi, M. M. (2009). School-based programs to reduce bullying and victimization. Paper presented at the (Campbell Systematic Reviews No. 6) Oslo, Norway.Google Scholar
- Frey, K. S., Hirschstein, M. K., Snell, J. L., Edstrom, L. V. S., MacKenzie, E. P., & Broderick, C. J. (2005). Reducing playground bullying and supporting beliefs: an experimental trial of the steps to respect program. Developmental Psychology, 41(3), 479–491. https://doi.org/10.1037/0012-16220.127.116.119.CrossRefPubMedGoogle Scholar
- Gladden, R. M., Vivolo-Kantor, A. M., Hamburger, M. E., & Lumpkin, C. D. (2014). Bullying surveillance among youths: uniform definitions for public health and recommended data elements, Version 1.0. Atlanta, GA: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention, U.S. Department of Education.Google Scholar
- Horner, R. H., Todd, A., Lewis-Palmer, T., Irvin, L., Sugai, G., & Boland, J. (2004). The school-wide evaluation tool (SET): a research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6(1), 3–12. https://doi.org/10.1177/10983007040060010201.CrossRefGoogle Scholar
- Mesa, J., Lewis-Palmer, T., & Reinke, W. (2005). Providing teachers with performance feedback on praise to reduce student problem behavior. Beyond Behavior, 15(1), 3–7.Google Scholar
- Miller, W., & Rollnick, S. (2002). Motivational interviewing: preparing people for change. New York: Guilford.Google Scholar
- National Academies of Sciences, E., and Medicine. (2016). Preventing bullying through science, policy, and practice. Washington, DC: National Academies Press.Google Scholar
- Nickerson, A. B. (2017). Preventing and intervening with bullying in schools: a framework for evidence-based practice. School Mental Health, 1–14. https://doi.org/10.1007/s12310-017-9221-8.
- Olweus, D. (1993). Bullying at school: what we know and what we can do. Oxford, England: Wiley-Blackwell.Google Scholar
- Pas, E. T., Bradshaw, C. P., & Cash, A. H. (2014). Coaching classroom-based preventive interventions. In M. D. Weist, N. A. Lever, C. P. Bradshaw, J. Sarno Owens, M. D. Weist, N. A. Lever, C. P. Bradshaw, & J. Sarno Owens (Eds.), Handbook of school mental health: research, training, practice, and policy (2nd ed., pp. 255–267). New York, NY, US: Springer Science + Business Media.CrossRefGoogle Scholar
- Pas, E. T., Bradshaw, C. P., Becker, K. D., Domitrovich, C., Berg, J., Musci, R., & Ialongo, N. S. (2015a). Identifying patterns of coaching to support the implementation of the good behavior game: the role of teacher characteristics. School Mental Health, 7(1), 61–73. https://doi.org/10.1007/s12310-015-9145-0.CrossRefGoogle Scholar
- Pas, E. T., Cash, A. H., O'Brennan, L., Debnam, K. J., & Bradshaw, C. P. (2015b). Profiles of classroom behavior in high schools: associations with teacher behavior management strategies and classroom composition. Journal of School Psychology, 53(2), 137–148. https://doi.org/10.1016/j.jsp.2014.12.005.CrossRefPubMedGoogle Scholar
- Pas, E. T., Johnson, S. R., Larson, K. E., Brandenburg, L., Church, R., & Bradshaw, C. P. (2016a). Reducing behavior problems among students with autism spectrum disorder: coaching teachers in a mixed-reality setting. Journal of Autism and Developmental Disorders, 46(12), 3640–3652. https://doi.org/10.1007/s10803-016-2898-y.CrossRefPubMedGoogle Scholar
- Pas, E. T., Larson, K. E., Reinke, W. M., Herman, K. C., & Bradshaw, C. P. (2016b). Implementation and acceptability of an adapted classroom check-up coaching model to promote culturally responsive classroom management. Education and Treatment of Children, 39(4), 467–491. https://doi.org/10.1353/etc.2016.0021.CrossRefGoogle Scholar
- Reinke, W. M., Herman, K. C., & Sprick, R. S. (2011a). Motivational interviewing for effective classroom management: the classroom check-up. New York: The Guilford Press.Google Scholar
- Reinke, W. M., Herman, K. C., Darney, D., Pitchford, J., Becker, K., Domitrovich, C., & Ialongo, N. (2012). Using the classroom check-up model to support implementation of PATHS to PAX. Advances in School Mental Health Promotion, 5(3), 220–232. https://doi.org/10.1080/1754730X.2012.707441.CrossRefGoogle Scholar
- Richards, C. (2016). Rough play, play fighting and surveillance: school playgrounds as sites of dissonance, controversy and fun Children's Games in the New Media Age (pp. 99–122). Abingdon: Routledge.Google Scholar
- Rusby, J. C., Taylor, T., & Milchak, C. (2001). Assessing school settings: interactions of students and teachers (ASSIST) observation system. Unpublished manual.Google Scholar
- Scott, S. E., Cortina, K. S., & Carlisle, J. F. (2012). Understanding coach-based professional development in reading first: how do coaches spend their time and how do teachers perceive coaches’ work? Literacy research and instruction, 51(1), 68–85. https://doi.org/10.1080/19388071.2011.569845.CrossRefGoogle Scholar
- Waasdorp, T. E., Pas, E. T., O' Brennan, L. M., & Bradshaw, C. P. (2011). A multilevel perspective on the climate of bullying: discrepancies among students, school staff, and parents. Journal of School Violence, 10(2), 115–132. https://doi.org/10.1080/15388220.2010.539164.CrossRefPubMedPubMedCentralGoogle Scholar
- Waasdorp, T. E., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of schoolwide positive behavioral interventions and supports on bullying and peer rejection: a randomized controlled effectiveness trial. Archives of Pediatrics and Adolescent Medicine, 166(2), 149–156. https://doi.org/10.1001/archpediatrics.2011.755.CrossRefPubMedGoogle Scholar
- Zhang, J., Musu-Gillette, L., & Oudekerk, B. (2016). Indicators of School Crime and Safety: 2015 (NCES 2016-079/NCJ 249758). Office of Justice Programs, U.S. Department of Justice, Washington, D.C.: National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics from https://nces.ed.gov/pubs2016/2016079.pdf Accessed 17 Jan 2018.